Data Engineer
I
Irth SolutionsSaaS Infrastructure
Remote (Quebec/Canada)Full-TimeMiddle
Salary75,000 CAD per year
Apply NowOpens the employer's application page
Job Details
- Languages
- English, French
- Experience
- 3 to 5 years
- Required Skills
- PythonSQLCloud ComputingSparkCI/CDData modelingDatabricks
Requirements
- 3 to 5 years of experience in data engineering with production-grade pipelines.
- Strong proficiency in Python and SQL.
- Hands-on experience with Databricks or equivalent Spark-based environments (schema design, Delta Lake, performance tuning).
- Experience integrating with external APIs at scale (authentication, rate limiting, error handling).
- Familiarity with data modeling, data quality, and schema evolution.
- Solid understanding of data pipeline reliability practices (monitoring, alerting, failure handling).
- Experience with at least one major cloud provider (Azure preferred).
- Comfortable working with unstructured/semi-structured text data at scale.
- Strong preference for candidates residing in Quebec.
- Fluency in French (spoken and written) as a strong asset in addition to English.
Responsibilities
- Design and maintain continuous ingestion pipelines from high-volume external APIs.
- Implement ingestion and transformation workflows using Databricks, Spark, and Delta Live Tables.
- Develop infrastructure for data deduplication, filtering, and schema evolution handling.
- Configure and manage Delta Lake storage, partitioning, and query optimization routines.
- Ensure pipeline reliability through monitoring, alerting, error handling, and performance tuning.
- Build and schedule workflows using orchestration tools like Databricks Workflows.
- Collaborate with data scientists to prepare structured inputs for downstream LLM/NLP models.
View Full Description & ApplyYou'll be redirected to the employer's site
