Senior Data Engineer
Company:Loopio Inc.
Location:Canada, UK, India, EST, PST
Languages:English
Seniority level:Senior, 5+ years
Experience:5+ years
Skills:
AWSDockerLeadershipPythonSoftware DevelopmentSQLAmazon RDSApache AirflowCloud ComputingDynamoDBETLJenkinsData engineeringSparkCI/CDCollaborationProblem SolvingMentoringData modelingCross-functional collaboration
Requirements:
5+ years of experience in data engineering, ETL, data architecture, and cloud-based data platforms. Strong proficiency in AWS (S3, RDS, Redshift, Glue, IAM, DynamoDB). Deep understanding of database design, query optimization, and data modelling. Expertise in distributed data processing frameworks (Spark, Databricks, Presto). Strong Python and PySpark development skills. Experience with CI/CD pipelines (Jenkins) and data workflow automation (Airflow, Databricks Jobs). Proven ability to build and optimize large-scale data infrastructure. Strong problem-solving skills and experience mentoring engineers.
Responsibilities:
Design, build, and optimize data pipelines for high-scale ingestion, transformation, and storage. Develop ETL workflows using Databricks, Redshift, and AWS cloud technologies. Develop and maintain a feature store for predictive models and AI-driven insights. Build automation tooling for data orchestration, testing, monitoring, and administration. Architect scalable data solutions leveraging AWS services and distributed computing frameworks. Partner with Data Scientists, ML Engineers, and business stakeholders to create AI-powered insights.
