Remoote.app

Work from anywhere. Start here.

JobsFree resume builderCompaniesLog in
Apply

Senior Data Engineer

Posted 9 days agoViewed

View full description

💎 Seniority level: Senior

📍 Location: Canada

🔍 Industry: Smart home technology

🗣️ Languages: English

🪄 Skills: PythonSQLApache AirflowETLGCPIoTMachine LearningApache KafkaData modeling

Requirements:
  • Proficiency in building data pipelines using Python, SQL, Apache Spark, Apache Kafka, and Apache Airflow.
  • Experience with cloud-based data platforms, GCP preferred.
  • Good understanding of the machine learning lifecycle and supporting data preparation.
  • Strong experience in data modeling and optimization for data warehousing solutions.
  • Excellent communication skills for collaboration and conveying technical concepts.
Responsibilities:
  • Design, build, and maintain scalable ETL/ELT pipelines for data ingestion and transformation.
  • Integrate diverse data sources into centralized data repositories.
  • Monitor, optimize, and ensure reliability of data workflows.
  • Collaborate with cross-functional teams to understand data requirements.
  • Implement data quality checks and create comprehensive documentation.
Apply

Related Jobs

Apply
🔥 Senior Data Engineer
Posted about 24 hours ago

📍 United States, Canada

🧭 Regular

💸 125000.0 - 160000.0 USD per year

🔍 Digital driver assistance services

🏢 Company: Agero👥 1001-5000💰 $4,750,000 over 2 years agoAutomotiveInsurTechInformation TechnologyInsurance

  • Bachelor's degree in a technical field and 5+ years or Master's degree with 3+ years of industry experience.
  • Extensive experience with Snowflake or other cloud-based data warehousing solutions.
  • Expertise in ETL/ELT pipelines using tools like Airflow, DBT, Fivetran.
  • Proficiency in Python for data processing and advanced SQL for managing databases.
  • Solid understanding of data modeling techniques and cost management strategies.
  • Experience with data quality frameworks and deploying data solutions in the cloud.
  • Familiarity with version control systems and implementing CI/CD pipelines.
  • Develop and maintain ETL/ELT pipelines to ingest data from diverse sources.
  • Monitor and optimize cloud costs while performing query optimization in Snowflake.
  • Establish modern data architectures including data lakes and warehouses.
  • Apply dimensional modeling techniques and develop transformations using DBT or Spark.
  • Write reusable and efficient code, and develop data-intensive UIs and dashboards.
  • Implement data quality frameworks and observability solutions.
  • Collaborate cross-functionally and document data flows, processes, and architecture.

AWSPythonSQLApache AirflowDynamoDBETLFlaskMongoDBSnowflakeFastAPIPandasCI/CDData modeling

Posted about 24 hours ago
Apply
Apply
🔥 Senior Data Engineer, Data Integration
Posted 8 days ago

📍 Canada

🔍 Home service technology

🏢 Company: Jobber👥 501-1000💰 $100,000,000 Series D almost 2 years agoSaaSMobileSmall and Medium BusinessesTask Management

  • Excellent ETL pipeline development skills and hands-on experience with Orchestration (Airflow).
  • Experience with CI/CD practices and optimizing data flow within high-volume infrastructures.
  • Experience with data ingestion systems (e.g., Fivetran, Airbyte) and reverse ETL systems (e.g., HighTouch, Census).
  • Expertise in dimensional modeling, star schemas, and warehousing concepts.
  • Experience with message queues (e.g., Kafka) and real-time stream processing.
  • Proficiency in designing and maintaining efficient data pipelines within cloud infrastructure (preferably AWS).
  • Strong SQL skills and knowledge of containerization (ECS orchestration) and AWS Lambda.
  • Ability to explain complex data concepts to technical and non-technical stakeholders.
  • Empower the Team: Develop tools, frameworks, and workflows to enhance data accessibility and enable data-driven decision-making.
  • Build robust alerting and monitoring systems for data quality and reliability.
  • Collaborate with various functions to support analyses that influence business decisions.
  • Work closely with software engineering teams for collaborative data development.
  • Accelerate Business Growth: Synchronize data between Jobber and external systems. Streamline ETL workflows using tools like Airflow and dbt.
  • Build data models for easier integration.
  • Strategize and innovate: Research emerging technologies to strengthen the data stack.
  • Participate in design and code reviews providing mentorship and knowledge sharing.
  • Ensure Data Integrity: Establish best practices for maintaining data quality.

AWSGraphQLSQLApache AirflowETLKafkaData engineeringNosqlCI/CDRESTful APIsData modeling

Posted 8 days ago
Apply
Apply
🔥 Senior Data Engineer, Data Platform
Posted 14 days ago

📍 Canada

🧭 Full-Time

🔍 Technology for small businesses

🏢 Company: Jobber👥 501-1000💰 $100,000,000 Series D almost 2 years agoSaaSMobileSmall and Medium BusinessesTask Management

  • Proven ability to lead and collaborate in team environments.
  • Strong coding skills in Python and SQL.
  • Expertise in building and maintaining ETL pipelines using tools like Airflow and dbt.
  • Experience with AWS tools such as Redshift, Glue, and Lambda.
  • Familiarity with handling large datasets using tools like Spark.
  • Experience with Terraform for infrastructure management.
  • Knowledge of dimensional modelling, star schemas, and data warehousing.
  • Design, develop, and maintain batch and real-time data pipelines within cloud infrastructure (preferably AWS).
  • Develop tools that automate processes and set up monitoring systems.
  • Collaborate with teams to extract actionable insights from data.
  • Lead initiatives to propose new technologies, participate in design and code reviews, and maintain data integrity.

AWSPythonSQLApache AirflowETLSparkTerraform

Posted 14 days ago
Apply
Apply
🔥 Senior Data Engineer
Posted 21 days ago

📍 South Africa, Mauritius, Kenya, Nigeria

🔍 Technology, Marketplaces

  • BSc degree in Computer Science, Information Systems, Engineering, or related technical field or equivalent work experience.
  • 3+ years related work experience.
  • Minimum of 2 years experience building and optimizing ‘big data’ data pipelines, architectures and maintaining data sets.
  • Experienced in Python.
  • Experienced in SQL (PostgreSQL, MS SQL).
  • Experienced in using cloud services: AWS, Azure or GCP.
  • Proficiency in version control, CI/CD and GitHub.
  • Understanding/experience in Glue and PySpark highly desirable.
  • Experience in managing data life cycle.
  • Proficiency in manipulating, processing and architecting large disconnected data sets for analytical requirements.
  • Ability to maintain and optimise processes supporting data transformation, data structures, metadata, dependency and workload management.
  • Good understanding of data management principles - data quality assurance and governance.
  • Strong analytical skills related to working with unstructured datasets.
  • Understanding of message queuing, stream processing, and highly scalable ‘big data’ datastores.
  • Strong attention to detail.
  • Good communication and interpersonal skills.
  • Suggest efficiencies and execute on implementation of internal process improvements in automating manual processes.
  • Implement enhancements and new features across data systems.
  • Improve streamline processes within data systems with support from Senior Data Engineer.
  • Test CI/CD process for optimal data pipelines.
  • Assemble large, complex data sets that meet functional / non-functional business requirements.
  • Highly efficient in ETL processes.
  • Develop and conduct unit tests on data pipelines as well as ensuring data consistency.
  • Develop and maintain automated monitoring solutions.
  • Support reporting and analytics infrastructure.
  • Maintain data quality and data governance as well as upkeep of overall maintenance of data infrastructure systems.
  • Maintain data warehouse and data lake metadata, data catalogue, and user documentation for internal business users.
  • Ensure best practice is implemented and maintained on database.

AWSPostgreSQLPythonSQLETLGitCI/CD

Posted 21 days ago
Apply
Apply
🔥 Senior Data Engineer (Remote, Canada)
Posted 4 months ago

📍 Canada

🔍 Technology / Artificial Intelligence

  • Proficiency with Open Source and Big Data technologies.
  • Hands-on experience with Hadoop, Cloudera, or similar platforms.
  • Expertise in Java development focused on efficient and scalable solutions.
  • Strong background in messaging systems like Kafka.
  • In-depth knowledge of SQL/NoSQL technologies.
  • Proven ability to design and develop workflows using Oozie or Airflow.
  • Demonstrated success in building secure and high-performance big data platforms.
  • Familiarity with cloud computing infrastructure such as AWS, GCP, or Snowflake.
  • Experience with analytical tools, languages or libraries.
  • Expertise in architecting large-scale data intelligence solutions, especially within Snowflake Cloud Data Warehouse, is a bonus.
  • Collaborate with engineering, operations, and product teams to implement new applications onto the data framework.
  • Enhance existing outdated components of the platform.
  • Conceive new functionalities for the products.
  • Work on large-scale data technologies and tackle complex problems.

AWSSQLArtificial IntelligenceCloud ComputingGCPHadoopJavaKafkaSnowflakeAirflowNosqlCommunication Skills

Posted 4 months ago
Apply
Apply
🔥 Senior Data Engineer, Data Services
Posted 6 months ago

📍 Central EU or Americas

🧭 Full-Time

🔍 Real estate investment

🏢 Company: Roofstock👥 501-1000💰 $240,000,000 Series E almost 3 years ago🫂 Last layoff almost 2 years agoRental PropertyPropTechMarketplaceReal EstateFinTech

  • BS or MS in a technical field: computer science, engineering or similar.
  • 8+ years technical experience working with data.
  • 5+ years strong experience building scalable data services and applications using SQL, Python, Java/Kotlin.
  • Deep understanding of microservices architecture and RESTful API development.
  • Experience with AWS services including messaging and familiarity with real-time data processing frameworks.
  • Significant experience building and deploying data-related infrastructure and robust data pipelines.
  • Strong understanding of data architecture and related challenges.
  • Experience with complex problems and distributed systems focusing on scalability and performance.
  • Strong communication and interpersonal skills.
  • Independent worker able to collaborate with cross-functional teams.
  • Improve and maintain the data services platform.
  • Deliver high-quality data services promptly, ensuring data governance and integrity while meeting objectives and maintaining SLAs.
  • Develop effective architectures and produce key code components contributing to technical solutions.
  • Integrate a diverse network of third-party tools into a cohesive, scalable platform.
  • Continuously enhance system performance and reliability by diagnosing and resolving operational issues.
  • Ensure rigorous testing of the team's work through automated methods.
  • Support data infrastructure and collaborate with the data team on scalable data pipelines.
  • Work within an Agile/Scrum framework with cross-functional teams to deliver value.
  • Influence the enterprise data platform architecture and standards.

AWSDockerPythonSQLAgileETLSCRUMSnowflakeAirflowData engineeringgRPCRESTful APIsMicroservices

Posted 6 months ago
Apply