Apply

Data Engineer

Posted 2 months agoViewed

View full description

πŸ’Ž Seniority level: Middle, 3+ years

πŸ“ Location: United States, Canada

πŸ’Έ Salary: 97000.0 - 140000.0 USD per year

πŸ” Industry: Software Development

🏒 Company: AgeroπŸ‘₯ 1001-5000πŸ’° $4,750,000 over 2 years agoAutomotiveInsurTechInformation TechnologyInsurance

πŸ—£οΈ Languages: English

⏳ Experience: 3+ years

πŸͺ„ Skills: AWSPythonSQLApache AirflowDynamoDBETLFlaskGitMongoDBSnowflakeApache KafkaFastAPISparkCI/CDData modeling

Requirements:
  • Extensive experience with Snowflake or similar data warehousing solutions
  • Expertise in building ETL/ELT pipelines using tools like Airflow or DBT
  • Proficiency in Python and advanced SQL skills
  • Solid understanding of data modeling techniques
  • Proven expertise in AWS services
Responsibilities:
  • Develop and maintain ETL/ELT pipelines
  • Monitor and optimize cloud costs
  • Develop and maintain data architectures
  • Establish data governance and quality frameworks
  • Work cross-functionally with stakeholders
Apply

Related Jobs

Apply

πŸ“ AL, AR, AZ, CA (exempt only), CO, CT, FL, GA, ID, IL, IN, IA, KS, KY, MA, ME, MD, MI, MN, MO, MT, NC, NE, NJ, NM, NV, NY, OH, OK, OR, PA, SC, SD, TN, TX, UT, VT, VA, WA, WI

🧭 Full-Time

πŸ” Insurance

🏒 Company: Kin Insurance

  • Depth of experience in modern big data environments.
  • Advanced knowledge and experience with SQL and Python is expected.
  • Insurance domain knowledge.
  • Design and develop data pipelines and modeling raw data for downstream ingestion.
  • Mentor and guide data engineers on your team and across the organization, while collaborating with other engineers, product managers, analysts, and stakeholders.
  • Lead a cross-functional project team with members from AppEng, DataEng, BI, and business stakeholders.

AWSPythonSQLApache AirflowETLCross-functional Team LeadershipData engineeringProblem SolvingMentoringDocumentationComplianceData visualizationData modelingData management

Posted about 1 hour ago
Apply
Apply
πŸ”₯ Data Engineer
Posted about 2 hours ago

πŸ“ United States

🧭 Contract

πŸ” Biotechnology

🏒 Company: AvomindπŸ‘₯ 11-50EmploymentHuman ResourcesRecruiting

  • Strong experience in data engineering and cloud platforms (preferably GCP).
  • Proficiency in programming languages like Python, SQL, and shell scripting.
  • Familiarity with data catalog tools (e.g., DataHub, Apache Atlas) and metadata management.
  • Experience with building and maintaining scalable ETL pipelines using orchestration tools (Dagster, Airflow).
  • Understanding of API development and integration.
  • Knowledge of data governance and data quality principles.
  • Background in biological or scientific data is a plus but not mandatory.
  • Design, build, and maintain ETL/ELT pipelines to process and transform data efficiently.
  • Develop and optimise scalable data architectures in the cloud.
  • Implement and maintain data cataloging solutions to ensure discoverability and governance.
  • Build APIs and integrations for seamless data exchange across systems.
  • Perform data quality checks and implement automated testing frameworks to ensure data accuracy and reliability.
  • Collaborate with teams to build self-service systems and promote data democratisation.
  • Document and maintain data engineering processes and best practices

PythonSQLETLGCPAirflowAPI testingData engineeringData modeling

Posted about 2 hours ago
Apply
Apply

πŸ“ United States

🧭 Full-Time

πŸ’Έ 150363.0 - 180870.0 USD per year

πŸ” Software Development

  • At least a Bachelors Degree or foreign equivalent in Computer Science, Computer Engineering, Electrical and Electronics Engineering, or a closely related technical field, and at least five (5) years of post-bachelor’s, progressive experience writing shell scripts; validating data; and engaging in data wrangling.
  • Experience must include at least three (3) years of experience debugging data; transforming data into Microsoft SQL server; developing processes to import data into HDFS using Sqoop; and using Java, UNIX Shell Scripts, and Python.
  • Experience must also include at least one (1) year of experience developing Hive scripts for data transformation on data lake projects; converting Hive scripts to Pyspark applications; automating in Hadoop; and implementing CI/CD pipelines.
  • Design, develop, test, and implement Big Data technical solutions.
  • Recommend the right technologies and solutions for a given use case, from the application layer to infrastructure.
  • Lead the delivery of compiling and installing database systems, integrating data from a variety of data sources (data warehouse, data marts) utilizing on-prem or cloud-based data structures.
  • Drive solution architecture and perform deployments of data pipelines and applications.
  • Author DDL and DML SQL spanning technical tacks.
  • Develop data transformation code and highly complex provisioning pipelines.
  • Ingest data from relational databases.
  • Execute automation strategy.

AWSPythonSQLHadoopJavaKafkaSnowflakeData engineeringSparkCI/CDScalaScriptingDebugging

Posted 1 day ago
Apply
Apply

πŸ“ USA

πŸ” Healthcare

🏒 Company: Innovaccer Inc.

  • SQL knowledge
  • ETL/ELT/Data pipeline knowledge
  • Python knowledge
  • Powershell / Bash knowledge
  • Excellent problem-solving and effective communication skills
  • Self-motivation, integrity and honesty
  • Collaborate with team, management, departments using virtual tools
  • Run Production data pipelines/processes, ensure the integrity of the data, and send out deliverables based on requirement/runbook documentation
  • Coordinate with the various technical teams to resolve issues/bugs/optimize said production processes
  • Coordinate with internal client facing team members to communicate the status of deliverables
  • Help develop/improve technical documentation to guide future software development projects and operations
  • Dedicated time to explore building out tech stack and capabilities where there are applicable use cases
  • Provide critical thinking, technical innovation, and extra attention to detail by serving as a trusted team member and peer code reviewer
  • Assists with external client communications when deliverables or receivables do not meet technical or project requirements, ensuring timely resolution and alignment

PythonSQLBashETLMicrosoft AzurePostgresData modeling

Posted 1 day ago
Apply
Apply

πŸ“ Worldwide

πŸ” Hospitality

🏒 Company: Lighthouse

  • 4+ years of professional experience using Python, Java, or Scala for data processing (Python preferred)
  • You stay up-to-date with industry trends, emerging technologies, and best practices in data engineering.
  • Improve, manage, and teach standards for code maintainability and performance in code submitted and reviewed
  • Ship large features independently, generate architecture recommendations and have the ability to implement them
  • Great communication: Regularly achieve consensus amongst teams
  • Familiarity with GCP, Kubernetes (GKE preferred),Β  CI/CD tools (Gitlab CI preferred), familiarity with the concept of Lambda Architecture.
  • Experience with Apache Beam or Apache Spark for distributed data processing or event sourcing technologies like Apache Kafka.
  • Familiarity with monitoring tools like Grafana & Prometheus.
  • Design and develop scalable, reliable data pipelines using the Google Cloud stack.
  • Optimise data pipelines for performance and scalability.
  • Implement and maintain data governance frameworks, ensuring data accuracy, consistency, and compliance.
  • Monitor and troubleshoot data pipeline issues, implementing proactive measures for reliability and performance.
  • Collaborate with the DevOps team to automate deployments and improve developer experience on the data front.
  • Work with data science and analytics teams to enable them to bring their research to production grade data solutions, using technologies like airflow, dbt or MLflow (but not limited to)
  • As a part of a platform team, you will communicate effectively with teams across the entire engineering organisation, to provide them with reliable foundational data models and data tools.
  • Mentor and provide technical guidance to other engineers working with data.

PythonSQLApache AirflowETLGCPKubernetesApache KafkaData engineeringCI/CDMentoringTerraformScalaData modeling

Posted 4 days ago
Apply
Apply
πŸ”₯ Data Engineer
Posted 5 days ago

πŸ“ Canada

🧭 Full-Time

πŸ” FinTech

🏒 Company: KOHO

  • 5+ years of mastery in data manipulation and analytics architecture
  • Advanced expertise in dbt (incremental modeling, materializations, snapshots, variables, macros, jinja)
  • Strong knowledge of SQL and how to write efficient SQL queries
  • Strong command of SQL, query optimization, and data warehouse design
  • Building strong relationships with stakeholders (the finance team), scope and prioritize their analytics requests.
  • Understanding business needs and translating them to requirements.
  • Using dbt (Core for development and Cloud for orchestration) to transform, test, deploy, and document financial data while applying software engineering best practices.
  • Troubleshooting variances in reports, and striving to eliminate them at the source.
  • Building game-changing data products that empower the finance team
  • Architecting solutions that transform complex financial data into actionable insights
  • Monitoring, optimizing and troubleshooting warehouse performance (AWS Redshift).
  • Creating scalable, self-service analytics solutions that democratize data access
  • Occasionally building dashboards and reports in Sigma and Drivetrain.
  • Defining processes, building tools, and offering training to empower all data users in the organization.

AWSPythonSQLETLData engineeringData visualizationData modelingFinanceData analytics

Posted 5 days ago
Apply
Apply

πŸ“ United States

🧭 Full-Time

πŸ’Έ 183600.0 - 216000.0 USD per year

πŸ” Software Development

  • 6+ years of experience in a data engineering role building products, ideally in a fast-paced environment
  • Good foundations in Python and SQL.
  • Experience with Spark, PySpark, DBT, Snowflake and Airflow
  • Knowledge of visualization tools, such as Metabase, Jupyter Notebooks (Python)
  • Collaborate on the design and improvements of the data infrastructure
  • Partner with product and engineering to advocate best practices and build supporting systems and infrastructure for the various data needs
  • Create data pipelines that stitch together various data sources in order to produce valuable business insights
  • Create real-time data pipelines in collaboration with the Data Science team

PythonSQLSnowflakeAirflowData engineeringSparkData visualizationData modeling

Posted 5 days ago
Apply
Apply

πŸ“ United States

🧭 Full-Time

πŸ” Healthcare

🏒 Company: Rad AIπŸ‘₯ 101-250πŸ’° $60,000,000 Series C 2 months agoArtificial Intelligence (AI)Enterprise SoftwareHealth Care

  • 4+ years relevant experience in data engineering.
  • Expertise in designing and developing distributed data pipelines using big data technologies on large scale data sets.
  • Deep and hands-on experience designing, planning, productionizing, maintaining and documenting reliable and scalable data infrastructure and data products in complex environments.
  • Solid experience with big data processing and analytics on AWS, using services such as Amazon EMR and AWS Batch.
  • Experience in large scale data processing technologies such as Spark.
  • Expertise in orchestrating workflows using tools like Metaflow.
  • Experience with various database technologies including SQL, NoSQL databases (e.g., AWS DynamoDB, ElasticSearch, Postgresql).
  • Hands-on experience with containerization technologies, such as Docker and Kubernetes.
  • Design and implement the data architecture, ensuring scalability, flexibility, and efficiency using pipeline authoring tools like Metaflow and large-scale data processing technologies like Spark.
  • Define and extend our internal standards for style, maintenance, and best practices for a high-scale data platform.
  • Collaborate with researchers and other stakeholders to understand their data needs including model training and production monitoring systems and develop solutions that meet those requirements.
  • Take ownership of key data engineering projects and work independently to design, develop, and maintain high-quality data solutions.
  • Ensure data quality, integrity, and security by implementing robust data validation, monitoring, and access controls.
  • Evaluate and recommend data technologies and tools to improve the efficiency and effectiveness of the data engineering process.
  • Continuously monitor, maintain, and improve the performance and stability of the data infrastructure.

AWSDockerSQLElasticSearchETLKubernetesData engineeringNosqlSparkData modeling

Posted 5 days ago
Apply
Apply

πŸ“ Worldwide

🧭 Full-Time

NOT STATED
  • Own the design and implementation of cross-domain data models that support key business metrics and use cases.
  • Partner with analysts and data engineers to translate business logic into performant, well-documented dbt models.
  • Champion best practices in testing, documentation, CI/CD, and version control, and guide others in applying them.
  • Act as a technical mentor to other analytics engineers, supporting their development and reviewing their code.
  • Collaborate with central data platform and embedded teams to improve data quality, metric consistency, and lineage tracking.
  • Drive alignment on model architecture across domainsβ€”ensuring models are reusable, auditable, and trusted.
  • Identify and lead initiatives to reduce technical debt and modernise legacy reporting pipelines.
  • Contribute to the long-term vision of analytics engineering at Pleo and help shape our roadmap for scalability and impact.

SQLData AnalysisETLData engineeringCI/CDMentoringDocumentationData visualizationData modelingData analyticsData management

Posted 5 days ago
Apply
Apply

πŸ“ United States

🧭 Full-Time

πŸ’Έ 183600.0 - 216000.0 USD per year

πŸ” Mental Healthcare

🏒 Company: HeadwayπŸ‘₯ 201-500πŸ’° $125,000,000 Series C over 1 year agoMental Health Care

  • 6+ years of experience in a data engineering role building products, ideally in a fast-paced environment
  • Good foundations in Python and SQL.
  • Experience with Spark, PySpark, DBT, Snowflake and Airflow
  • Knowledge of visualization tools, such as Metabase, Jupyter Notebooks (Python)
  • A knack for simplifying data, expressing information in charts and tables
  • Collaborate on the design and improvements of the data infrastructure
  • Partner with product and engineering to advocate best practices and build supporting systems and infrastructure for the various data needs
  • Create data pipelines that stitch together various data sources in order to produce valuable business insights
  • Create real-time data pipelines in collaboration with the Data Science team

PythonSQLETLSnowflakeAirflowData engineeringRDBMSSparkRESTful APIsData visualizationData modeling

Posted 6 days ago
Apply