Lead Data Engineer

Posted 2 months agoViewed

View full description

💎 Seniority level: Senior, 6+ years

📍 Location: United States, Canada

🔍 Industry: AI-first SaaS

🗣️ Languages: English

⏳ Experience: 6+ years

🪄 Skills: PythonKafkaData engineeringGrafanaPrometheusSparkScala

Requirements:

6+ years experience in data engineering
Expertise in Python and Scala for data pipeline development
Experience with Apache Spark and Databricks
Hands-on experience with Prometheus, Grafana, and ELK stack/OpenSearch
Experience with Kafka and Flink

Responsibilities:

Develop and implement advanced data models
Collaborate on data-driven solutions for service optimization
Conduct experiments to enhance service predictions
Present findings and recommendations to stakeholders
Mentor junior data scientists

Apply

Related Jobs

Apply

🔥 Lead Data Engineer

Posted 6 days ago

📍 Ukraine, Canada

🧭 Full-Time

🔍 Online Car Market

🏢 Company: N-iX👥 1001-5000 IT Services and IT Consulting

🔧 Requirements

8+ years of experience in software engineering.
Proven experience leading complex data projects, organizing teams, and architecting and designing new solutions.
Extensive expertise in AWS services, especially Glue, Athena.
Proficiency in Python, PySpark, GCP.
Experience with SQL, Airflow, data streaming frameworks (Kafka or Firehose).
Experience with SQL Server Data Tools.
Experience in building and maintaining data pipelines.
Experience in building data platforms, not just using them.
Proficiency in data modeling techniques and best practices

💡 Responsibilities

Work during the Canadian time zone.
Simplify existing tools, make them more modular, work with technical debt.
Propose improvements, be actively involved in team discussions and decision-making, drive initiatives.
Work closely with a team of Senior Engineers, Product Manager and Engineering Manager.

AWSLeadershipPythonSQLApache AirflowGCPApache KafkaData modelingSoftware Engineering

Posted 6 days ago

Apply

🔥 Lead Data Engineer I

Posted 3 months ago

📍 United States of America

🧭 Full-Time

💸 140000.0 - 170000.0 USD per year

🔍 Insurance

🏢 Company: joinroot

🔧 Requirements

4+ years as a software engineer.
2+ years leading software teams.
Expertise in Python, Terraform, SQL, and Spark.
Expertise in Cloud Architecture.
Experience with telematics or sensor data collection systems.
Proven leadership of projects across multiple teams and functional domains.
Excellent communication skills with engineering colleagues and senior business leaders.

💡 Responsibilities

Partner with Marketing, Product, Data Science, Analytics, and Insurance experts to set the strategy for the quarters to come.
Identify and socialize important technical initiatives that increase the effectiveness of products, systems, and teams.
Coach and guide engineers in planning experiments and projects aligned with strategic objectives.
Contribute code each development cycle to advance the team’s impact.
Lead incident response to improve system resiliency.
Coordinate with Staff Engineers to establish and evangelize standards and best practices.

LeadershipPythonSQLStrategyData scienceSparkCommunication SkillsCollaborationTerraform

Posted 3 months ago

Apply

🔥 Lead, Data Engineer (Client Deployment) (United States)

Posted 4 months ago

📍 United States

🔍 Data Management

🏢 Company: Demyst👥 51-100💰 about 2 years agoBig Data Financial Services Broadcasting Data Integration Analytics Information Technology FinTech Software

🔧 Requirements

Bachelor's degree or higher in Computer Science, Data Engineering, or related fields. Equivalent work experience is also highly valued.
5-10 years of experience in data engineering, software engineering, or client deployment roles, with at least 3 years in a leadership capacity.
Strong leadership skills, including the ability to mentor and motivate a team, lead through change, and drive outcomes.
Expertise in designing, building, and optimizing ETL/ELT data pipelines using Python, JavaScript, Golang, Scala, or similar languages.
Experience in managing large-scale data processing environments, including Databricks and Spark.
Proven experience with Apache Airflow to orchestrate data pipelines and manage workflow automation.
Deep knowledge of cloud services, particularly AWS (EC2/ECS, Lambda, S3), and their role in data engineering.
Hands-on experience with both SQL and NoSQL databases, with a deep understanding of data modeling and architecture.
Strong ability to collaborate with clients and cross-functional teams, delivering technical solutions that meet business needs.
Proven experience in unit testing, integration testing, and engineering best practices to ensure high-quality code.
Familiarity with agile project management tools (JIRA, Confluence, etc.) and methodologies.
Experience with data visualization and analytics tools such as Jupyter Lab, Metabase, Tableau.
Strong communicator and problem solver, comfortable working in distributed teams.

💡 Responsibilities

Lead the configuration, deployment, and maintenance of data solutions on the Demyst platform to support client use cases.
Supervise and mentor the local and distributed data engineering team, ensuring best practices in data architecture, pipeline development, and deployment.
Recruit, train, and evaluate technical talent, fostering a high-performing, collaborative team culture.
Contribute hands-on to coding, code reviews, and technical decision-making, ensuring scalability and performance.
Design, build, and optimize data pipelines, leveraging tools like Apache Airflow to automate workflows and manage large datasets effectively.
Work closely with clients to advise on data engineering best practices, including data cleansing, transformation, and storage strategies.
Implement solutions for data ingestion from various sources, ensuring the consistency, accuracy, and availability of data.
Lead critical client projects, managing engineering resources, project timelines, and client engagement.
Provide technical guidance and support for complex enterprise data integrations with third-party systems (e.g., AI platforms, data providers, decision engines).
Ensure compliance with data governance and security protocols when handling sensitive client data.
Develop and maintain documentation for solutions and business processes related to data engineering workflows.
Other duties as required.

AWSLeadershipProject ManagementPythonSQLAgileApache AirflowETLJavascriptJiraTableauStrategyAirflowData engineeringGoNosqlSpark

Posted 4 months ago

Apply