Lead Data Engineer

Posted 2024-10-23

View full description

💎 Seniority level: Lead, 10+ years

📍 Location: United States

🏢 Company: Sparc The World

⏳ Experience: 10+ years

🪄 Skills: PythonSQLStrategyData engineeringData scienceNosqlCollaboration

Requirements:

Expertise in data modeling, data architecture, and data visualization tools.
Extensive experience with SQL and NoSQL database management systems.
Strong proficiency in programming languages relevant to data engineering such as Python, R, Scala.
Strong understanding of data governance, quality standards, and lineage concepts.
Proven ability to develop and execute effective data strategies.
Excellent communication and collaboration skills.
Experience supporting AI/ML data projects and Data Science teams.
10+ years of programming and development focusing on data-driven solutions.

Responsibilities:

Develop and implement a comprehensive data strategy that aligns with growth objectives.
Lead the design and maintenance of scalable data models and architectures.
Collaborate closely with Product to identify data needs and integrate new sources.
Champion data visualization tools to enhance decision-making.
Establish data lineage and quality frameworks for data accuracy.
Drive adoption of best practices in data management and analytics.
Support AI/ML initiatives in collaboration with Data Scientists.
Foster a culture of continuous improvement and learning within the data team.

Apply

Related Jobs

Apply

🔥 Lead Data Engineer

Posted 2024-11-07

📍 North America, Latin America, Europe

🔍 Data consulting

Bachelor’s degree in engineering, computer science or equivalent area.
5+ years in related technical roles such as data management, database development, and ETL.
Expertise in evaluating and integrating data ingestion technologies.
Experience in designing and developing data warehouses with various platforms.
Proficiency in building ETL/ELT ingestion pipelines with tools like DataStage or Informatica.
Cloud experience on AWS; Azure and GCP experience is a plus.
Proficiency in Python scripting; Scala is required.

Designing and developing Snowflake Data Cloud solutions.
Creating data ingestion pipelines and working on data architecture.
Ensuring data governance and security throughout customer projects.
Leading technical teams and collaborating with clients on data initiatives.

AWSLeadershipPythonSQLAgileETLOracleSnowflakeData engineeringSparkCollaboration

Posted 2024-11-07

Apply

🔥 Lead Data Engineer

Posted 2024-10-22

📍 Canada, US

🧭 Full-Time

💸 170000 - 190000 CAD / USD per year

🔍 Blockchain infrastructure

🏢 Company: Figment

Experience with the data transformation tool DBT, including designing complex data transformations.
Programming experience in Python with advanced libraries and frameworks.
Experience with data orchestration tools like Dagster or Airflow.
Familiarity with data warehousing solutions such as Snowflake, BigQuery, or Redshift.
Extensive data engineering experience, including managing data pipelines and CI/CD infrastructure across AWS.

Lead the design and implementation of reliable data pipelines and data storage solutions.
Manage specific data pipelines and oversee the technical aspects of data operations.
Ensure data processes are optimized and align with business requirements.
Identify areas for process improvements and suggest tools to enhance efficiency.
Continuously improve data infrastructure automation for reliable processing.
Lead the development and maintenance of data pipelines and ETL processes using technologies like Dagster and DBT.

AWSLeadershipPythonBlockchainETLSnowflakeAirflowData engineeringCollaborationCI/CD

Posted 2024-10-22

Apply

🔥 Lead, Data Engineer (Client Deployment) (United States)

Posted 2024-10-15

📍 United States

🔍 Data Management

🏢 Company: Demyst

Bachelor's degree or higher in Computer Science, Data Engineering, or related fields. Equivalent work experience is also highly valued.
5-10 years of experience in data engineering, software engineering, or client deployment roles, with at least 3 years in a leadership capacity.
Strong leadership skills, including the ability to mentor and motivate a team, lead through change, and drive outcomes.
Expertise in designing, building, and optimizing ETL/ELT data pipelines using Python, JavaScript, Golang, Scala, or similar languages.
Experience in managing large-scale data processing environments, including Databricks and Spark.
Proven experience with Apache Airflow to orchestrate data pipelines and manage workflow automation.
Deep knowledge of cloud services, particularly AWS (EC2/ECS, Lambda, S3), and their role in data engineering.
Hands-on experience with both SQL and NoSQL databases, with a deep understanding of data modeling and architecture.
Strong ability to collaborate with clients and cross-functional teams, delivering technical solutions that meet business needs.
Proven experience in unit testing, integration testing, and engineering best practices to ensure high-quality code.
Familiarity with agile project management tools (JIRA, Confluence, etc.) and methodologies.
Experience with data visualization and analytics tools such as Jupyter Lab, Metabase, Tableau.
Strong communicator and problem solver, comfortable working in distributed teams.

Lead the configuration, deployment, and maintenance of data solutions on the Demyst platform to support client use cases.
Supervise and mentor the local and distributed data engineering team, ensuring best practices in data architecture, pipeline development, and deployment.
Recruit, train, and evaluate technical talent, fostering a high-performing, collaborative team culture.
Contribute hands-on to coding, code reviews, and technical decision-making, ensuring scalability and performance.
Design, build, and optimize data pipelines, leveraging tools like Apache Airflow to automate workflows and manage large datasets effectively.
Work closely with clients to advise on data engineering best practices, including data cleansing, transformation, and storage strategies.
Implement solutions for data ingestion from various sources, ensuring the consistency, accuracy, and availability of data.
Lead critical client projects, managing engineering resources, project timelines, and client engagement.
Provide technical guidance and support for complex enterprise data integrations with third-party systems (e.g., AI platforms, data providers, decision engines).
Ensure compliance with data governance and security protocols when handling sensitive client data.
Develop and maintain documentation for solutions and business processes related to data engineering workflows.
Other duties as required.

AWSLeadershipProject ManagementPythonSQLAgileApache AirflowETLJavascriptJavaScriptJiraTableauStrategyAirflowData engineeringGolangNosqlSpark

Posted 2024-10-15

Apply

🔥 Lead, Data Engineer (Client Deployment) (United States)

Posted 2024-10-01

📍 United States

🧭 Full-Time

🔍 Data Engineering

🏢 Company: Demyst

Bachelor's degree or higher in Computer Science, Data Engineering, or related fields. Equivalent work experience is also highly valued.
5-10 years of experience in data engineering, software engineering, or client deployment roles, with at least 3 years in a leadership capacity.
Strong leadership skills, including the ability to mentor and motivate a team, lead through change, and drive outcomes.
Expertise in designing, building, and optimizing ETL/ELT data pipelines using Python, JavaScript, Golang, Scala, or similar languages.
Experience in managing large-scale data processing environments, including Databricks and Spark.
Proven experience with Apache Airflow to orchestrate data pipelines and manage workflow automation.
Deep knowledge of cloud services, particularly AWS (EC2/ECS, Lambda, S3), and their role in data engineering.
Hands-on experience with both SQL and NoSQL databases, with a deep understanding of data modeling and architecture.
Strong ability to collaborate with clients and cross-functional teams, delivering technical solutions that meet business needs.
Proven experience in unit testing, integration testing, and engineering best practices to ensure high-quality code.
Familiarity with agile project management tools (JIRA, Confluence, etc.) and methodologies.
Experience with data visualization and analytics tools such as Jupyter Lab, Metabase, Tableau.
Strong communicator and problem solver, comfortable working in distributed teams.

Lead the configuration, deployment, and maintenance of data solutions on the Demyst platform to support client use cases.
Supervise and mentor the local and distributed data engineering team, ensuring best practices in data architecture, pipeline development, and deployment.
Recruit, train, and evaluate technical talent, fostering a high-performing, collaborative team culture.
Contribute hands-on to coding, code reviews, and technical decision-making, ensuring scalability and performance.
Design, build, and optimize data pipelines, leveraging tools like Apache Airflow, to automate workflows and manage large datasets effectively.
Work closely with clients to advise on data engineering best practices, including data cleansing, transformation, and storage strategies.
Implement solutions for data ingestion from various sources, ensuring the consistency, accuracy, and availability of data.
Lead critical client projects, managing engineering resources, project timelines, and client engagement.
Provide technical guidance and support for complex enterprise data integrations with third-party systems (e.g., AI platforms, data providers, decision engines).
Ensure compliance with data governance and security protocols when handling sensitive client data.
Develop and maintain documentation for solutions and business processes related to data engineering workflows.

AWSLeadershipProject ManagementPythonSQLAgileApache AirflowETLJavascriptJavaScriptJiraTableauStrategyAirflowData engineeringGolangNosqlSpark

Posted 2024-10-01

Apply

Lead Data Engineer

Requirements:

Responsibilities:

Related Jobs

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities