Remoote.app

Work from anywhere. Start here.

Free resume builderCompaniesLog in
Apply

Lead Data Engineer

Posted about 13 hours agoViewed

View full description

💎 Seniority level: Lead, 10+ years

📍 Location: United States

💸 Salary: 152500.0 - 178000.0 USD per year

🔍 Industry: Software Development

🗣️ Languages: English

⏳ Experience: 10+ years

🪄 Skills: AWSDockerLeadershipPostgreSQLPythonSoftware DevelopmentSQLAgileBashCloud ComputingGitJavaJenkinsKubernetesMongoDBRabbitmqApache KafkaData engineeringCommunication SkillsCI/CDProblem SolvingRESTful APIsMentoringTerraformMicroservicesJSONData visualizationTeam managementAnsibleData modelingSoftware EngineeringData analyticsData management

Requirements:
  • 10+ years of professional software development or data engineering experience (10+ with a STEM B.S. or 8+ with a relevant Master's degree)
  • Strong proficiency in Python and familiarity with Java and Bash scripting
  • Hands-on experience implementing database technologies, messaging systems, and stream computing software (e.g., PostgreSQL, PostGIS, MongoDB, DuckDB, KsqlDB, RabbitMQ)
  • Experience with data fabric development using publish-subscribe models (e.g., Apache NiFi, Apache Pulsar, Apache Kafka and Kafka-based data service architecture)
  • Proficiency with containerization technologies (e.g., Docker, Docker-Compose, RKE2, Kubernetes, and Microk8s)
  • Experience with version control systems (e.g., Git), CI/CD tools (e.g., Jenkins), and collaborative development workflows
  • Strong knowledge of data modeling and database optimization techniques
  • Familiarity with data serialization languages (e.g., JSON, GeoJSON, YAML, XML)
  • Excellent problem-solving and analytical skills that have been applied to high visibility, important data engineering projects
  • Strong communication skills and ability to lead the work of other engineers in a collaborative environment
  • Demonstrated experience in coordinating team activities, setting priorities, and managing tasks to ensure balanced workloads and effective team performance
  • Experience managing and mentoring development teams in an Agile environment
  • Ability to make effective architecture decisions and document them clearly
  • Must be a US Citizen and eligible to obtain and maintain a US Security Clearance
Responsibilities:
  • Develop and continuously improve a data service that underpins cloud-based applications
  • Support data and database modeling efforts
  • Contribute to the development and maintenance of reusable component libraries and shared codebase
  • Participate in the entire software development lifecycle, including requirement gathering, design, development, testing, and deployment, using an agile, iterative process
  • Collaborate with developers, designers, testers, project managers, product owners, and project sponsors to integrate the data service to end user applications
  • Communicate tasking estimation and progress regularly to a development lead and product owner through appropriate tools
  • Ensure seamless integration between database and messaging systems and the frontend / UI they support
  • Ensure data quality, reliability, and performance through code reviews and effective testing strategies
  • Write high-quality code, applying best practices, coding standards, and design patterns
  • Team with other developers, fostering a culture of continuous learning and professional growth
Apply

Related Jobs

Apply
🔥 Lead Data Engineer
Posted 4 days ago

📍 United States, Canada

🧭 Full-Time

🔍 Software Development

🏢 Company: CollegeVine👥 51-100💰 $24,000,000 over 2 years agoHigher EducationArtificial Intelligence (AI)SaaSGenerative AIEnterprise Software

  • 10+ years of software engineering experience (at least 5 in data)
  • Proficiency in Python, Scala, and of course SQL
  • Deep expertise with Spark or a similar distributed processing framework, having built and tuned production workloads in the framework
  • Experience designing, communicating, and implementing data platforms (ideally from the ground up)
  • Extremely comfortable managing complex data projects and stakeholder expectations (you’ll frequently be working directly with CollegeVine’s C-level).
  • Own problems end-to-end, deliver results quickly, and be willing to pick up whatever knowledge you're missing to get the job done
  • Partner with subject matter experts across the company including security, product, design, and customer success
  • Drive our data architecture and engineering decisions, bringing your strong experience and knowledge to bear
  • Focus on solving the problems at hand, not just writing code (although that’s typically how CollegeVine delivers solutions)
  • Mentor other engineers (data or otherwise) on effective data/distributed systems engineering practices

LeadershipPythonSQLCloud ComputingETLMachine LearningAlgorithmsData engineeringData StructuresRDBMSREST APISparkCI/CDProblem SolvingMentoringScalaData visualizationData modelingSoftware Engineering

Posted 4 days ago
Apply
Apply
🔥 Lead Data Engineer
Posted 5 days ago

📍 United States

🧭 Full-Time

💸 152500.0 - 178000.0 USD per year

🔍 Software Development

🏢 Company: Hypergiant👥 101-250💰 Corporate almost 6 years agoArtificial Intelligence (AI)Machine LearningInformation TechnologyMilitary

  • 10+ years of professional software development or data engineering experience (10+ with a STEM B.S. or 8+ with a relevant Master's degree)
  • Strong proficiency in Python and familiarity with Java and Bash scripting
  • Hands-on experience implementing database technologies, messaging systems, and stream computing software (e.g., PostgreSQL, PostGIS, MongoDB, DuckDB, KsqlDB, RabbitMQ)
  • Experience with data fabric development using publish-subscribe models (e.g., Apache NiFi, Apache Pulsar, Apache Kafka and Kafka-based data service architecture)
  • Proficiency with containerization technologies (e.g., Docker, Docker-Compose, RKE2, Kubernetes, and Microk8s)
  • Experience with version control systems (e.g., Git), CI/CD tools (e.g., Jenkins), and collaborative development workflows
  • Strong knowledge of data modeling and database optimization techniques
  • Familiarity with data serialization languages (e.g., JSON, GeoJSON, YAML, XML)
  • Excellent problem-solving and analytical skills that have been applied to high visibility, important data engineering projects
  • Strong communication skills and ability to lead the work of other engineers in a collaborative environment
  • Demonstrated experience in coordinating team activities, setting priorities, and managing tasks to ensure balanced workloads and effective team performance
  • Experience managing and mentoring development teams in an Agile environment
  • Ability to make effective architecture decisions and document them clearly
  • Develop and continuously improve a data service that underpins cloud-based applications
  • Support data and database modeling efforts
  • Contribute to the development and maintenance of reusable component libraries and shared codebase
  • Participate in the entire software development lifecycle, including requirement gathering, design, development, testing, and deployment, using an agile, iterative process
  • Collaborate with developers, designers, testers, project managers, product owners, and project sponsors to integrate the data service to end user applications
  • Communicate tasking estimation and progress regularly to a development lead and product owner through appropriate tools
  • Ensure seamless integration between database and messaging systems and the frontend / UI they support
  • Ensure data quality, reliability, and performance through code reviews and effective testing strategies
  • Write high-quality code, applying best practices, coding standards, and design patterns
  • Team with other developers, fostering a culture of continuous learning and professional growth

AWSDockerLeadershipPostgreSQLPythonSoftware DevelopmentSQLAgileBashGitJavaJenkinsKubernetesMongoDBRabbitmqApache KafkaData engineeringCommunication SkillsCI/CDAgile methodologiesRESTful APIsTerraformJSONData visualizationAnsibleData modelingData management

Posted 5 days ago
Apply
Apply
🔥 Lead Data Engineer
Posted 2 months ago

📍 United States

🧭 Full-Time

💸 175000.0 - 215000.0 USD per year

🔍 Mental Health Care

🏢 Company: Charlie Health👥 501-1000💰 $850,000 Seed almost 5 years agoMental Health Care

  • Bachelor’s degree in Computer Science, Mathematics, or other technical discipline, or equivalent practical experience.
  • 7+ years experience as a software engineer, with at least 5 years of experience in a data engineering role.
  • Deep expertise in SQL, understanding CTEs, aggregation functions, window functions, partitioning, and clustering.
  • High proficiency in Python and experience with common data engineering libraries such as Pandas, Numpy, and Great Expectations.
  • Experience with a modern data stack and tools such as FiveTran, Snowflake, DBT, Dagster, Hightouch, and Tableau.
  • Experience with data exploration, profiling, governance, visualization, and activation.
  • Proven ability to thrive in an ambiguous and rapidly changing environment.
  • Experience working with sensitive data in a regulated environment.
  • Expertise in healthcare is a plus.
  • Develop, release, and maintain high-quality data pipelines using Python, FiveTran, DBT, and Snowflake.
  • Own and guide the development of the data infrastructure.
  • Develop custom integrations using Dagster.
  • Configure reverse ETL integrations using Hightouch.
  • Identify bottlenecks and implement improvements to data engineering processes, tools, and procedures.
  • Promote collaboration and learning across teams through mentoring and knowledge sharing.
  • Participate in on-call rotation to ensure data availability.

PythonSQLNumpySnowflakePandas

Posted 2 months ago
Apply
Apply
🔥 Lead Data Engineer I
Posted 3 months ago

📍 United States of America

🧭 Full-Time

💸 140000.0 - 170000.0 USD per year

🔍 Insurance

🏢 Company: joinroot

  • 4+ years as a software engineer.
  • 2+ years leading software teams.
  • Expertise in Python, Terraform, SQL, and Spark.
  • Expertise in Cloud Architecture.
  • Experience with telematics or sensor data collection systems.
  • Proven leadership of projects across multiple teams and functional domains.
  • Excellent communication skills with engineering colleagues and senior business leaders.
  • Partner with Marketing, Product, Data Science, Analytics, and Insurance experts to set the strategy for the quarters to come.
  • Identify and socialize important technical initiatives that increase the effectiveness of products, systems, and teams.
  • Coach and guide engineers in planning experiments and projects aligned with strategic objectives.
  • Contribute code each development cycle to advance the team’s impact.
  • Lead incident response to improve system resiliency.
  • Coordinate with Staff Engineers to establish and evangelize standards and best practices.

LeadershipPythonSQLStrategyData scienceSparkCommunication SkillsCollaborationTerraform

Posted 3 months ago
Apply
Apply
🔥 Lead, Data Engineer (Client Deployment) (United States)
Posted 5 months ago

📍 United States

🔍 Data Management

🏢 Company: Demyst👥 51-100💰 about 2 years agoBig DataFinancial ServicesBroadcastingData IntegrationAnalyticsInformation TechnologyFinTechSoftware

  • Bachelor's degree or higher in Computer Science, Data Engineering, or related fields. Equivalent work experience is also highly valued.
  • 5-10 years of experience in data engineering, software engineering, or client deployment roles, with at least 3 years in a leadership capacity.
  • Strong leadership skills, including the ability to mentor and motivate a team, lead through change, and drive outcomes.
  • Expertise in designing, building, and optimizing ETL/ELT data pipelines using Python, JavaScript, Golang, Scala, or similar languages.
  • Experience in managing large-scale data processing environments, including Databricks and Spark.
  • Proven experience with Apache Airflow to orchestrate data pipelines and manage workflow automation.
  • Deep knowledge of cloud services, particularly AWS (EC2/ECS, Lambda, S3), and their role in data engineering.
  • Hands-on experience with both SQL and NoSQL databases, with a deep understanding of data modeling and architecture.
  • Strong ability to collaborate with clients and cross-functional teams, delivering technical solutions that meet business needs.
  • Proven experience in unit testing, integration testing, and engineering best practices to ensure high-quality code.
  • Familiarity with agile project management tools (JIRA, Confluence, etc.) and methodologies.
  • Experience with data visualization and analytics tools such as Jupyter Lab, Metabase, Tableau.
  • Strong communicator and problem solver, comfortable working in distributed teams.
  • Lead the configuration, deployment, and maintenance of data solutions on the Demyst platform to support client use cases.
  • Supervise and mentor the local and distributed data engineering team, ensuring best practices in data architecture, pipeline development, and deployment.
  • Recruit, train, and evaluate technical talent, fostering a high-performing, collaborative team culture.
  • Contribute hands-on to coding, code reviews, and technical decision-making, ensuring scalability and performance.
  • Design, build, and optimize data pipelines, leveraging tools like Apache Airflow to automate workflows and manage large datasets effectively.
  • Work closely with clients to advise on data engineering best practices, including data cleansing, transformation, and storage strategies.
  • Implement solutions for data ingestion from various sources, ensuring the consistency, accuracy, and availability of data.
  • Lead critical client projects, managing engineering resources, project timelines, and client engagement.
  • Provide technical guidance and support for complex enterprise data integrations with third-party systems (e.g., AI platforms, data providers, decision engines).
  • Ensure compliance with data governance and security protocols when handling sensitive client data.
  • Develop and maintain documentation for solutions and business processes related to data engineering workflows.
  • Other duties as required.

AWSLeadershipProject ManagementPythonSQLAgileApache AirflowETLJavascriptJiraTableauStrategyAirflowData engineeringGoNosqlSpark

Posted 5 months ago
Apply