Why is remote work so nice? Let's try to see!
Apply
Lead Data Engineer
Posted 5 months agoViewed
💎 Seniority level: Lead, Extensive experience
💸 Salary: 175000.0 - 220000.0 USD per year
🔍 Industry: Software Development
🏢 Company: Lumos👥 51-100💰 $35,000,000 Series B 10 months agoSecurityInformation TechnologyIdentity ManagementCollaborationSoftware
🗣️ Languages: English
⏳ Experience: Extensive experience
Requirements:
- Extensive experience designing and implementing medallion architectures (bronze, silver, gold layers) or similar data warehouse paradigms.
- Proficiency in deploying data pipelines using CI/CD tools and integrating automated data quality checks, version control, and deployment automation to ensure reliable and repeatable data processes.
- Expertise in advanced SQL, ETL processes, and data transformation techniques. Strong programming skills in Python.
- Demonstrated ability to work closely with AI engineers, data scientists, product engineers, product managers, and other stakeholders to ensure that data pipelines meet the needs of all teams.
Responsibilities:
- Architect, build, and maintain cutting-edge data pipelines that empower our AI products, in-product analytics, and internal reporting.
- Ensure the scalability, reliability, and quality of our analytics data infrastructure, enabling the seamless integration of usage, spend, compliance, and access data to drive business insights and deliver exceptional value to our customers.
- Play a pivotal role in transforming complex data into actionable intelligence, fueling Lumos' growth and innovation.
Related Jobs
Apply
🔥 Staff/Lead Data Engineer
Posted 15 days ago
🧭 Full-Time
💸 220000.0 - 250000.0 USD per year
🔍 Software Development
🏢 Company: Machinify👥 51-100💰 $10,000,000 Series A over 6 years agoArtificial Intelligence (AI)Business IntelligencePredictive AnalyticsSaaSMachine LearningAnalytics
- Deep experience as a hands-on Data Engineer building production data pipelines
- Experience managing the delivery of complex data
- Experience in ETL orchestration and workflow management tools with a strong preference for Apache Airflow
- Experience in Spark or other distributed computing frameworks
- SQL and Python
- Advanced SQL performance tuning
- Kubernetes and building Docker images
- AWS & GCP
- Experience working with APIs to collect or ingest data
- Manage SLA for all pipelines in allocated areas of ownership
- Streaming technologies like kafka , spark streaming etc
- ELK stack , Grafana etc
- Independently understand all aspects of a business problem including those unrelated to their area of expertise, weigh pros and cons of different approaches and suggest ones likely to succeed
- Work with a cross-functional organization including engineering, delivering, subject-matter experts, product managers, as well as platform engineers to deliver a scalable framework.
- Map the customer data into Machinify canonical form. Identify and ingest non canonical fields and generalize the process to a minimal level of customization.
- Proactively design and adapt the canonical form to suit changing query patterns and needs.
- Ultimately own data availability and quality for the Data Science organization.
Posted 15 days ago
ApplyApply
🔥 Lead Data Engineer
Posted 22 days ago
🔍 AI Strategy and Applied AI
🏢 Company: Board of Innovation👥 101-250Management ConsultingConsultingInnovation ManagementCreative Agency
- Bachelor’s or Master’s degree in Computer Science, Data Engineering, or a related field.
- A minimum of 6 years of professional experience in data engineering.
- Proven experience working in a consulting or agency environment on project-based work.
- Experience in Python, SQL, and data transformation libraries like pandas or PySpark.
- Hands-on experience with data pipeline orchestration tools like Apache Airflow or Prefect.
- Solid understanding of database design and optimization for relational and non-relational databases.
- Familiarity with API integration for ingesting and processing data.
- Develop and manage ETL pipelines to extract, transform, and load data from various internal and external sources into harmonized datasets.
- Design, optimize, and maintain databases and data storage systems (e.g. PostgreSQL, MongoDB, Azure Data Lake, or AWS S3).
- Collaborate with AI Application Engineers to prepare data for use in foundational model workflows (e.g. embeddings and retrieval-augmented generation setups).
- Ensure data integrity, quality, and security across all pipelines and workflows.
- Monitor, debug, and optimize data pipelines for performance and reliability.
Posted 22 days ago
ApplyApply
🔥 Lead Data Engineer
Posted about 1 month ago
📍 Ukraine, Canada
🧭 Full-Time
🔍 Online Car Market
🏢 Company: N-iX👥 1001-5000IT Services and IT Consulting
- 8+ years of experience in software engineering.
- Proven experience leading complex data projects, organizing teams, and architecting and designing new solutions.
- Extensive and proven expertise in AWS services, especially S3, Glue, Athena.
- Proficiency in Python, PySpark, GCP.
- Experience with SQL, Airflow, data streaming frameworks (Kafka or Firehose).
- Simplify existing tools, make them more modular, work with technical debt.
- Propose improvements, be actively involved in team discussions and decision-making, drive initiatives.
- Work closely with a team of Senior Engineers, Product Manager and Engineering Manager.
AWSLeadershipPythonSQLApache AirflowGCPApache KafkaData modelingSoftware Engineering
Posted about 1 month ago
ApplyApply
🔥 Lead Data Engineer
Posted 2 months ago
📍 United States
🔍 Defense and Financial Technology
🏢 Company: 540
- Bachelor's Degree.
- 8+ years of related experience.
- Well-versed in Python.
- Experience building and managing data pipelines.
- Proficient in data analytics tools such as Databricks.
- Experience building dashboards using PowerBI and/or similar tools.
- Experience working via the terminal / command line.
- Experience consuming data via APIs.
- Hands-on experience using Jira and Confluence.
- Working directly with government leadership managing teams, customers, and data requirements.
- Assisting Audit teams with monthly data ingestions from Army systems.
- Management of data initiatives and small projects from start to finish.
- Working with Army FM&C Lead to prioritize Advana data product requirements.
- Developing recurring and ad hoc financial datasets.
- Developing Advana datasets and analytical products to enable the Army reporting on all Financial System data.
- Reviewing data pipeline code via GitLab to ensure it meets team and code standards.
- Overseeing overall architecture and technical direction for FM&C data projects.
AWSPython
Posted 2 months ago
ApplyApply
🔥 Lead Data Engineer I
Posted 4 months ago
📍 United States of America
🧭 Full-Time
💸 140000.0 - 170000.0 USD per year
🔍 Insurance
🏢 Company: joinroot
- 4+ years as a software engineer.
- 2+ years leading software teams.
- Expertise in Python, Terraform, SQL, and Spark.
- Expertise in Cloud Architecture.
- Experience with telematics or sensor data collection systems.
- Proven leadership of projects across multiple teams and functional domains.
- Excellent communication skills with engineering colleagues and senior business leaders.
- Partner with Marketing, Product, Data Science, Analytics, and Insurance experts to set the strategy for the quarters to come.
- Identify and socialize important technical initiatives that increase the effectiveness of products, systems, and teams.
- Coach and guide engineers in planning experiments and projects aligned with strategic objectives.
- Contribute code each development cycle to advance the team’s impact.
- Lead incident response to improve system resiliency.
- Coordinate with Staff Engineers to establish and evangelize standards and best practices.
LeadershipPythonSQLStrategyData scienceSparkCommunication SkillsCollaborationTerraform
Posted 4 months ago
ApplyApply
🔥 Lead, Data Engineer (Client Deployment) (United States)
Posted 5 months ago
📍 United States
🔍 Data Management
🏢 Company: Demyst👥 51-100💰 about 2 years agoBig DataFinancial ServicesBroadcastingData IntegrationAnalyticsInformation TechnologyFinTechSoftware
- Bachelor's degree or higher in Computer Science, Data Engineering, or related fields. Equivalent work experience is also highly valued.
- 5-10 years of experience in data engineering, software engineering, or client deployment roles, with at least 3 years in a leadership capacity.
- Strong leadership skills, including the ability to mentor and motivate a team, lead through change, and drive outcomes.
- Expertise in designing, building, and optimizing ETL/ELT data pipelines using Python, JavaScript, Golang, Scala, or similar languages.
- Experience in managing large-scale data processing environments, including Databricks and Spark.
- Proven experience with Apache Airflow to orchestrate data pipelines and manage workflow automation.
- Deep knowledge of cloud services, particularly AWS (EC2/ECS, Lambda, S3), and their role in data engineering.
- Hands-on experience with both SQL and NoSQL databases, with a deep understanding of data modeling and architecture.
- Strong ability to collaborate with clients and cross-functional teams, delivering technical solutions that meet business needs.
- Proven experience in unit testing, integration testing, and engineering best practices to ensure high-quality code.
- Familiarity with agile project management tools (JIRA, Confluence, etc.) and methodologies.
- Experience with data visualization and analytics tools such as Jupyter Lab, Metabase, Tableau.
- Strong communicator and problem solver, comfortable working in distributed teams.
- Lead the configuration, deployment, and maintenance of data solutions on the Demyst platform to support client use cases.
- Supervise and mentor the local and distributed data engineering team, ensuring best practices in data architecture, pipeline development, and deployment.
- Recruit, train, and evaluate technical talent, fostering a high-performing, collaborative team culture.
- Contribute hands-on to coding, code reviews, and technical decision-making, ensuring scalability and performance.
- Design, build, and optimize data pipelines, leveraging tools like Apache Airflow to automate workflows and manage large datasets effectively.
- Work closely with clients to advise on data engineering best practices, including data cleansing, transformation, and storage strategies.
- Implement solutions for data ingestion from various sources, ensuring the consistency, accuracy, and availability of data.
- Lead critical client projects, managing engineering resources, project timelines, and client engagement.
- Provide technical guidance and support for complex enterprise data integrations with third-party systems (e.g., AI platforms, data providers, decision engines).
- Ensure compliance with data governance and security protocols when handling sensitive client data.
- Develop and maintain documentation for solutions and business processes related to data engineering workflows.
- Other duties as required.
AWSLeadershipProject ManagementPythonSQLAgileApache AirflowETLJavascriptJiraTableauStrategyAirflowData engineeringGoNosqlSpark
Posted 5 months ago
ApplyRelated Articles
Posted 7 months ago
Insights into the evolving landscape of remote work in 2024 reveal the importance of certifications and continuous learning. This article breaks down emerging trends, sought-after certifications, and provides practical solutions for enhancing your employability and expertise. What skills will be essential for remote job seekers, and how can you navigate this dynamic market to secure your dream role?
Posted 7 months ago
Explore the challenges and strategies of maintaining work-life balance while working remotely. Learn about unique aspects of remote work, associated challenges, historical context, and effective strategies to separate work and personal life.
Posted 7 months ago
Google is gearing up to expand its remote job listings, promising more opportunities across various departments and regions. Find out how this move can benefit job seekers and impact the market.
Posted 7 months ago
Learn about the importance of pre-onboarding preparation for remote employees, including checklist creation, documentation, tools and equipment setup, communication plans, and feedback strategies. Discover how proactive pre-onboarding can enhance job performance, increase retention rates, and foster a sense of belonging from day one.