Why is remote work so nice? Let's try to see!
Apply
Lead Data Engineer
Posted over 1 year agoViewed
ApplyRelated Jobs
Apply
🔥 Lead Data Engineer
Posted 13 days ago
📍 United States
🧭 Full-Time
💸 152500.0 - 178000.0 USD per year
🔍 Software Development
🏢 Company: Hypergiant👥 101-250💰 Corporate almost 6 years agoArtificial Intelligence (AI)Machine LearningInformation TechnologyMilitary
- 10+ years of professional software development or data engineering experience (10+ with a STEM B.S. or 8+ with a relevant Master's degree)
- Strong proficiency in Python and familiarity with Java and Bash scripting
- Hands-on experience implementing database technologies, messaging systems, and stream computing software (e.g., PostgreSQL, PostGIS, MongoDB, DuckDB, KsqlDB, RabbitMQ)
- Experience with data fabric development using publish-subscribe models (e.g., Apache NiFi, Apache Pulsar, Apache Kafka and Kafka-based data service architecture)
- Proficiency with containerization technologies (e.g., Docker, Docker-Compose, RKE2, Kubernetes, and Microk8s)
- Experience with version control systems (e.g., Git), CI/CD tools (e.g., Jenkins), and collaborative development workflows
- Strong knowledge of data modeling and database optimization techniques
- Familiarity with data serialization languages (e.g., JSON, GeoJSON, YAML, XML)
- Excellent problem-solving and analytical skills that have been applied to high visibility, important data engineering projects
- Strong communication skills and ability to lead the work of other engineers in a collaborative environment
- Demonstrated experience in coordinating team activities, setting priorities, and managing tasks to ensure balanced workloads and effective team performance
- Experience managing and mentoring development teams in an Agile environment
- Ability to make effective architecture decisions and document them clearly
- Develop and continuously improve a data service that underpins cloud-based applications
- Support data and database modeling efforts
- Contribute to the development and maintenance of reusable component libraries and shared codebase
- Participate in the entire software development lifecycle, including requirement gathering, design, development, testing, and deployment, using an agile, iterative process
- Collaborate with developers, designers, testers, project managers, product owners, and project sponsors to integrate the data service to end user applications
- Communicate tasking estimation and progress regularly to a development lead and product owner through appropriate tools
- Ensure seamless integration between database and messaging systems and the frontend / UI they support
- Ensure data quality, reliability, and performance through code reviews and effective testing strategies
- Write high-quality code, applying best practices, coding standards, and design patterns
- Team with other developers, fostering a culture of continuous learning and professional growth
AWSDockerLeadershipPostgreSQLPythonSoftware DevelopmentSQLAgileBashGitJavaJenkinsKubernetesMongoDBRabbitmqApache KafkaData engineeringCommunication SkillsCI/CDAgile methodologiesRESTful APIsTerraformJSONData visualizationAnsibleData modelingData management
Posted 13 days ago
ApplyApply
🔥 Lead Data Engineer/Analyst - Contract
Posted about 1 month ago
🧭 Contract
🔍 Consulting
🏢 Company: Tech Holding👥 101-250ConsultingFinanceInformation Technology
- 7+ years of experience in data engineering, focusing on large-scale data pipelines and systems.
- Expertise in SQL for complex data analysis and manipulation.
- Strong programming skills in Python and shell scripting, with experience in Spark for distributed data processing.
- Hands-on experience with ETL tools and cloud platforms like AWS (e.g., S3, SageMaker, EMR).
- Proficiency in working with relational databases and data warehouses (e.g., Snowflake, Redshift).
- Ability to scale data processing and modeling solutions to handle millions of rows efficiently.
- Strong problem-solving skills with the ability to devise innovative solutions.
- Excellent communication and interpersonal skills, capable of working with technical and non-technical teams.
- Proven ability to work independently and manage priorities effectively in a fast-paced environment.
- Design, develop, and optimize data pipelines and workflows for seamless data movement and processing.
- Build and maintain scalable data architectures, including data warehouses (e.g., Snowflake, Redshift) and big data platforms (e.g., Databricks, Spark).
- Implement ETL processes and automate data workflows to ensure high performance and reliability.
- Extract, transform, and manipulate datasets using SQL and Python to create features for analytics and machine learning.
- Analyze large datasets to identify trends, insights, and opportunities for improving data systems and business processes.
- Build dashboards and visualizations to communicate data insights and results effectively.
- Develop, train, and deploy machine learning models within data pipelines to enhance predictive capabilities.
- Ensure model scalability and performance when working with large datasets and distributed systems.
- Utilize AWS tools (e.g., SageMaker, EMR) for managing machine learning workflows and data processing.
- Mentor junior team members on technical projects and coding best practices.
- Partner with stakeholders across analytics, product, finance, and marketing teams to deliver end-to-end data solutions.
- Actively participate in peer reviews, architectural discussions, and documentation efforts
Posted about 1 month ago
ApplyApply
🔥 Principal/Lead Data Engineer
Posted 3 months ago
📍 Romania
🧭 Full-Time
🔍 Cloud Data Engineering
🏢 Company: Qodea
- Strong experience as a Senior / Principal Cloud Data Engineer, with a solid track record of migrating large volumes of of data through the use of cloud data services and modern tooling
- Experience working on projects within large enterprise organisations either as an internal resource or as a 3rd party consultant
- Experience in performing a technical leadership role on projects and contributing to technical decision making during in-flight projects
- A track record of being involved in a wide range of projects with various tools and technologies, and solving a broad range of problems using your technical skills
- Demonstrable experience of utilising strong communication and stakeholder management skills when engaging with customers
- Significant experience of coding in Python and Scala or Java
- Experience with big data processing tools such as Hadoop or Spark
- Cloud experience; GCP specifically in this case, including services such as Cloud Run, Cloud Functions, BigQuery, GCS, Secret Manager, Vertex AI etc.
- Experience with Terraform
- Prior experience in a customer-facing consultancy role would be highly desirable
- Experience in working with agile software engineering teams
- Lead client engagements and team lead on client-facing delivery projects
- Consult, design, coordinate architecture to modernise infrastructure for performance, scalability, latency, and reliability
- Identify, scope, and participate in the design and delivery of cloud data platform solutions
- Deliver highly scalable big data architecture solutions using Google Cloud Technology
- Create and maintain appropriate standards and best practices around Google Cloud SQL, BigQuery, and other data technologies
- Design and execute a platform modernization approach for customers' data environments
- Document and share technical best practices/insights with engineering colleagues and the Data Engineering community
- Mentor and develop engineers within the Qodea Data Team and within our customers' engineering teams
- Act as the point of escalation with client-facing problems that need solving
LeadershipPythonSQLAgileGCPHadoopJavaData engineeringSparkTerraform
Posted 3 months ago
ApplyApply
🔥 Lead Data Engineer I
Posted 3 months ago
📍 United States of America
🧭 Full-Time
💸 140000.0 - 170000.0 USD per year
🔍 Insurance
🏢 Company: joinroot
- 4+ years as a software engineer.
- 2+ years leading software teams.
- Expertise in Python, Terraform, SQL, and Spark.
- Expertise in Cloud Architecture.
- Experience with telematics or sensor data collection systems.
- Proven leadership of projects across multiple teams and functional domains.
- Excellent communication skills with engineering colleagues and senior business leaders.
- Partner with Marketing, Product, Data Science, Analytics, and Insurance experts to set the strategy for the quarters to come.
- Identify and socialize important technical initiatives that increase the effectiveness of products, systems, and teams.
- Coach and guide engineers in planning experiments and projects aligned with strategic objectives.
- Contribute code each development cycle to advance the team’s impact.
- Lead incident response to improve system resiliency.
- Coordinate with Staff Engineers to establish and evangelize standards and best practices.
LeadershipPythonSQLStrategyData scienceSparkCommunication SkillsCollaborationTerraform
Posted 3 months ago
ApplyApply
🔥 Lead Data Engineer
Posted 4 months ago
🧭 Full-Time
💸 190000.0 - 245000.0 USD per year
🔍 Software Development
🏢 Company: Lumos👥 51-100💰 $35,000,000 Series B 10 months agoSecurityInformation TechnologyIdentity ManagementCollaborationSoftware
- Extensive experience designing and implementing medallion architectures (bronze, silver, gold layers) or similar data warehouse paradigms.
- Skilled in optimizing data pipelines for both batch and real-time processing.
- Proficiency in deploying data pipelines using CI/CD tools and integrating automated data quality checks, version control, and deployment automation to ensure reliable and repeatable data processes.
- Expertise in advanced SQL, ETL processes, and data transformation techniques.
- Strong programming skills in Python.
- Demonstrated ability to work closely with AI engineers, data scientists, product engineers, product managers, and other stakeholders to ensure that data pipelines meet the needs of all teams.
- Architect, build, and maintain cutting-edge data pipelines that empower our AI products, in-product analytics, and internal reporting.
- Ensure the scalability, reliability, and quality of our analytics data infrastructure, enabling the seamless integration of usage, spend, compliance, and access data to drive business insights and deliver exceptional value to our customers.
- Play a pivotal role in transforming complex data into actionable intelligence, fueling Lumos' growth and innovation.
Posted 4 months ago
ApplyApply
🔥 Lead, Data Engineer (Client Deployment) (United States)
Posted 5 months ago
📍 United States
🔍 Data Management
🏢 Company: Demyst👥 51-100💰 about 2 years agoBig DataFinancial ServicesBroadcastingData IntegrationAnalyticsInformation TechnologyFinTechSoftware
- Bachelor's degree or higher in Computer Science, Data Engineering, or related fields. Equivalent work experience is also highly valued.
- 5-10 years of experience in data engineering, software engineering, or client deployment roles, with at least 3 years in a leadership capacity.
- Strong leadership skills, including the ability to mentor and motivate a team, lead through change, and drive outcomes.
- Expertise in designing, building, and optimizing ETL/ELT data pipelines using Python, JavaScript, Golang, Scala, or similar languages.
- Experience in managing large-scale data processing environments, including Databricks and Spark.
- Proven experience with Apache Airflow to orchestrate data pipelines and manage workflow automation.
- Deep knowledge of cloud services, particularly AWS (EC2/ECS, Lambda, S3), and their role in data engineering.
- Hands-on experience with both SQL and NoSQL databases, with a deep understanding of data modeling and architecture.
- Strong ability to collaborate with clients and cross-functional teams, delivering technical solutions that meet business needs.
- Proven experience in unit testing, integration testing, and engineering best practices to ensure high-quality code.
- Familiarity with agile project management tools (JIRA, Confluence, etc.) and methodologies.
- Experience with data visualization and analytics tools such as Jupyter Lab, Metabase, Tableau.
- Strong communicator and problem solver, comfortable working in distributed teams.
- Lead the configuration, deployment, and maintenance of data solutions on the Demyst platform to support client use cases.
- Supervise and mentor the local and distributed data engineering team, ensuring best practices in data architecture, pipeline development, and deployment.
- Recruit, train, and evaluate technical talent, fostering a high-performing, collaborative team culture.
- Contribute hands-on to coding, code reviews, and technical decision-making, ensuring scalability and performance.
- Design, build, and optimize data pipelines, leveraging tools like Apache Airflow to automate workflows and manage large datasets effectively.
- Work closely with clients to advise on data engineering best practices, including data cleansing, transformation, and storage strategies.
- Implement solutions for data ingestion from various sources, ensuring the consistency, accuracy, and availability of data.
- Lead critical client projects, managing engineering resources, project timelines, and client engagement.
- Provide technical guidance and support for complex enterprise data integrations with third-party systems (e.g., AI platforms, data providers, decision engines).
- Ensure compliance with data governance and security protocols when handling sensitive client data.
- Develop and maintain documentation for solutions and business processes related to data engineering workflows.
- Other duties as required.
AWSLeadershipProject ManagementPythonSQLAgileApache AirflowETLJavascriptJiraTableauStrategyAirflowData engineeringGoNosqlSpark
Posted 5 months ago
ApplyRelated Articles
Posted 7 months ago
Insights into the evolving landscape of remote work in 2024 reveal the importance of certifications and continuous learning. This article breaks down emerging trends, sought-after certifications, and provides practical solutions for enhancing your employability and expertise. What skills will be essential for remote job seekers, and how can you navigate this dynamic market to secure your dream role?
Posted 7 months ago
Explore the challenges and strategies of maintaining work-life balance while working remotely. Learn about unique aspects of remote work, associated challenges, historical context, and effective strategies to separate work and personal life.
Posted 7 months ago
Google is gearing up to expand its remote job listings, promising more opportunities across various departments and regions. Find out how this move can benefit job seekers and impact the market.
Posted 7 months ago
Learn about the importance of pre-onboarding preparation for remote employees, including checklist creation, documentation, tools and equipment setup, communication plans, and feedback strategies. Discover how proactive pre-onboarding can enhance job performance, increase retention rates, and foster a sense of belonging from day one.