Senior Data Engineer

Posted 2024-10-16

View full description

💎 Seniority level: Senior, Minimum of 10 years

📍 Location: United States

🔍 Industry: Technology

🏢 Company: CCS, LLC

🗣️ Languages: English

⏳ Experience: Minimum of 10 years

🪄 Skills: SQLAgileData AnalysisETLAzureData analysisCommunication SkillsCollaboration

Requirements:

Minimum of 10 years industry experience in data and reporting, with knowledge of healthcare data as a plus.
10+ years in data processes/pipelines, ETL, and query optimization.
5+ years with Databricks, PySpark/SparkSQL, and Azure Cloud technologies.
Experience with data validation, writing complex queries, and familiarity with HIPAA compliance.
Excellent verbal and written communication skills, and proficiency in MS Office.

Responsibilities:

Work with the agile team to participate in agile ceremonies like grooming, planning, standup, retrospective, and demos.
Design and develop modern data ingestion pipelines using tools like ADF, Databricks, MS-SQL.
Create configurable designs to apply various business rules based on clients' needs.
Conduct data validation and quality checks to ensure accuracy in the data warehouse.
Work on large data sets ensuring configurable ingestion, dynamic rules, and data transformation.

Apply

Related Jobs

Apply

🔥 Senior Data Engineer

Posted 2024-11-24

📍 United States, Latin America, India

🧭 Full-Time

🔍 Modern data stack and cloud data services

At least 4+ years experience as a Software Engineer, Data Engineer, or Data Analyst.
Ability to develop end-to-end technical solutions in production environments.
Proficient in Java, Python, or Scala.
Experience with core cloud data platforms like Snowflake, AWS, Azure, Databricks, and GCP.
Strong SQL skills, capable of writing, debugging, and optimizing queries.
Client-facing communication skills for presentations and documentation.
Bachelor's degree in Computer Science or a related field.

Develop end-to-end technical solutions and ensure their performance, security, and scalability.
Help with robust data integration and contribute to the production deployment of solutions.
Create and deliver detailed presentations to clients.
Document solutions including POCs, roadmaps, diagrams, and logical system views.

AWSPythonSoftware DevelopmentSQLElasticSearchGCPHadoopJavaKafkaSnowflakeAirflowAzureCassandraElasticsearchNosqlSparkCommunication SkillsDocumentation

Posted 2024-11-24

Apply

🔥 Senior Data Engineer

Posted 2024-11-22

📍 Ontario

🔍 Customer engagement platform

🏢 Company: Braze

5+ years of hands-on experience in data engineering, cloud data warehouses, and ETL development.
Proven expertise in designing and optimizing data pipelines and architectures.
Strong proficiency in advanced SQL and data modeling techniques.
A track record of leading impactful data projects from conception to deployment.
Effective collaboration skills with cross-functional teams and stakeholders.
In-depth understanding of technical architecture and data flow in a cloud-based environment.
Ability to mentor and guide junior team members on best practices for data engineering and development.
Passion for building scalable data solutions that enhance customer experiences and drive business growth.
Strong analytical and problem-solving skills, with a keen eye for detail and accuracy.
Extensive experience working with and aggregating large event-level data.
Familiarity with data governance principles and ensuring compliance with industry regulations.
Preferable experience with Kubernetes for container orchestration and Airflow for workflow management.

Lead the design, implementation, and monitoring of scalable data pipelines and architectures using tools like Snowflake and dbt.
Develop and maintain robust ETL processes to ensure high-quality data ingestion, transformation, and storage.
Collaborate closely with data scientists, analysts, and other engineers to design and implement data solutions that drive customer engagement and retention.
Optimize and manage data flows and integrations across various platforms and applications.
Ensure data quality, consistency, and governance by implementing best practices and monitoring systems.
Work extensively with large-scale event-level data, aggregating and processing it to support business intelligence and analytics.
Implement and maintain data products using advanced techniques and tools.
Collaborate with cross-functional teams including engineering, product management, sales, marketing, and customer success to deliver valuable data solutions.
Continuously evaluate and integrate new data technologies and tools to enhance our data infrastructure and capabilities.

SQLBusiness IntelligenceETLSnowflakeData engineeringCollaborationCompliance

Posted 2024-11-22

Apply

🔥 Senior Data Engineer

Posted 2024-11-22

📍 United States of America

💸 90000 - 154000 USD per year

🏢 Company: VSPVisionCareers

Bachelor’s degree in computer science, data science, statistics, economics or related area.
Excellent written and verbal communication skills.
6+ years of experience in development teams focusing on analytics.
6+ years of hands-on experience in data preparation and SQL.
Knowledge of data architectures like event-driven architecture and real-time data.
Familiarity with DataOps practices and multiple data integration platforms.

Design, build, and optimize data pipelines for analytics.
Collaborate with multi-disciplinary teams for data integration.
Analyze data requirements to develop scalable pipeline solutions.
Profile data for accuracy and completeness in data gathering.
Drive automation of data tasks to enhance productivity.
Participate in architecture and design reviews.

AWSSQLAgileETLKafkaOracleSCRUMSnowflakeApache KafkaData scienceData StructuresCommunication SkillsCollaboration

Posted 2024-11-22

Apply

🔥 Senior Data Engineer

Posted 2024-11-20

📍 USA

🧭 Full-Time

💸 140000 - 160000 USD per year

🔍 Health tech

🏢 Company: Carrum Health

10+ years professional experience as a data engineer, including ownership of data products.
Proficiency with data engineering technologies including Python, PostgreSQL, AWS Athena, and Docker.
History of designing systems focused on data quality and scalability.
Experience in the healthcare space or another highly-regulated industry.
Strong interpersonal skills and the ability to work collaboratively.

Develop and maintain creative solutions to evolving challenges that span data ingestion, processing, and modeling.
Collaborate closely with internal and external stakeholders to understand data needs.
Grow and maintain pipelines that power analytics, machine learning, and data products.
Participate early in the development of data foundation and infrastructure, implementing quality control and reporting systems.
Understand HIPAA compliance and support Carrum’s success in this area.

AWSDockerPostgreSQLPythonBashETLMachine LearningRubyTableauData engineeringData scienceDevOpsCompliance

Posted 2024-11-20

Apply

🔥 Senior Data Engineer

Posted 2024-11-15

📍 United States, Canada

🔍 Advanced analytics consulting

🏢 Company: Tiger Analytics

Bachelor’s degree in Computer Science or similar field.
8+ years of experience in a Data Engineer role.
Experience with relational SQL and NoSQL databases like MySQL, Postgres.
Strong analytical skills and advanced SQL knowledge.
Development of ETL pipelines using Python & SQL.
Good experience with Customer Data Platforms (CDP).
Experience in SQL optimization and performance tuning.
Data modeling and building high-volume ETL pipelines.
Working experience with any cloud platform.
Experience with Google Tag Manager and Power BI is a plus.
Experience with object-oriented scripting languages: Python, Java, Scala, etc.
Experience extracting/querying/joining large data sets at scale.
Strong communication and organizational skills.

Designing, building, and maintaining scalable data pipelines on cloud infrastructure.
Working closely with cross-functional teams.
Supporting data analytics, machine learning, and business intelligence initiatives.

PythonSQLBusiness IntelligenceETLJavaMySQLPostgresNosqlAnalytical SkillsOrganizational skills

Posted 2024-11-15

Apply

🔥 Senior Data Engineer I

Posted 2024-11-14

📍 Arizona, California, Connecticut, Colorado, Florida, Georgia, Hawaii, Illinois, Maryland, Massachusetts, Michigan, Minnesota, Missouri, New Hampshire, New York, North Carolina, North Dakota, Ohio, Oregon, Pennsylvania, Rhode Island, South Carolina, Texas, Utah, Vermont, Virginia, Washington, Washington D.C. and Wisconsin

🧭 Full-Time

💸 157791 - 183207 USD per year

🔍 Nonprofit, technology for political campaigns

🏢 Company: ActBlue

3-5 years of experience in data engineering or related roles.
Experience building, deploying, and running Machine Learning models in a production environment.
Experience maintaining and deploying server-side web applications.
Good collaboration skills with remote teams and a team player mentality.
Eagerness to learn, support teammates’ growth, and an understanding of performance, scalability, and security.

Implement and deliver complex, high-impact data platform projects, managing them through their full lifecycle with minimal guidance.
Work closely with application developers, database administrators, and data scientists to create robust infrastructure for data-driven insights.
Identify and understand end-user data needs, design solutions, and build scalable data pipelines.
Create data frameworks and services for engineers and data scientists to ensure scalability and consistency.
Collaborate with data scientists to advance the production-level Machine Learning platform.
Cultivate strong relationships with stakeholders and engineering teams to inform technical decisions.

AWSPythonMachine LearningData engineeringTerraform

Posted 2024-11-14

Apply

🔥 Senior Data Engineer

Posted 2024-11-14

📍 US

🧭 Full-Time

🔍 Cloud integration technology

🏢 Company: Cleo (US)

5-7+ years of experience in data engineering focusing on AI/ML models.
Hands-on expertise in data transformation and building data pipelines.
Leadership experience in mentoring data engineering teams.
Strong experience with cloud platforms and big data technologies.

Lead the design and build of scalable, reliable, and efficient data pipelines.
Set data infrastructure strategy for data warehouses and lakes.
Hands-on data transformation for AI/ML models.
Build data structures and manage metadata.
Implement data quality controls.
Collaborate with cross-functional teams to meet data requirements.
Optimize ETL processes for AI/ML.
Ensure data pipelines support model training needs.
Define data governance practices.

LeadershipArtificial IntelligenceETLMachine LearningStrategyData engineeringData StructuresMentoring

Posted 2024-11-14

Apply

🔥 Senior Data Engineer

Posted 2024-11-11

📍 USA

🧭 Full-Time

🔍 Energy analytics and forecasting

Senior level experience within data engineering with primary focus using Python.
Experience with cloud-based infrastructure (Kubernetes/Docker) and data services (GCP, AWS, Azure, et al).
Experience building data pipelines with a proven track record of delivering results that impact the business.
Experience working on complex large codebase with a focus on refactoring and enhancements.
Experience building data monitoring pipelines with a focus on scalability.

Rebuilding systems to identify more efficient ways to process data.
Automate the entire forecasting pipeline, including data collection, preprocessing, model training, and deployment.
Continuously monitor system performance and optimize data processing workflows to reduce latency and improve efficiency.
Set up real-time monitoring for data feeds to detect anomalies or issues promptly.
Utilize distributed computing and parallel processing to handle large-scale data.
Design your data infrastructure to be scalable to accommodate future growth in data volume and sources.

AWSDockerPythonGCPKubernetesAzureData engineering

Posted 2024-11-11

Apply

🔥 Senior Data Engineer

Posted 2024-11-07

📍 Canada, UK, US

🔍 Smart home technology

🏢 Company: ecobee

Proficiency in building data pipelines using Python and SQL.
Experience with Apache Spark, Apache Kafka, and Apache Airflow.
Experience with cloud-based data platforms, preferably GCP.
Familiarity with SQL-based operational databases.
Good understanding of machine learning lifecycle.
Strong experience in data modeling and schema design.
Experience with both batch and real-time data processing.
Excellent communication skills for collaborative work.

Design, build, and maintain scalable and efficient ETL/ELT pipelines.
Implement data extraction and processing solutions for analytics and machine learning.
Integrate diverse data sources into centralized data repositories.
Develop and maintain data warehousing solutions.
Monitor and optimize data workflows for performance and reliability.
Implement monitoring and logging for data pipelines.
Collaborate with cross-functional teams to understand data requirements.
Translate business requirements into technical specifications.
Implement data quality checks and cleansing procedures.
Create and maintain documentation for data pipelines.
Share knowledge and best practices within the team.
Architect data pipelines for massive IoT data streams.

LeadershipPythonSQLApache AirflowETLGCPIoTKafkaMachine LearningAirflowApache KafkaData engineeringSparkCommunication SkillsCollaboration

Posted 2024-11-07

Apply

🔥 Senior Data Engineer

Posted 2024-11-07

📍 Mexico, Gibraltar, Colombia, USA, Brazil, Argentina

🧭 Full-Time

🔍 FinTech

🏢 Company: Bitso

Proven English fluency.
3+ years professional working experience with analytics, ETLs, and data systems.
3+ years with SQL databases, data lake, big data, and cloud infrastructure.
3+ years experience with Spark.
BS or Master's in Computer Science or similar.
Strong proficiency in SQL, Python, and AWS.
Strong data modeling skills.

Build processes required for optimal extraction, transformation, and loading of data from various sources using SQL, Python, Spark.
Identify, design, and implement internal process improvements while optimizing data delivery and redesigning infrastructure for scalability.
Ensure data integrity, quality, and security.
Work with stakeholders to assist with data-related technical issues and support their data needs.
Manage data separation and security across multiple data sources.

AWSPythonSQLBusiness IntelligenceMachine LearningData engineeringData StructuresSparkCommunication Skills

Posted 2024-11-07

Apply

Senior Data Engineer

Requirements:

Responsibilities:

Related Jobs

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities