Senior Data Engineer

Posted 2024-10-17

View full description

💎 Seniority level: Senior, 5+ years experience

📍 Location: USA, CAN, MEX

🔍 Industry: Transportation technology

🏢 Company: Fleetio

🗣️ Languages: English

⏳ Experience: 5+ years experience

🪄 Skills: AWSProject ManagementPythonSQLBusiness IntelligenceDesign PatternsKafkaSnowflakeTableauData engineeringServerlessCommunication SkillsCI/CD

Requirements:

5+ years experience working in a data engineering or data-focused software engineering role.
Experience transforming raw data into clean models using standard tools of the modern data stack.
Deep understanding of ELT and data modeling concepts.
Experience with streaming data and pipelines (Kafka or Kinesis).
Proficiency in Python with a proven track record of delivering production-ready Python applications.
Experience in designing, building, and administering modern data pipelines and data warehouses.
Experience with dbt.
Familiarity with semantic layers like Cube or MetricFlow.
Experience with Snowflake, BigQuery, or Redshift.
Knowledge of version control tools such as GitHub or GitLab.
Experience with ELT tools like Stitch or Fivetran.
Experience with orchestration tools such as Prefect or Dagster.
Knowledge of CI/CD and IaaC tooling such as GitHub Actions and Terraform.
Experience with business intelligence solutions (Metabase, Looker, Tableau, Periscope, Mode).
Familiarity with serverless cloud functions (AWS Lambda, Google Cloud Functions, etc.).
Excellent communication and project management skills with a customer service-focused mindset.

Responsibilities:

Enable and scale self-serve analytics for all Fleetio team members by modeling data and metrics via tools like dbt.
Develop data destinations, custom integrations, and maintain open source packages for customer data integration.
Maintain and develop custom data pipelines from operational source systems for both streaming and batch sources.
Work on the development of internal data infrastructure, improving data hygiene and integrity through ELT pipeline monitoring.
Architect, design, and implement core components of data platform including data observability and data science products.
Develop and maintain streaming data pipelines from various databases and sources.
Collaborate across the company to tailor data needs and ensure data is appropriately modeled and available.
Document best practices and coach others on data modeling and SQL query optimization, managing roles, permissions, and deprecated projects.

Apply

Related Jobs

Apply

🔥 Senior Data Engineer I

Posted 2024-11-14

📍 Arizona, California, Connecticut, Colorado, Florida, Georgia, Hawaii, Illinois, Maryland, Massachusetts, Michigan, Minnesota, Missouri, New Hampshire, New York, North Carolina, North Dakota, Ohio, Oregon, Pennsylvania, Rhode Island, South Carolina, Texas, Utah, Vermont, Virginia, Washington, Washington D.C. and Wisconsin

🧭 Full-Time

💸 157791 - 183207 USD per year

🔍 Nonprofit, technology for political campaigns

🏢 Company: ActBlue

3-5 years of experience in data engineering or related roles.
Experience building, deploying, and running Machine Learning models in a production environment.
Experience maintaining and deploying server-side web applications.
Good collaboration skills with remote teams and a team player mentality.
Eagerness to learn, support teammates’ growth, and an understanding of performance, scalability, and security.

Implement and deliver complex, high-impact data platform projects, managing them through their full lifecycle with minimal guidance.
Work closely with application developers, database administrators, and data scientists to create robust infrastructure for data-driven insights.
Identify and understand end-user data needs, design solutions, and build scalable data pipelines.
Create data frameworks and services for engineers and data scientists to ensure scalability and consistency.
Collaborate with data scientists to advance the production-level Machine Learning platform.
Cultivate strong relationships with stakeholders and engineering teams to inform technical decisions.

AWSPythonMachine LearningData engineeringTerraform

Posted 2024-11-14

Apply

🔥 Senior Data Engineer

Posted 2024-11-14

📍 US

🧭 Full-Time

🔍 Cloud integration technology

🏢 Company: Cleo (US)

5-7+ years of experience in data engineering focusing on AI/ML models.
Hands-on expertise in data transformation and building data pipelines.
Leadership experience in mentoring data engineering teams.
Strong experience with cloud platforms and big data technologies.

Lead the design and build of scalable, reliable, and efficient data pipelines.
Set data infrastructure strategy for data warehouses and lakes.
Hands-on data transformation for AI/ML models.
Build data structures and manage metadata.
Implement data quality controls.
Collaborate with cross-functional teams to meet data requirements.
Optimize ETL processes for AI/ML.
Ensure data pipelines support model training needs.
Define data governance practices.

LeadershipArtificial IntelligenceETLMachine LearningStrategyData engineeringData StructuresMentoring

Posted 2024-11-14

Apply

🔥 Senior Data Engineer

Posted 2024-11-11

📍 USA

🧭 Full-Time

🔍 Energy analytics and forecasting

Senior level experience within data engineering with primary focus using Python.
Experience with cloud-based infrastructure (Kubernetes/Docker) and data services (GCP, AWS, Azure, et al).
Experience building data pipelines with a proven track record of delivering results that impact the business.
Experience working on complex large codebase with a focus on refactoring and enhancements.
Experience building data monitoring pipelines with a focus on scalability.

Rebuilding systems to identify more efficient ways to process data.
Automate the entire forecasting pipeline, including data collection, preprocessing, model training, and deployment.
Continuously monitor system performance and optimize data processing workflows to reduce latency and improve efficiency.
Set up real-time monitoring for data feeds to detect anomalies or issues promptly.
Utilize distributed computing and parallel processing to handle large-scale data.
Design your data infrastructure to be scalable to accommodate future growth in data volume and sources.

AWSDockerPythonGCPKubernetesAzureData engineering

Posted 2024-11-11

Apply

🔥 Senior Data Engineer PGD-19865

Posted 2024-11-09

📍 Argentina, Colombia, Costa Rica, Mexico

🔍 Data Analytics

Proficient with SQL and data visualization tools (i.e. Tableau, PowerBI, Google Data Studio).
Programming skills mainly SQL.
Knowledge and experience with Python and/or R a plus.
Experience with tools like Alteryx a plus.
Experience working with Google Cloud and AWS a plus.
Familiarity with Gitlab.

Analyze data and consult with subject matter experts to design and develop business rules for data processing.
Setup and/or maintain any existing dataflows in data wrangling tools like Alteryx or Google Dataprep.
Create and/or maintain SQL scripts.
Monitor, troubleshoot, and remediate data quality across marketing data systems ensuring full understanding of client deliverables.
Design and execute data quality checks.
Keep up to date on digital media operations - new partners, buy changes, etc.
Maintain ongoing management and stewardship of data governance, processing, and reporting.
Govern taxonomy additions, application, and use.
Serve as a knowledge expert for operational processes and identify areas of improvement.
Evaluate opportunities for simplification and/or automation for reporting and various processes.

PythonSQLGCPMicrosoft Power BITableau

Posted 2024-11-09

Apply

🔥 Senior Data Engineer PGD-19865

Posted 2024-11-09

📍 Argentina, Colombia, Costa Rica, Mexico

🔍 Data Analytics

Proficient with SQL and data visualization tools such as Tableau, PowerBI, Google Data Studio.
Strong programming skills primarily in SQL.
Knowledge and experience with Python and/or R is a plus.
Experience with tools like Alteryx is a plus.
Experience working with Google Cloud and AWS is a plus.
Familiarity with Gitlab.

Analyze data and consult with subject matter experts to design and develop business rules for data processing.
Setup and/or maintain existing dataflows in tools like Alteryx or Google Dataprep.
Create and/or maintain SQL scripts.
Monitor, troubleshoot, and remediate data quality across marketing data systems.
Design and execute data quality checks.
Keep up to date on digital media operations.
Maintain ongoing management and stewardship of data governance.
Govern taxonomy additions and applications.
Serve as a knowledge expert for operational processes.
Evaluate opportunities for simplification and automation.

PythonSQLGCPKubernetesMicrosoft Power BITableau

Posted 2024-11-09

Apply

🔥 Senior Data Engineer PGD-19865

Posted 2024-11-09

📍 Argentina, Colombia, Costa Rica, Mexico

🔍 Data Analytics

Proficient with SQL and data visualization tools (i.e. Tableau, PowerBI, Google Data Studio).
Programming skills mainly SQL.
Knowledge and experience with Python and/or R a plus.
Experience with tools like Alteryx a plus.
Experience working with Google Cloud and AWS a plus.

Analyze data and consult with subject matter experts to design and develop business rules for data processing.
Setup and/or maintain any existing dataflows in data wrangling tools like Alteryx or Google Dataprep.
Create and/or maintain SQL scripts.
Monitor, troubleshoot, and remediate data quality across marketing data systems ensuring full understanding of client deliverables.
Design and execute data quality checks.
Maintain ongoing management and stewardship of data governance, processing, and reporting.
Govern taxonomy additions, application, and use.
Serve as a knowledge expert for operational processes and identify areas of improvement to ensure appropriate turnaround times and data quality standards are being met.
Evaluate opportunities for simplification and/or automation for reporting and various processes.

AWSPythonSQLGCPMicrosoft Power BITableauAmazon Web ServicesData engineering

Posted 2024-11-09

Apply

🔥 Senior Data Engineer PGD-19865

Posted 2024-11-09

📍 Argentina, Colombia, Costa Rica, Mexico

🔍 Data Analytics

Proficient with SQL and data visualization tools (i.e. Tableau, PowerBI, Google Data Studio).
Programming skills mainly SQL.
Knowledge and experience with Python and/or R a plus.
Experience with Alteryx a plus.
Experience working with Google Cloud and AWS a plus.

Analyze data and consult with subject matter experts to design and develop business rules for data processing.
Setup and/or maintain existing dataflows in data wrangling tools like Alteryx or Google Dataprep.
Create and/or maintain SQL scripts.
Monitor, troubleshoot, and remediate data quality across marketing data systems.
Design and execute data quality checks.
Maintain ongoing management and stewardship of data governance, processing, and reporting.
Govern taxonomy additions, application, and use.
Serve as a knowledge expert for operational processes and identify improvement areas.
Evaluate opportunities for simplification and/or automation for reporting and processes.

AWSPythonSQLData AnalysisGCPMicrosoft Power BITableauAmazon Web ServicesData analysisData engineering

Posted 2024-11-09

Apply

🔥 Senior Data Engineer

Posted 2024-11-07

📍 Canada, UK, US

🔍 Smart home technology

🏢 Company: ecobee

Proficiency in building data pipelines using Python and SQL.
Experience with Apache Spark, Apache Kafka, and Apache Airflow.
Experience with cloud-based data platforms, preferably GCP.
Familiarity with SQL-based operational databases.
Good understanding of machine learning lifecycle.
Strong experience in data modeling and schema design.
Experience with both batch and real-time data processing.
Excellent communication skills for collaborative work.

Design, build, and maintain scalable and efficient ETL/ELT pipelines.
Implement data extraction and processing solutions for analytics and machine learning.
Integrate diverse data sources into centralized data repositories.
Develop and maintain data warehousing solutions.
Monitor and optimize data workflows for performance and reliability.
Implement monitoring and logging for data pipelines.
Collaborate with cross-functional teams to understand data requirements.
Translate business requirements into technical specifications.
Implement data quality checks and cleansing procedures.
Create and maintain documentation for data pipelines.
Share knowledge and best practices within the team.
Architect data pipelines for massive IoT data streams.

LeadershipPythonSQLApache AirflowETLGCPIoTKafkaMachine LearningAirflowApache KafkaData engineeringSparkCommunication SkillsCollaboration

Posted 2024-11-07

Apply

🔥 Senior Data Engineer

Posted 2024-11-07

📍 Mexico, Gibraltar, Colombia, USA, Brazil, Argentina

🧭 Full-Time

🔍 FinTech

🏢 Company: Bitso

Proven English fluency.
3+ years professional working experience with analytics, ETLs, and data systems.
3+ years with SQL databases, data lake, big data, and cloud infrastructure.
3+ years experience with Spark.
BS or Master's in Computer Science or similar.
Strong proficiency in SQL, Python, and AWS.
Strong data modeling skills.

Build processes required for optimal extraction, transformation, and loading of data from various sources using SQL, Python, Spark.
Identify, design, and implement internal process improvements while optimizing data delivery and redesigning infrastructure for scalability.
Ensure data integrity, quality, and security.
Work with stakeholders to assist with data-related technical issues and support their data needs.
Manage data separation and security across multiple data sources.

AWSPythonSQLBusiness IntelligenceMachine LearningData engineeringData StructuresSparkCommunication Skills

Posted 2024-11-07

Apply

🔥 Senior Data Engineer - Data Corpus

Posted 2024-11-07

📍 US, Germany, UK

🧭 Full-Time

🔍 Music

🏢 Company: SoundCloud

Senior Level Data Professional with a minimum of 4 years of experience (ideal 6+ years).
Experience with Cloud technologies, specifically GCP (required), with AWS/Azure as a plus.
Experience working with BigQuery and advanced SQL knowledge.
Proficiency in Python and Airflow.
Experience with big data at terabyte/petabyte scale.
Data Architecture/solution design experience.
Familiarity with Agile methodology and Jira.
Experience in data warehousing and analytical data modeling.
Knowledge of CI/CD pipelines and Git.
Experience in building reliable ETL pipelines and datasets for BI tools (Looker preferred).
Basic statistical knowledge and ability to produce high-quality technical documentation.

Build and maintain a unified and standardized data warehouse, Corpus, at SoundCloud.
Abstract the complexity of SoundCloud’s vast data ecosystem.
Collaboration with business reporting, data science, and product teams.
Gather and refine requirements, design data architecture and solutions.
Build ETL pipelines using Airflow to land data in BigQuery.
Model and build the business-ready data layer for dashboarding tools.

PythonSQLAgileETLGCPGitJiraAirflowCI/CD

Posted 2024-11-07

Apply

Senior Data Engineer

Requirements:

Responsibilities:

Related Jobs

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities