Senior Data Engineer

Posted 4 days agoViewed

View full description

💎 Seniority level: Senior, 5-10+ years

📍 Location: United States

🗣️ Languages: English

⏳ Experience: 5-10+ years

🪄 Skills: PythonSQLETLSnowflakeData engineeringData visualizationData modeling

Requirements:

Experience with Dataiku.
Expertise in IDMC (Informatica Data Management Cloud).
Strong knowledge of SQL statements.
Basic experience with Python.
Knowledge of cloud-based data warehousing solutions, like Snowflake

Responsibilities:

Utilize Dataiku for data preparation, analysis, and workflow automation.
Deconstruct complex SQL statements to understand data flows and transformation logic.
Migrate data to IDMC (Informatica Data Management Cloud) ensuring quality and validation.
Use Snowflake to extract data and convert it into formats compatible with IDMC.
Work closely with the business team to validate data accuracy and ensure alignment with business requirements.
Provide support for 1-2 products, such as VSM and Flash, ensuring data-related needs are met.

Apply

Related Jobs

Apply

🔥 Senior Data Engineer

Posted 2 days ago

📍 United States of America

💸 78750.0 - 133875.0 USD per year

🏢 Company: vspvisioncareers

🔧 Requirements

6+ years’ experience working in development team providing analytical capabilities
6+ years of hands-on experience in the data space, spanning data preparation, SQL, integration tools, ETL/ELT/data pipeline design
SQL coding experience
Experience working in an agile development environment (Scrum, Kanban) with a focus on Continuous Integration and Delivery
Knowledge about various data architectures, patterns, and capabilities such as event-driven architecture, real-time data flows, non-relational repositories, data virtualization, cloud storage, etc
Knowledge of and experience with multiple data integration platforms (IBM InfoSphere DataStage, Oracle Data Integrator, Informatica PowerCenter, MS SSIS, AWS Glue, Denodo), and data warehouse MPP platforms such as Snowflake, Netezza, Teradata, Redshift, etc

💡 Responsibilities

Collaborate within an agile, multi-disciplinary team to deliver optimal data integration and transformation solutions
Analyze data requirements (functional and non-functional) to develop and design robust, scalable, automated, fault-tolerant data pipeline solutions for business and technology initiatives
Design, build, maintain, and operationalize data pipelines for high volume and complex data using appropriate tools and practices in development, test, and production environments
Develop and design data mappings, programs, routines, and SQL to acquire data from legacy, web, cloud, and purchased package environments into the analytics environment
Drive automation of data pipeline preparation and integration tasks to minimize manual and error-prone processes and improve productivity using modern data preparation, integration, and AI-enabled metadata management tools and techniques
Participate in architecture, governance, and design reviews, identifying opportunities and making recommendations
Collaborate with architects to design and model application data structures, storage, and integration in accordance with enterprise-wide architecture standards across legacy, web, cloud, and purchased package environments

AWSSQLAgileETLSnowflakeApache KafkaData engineeringCI/CDRESTful APIsData visualizationData modelingData management

Posted 2 days ago

Apply

🔥 Senior Data Engineer

Posted 4 days ago

📍 AL, AZ, AR, CA, CO, CT, DC, FL, GA, IL, KS, KY, MA, MI, MN, MO, NV, NJ, NY, NC, OH, OR, PA, SC, TN, TX, UT, VA, or WA

💸 116500.0 - 196700.0 USD per year

🔍 Data Analytics

🔧 Requirements

3-5 years of experience in data engineering, data analytics, or business analysis roles.
3 years of experience writing SQL for data warehousing.
2-3 years of experience working on data warehouses to support analytics.
2-3 years of experience building and maintaining enterprise data models, such as fact/dimensional star schemas.
Proficiency in cloud data warehouses (Redshift, BigQuery, Snowflake).
Strong SQL query writing and data modeling capabilities.
Experience with Python for scripting, data manipulation, and API interaction.
Familiarity with version control systems (e.g., Git, GitHub, GitLab).
Experience with workflow orchestration tools (e.g., Apache Airflow).
Knowledge of cloud platforms, particularly AWS.

💡 Responsibilities

Create and maintain Apache Airflow data pipeline jobs.
Monitor and update Fivetran extract and load runs.
Create and govern dbt models, testing, and job runs.
Monitor and maintain data quality processes and procedures.
Own, manage, and govern Snowflake data warehouse, including permission grants.
Maintain and govern usage of data sent from Snowflake to source systems ("Reverse ETL").
Collaborate with stakeholders to maintain and monitor existing data pipelines.
Evaluate automation opportunities within the business, using available data tools.
Provide data set and tooling recommendations where appropriate.
Provision data sets for analytic, master data, and reporting use cases.

AWSPythonSQLApache AirflowETLSnowflakeData engineeringData visualizationData modelingData analytics

Posted 4 days ago

Apply

🔥 Senior Data Engineer

Posted 4 days ago

📍 United States

🧭 Full-Time

💸 135000.0 - 145000.0 USD per year

🔍 Life Science

🏢 Company: Medispend

🔧 Requirements

Hands-on knowledge of data integration platforms
Experience with enterprise systems (ERP, CRM, etc.)
Substantial programming experience with Python based data orchestration and transformation frameworks (i.e. Airflow, AWS Glue, Prefect, Dagster, Spark, Polars, Databricks, etc.)
Strong working knowledge of traditional RDBMS data warehousing as well as other platforms like Snowflake, RedShift

💡 Responsibilities

Gather requirements and design the integration of a new data source
Design and build data transformations
Estimate levels of effort for prospective client implementations
Evaluate new open source data management tools
Determine root cause for a failed integration
Conduct peer review for code check-ins
Design and build a data migration framework
Monitor infrastructure capacity of the data transformation platform
Compile and analyze data transformation success/failure rates

AWSProject ManagementPythonSQLApache AirflowCloud ComputingETLSnowflakeJiraAlgorithmsData engineeringData StructuresPostgresRDBMSREST APISparkCommunication SkillsAnalytical SkillsCollaborationCI/CDProblem SolvingAgile methodologiesDevOpsData visualizationData modelingScriptingData analyticsData managementSaaS

Posted 4 days ago

Apply

🔥 Senior Data Engineer

Posted 6 days ago

📍 US

🧭 Full-Time

🏢 Company: Vettura

🔧 Requirements

3–6 years of experience in backend or data engineering roles.
Strong expertise in API integrations and ETL systems.
Proficiency in Node.js, TypeScript, and Python.
Experience with distributed systems and cloud-native databases (e.g., Yugabyte, Opensearch, Redis).
Familiarity with modern orchestration and identity tools (Argo/Temporal, Keycloak).
Knowledge of GraphQL, OAuth 2.0, and JWT for data access control.

💡 Responsibilities

Design and develop scalable ETL pipelines and connectors for real-time and batch data ingestion.
Integrate with internal and third-party APIs, ensuring secure and reliable data flow.
Collaborate with teams to architect high-performance data systems using tools like Yugabyte, Opensearch, and Redis.
Implement authentication and authorization using OAuth, JWT, and Keycloak.
Use orchestration tools like Argo or Temporal to automate data workflows.
Develop and maintain backend services using Node.js, TypeScript, and Python.
Ensure data security, compliance, and accessibility at all stages.

AWSBackend DevelopmentDockerNode.jsPythonCloud ComputingETLOAuthTypeScriptAPI testingData engineeringPostgresRedisCI/CDRESTful APIsJSONData management

Posted 6 days ago

Apply

🔥 Senior Data Engineer

Posted 7 days ago

📍 United States

🧭 Full-Time

🔍 Fintech

🏢 Company: Plum Inc

🔧 Requirements

5+ years of experience building and maintaining production-grade data pipelines.
Proven expertise in Python and SQL for data engineering tasks.
Strong understanding of lakehouse architecture and data modeling concepts.
Experience working with Databricks, Delta Lake, and Apache Spark.
Hands-on experience with AWS cloud infrastructure.
Track record of integrating data from external systems, APIs, and databases.

💡 Responsibilities

Design and architect end-to-end data processing pipelines: ingestion, transformation, and delivery to the Delta Lakehouse.
Integrate with external systems (e.g., CRMs, file systems, APIs) to automate ingestion of diverse data sources.
Develop robust data workflows using Python and Databricks Workflows.
Implement modular, maintainable ETL processes following SDLC best practices and Git-based version control.
Contribute to the evolution of our Lakehouse architecture to support downstream analytics and machine learning use cases.
Monitor, troubleshoot, and optimize data workflows in production.
Collaborate with cross-functional teams to translate data needs into scalable solutions.

AWSPythonSQLBashETLData engineeringCommunication SkillsProblem SolvingRESTful APIsData modeling

Posted 7 days ago

Apply

🔥 Senior Data Engineer - Integrations Services

Posted 7 days ago

📍 North America

🔍 Advertising

🔧 Requirements

Adtech experience or Martech experience is an asset
Proven track record of building scalable data pipelines and services
Experience with working with large data in low-latency environments
Software development experience in distributed systems, architecting scalable microservices and data pipelines in a successful high-growth technology company
A strong understanding of computer science fundamentals, data structures, and algorithms
Up to date knowledge and experience in the latest technologies, tools, and frameworks
Experience with data engineering tools and frameworks (Spark, Flink, Apache Beam, BigQuery, Redshift, dbt, etc)
Experience with Go and high performance databases is a plus

💡 Responsibilities

Working with a team of engineers to build first-party and third-party data integration with external data sources
Build and architect scalable low-latency backend systems and big data pipelines
Provide technical guidance in designing scalable solutions and best practices
Make a positive impact on the team's productivity and growth
Promote software development best-practices and conduct rigorous code reviews
Rigorously identify and solve technical challenges
Write performance efficient and memory optimized code that can scale with the volume of data that we ingest

Backend DevelopmentSoftware DevelopmentSQLETLKafkaAlgorithmsData engineeringData StructuresGoSparkRESTful APIsMicroservicesData modeling

Posted 7 days ago

Apply

🔥 Senior Data Engineer

Posted 8 days ago

📍 AL, AR, AZ, CA (exempt only), CO, CT, FL, GA, ID, IL, IN, IA, KS, KY, MA, ME, MD, MI, MN, MO, MT, NC, NE, NJ, NM, NV, NY, OH, OK, OR, PA, SC, SD, TN, TX, UT, VT, VA, WA, and WI

🧭 Full-Time

🔍 Insurance

🏢 Company: Kin Insurance

🔧 Requirements

4+ years of hands-on data engineering experience related to: Data structures and cloud platform environments and best practices (AWS strongly preferred, Azure, or GCP)
ETL performance tuning and cost optimization
Data lake and lakehouse patterns including open table formats (e.g. Iceberg, Hudi, Delta)
Proficiency in Python (Pandas, NumPy, etc.) and SQL for advanced data processing and querying
Expertise in distributed data processing/storage (e.g., Apache Spark, Kafka, Hadoop, or similar)

💡 Responsibilities

Designing and developing scalable data pipelines and models for downstream analytics and reporting
Leading and collaborating with a cross-functional project team to implement data validation, QA standards, and effective data lifecycle management
Optimizing pipeline performance, cost, and data quality in a large-scale data environment
Migrating data warehouse (DBT, Redshift) architecture to Lakehouse (e.g: S3, Glue, Databricks, Unity catalog) architecture
Mentoring data engineers and promoting best practices in software engineering, documentation, and metadata management
Ensuring data security and compliance with regulations (e.g., GDPR, CCPA, GLBA) through robust pipeline design and access monitoring
Translating ambiguous business requirements into technical solutions using marketing domain knowledge

AWSPythonSQLETLKafkaData engineeringData StructuresCommunication SkillsMentoringData modeling

Posted 8 days ago

Apply

🔥 Senior Data Engineer

Posted 8 days ago

📍 Worldwide

🧭 Full-Time

💸 140000.0 - 175000.0 USD per year

🔍 Software Development

🏢 Company: Figment👥 11-50 Hospitality Travel Accommodations Art

🔧 Requirements

Extensive experience with data engineering, including building and managing data pipelines and ETL processes.
Proficiency in the Python programming language and SQL.
Experience developing highly concurrent and performant applications ensuring scalability and efficient resource utilization in distributed or multi-threaded systems.
Experience implementing robust microservices following best practices in error handling, logging, and testing for production-grade systems.
Experience with using CI/CD pipelines for automated data infrastructure provisioning and application deployment.
Experience with the data orchestration tool Dagster or Airflow.
Experience designing and orchestrating complex DAGs to manage dependencies, triggers, and retries for data workflows, ensuring reliable and efficient pipeline execution.
Experience with the data transformation tool DBT.
Experience designing and implementing complex data transformations using advanced DBT models, materializations, and configurations to streamline data workflows and improve performance.
Experience optimizing and troubleshoot DBT pipelines for scale, ensuring that transformations run efficiently in production environments, handling large datasets without issues.
Experience with cloud data warehousing platforms (e.g. Snowflake)
Experience architecting and optimizing Snowflake environments for performance, including designing partitioning strategies, clustering keys, and storage optimizations for cost-effective scaling.
Has an understanding of security and governance policies within Snowflake, including data encryption, access control, and audit logging to meet compliance and security best practices.

💡 Responsibilities

Implement and maintain reliable data pipelines and data storage solutions.
Implement data modeling and integrate technologies according to project needs.
Manage specific data pipelines and oversees the technical aspects of data operations
Ensure data processes are optimized and align with business requirements
Identify areas for process improvements and suggests tools and technologies to enhance efficiency
Continuously improve data infrastructure automation, ensuring reliable and efficient data processing.
Develop and maintain data pipelines and ETL processes using technologies such as Dagster and DBT to ensure efficient data flow and processing.
Automate data ingestion, transformation, and loading processes to support blockchain data analytics and reporting.
Utilize Snowflake data warehousing solutions to manage and optimize data storage and retrieval.
Collaborate with Engineering Leadership and Product teams to articulate data strategies and progress.
Promote best practices in data engineering, cloud infrastructure, networking, and security.

PythonSQLCloud ComputingETLSnowflakeData engineeringCI/CDRESTful APIsMicroservicesData modeling

Posted 8 days ago

Apply

🔥 Senior Data Engineer

Posted 9 days ago

📍 United States

🧭 Full-Time

💸 112455.0 - 151200.0 USD per year

🏢 Company: Samsara👥 1001-5000💰 Secondary Market over 4 years ago🫂 Last layoff about 5 years agoCloud Data Services Business Intelligence Internet of Things SaaS Software

🔧 Requirements

5+ years in Python, SQL.
Exposure to ETL tools such as Fivetran, DBT or equivalent.
Exposure to python based API frameworks for data pipelines.
RDBMS: MySQL, AWS RDS/Aurora MySQL, PostgreSQL, Oracle, MS SQL-Server or equivalent.
Cloud: AWS, Azure and/or GCP.
Data warehouse: Databricks, Snowflake or equivalent.

💡 Responsibilities

Develop and maintain E2E data pipelines, backend ingestion and participate in the build of Samsara’s Data Platform to enable advanced automation and analytics.
Work with data from a variety of sources including but not limited to: CRM data, Product data, Marketing data, Order flow data, Support ticket volume data.
Manage critical data pipelines to enable our growth initiatives and advanced analytics.
Facilitate data integration and transformation requirements for moving data between applications; ensuring interoperability of applications with data layers and data lake.
Develop and improve the current data architecture, data quality, monitoring, observability and data availability.
Write data transformations in SQL/Python to generate data products consumed by customer systems and Analytics, Marketing Operations, Sales Operations teams.
Champion, role model, and embed Samsara’s cultural principles (Focus on Customer Success, Build for the Long Term, Adopt a Growth Mindset, Be Inclusive, Win as a Team) as we scale globally and across new offices.

AWSPythonSQLApache AirflowCloud ComputingETLSnowflakeData engineeringRDBMSREST APIData modeling

Posted 9 days ago

Apply

🔥 Senior Data Engineer

Posted 10 days ago

📍 Americas

🧭 Full-Time

🔍 Software Development

🏢 Company: Virtasant Inc.

🔧 Requirements

Strong proficiency in Python and SQL (non-negotiable).
Experience building ETL pipelines in production environments.
Familiarity with Databricks and big data tools (preferred).
Knowledge of visualization tools like Superset, Dash, Plotly.
Strong knowledge of Pandas; PySpark is a plus.
Focus on batch data processing (not real-time streaming).

💡 Responsibilities

Building and maintaining ETL pipelines using Python and SQL.
Managing, transforming, and optimizing structured datasets within Databricks.
Writing performant SQL queries to support reporting and analytics.
Developing dashboards and reporting tools using Superset, Dash, and Plotly.
Identifying and resolving inefficiencies in data workflows.
Enabling stakeholders to make data-driven decisions through automated reporting.
Collaborating closely with Client teams to understand data needs and deliver solutions.

PythonSQLETLPandas

Posted 10 days ago

Apply