Senior Data Engineer

Posted 1 day agoViewed

View full description

💎 Seniority level: Senior, 5+ years

📍 Location: United States

💸 Salary: 145000.0 - 200000.0 USD per year

🔍 Industry: Daily Fantasy Sports

🏢 Company: PrizePicks👥 101-250💰 Corporate about 2 years agoGaming Fantasy Sports Sports

🗣️ Languages: English

⏳ Experience: 5+ years

🪄 Skills: LeadershipPostgreSQLPythonSQLApache AirflowBashCloud ComputingETLGCPGitKafkaKubernetesData engineeringData scienceREST APICI/CDRESTful APIsMentoringTerraformData modeling

Requirements:

5+ years of experience in a data Engineering, or data-oriented software engineering role creating and pushing end-to-end data engineering pipelines.
2+ years of experience acting as technical lead and providing mentorship and feedback to junior engineers.
Extensive experience building and optimizing cloud-based data streaming pipelines and infrastructure.
Extensive experience exposing real-time predictive model outputs to production-grade systems leveraging large-scale distributed data processing and model training.
Experience in most of the following: SQL/NoSQL databases/warehouses: Postgres, BigQuery, BigTable, Materialize, AlloyDB, etc Replication/ELT services: Data Stream, Hevo, etc. Data Transformation services: Spark, Dataproc, etc Scripting languages: SQL, Python, Go. Cloud platform services in GCP and analogous systems: Cloud Storage, Cloud Compute Engine, Cloud Functions, Kubernetes Engine etc. Data Processing and Messaging Systems: Kafka, Pulsar, Flink Code version control: Git Data pipeline and workflow tools: Argo, Airflow, Cloud Composer. Monitoring and Observability platforms: Prometheus, Grafana, ELK stack, Datadog Infrastructure as Code platforms: Terraform, Google Cloud Deployment Manager. Other platform tools such as Redis, FastAPI, and Streamlit.

Responsibilities:

Enhance the capabilities of our existing Core Data platforms and develop new integrations with both internal and external APIs within the Data organization.
Work closely with DevOps, architects, and engineers to ensure the success of the Core Data platform.
Collaborate with Analytics Engineers to enhance data transformation processes, streamline CI/CD pipelines, and optimize team collaboration workflows.
Architect and implement Infrastructure as Code (IaC) solutions to automate and streamline the deployment and management of data infrastructure.
Develop and manage CI/CD pipelines to automate and streamline the deployment of data solutions.
Ensure code is thoroughly tested, effectively integrated, and efficiently deployed, in alignment with industry best practices for version control, automation, and quality assurance.
Serve as a Data Engineering thought leader within the broader PrizePicks technology organization by staying current with emerging technologies, implementing innovative solutions, and sharing knowledge and best practices with junior team members and collaborators.
Provide on-call support as part of a shared rotation between the Data and Analytics Engineering teams to maintain system reliability and respond to critical issues.

Apply

Related Jobs

Apply

🔥 Senior Data Engineer

Posted about 5 hours ago

📍 Worldwide

🧭 Full-Time

🔍 Software Development

🏢 Company: Kit👥 11-50💰 over 1 year agoEducation Financial Services Apps

🔧 Requirements

Strong command of SQL, including DDL and DML.
Proficient in Python
Strong understanding of DBMS internals, including an appreciation for platform-specific nuances.
A willingness to work with Redshift and deeply understand its nuances.
Familiarity with our key tools (Redshift, Segment, dbt, github)
8+ years in data, with at least 3 years specializing in Data Engineering
Proven track record managing and optimizing OLAP clusters
Experience refactoring problematic data pipelines without disrupting business operations
History of implementing data quality frameworks and validation processes

💡 Responsibilities

Dive into our Redshift warehouse, dbt models, and workflows.
Evaluate the CRM data lifecycle, including source extraction, warehouse ingestion, transformation, and reverse ETL.
Refine and start implementing your design for source extraction and warehouse ingestion.
Complete the implementation of the CRM source extraction/ingestion project and use the learnings to refine your approach in preparation for other, similar initiatives including, but by no means limited to web traffic events and product usage logs.

PythonSQLETLGitData engineeringRDBMSData modelingData management

Posted about 5 hours ago

Apply

🔥 Senior Data Engineer

Posted about 9 hours ago

📍 United States of America

🏢 Company: IDEXX

🔧 Requirements

Bachelor’s degree in Computer Science, Computer Engineering, Information Systems, Information Systems Engineering or a related field and 5 years of experience or Master’s degree in Computer Science, Computer Engineering, Information Systems, Information Systems Engineering or a related field and 3 years of related professional experience.
Advanced SQL knowledge and experience working with relational databases, including Snowflake, Oracle, Redshift.
Experience with AWS or Azure cloud platforms
Experience with data pipeline and workflow scheduling tools: Apache Airflow, Informatica.
Experience with ETL/ELT tools and data processing techniques
Experience in database design, development, and modeling
3 years of related professional experience with object-oriented languages: Python, Java, and Scala

💡 Responsibilities

Design and implement scalable, reliable distributed data processing frameworks and analytical infrastructure
Design metadata and schemas for assigned projects based on a logical model
Create scripts for physical data layout
Write scripts to load test data
Validate schema design
Develop and implement node cluster models for unstructured data storage and metadata
Design advanced level Structured Query Language (SQL), data definition language (DDL) and Python scripts
Define, design, and implement data management, storage, backup and recovery solutions
Design automated software deployment functionality
Monitor structural performance and utilization, identifying problems and implements solutions
Lead the creation of standards, best practices and new processes for operational integration of new technology solutions
Ensures environments are compliant with defined standards and operational procedures
Implement measures to ensure data accuracy and accessibility, constantly monitoring and refining the performance of data management systems

AWSPythonSQLApache AirflowCloud ComputingETLJavaOracleSnowflakeAzureData engineeringScalaData modelingData management

Posted about 9 hours ago

Apply

🔥 Senior Data Engineer (Remote - US)

Posted 3 days ago

📍 United States

🧭 Full-Time

💸 135000.0 - 160000.0 USD per year

🔍 Healthcare

🏢 Company: Jobgether👥 11-50💰 $1,493,585 Seed about 2 years agoInternet

🔧 Requirements

5+ years of experience in data engineering roles, preferably in fast-paced or data-centric environments
Proficient in SQL and experienced with data warehouses such as Snowflake or Redshift
Strong experience with cloud platforms (AWS, GCP, or Azure)
Familiarity with workflow management tools like Apache Airflow or Luigi
Knowledge of data modeling, warehousing architecture, and pipeline automation best practices
Degree in Computer Science, Engineering, Mathematics, or related field (Master’s preferred)
Familiarity with healthcare data standards like FHIR or HL7 is a plus
Strong problem-solving skills and ability to adapt in a dynamic environment

💡 Responsibilities

Build, optimize, and maintain highly scalable and reliable data pipelines
Collaborate with data scientists and analysts to meet data needs across the business
Automate data cleansing, validation, transformation, and mining processes
Improve internal data workflows and automate manual processes to enhance scalability
Troubleshoot data issues, ensure security compliance, and support infrastructure-related inquiries
Deliver high-quality data solutions that empower cross-functional teams with actionable insights

AWSSQLApache AirflowETLGCPSnowflakeAzureData engineeringData modeling

Posted 3 days ago

Apply

🔥 Senior Data Engineer

Posted 4 days ago

📍 Boston, MA; Vancouver, BC; Chicago, IL; and Vancouver, WA

💸 160000.0 - 190000.0 USD per year

🔍 Social Media Marketing

🏢 Company: Later👥 1-10 Consumer Electronics iOS Apps Software

🔧 Requirements

Minimum of 5 years in data engineering or related fields, with a strong focus on building data infrastructure and pipelines.
Bachelor’s degree in Computer Science, Engineering, or a related technical field; advanced degree preferred.

💡 Responsibilities

Design and build a robust data warehouse architecture.
Design, build, and maintain scalable data pipelines for both batch and real-time processing, ensuring high availability and reliability.
Develop reliable transformation layers and data pipelines from ambiguous business processes using tools like DBT.
Establish optimized data architectures using cloud technologies, and implement both batch and streaming data processing systems.
Enforce data quality checks and governance practices to maintain data integrity and compliance.
Work with data scientists, product managers, and business stakeholders to understand data needs and deliver actionable insights.
Analyze and optimize data pipelines for performance and cost-effectiveness.

AWSSQLApache AirflowCloud ComputingETLData engineering

Posted 4 days ago

Apply

🔥 Senior Data Engineer

Posted 9 days ago

📍 United States of America

💸 78750.0 - 133875.0 USD per year

🏢 Company: vspvisioncareers

🔧 Requirements

6+ years’ experience working in development team providing analytical capabilities
6+ years of hands-on experience in the data space, spanning data preparation, SQL, integration tools, ETL/ELT/data pipeline design
SQL coding experience
Experience working in an agile development environment (Scrum, Kanban) with a focus on Continuous Integration and Delivery
Knowledge about various data architectures, patterns, and capabilities such as event-driven architecture, real-time data flows, non-relational repositories, data virtualization, cloud storage, etc
Knowledge of and experience with multiple data integration platforms (IBM InfoSphere DataStage, Oracle Data Integrator, Informatica PowerCenter, MS SSIS, AWS Glue, Denodo), and data warehouse MPP platforms such as Snowflake, Netezza, Teradata, Redshift, etc

💡 Responsibilities

Collaborate within an agile, multi-disciplinary team to deliver optimal data integration and transformation solutions
Analyze data requirements (functional and non-functional) to develop and design robust, scalable, automated, fault-tolerant data pipeline solutions for business and technology initiatives
Design, build, maintain, and operationalize data pipelines for high volume and complex data using appropriate tools and practices in development, test, and production environments
Develop and design data mappings, programs, routines, and SQL to acquire data from legacy, web, cloud, and purchased package environments into the analytics environment
Drive automation of data pipeline preparation and integration tasks to minimize manual and error-prone processes and improve productivity using modern data preparation, integration, and AI-enabled metadata management tools and techniques
Participate in architecture, governance, and design reviews, identifying opportunities and making recommendations
Collaborate with architects to design and model application data structures, storage, and integration in accordance with enterprise-wide architecture standards across legacy, web, cloud, and purchased package environments

AWSSQLAgileETLSnowflakeApache KafkaData engineeringCI/CDRESTful APIsData visualizationData modelingData management

Posted 9 days ago

Apply

🔥 Senior Data Engineer

Posted 10 days ago

📍 United States

🧭 Full-Time

💸 135000.0 - 145000.0 USD per year

🔍 Life Science

🏢 Company: Medispend

🔧 Requirements

Hands-on knowledge of data integration platforms
Experience with enterprise systems (ERP, CRM, etc.)
Substantial programming experience with Python based data orchestration and transformation frameworks (i.e. Airflow, AWS Glue, Prefect, Dagster, Spark, Polars, Databricks, etc.)
Strong working knowledge of traditional RDBMS data warehousing as well as other platforms like Snowflake, RedShift

💡 Responsibilities

Gather requirements and design the integration of a new data source
Design and build data transformations
Estimate levels of effort for prospective client implementations
Evaluate new open source data management tools
Determine root cause for a failed integration
Conduct peer review for code check-ins
Design and build a data migration framework
Monitor infrastructure capacity of the data transformation platform
Compile and analyze data transformation success/failure rates

AWSProject ManagementPythonSQLApache AirflowCloud ComputingETLSnowflakeJiraAlgorithmsData engineeringData StructuresPostgresRDBMSREST APISparkCommunication SkillsAnalytical SkillsCollaborationCI/CDProblem SolvingAgile methodologiesDevOpsData visualizationData modelingScriptingData analyticsData managementSaaS

Posted 10 days ago

Apply

🔥 Senior Data Engineer

Posted 11 days ago

📍 United States

🧭 Contract

🔧 Requirements

Experience with Dataiku.
Expertise in IDMC (Informatica Data Management Cloud).
Strong knowledge of SQL statements.
Basic experience with Python.
Knowledge of cloud-based data warehousing solutions, like Snowflake

💡 Responsibilities

Utilize Dataiku for data preparation, analysis, and workflow automation.
Deconstruct complex SQL statements to understand data flows and transformation logic.
Migrate data to IDMC (Informatica Data Management Cloud) ensuring quality and validation.
Use Snowflake to extract data and convert it into formats compatible with IDMC.
Work closely with the business team to validate data accuracy and ensure alignment with business requirements.
Provide support for 1-2 products, such as VSM and Flash, ensuring data-related needs are met.

PythonSQLETLSnowflakeData engineeringData visualizationData modeling

Posted 11 days ago

Apply

🔥 Senior Data Engineer

Posted 13 days ago

📍 US

🧭 Full-Time

🏢 Company: Vettura

🔧 Requirements

3–6 years of experience in backend or data engineering roles.
Strong expertise in API integrations and ETL systems.
Proficiency in Node.js, TypeScript, and Python.
Experience with distributed systems and cloud-native databases (e.g., Yugabyte, Opensearch, Redis).
Familiarity with modern orchestration and identity tools (Argo/Temporal, Keycloak).
Knowledge of GraphQL, OAuth 2.0, and JWT for data access control.

💡 Responsibilities

Design and develop scalable ETL pipelines and connectors for real-time and batch data ingestion.
Integrate with internal and third-party APIs, ensuring secure and reliable data flow.
Collaborate with teams to architect high-performance data systems using tools like Yugabyte, Opensearch, and Redis.
Implement authentication and authorization using OAuth, JWT, and Keycloak.
Use orchestration tools like Argo or Temporal to automate data workflows.
Develop and maintain backend services using Node.js, TypeScript, and Python.
Ensure data security, compliance, and accessibility at all stages.

AWSBackend DevelopmentDockerNode.jsPythonCloud ComputingETLOAuthTypeScriptAPI testingData engineeringPostgresRedisCI/CDRESTful APIsJSONData management

Posted 13 days ago

Apply

🔥 Senior Data Engineer

Posted 14 days ago

📍 United States

🧭 Full-Time

🔍 Fintech

🏢 Company: Plum Inc

🔧 Requirements

5+ years of experience building and maintaining production-grade data pipelines.
Proven expertise in Python and SQL for data engineering tasks.
Strong understanding of lakehouse architecture and data modeling concepts.
Experience working with Databricks, Delta Lake, and Apache Spark.
Hands-on experience with AWS cloud infrastructure.
Track record of integrating data from external systems, APIs, and databases.

💡 Responsibilities

Design and architect end-to-end data processing pipelines: ingestion, transformation, and delivery to the Delta Lakehouse.
Integrate with external systems (e.g., CRMs, file systems, APIs) to automate ingestion of diverse data sources.
Develop robust data workflows using Python and Databricks Workflows.
Implement modular, maintainable ETL processes following SDLC best practices and Git-based version control.
Contribute to the evolution of our Lakehouse architecture to support downstream analytics and machine learning use cases.
Monitor, troubleshoot, and optimize data workflows in production.
Collaborate with cross-functional teams to translate data needs into scalable solutions.

AWSPythonSQLBashETLData engineeringCommunication SkillsProblem SolvingRESTful APIsData modeling

Posted 14 days ago

Apply

🔥 Senior Data Engineer - Integrations Services

Posted 14 days ago

📍 North America

🔍 Adtech

🔧 Requirements

Adtech experience or Martech experience is an asset
Proven track record of building scalable data pipelines and services
Experience with working with large data in low-latency environments
Software development experience in distributed systems, architecting scalable microservices and data pipelines in a successful high-growth technology company
A strong understanding of computer science fundamentals, data structures, and algorithms
Up to date knowledge and experience in the latest technologies, tools, and frameworks
Experience with data engineering tools and frameworks (Spark, Flink, Apache Beam, BigQuery, Redshift, dbt, etc)
Experience with Go and high performance databases is a plus

💡 Responsibilities

Working with a team of engineers to build first-party and third-party data integration with external data sources
Build and architect scalable low-latency backend systems and big data pipelines
Provide technical guidance in designing scalable solutions and best practices
Make a positive impact on the team's productivity and growth
Promote software development best-practices and conduct rigorous code reviews
Rigorously identify and solve technical challenges
Write performance efficient and memory optimized code that can scale with the volume of data that we ingest

Backend DevelopmentSoftware DevelopmentSQLETLKafkaAlgorithmsData engineeringData StructuresGoSparkRESTful APIsMicroservicesData modeling

Posted 14 days ago

Apply