Data Engineer

Posted 1 day agoViewed

View full description

💎 Seniority level: Junior, 1+ years

📍 Location: United States, EST

🔍 Industry: Pharmaceuticals

🏢 Company: Azurity Pharmaceuticals - US

⏳ Experience: 1+ years

🪄 Skills: PythonSQLData MiningETLData engineeringData modelingScriptingData analytics

Requirements:

Bachelor’s degree or foreign equivalent in computer science, computer information systems, mathematics, or a related field, plus one year of experience in the job offered or as a data engineer creating data models and data flow diagrams; performing data mining; using segmentation techniques, Python Scripting and SQL; building ETL data pipelines from scratch; building an ETL testing pipeline.

Responsibilities:

Serve as the principal data engineer for data warehouse initiatives.
Partner with product owners and business stakeholders to analyze corporate needs.
Provide data architecture and solutions that meet business data insights and reporting requirements.
Design, build, and maintain data pipelines, workflows, data modeling, and database structures.
Implement data storage solutions and ensure data quality and consistency.
Optimize data flow and collection for efficiency.
Collaborate with data scientists, software developers, and commercial operations colleagues to create coherent solutions.
Find and recommend process improvements, particularly transitioning from manual processes to automation.
Ensure compliance with information security and privacy policies in data architecture and data warehouse capabilities.
Provide scripting and software development support as needed (P-SQL, T-SQL, Python).
Create and maintain documentation, including sources to target data mapping and systems architecture diagrams.
Monitor, support, and troubleshoot data processing systems to improve data analytics platforms.

Apply

Related Jobs

Apply

🔥 Data Engineer Intern

Posted 1 day ago

📍 United States

🧭 Internship

💸 30.0 USD per hour

🔍 Streaming Media

🏢 Company: Wurl👥 101-250💰 $11,000,000 Series B over 4 years agoAdvertising TV Video Streaming Media and Entertainment Content Marketing Content Delivery Network

🔧 Requirements

Pursuing or recently completed a Bachelor’s or Master’s degree in Computer Science, Data Science, or a related technical field.
Proficiency in Python and SQL.
Familiarity with data engineering concepts and tools (e.g., ETL processes, data warehousing).
Interest in digital advertising, performance marketing, and streaming media technologies.
Strong analytical and problem-solving skills.
Excellent communication and collaboration abilities.

💡 Responsibilities

Collaborate with the Data Engineering team to develop and maintain scalable data pipelines that support performance marketing efforts on CTV.
Assist in implementing attribution models that tie CTV viewership to campaigns.
Work with cross-functional teams to integrate AI-driven targeting into our data infrastructure.
Analyze viewer engagement data to identify opportunities for optimizing return on ad spend (ROAS).
Contribute to the development of tools and dashboards that provide insights into campaign performance and audience behavior.

PythonSQLETLData engineering

Posted 1 day ago

Apply

🔥 Data Engineer / Scientist

Posted 1 day ago

📍 US

🧭 Full-Time

🔍 Software Development

🏢 Company: Common Room

🔧 Requirements

3+ years of experience building data or/or ML-based models leveraging industry best practices.
Exceptional SQL skills and experience with large-scale data transformation.

💡 Responsibilities

Build data or/or ML-based models leveraging industry best practices.
Shape the future of Common Room’s data capabilities by integrating data science into our product development process.
Partner with customers to solve unique data challenges to optimize their workflows, and validate methodologies to develop pragmatic, scalable solutions for customers.
Partner with the product team to convert data insights into product improvements

AWSNode.jsPythonSQLKafkaMachine LearningSnowflakeTypeScriptClickhouseData engineeringData sciencePostgresReactData modeling

Posted 1 day ago

Apply

🔥 Clinical Data Engineer

Posted 2 days ago

📍 US

🧭 Full-Time

💸 125000.0 - 200000.0 USD per year

🔍 Software Development

🏢 Company: PredictionHealth

🔧 Requirements

Reason around messy, clinical datasets and conduct analysis to communicate out findings
Learn and adapt quickly, collaborate with a team.

💡 Responsibilities

Reason around messy, clinical datasets
Identify patterns in datasets and conduct analysis to communicate out findings
Independently generate your own hypothesis and work to answer them with data
Scope, plan, and deliver on complex projects with minimal direction
Learn and adapt quickly
Collaborate with a team of individuals with diverse skill sets in a fast-paced environment

AWSPythonSQLCloud ComputingData AnalysisETLData engineeringData visualizationData modelingData analytics

Posted 2 days ago

Apply

🔥 Data Engineer

Posted 2 days ago

📍 United States

💸 97000.0 - 153595.0 USD per year

🔍 Healthcare

🏢 Company: healthfirst

🔧 Requirements

Work experience in a data engineering
Work experience in data programing languages such as Java or Python
Work experience in a Big Data ecosystem processing data including file systems, data structures/databases, automation, security, messaging, movement, etc.
Work experience in a production cloud infrastructure
Bachelor’s Degree in computer engineering or related field (preferred)
Hands on experience in leading healthcare data transformation initiatives from on-premise to cloud deployment (preferred)
Demonstrated experience working in an Agile environment as a Data Engineer (preferred)
Hands on work with Amazon Web Services, including creating Redshift data structures, accessing them with Spectrum and storing data in S3 (preferred)
Proven results using an analytical perspective to identify engineering patterns within complex strategies and ideas, and break them down into engineered code components (preferred)
Knowledge of provider-sponsored health insurance systems/processes and the Healthcare industry (preferred)
Experience developing, prototyping, and testing engineered processes, products, or services (preferred)
Proficiency with relational, graph and noSQL databases; expertise in SQL (preferred)
Demonstrates critical thinking skills with ability to problem solve (preferred)
Excellent interpersonal skills with proven ability to influence with impact across functions and disciplines (preferred)
Skilled in Microsoft Office including Project, PowerPoint, Word, Excel and Visio (preferred)

💡 Responsibilities

Finds trends in datasets and develops workflows and algorithms to make raw data useful to the enterprise
Designs and implements standardized data management procedures around data staging, data ingestion, data preparation, data provisioning, and data destruction (scripts, programs, automation, assisted by automation, etc.)
Ensures quality of technical solutions as data moves across Healthfirst environments
Provides insight into the changing data environment, data processing, data storage and utilization requirements for the company and offer suggestions for solutions
Ensures managed analytic assets support Healthfirst’s strategic goals by creating and verifying data acquisition requirements and strategy
Develops, constructs, tests and maintains architectures
Aligns architecture with business requirements and uses programming language and tools
Identifies ways to improve data reliability, efficiency, and quality
Conducts research for industry and business questions
Deploys sophisticated analytics programs, machine learning and statistical methods
Prepares data for predictive and prescriptive modeling and finds hidden patterns using data
Uses data to discover tasks that can be automated
Creates data monitoring capabilities for each business process and works with data consumers on updates
Aligns data architecture to Healthfirst solution architecture; contributes to overall solution architecture
Helps maintain the integrity and security of the company data
Additional duties as assigned or required

AWSPythonSQLAgileApache AirflowCloud ComputingETLJavaJava EEAmazon Web ServicesData engineeringData StructuresNosqlData visualizationData modelingScriptingData analyticsData management

Posted 2 days ago

Apply

🔥 Senior Data Engineer

Posted 2 days ago

📍 United States of America

💸 78750.0 - 133875.0 USD per year

🏢 Company: vspvisioncareers

🔧 Requirements

6+ years’ experience working in development team providing analytical capabilities
6+ years of hands-on experience in the data space, spanning data preparation, SQL, integration tools, ETL/ELT/data pipeline design
SQL coding experience
Experience working in an agile development environment (Scrum, Kanban) with a focus on Continuous Integration and Delivery
Knowledge about various data architectures, patterns, and capabilities such as event-driven architecture, real-time data flows, non-relational repositories, data virtualization, cloud storage, etc
Knowledge of and experience with multiple data integration platforms (IBM InfoSphere DataStage, Oracle Data Integrator, Informatica PowerCenter, MS SSIS, AWS Glue, Denodo), and data warehouse MPP platforms such as Snowflake, Netezza, Teradata, Redshift, etc

💡 Responsibilities

Collaborate within an agile, multi-disciplinary team to deliver optimal data integration and transformation solutions
Analyze data requirements (functional and non-functional) to develop and design robust, scalable, automated, fault-tolerant data pipeline solutions for business and technology initiatives
Design, build, maintain, and operationalize data pipelines for high volume and complex data using appropriate tools and practices in development, test, and production environments
Develop and design data mappings, programs, routines, and SQL to acquire data from legacy, web, cloud, and purchased package environments into the analytics environment
Drive automation of data pipeline preparation and integration tasks to minimize manual and error-prone processes and improve productivity using modern data preparation, integration, and AI-enabled metadata management tools and techniques
Participate in architecture, governance, and design reviews, identifying opportunities and making recommendations
Collaborate with architects to design and model application data structures, storage, and integration in accordance with enterprise-wide architecture standards across legacy, web, cloud, and purchased package environments

AWSSQLAgileETLSnowflakeApache KafkaData engineeringCI/CDRESTful APIsData visualizationData modelingData management

Posted 2 days ago

Apply

🔥 Staff Data Engineer

Posted 3 days ago

📍 United States

🧭 Full-Time

💸 160000.0 - 230000.0 USD per year

🔍 Daily Fantasy Sports

🏢 Company: PrizePicks👥 101-250💰 Corporate about 2 years agoGaming Fantasy Sports Sports

🔧 Requirements

7+ years of experience in a data Engineering, or data-oriented software engineering role creating and pushing end-to-end data engineering pipelines.
3+ years of experience acting as technical lead and providing mentorship and feedback to junior engineers.
Extensive experience building and optimizing cloud-based data streaming pipelines and infrastructure.
Extensive experience exposing real-time predictive model outputs to production-grade systems leveraging large-scale distributed data processing and model training.
Experience in most of the following:
Excellent organizational, communication, presentation, and collaboration experience with organizational technical and non-technical teams
Graduate degree in Computer Science, Mathematics, Informatics, Information Systems or other quantitative field

💡 Responsibilities

Enhance the capabilities of our existing Core Data Platform and develop new integrations with both internal and external APIs within the Data organization.
Develop and maintain advanced data pipelines and transformation logic using Python and Go, ensuring efficient and reliable data processing.
Collaborate with Data Scientists and Data Science Engineers to support the needs of advanced ML development.
Collaborate with Analytics Engineers to enhance data transformation processes, streamline CI/CD pipelines, and optimize team collaboration workflows Using DBT.
Work closely with DevOps and Infrastructure teams to ensure the maturity and success of the Core Data platform.
Guide teams in implementing and maintaining comprehensive monitoring, alerting, and documentation practices, and coordinate with Engineering teams to ensure continuous feature availability.
Design and implement Infrastructure as Code (IaC) solutions to automate and streamline data infrastructure deployment, ensuring scalable, consistent configurations aligned with data engineering best practices.
Build and maintain CI/CD pipelines to automate the deployment of data solutions, ensuring robust testing, seamless integration, and adherence to best practices in version control, automation, and quality assurance.
Experienced in designing and automating data governance workflows and tool integrations across complex environments, ensuring data integrity and protection throughout the data lifecycle
Serve as a Staff Engineer within the broader PrizePicks technology organization by staying current with emerging technologies, implementing innovative solutions, and sharing knowledge and best practices with junior team members and collaborators.
Ensure code is thoroughly tested, effectively integrated, and efficiently deployed, in alignment with industry best practices for version control, automation, and quality assurance.
Mentor and support junior engineers by providing guidance, coaching and educational opportunities
Provide on-call support as part of a shared rotation between the Data and Analytics Engineering teams to maintain system reliability and respond to critical issues.

AWSBackend DevelopmentDockerLeadershipPythonSQLApache AirflowCloud ComputingETLGitKafkaKubernetesRabbitmqAlgorithmsApache KafkaData engineeringData StructuresGoPostgresSparkCommunication SkillsAnalytical SkillsCollaborationCI/CDProblem SolvingRESTful APIsMentoringLinuxDevOpsTerraformExcellent communication skillsStrong communication skillsData visualizationData modelingScriptingSoftware EngineeringData analyticsData management

Posted 3 days ago

Apply

🔥 Senior Data Engineer

Posted 3 days ago

📍 AL, AZ, AR, CA, CO, CT, DC, FL, GA, IL, KS, KY, MA, MI, MN, MO, NV, NJ, NY, NC, OH, OR, PA, SC, TN, TX, UT, VA, or WA

💸 116500.0 - 196700.0 USD per year

🔍 Data Analytics

🔧 Requirements

3-5 years of experience in data engineering, data analytics, or business analysis roles.
3 years of experience writing SQL for data warehousing.
2-3 years of experience working on data warehouses to support analytics.
2-3 years of experience building and maintaining enterprise data models, such as fact/dimensional star schemas.
Proficiency in cloud data warehouses (Redshift, BigQuery, Snowflake).
Strong SQL query writing and data modeling capabilities.
Experience with Python for scripting, data manipulation, and API interaction.
Familiarity with version control systems (e.g., Git, GitHub, GitLab).
Experience with workflow orchestration tools (e.g., Apache Airflow).
Knowledge of cloud platforms, particularly AWS.

💡 Responsibilities

Create and maintain Apache Airflow data pipeline jobs.
Monitor and update Fivetran extract and load runs.
Create and govern dbt models, testing, and job runs.
Monitor and maintain data quality processes and procedures.
Own, manage, and govern Snowflake data warehouse, including permission grants.
Maintain and govern usage of data sent from Snowflake to source systems ("Reverse ETL").
Collaborate with stakeholders to maintain and monitor existing data pipelines.
Evaluate automation opportunities within the business, using available data tools.
Provide data set and tooling recommendations where appropriate.
Provision data sets for analytic, master data, and reporting use cases.

AWSPythonSQLApache AirflowETLSnowflakeData engineeringData visualizationData modelingData analytics

Posted 3 days ago

Apply

🔥 Senior Data Engineer

Posted 4 days ago

📍 United States

🧭 Full-Time

💸 135000.0 - 145000.0 USD per year

🔍 Life Science

🏢 Company: Medispend

🔧 Requirements

Hands-on knowledge of data integration platforms
Experience with enterprise systems (ERP, CRM, etc.)
Substantial programming experience with Python based data orchestration and transformation frameworks (i.e. Airflow, AWS Glue, Prefect, Dagster, Spark, Polars, Databricks, etc.)
Strong working knowledge of traditional RDBMS data warehousing as well as other platforms like Snowflake, RedShift

💡 Responsibilities

Gather requirements and design the integration of a new data source
Design and build data transformations
Estimate levels of effort for prospective client implementations
Evaluate new open source data management tools
Determine root cause for a failed integration
Conduct peer review for code check-ins
Design and build a data migration framework
Monitor infrastructure capacity of the data transformation platform
Compile and analyze data transformation success/failure rates

AWSProject ManagementPythonSQLApache AirflowCloud ComputingETLSnowflakeJiraAlgorithmsData engineeringData StructuresPostgresRDBMSREST APISparkCommunication SkillsAnalytical SkillsCollaborationCI/CDProblem SolvingAgile methodologiesDevOpsData visualizationData modelingScriptingData analyticsData managementSaaS

Posted 4 days ago

Apply

🔥 Senior Data Engineer

Posted 4 days ago

📍 United States

🧭 Contract

🔧 Requirements

Experience with Dataiku.
Expertise in IDMC (Informatica Data Management Cloud).
Strong knowledge of SQL statements.
Basic experience with Python.
Knowledge of cloud-based data warehousing solutions, like Snowflake

💡 Responsibilities

Utilize Dataiku for data preparation, analysis, and workflow automation.
Deconstruct complex SQL statements to understand data flows and transformation logic.
Migrate data to IDMC (Informatica Data Management Cloud) ensuring quality and validation.
Use Snowflake to extract data and convert it into formats compatible with IDMC.
Work closely with the business team to validate data accuracy and ensure alignment with business requirements.
Provide support for 1-2 products, such as VSM and Flash, ensuring data-related needs are met.

PythonSQLETLSnowflakeData engineeringData visualizationData modeling

Posted 4 days ago

Apply

🔥 HR Data Engineer

Posted 5 days ago

📍 United States

💸 94800.0 - 151400.0 USD per year

🏢 Company: careers_gm

🔧 Requirements

5+ years of experience in HR Data Engineer role leading HR data engineering transformation and implementing data pipelines and data solutions in the People Analytics/HR domain
Very good understanding of HR data and HR employee lifecycle processes (talent acquisition, talent development, workforce planning, engagement, employee listening, external benchmarking etc.)
Very good understanding of HCM data architecture, models and data pipelines and experience designing and implementing data integrations and ETLs with Workday (RaaS, APIs)
Experience designing and automating data and analytics solutions that can provide insights and recommendations at scale
Proficiency in SQL, R/Python and ETL tools
Deep expertise in modern data platforms (particularly Databricks) and end-to-end data architecture (DLT Streaming Pipelines, Workflows, Notebooks, DeltaLake, Unity Catalog)
Experience with different authentication (Basic Auth, Oauth, etc.) and encryption methods and tools (GPG, Voltage, etc.)
Very strong data analytics skills and ability to leverage multiple internal and external data sources to enable data-driven insights and inform strategic talent decisions
Knowledge of compliance and regulatory requirements associated with data management
Experience working in environments requiring strict confidentiality and handling of sensitive data
Great communication skills and ability to explain complex technical concepts to non-technical stakeholders.
Degree with quantitative focus (e.g., Mathematics, Statistics) and/or degree in Human Resources is a plus

💡 Responsibilities

Design, develop, and maintain ETL/ELT processes for HR data from multiple systems including Workday to empower data-driven decision-making
Drive implementation of robust HR data models and pipelines optimized for reporting and analytics, ensuring data quality, reliability, and security for on-prem and Azure cloud solutions.
Develop pipelines and testing automation to ensure HR data quality and integrity across multiple data sources
Collaborate with People Analytics and HR business partners to understand data requirements and deliver reliable solutions. Collaborate with technical teams to build the best-in-class data environment and technology stack for People Analytics teams.
Ensure data integrity, quality, consistency, security, and compliance (e.g., GDPR, CCPA, HIPAA where applicable).
Design and implement secure processes for handling sensitive information in our data tech stack while maintaining appropriate access controls and confidentiality
Automate manual HR reporting and improve data accessibility through scalable data pipelines across the entire HR employee lifecycle
Troubleshoot and resolve data-related issues quickly and efficiently.
Contribute to HR tech stack evaluations and migrations, especially around data capabilities and API integrations.
Incorporate external data sources into internal datasets for comprehensive analysis
Manage and optimize platform architecture including Databricks environment configuration and performance optimization
Stay up to date with emerging trends and advancements in data engineering – both technically and in the HR and People Analytics/sciences domain

PythonSQLApache AirflowETLOAuthAzureData engineeringData scienceREST APIComplianceData visualizationData modelingData analyticsData management

Posted 5 days ago

Apply