Data Engineer

Posted 2024-11-09

View full description

💎 Seniority level: Senior, 3+ years or 6+ years

📍 Location: United States

🔍 Industry: Healthcare

🏢 Company: OneImaging

🗣️ Languages: English

⏳ Experience: 3+ years or 6+ years

🪄 Skills: AWSBackend DevelopmentLeadershipNode.jsApache AirflowETLKafkaKubernetesMongoDBRabbitmqAirflowData engineeringPostgresCommunication Skills

Requirements:

B.S. / M.S in Computer Science, Engineering, Applied Math & Statistics, or a related computational field.
3+ years of experience in backend-focused software engineering/data engineering or 6+ years of experience.
Proven experience in backend development with strong proficiency in Node.js (Express), MongoDB, Mongoose, and Postgres.
Familiarity with scaling approaches for FTP and delivering assets to the API server.
Hands-on experience with AWS services including S3, Route 53, etc.
Experience with ETL processes using Apache Airflow or similar.
Knowledge of integrating databases with BI tools and data visualization frameworks.
Understanding of event-driven architecture using Kafka, SQS, or RabbitMQ.
Capability in data extraction from scanned documents using OCR.
Strong understanding of data security and compliance best practices.
Ability to work independently and collaboratively.

Responsibilities:

Implement and manage ETL scripts and processes for data ingestion, validation, transformation, database updates, and reporting results using Apache Airflow.
Develop and maintain scalable API adjacent services using Node.js (Express), MongoDB, Mongoose, and Postgres.
Make recommendations to optimize data models & schemas in MongoDB and Postgres.
Manage database migrations and helper scripts to ensure smooth updates.
Optimize query and controller performance.
Collaborate on developing event-driven communication between services using Kafka, SQS, or RabbitMQ.
Support database integration with BI tools and data visualization frameworks.
Maintain and optimize API services for frontend interactions.
Create documentation for key platform areas.

Apply

Related Jobs

Apply

🔥 Principal Data Engineer

Posted 2024-12-03

📍 United States

💸 210000 - 220000 USD per year

🔍 Healthcare

🏢 Company: Transcarent

Experienced: 10+ years in data engineering with a strong background in building and scaling data architectures.
Technical Expertise: Advanced working knowledge of SQL, relational databases, big data tools (e.g., Spark, Kafka), and cloud-based data warehousing (e.g., Snowflake).
Architectural Visionary: Experience in service-oriented and event-based architecture with strong API development skills.
Problem Solver: Manage and optimize processes for data transformation, metadata management, and workload management.
Collaborative Leader: Strong communication skills to present ideas clearly and lead cross-functional teams.
Project Management: Strong organizational skills, capable of leading multiple projects simultaneously.

Lead the Design and Implementation: Architect and implement cutting-edge data processing platforms and enterprise-wide data solutions using modern data architecture principles.
Scale Data Platform: Develop a scalable Platform for data extraction, transformation, and loading from various sources, ensuring data integrity and accessibility.
AI / ML platform: Design and build scalable AI and ML platforms for Transcarent use cases.
Collaborate Across Teams: Work with Executive, Product, Clinical, Data, and Design teams to meet their data infrastructure needs.
Optimize Data Pipelines: Build and optimize complex data pipelines for high performance and reliability.
Innovate and Automate: Create and maintain data tools and pipelines for analytics and data science.
Mentor and Lead: Provide leadership and mentorship to the data engineering team.

AWSLeadershipProject ManagementPythonSQLJavaKafkaSnowflakeC++AirflowData engineeringSparkCommunication SkillsProblem SolvingOrganizational skills

Posted 2024-12-03

Apply

🔥 Principal Data Engineer - Remote US

Posted 2024-12-03

📍 United States

🔍 Sales Technology

🏢 Company: Seamless.AI👥 501-1000💰 $75.0m Series A on 2021-05-01Sales Automation Artificial Intelligence (AI)Lead Generation Machine Learning Information Technology Software

Bachelor's degree in Computer Science, Information Systems, or related fields.
7+ years of experience as a Data Engineer focusing on ETL processes.
Professional experience with Spark and AWS pipeline development.
Strong proficiency in Python and experience with related libraries (e.g., pandas, NumPy, PySpark).
Hands-on experience with AWS Glue or similar ETL tools.
Solid understanding of data modeling, warehousing, and architecture principles.
Expertise in working with large data sets and distributed computing frameworks.
Experience developing and training machine learning models.
Strong proficiency in SQL and familiarity with data matching methodologies.
Experience with data governance and security practices.
Strong problem-solving and communication skills.

Design, develop, and maintain robust and scalable ETL pipelines to acquire, transform, and load data from various sources.
Collaborate with cross-functional teams to understand data requirements and develop data acquisition strategies.
Implement data transformation logic using programming languages like Python.
Utilize AWS Glue or similar tools to manage ETL jobs and workflows.
Optimize and tune ETL processes for performance with large datasets.
Apply data matching and aggregation methodologies to ensure data accuracy.
Implement data governance practices for compliance and security.
Collaborate with the data engineering team to adopt new technologies.

AWSPythonSQLArtificial IntelligenceETLMachine LearningNumpyData engineeringPandasSparkAnalytical SkillsCollaborationOrganizational skillsCompliance

Posted 2024-12-03

Apply

🔥 Senior Data Engineer

Posted 2024-12-03

📍 United States, Latin America, India

🔍 Cloud Data Technologies

🏢 Company: phData

At least 4+ years experience as a Software Engineer, Data Engineer or Data Analyst.
Programming expertise in Java, Python and/or Scala.
Core cloud data platforms: Snowflake, AWS, Azure, Databricks, and GCP.
SQL proficiency and the ability to write, debug, and optimize SQL queries.
Experience creating and delivering detailed presentations.

Develop end-to-end technical solutions into production.
Ensure performance, security, scalability, and robust data integration.
Client-facing communication and presentation delivery.
Create detailed solution documentation.

AWSPythonSoftware DevelopmentSQLElasticSearchGCPHadoopJavaKafkaSnowflakeAirflowAzureCassandraElasticsearchNosqlSparkCommunication SkillsDocumentation

Posted 2024-12-03

Apply

🔥 Data Engineer

Posted 2024-11-30

📍 CA, US / TX, US / FL, US

🧭 Temporary

🔍 Healthcare

🏢 Company: Tekton Labs

Expertise in ETL processes and data integration.
Strong understanding of healthcare workflows and reporting requirements.
Proficiency in SQL and Python.
Hands-on experience with BI tools such as Freshpaint, Tableau, or Power BI.

Assess existing reporting infrastructure and refactor legacy systems to optimize performance.
Design schemas for healthcare workflows and centralize session form data.
Develop ETL processes for integrating patient data into reporting pipelines.
Deliver the first version (V1) of NIH reports and business performance reports.
Create an interactive BI dashboard using tools like Freshpaint, Tableau, or Power BI.
Define funnel metrics including conversion and drop-off rates and establish baseline metrics.

PythonSQLETLTableauDocumentation

Posted 2024-11-30

Apply

🔥 Senior Azure Data Engineer

Posted 2024-11-27

📍 United States

🧭 Full-Time

💸 125000 - 165000 USD per year

🏢 Company: FSAStore.com

Bachelor’s degree in Computer Science, Engineering, or related field.
Minimum of 5+ years experience in data engineering roles with a focus on building and maintaining data pipelines.
Strong proficiency in Azure Synapse Analytics and Azure DevOps.
Expertise in Apache Spark pools.
Proficient in Python scripting and programming languages such as SQL and Java.
Senior level knowledge of data modeling and data warehousing concepts.
Strong communication skills to convey technical concepts to non-technical stakeholders.
Experience with data modeling, warehousing, and building ETL pipelines.
Familiarity with Azure Data Factory and Azure Data Lake Storage.

Design, develop, and maintain robust data pipelines and infrastructure to support data integration, processing, and storage.
Collaborate with analysts and stakeholders to understand data requirements and translate them into technical solutions.
Implement best practices for data governance, security, and quality assurance.
Explore and evaluate new technologies and tools to enhance data infrastructure.
Design and implement data models on Azure Synapse Analytics.
Utilize Azure DevOps for CI/CD and agile project management.
Manage and optimize Apache Spark pools for high-performance processing.
Develop and maintain ETL processes using Python.
Ensure data quality and integrity across platforms.
Collaborate with cross-functional teams.
Stay current with industry trends in Azure data services.

Project ManagementPythonSQLAgileETLJavaAzureData engineeringSparkCommunication SkillsDevOps

Posted 2024-11-27

Apply

🔥 Senior Data Engineer - Search

Posted 2024-11-26

📍 US, Germany, UK

🧭 Full-Time

🔍 Music streaming

🏢 Company: SoundCloud

Strong background in data engineering with at least 5 years of experience.
Experience in designing and optimizing data pipelines, data architecture and modeling, and ETL processes for large-scale distributed systems.
Proficient in SQL and Python, ideally with experience in Scala or Java.
Experience in training, prototyping, and deploying machine learning models.
Experience in Search or Recommender Systems is a plus.

Build and optimize high-end data pipelines that drive key performance indicators (KPIs) and train Machine Learning models.
Work on real-time processing of data pipelines for search features used by millions of users.
Collaborate with cross-functional teams to ensure structured data for search systems and machine learning models.

PythonSQLAgileETLMachine LearningAlgorithmsData engineeringData StructuresSpark

Posted 2024-11-26

Apply

🔥 Data Engineer (Remote in US)

Posted 2024-11-26

📍 US

💸 84000 - 120000 USD per year

🔍 Consumer insights

Strong PL/SQL, SQL development skills.
Proficient in multiple programming languages used in data engineering such as Python and Java.
Minimum 3-5 years of experience in Data engineering with Oracle and MS SQL.
Experience with data warehousing concepts and technologies including cloud-based services (e.g. Snowflake).
Experience with cloud platforms like Azure and knowledge of infrastructure.
Experience with data orchestration tools (e.g. Azure Data Factory, DataBricks workflows).
Understanding of data privacy regulations and best practices.
Experience working with remote teams.
Experience working on a team with a CI/CD process.
Familiarity using tools like Git and Jira.
Great problem-solving abilities and work ethics.
Bachelor's degree in Computer Science or Computer Engineering.

Design, implement and maintain scalable pipelines and architecture to collect, process, and store data from various sources.
Unit test and document solutions that meet product quality standards prior to release to QA.
Identify and resolve performance bottlenecks in pipelines to ensure efficient and timely data delivery.
Implement data quality checks and validations to ensure accuracy, completeness, and consistency of data delivery.
Work with Data Architect on data governance, quality, and security best practices.
Collaborate with cross-functional teams to identify and address data needs.
Define and document technical requirements.

PythonSQLETLGitJavaOracleQASnowflakeJiraAzureData engineeringCI/CD

Posted 2024-11-26

Apply

🔥 Senior Data Engineer, Remote

Posted 2024-11-26

📍 United States

🔍 Data and technology

5+ years of experience making contributions in the form of code.
Experience with algorithms and data structures and knowing when to apply them.
Deep familiarity with Scala or Java.
Experience working with high-scale systems: realtime and batch.
Interest in data engineering to develop ingestion engines, ETL pipelines, and organizing data.
Experience in Machine Learning techniques and tools is a plus.

Be a senior member of the team by contributing to the architecture, design, and implementation of EMS systems.
Mentor junior engineers and promote their growth.
Lead technical projects, managing the planning, execution, and success of complex technical projects.
Collaborate with other engineering, product, and data science teams to ensure we're building the best products.
Be on call if required and accommodate Eastern Time Zone.

SQLETLGCPAlgorithmsData engineeringData StructuresSparkCollaboration

Posted 2024-11-26

Apply

🔥 Senior Data Engineer

Posted 2024-11-24

📍 United States, Latin America, India

🧭 Full-Time

🔍 Modern data stack and cloud data services

At least 4+ years experience as a Software Engineer, Data Engineer, or Data Analyst.
Ability to develop end-to-end technical solutions in production environments.
Proficient in Java, Python, or Scala.
Experience with core cloud data platforms like Snowflake, AWS, Azure, Databricks, and GCP.
Strong SQL skills, capable of writing, debugging, and optimizing queries.
Client-facing communication skills for presentations and documentation.
Bachelor's degree in Computer Science or a related field.

Develop end-to-end technical solutions and ensure their performance, security, and scalability.
Help with robust data integration and contribute to the production deployment of solutions.
Create and deliver detailed presentations to clients.
Document solutions including POCs, roadmaps, diagrams, and logical system views.

AWSPythonSoftware DevelopmentSQLElasticSearchGCPHadoopJavaKafkaSnowflakeAirflowAzureCassandraElasticsearchNosqlSparkCommunication SkillsDocumentation

Posted 2024-11-24

Apply

🔥 Data Engineer (Remote in US)

Posted 2024-11-23

📍 US

💸 84000 - 120000 USD per year

🔍 Consumer insights

Strong PL/SQL, SQL development skills.
Proficient in multiple data engineering languages such as Python and Java.
3-5 years of experience in data engineering specific to Oracle and MS SQL.
Experience with data warehousing technologies and cloud-based services like Snowflake.
Experience with cloud platforms such as Azure and infrastructure knowledge.
Familiarity with data orchestration tools like Azure Data Factory and DataBricks workflows.
Understanding of data privacy regulations and best practices.
Experience working on a remote team.
Familiarity with CI/CD processes and tools like Git, Jira.

Design, implement, and maintain scalable data pipelines and architecture.
Unit test and document solutions to meet product quality standards.
Identify and resolve performance bottlenecks in data pipelines.
Implement data quality checks and validation processes.
Collaborate with cross-functional teams to address data needs.
Ensure technology solutions support customer and organizational needs.
Define and document technical requirements.

PythonSQLETLGitJavaOracleQASnowflakeJiraAzureData engineeringCI/CD

Posted 2024-11-23

Apply

Data Engineer

Requirements:

Responsibilities:

Related Jobs

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities