Apache Airflow Jobs

Find remote positions requiring Apache Airflow skills. Browse through opportunities where you can utilize your expertise and grow your career.

67 jobs found. to receive daily emails with new job openings that match your preferences.

67 jobs found.

Set alerts to receive daily emails with new job openings that match your preferences.

Apply

🔥 DevOps (MLOps) Engineer

Posted 2024-11-21

📍 United States

🏢 Company: ARFA Solutions, LLC

Bachelor's degree in Computer Science, Software Engineering, Data Science, or a related field.
5+ years of experience in DevOps or MLOps roles.
Hands-on experience with machine learning frameworks such as TensorFlow, PyTorch, or Scikit-learn.
Proficiency in programming languages such as Python, R, or Java.
Strong understanding of cloud platforms (AWS, Azure, or GCP) and their machine learning services.
Experience with containerization technologies, particularly Docker, and orchestration tools like Kubernetes.
Knowledge of CI/CD tools like Jenkins, GitLab CI/CD, or Azure DevOps.
Familiarity with data pipeline tools and frameworks, such as Apache Airflow or Kubeflow.
Strong problem-solving and analytical skills.
Excellent communication and teamwork abilities.

Design, implement, and manage MLOps pipelines for deploying and monitoring machine learning models.
Collaborate with data scientists to understand model requirements and operationalize models into production environments.
Automate model training, testing, and deployment processes through CI/CD principles.
Monitor and evaluate model performance in production and implement solutions for improvement.
Implement data versioning, model versioning, and tracking systems to ensure reproducibility.
Ensure security and compliance in the deployment of machine learning models.
Document processes and improve collaboration between teams for efficient workflows.
Stay up-to-date with industry trends and technologies in MLOps and machine learning.

AWS Docker Python Apache Airflow GCP Java Jenkins Kubeflow Kubernetes Machine Learning PyTorch Airflow Azure Data science Tensorflow Analytical Skills Collaboration CI/CD DevOps Compliance

Posted 2024-11-21

Apply

🔥 Senior Data Engineer

Posted 2024-11-21

📍 Belgium, Spain

🔍 Hospitality industry

🏢 Company: Lighthouse

5+ years of professional experience using Python, Java, or Scala for data processing (Python preferred)
Experience with writing data processing pipelines and with cloud platforms like AWS, GCP, or Azure
Experience with data pipeline orchestration tools like Apache Airflow (preferred), Dagster or Prefect
Deep understanding of data warehousing strategies
Experience with transformation tools like dbt to manage data transformation in your data pipelines
Some experience in managing infrastructure with IaC tools like Terraform
Stay updated with industry trends, emerging technologies, and best practices in data engineering
Improve, manage, and teach standards for code maintainability and performance in code submitted and reviewed
Ship large features independently, generate architecture recommendations with the ability to implement them
Strong communicator that can describe complex topics in a simple way to a variety of technical and non-technical stakeholders.

Design and develop scalable, reliable data pipelines using the Google Cloud stack.
Ingest, process, and store structured and unstructured data from various sources into our data-lakes and data warehouses.
Optimise data pipelines for cost, performance and scalability.
Implement and maintain data governance frameworks, ensuring data accuracy, consistency, and compliance.
Monitor and troubleshoot data pipeline issues, implementing proactive measures for reliability and performance.
Mentor and provide technical guidance to other engineers working with data.
Partner with Product, Engineering & Data Science teams to operationalise new solutions.

Python Apache Airflow GCP Java Kafka Kubernetes Airflow Data engineering Grafana Prometheus Spark CI/CD Terraform Documentation Compliance

Posted 2024-11-21

Apply

🔥 Cloud DevOps Engineer (AWS) (Remote, Canada)

Posted 2024-11-20

📍 Canada

🧭 Full-Time

🔍 Technology

Bachelor’s or Master’s degree in Computer Science, Data Engineering, or a related field.
5+ years of experience in Data Engineering, with at least 3+ years in AWS environments.
Strong knowledge of AWS services including SageMaker, Lambda, Glue, and Redshift.
Hands-on experience deploying machine learning models in AWS SageMaker.
Proficiency in DevOps practices, CI/CD pipelines, Docker, and infrastructure-as-code tools.
Advanced SQL skills and experience with complex ETL workflows.
Proficiency in Python and skills in Java or Scala.
Experience with Apache Airflow for data orchestration.
Effective communication skills and a result-oriented approach.

Design, develop, and maintain ETL pipelines ensuring reliable data flow and high-quality data for analytics.
Build and optimize data models to efficiently handle large data volumes in Snowflake.
Create complex SQL queries for data processing and analytics.
Manage orchestration and scheduling using Apache Airflow.
Document data pipelines and architecture.
Architect, build, and maintain data science infrastructure on AWS.
Collaborate with Data Scientists on deploying ML models using SageMaker.
Automate ML model deployment and monitoring with CI/CD and IaC tools.
Set up monitoring solutions to ensure effective operation of data pipelines.

AWS Docker Python SQL Apache Airflow Artificial Intelligence ETL Git Java Machine Learning Snowflake Airflow Data engineering Data science CI/CD DevOps Terraform Documentation

Posted 2024-11-20

Apply

🔥 Lead Engineer, Back-end

Posted 2024-11-13

📍 New York City, NOT STATED

🧭 Full-Time

💸 150000 - 175000 USD per year

🔍 Digital creative agency

🏢 Company: Code and Theory

8 years minimum experience in software programming, specializing in back-end development.
Proven experience with Python; Go or Java is a plus.
Deep understanding of designing complex workflows; experience with LangChain is a plus.
Familiarity with AI frameworks like TensorFlow or PyTorch, and working knowledge of LLMs.
Experience with RESTful APIs, gRPC, and asynchronous communication.
Proficiency with SQL and NoSQL databases, including state management.
Experience with message brokers like RabbitMQ or Apache Kafka.
Experience with Docker and Kubernetes for application deployment.
Understanding of secure coding practices and data encryption.
Experience with Retrieval-Augmented Generation (RAG) systems.
Working knowledge of CI/CD pipelines and cloud platforms.
Experience with monitoring and logging tools like Prometheus or ELK Stack.
Strong experience with workflow orchestration tools like Prefect or Apache Airflow.
Proven experience in building distributed systems and microservices architecture.

Be a hands-on leader to engineering teams in successfully delivering scalable, maintainable, and secure features to customers.
Integrate Foundation Model LLMs and internal RAG systems into backend services.
Implement workflow orchestration logic to manage task dependencies.
Collaborate with AI specialists for effective integration.
Ensure system scalability and efficiency for handling high loads.
Implement asynchronous processing, caching, and optimize communication protocols.
Set up logging, monitoring, and alerting mechanisms.
Adhere to data privacy and security best practices.
Write clear technical documentation.
Develop and conduct thorough testing.
Complete tasks in a timely manner and foster collaboration.

AWS Docker Python Software Development SQL Apache Airflow Kafka Kubernetes PyTorch Rabbitmq Airflow Apache Kafka Azure Grafana gRPC Prometheus Nosql Tensorflow Collaboration CI/CD RESTful APIs DevOps Terraform Documentation Microservices Compliance

Posted 2024-11-13

Apply

🔥 Software Engineer (Data Engineering)

Posted 2024-11-10

📍 San Francisco Bay Area

🔍 Financial planning and decision-making software

Confidence with a chosen programming language.
Ability to quickly learn new technologies.
Strong software engineering and computer science fundamentals.
Extensive experience with common big data workflow frameworks and solutions.

Laying the foundation of an exceptional data engineering practice.
Collaborating with the team to enhance big data workflow frameworks and solutions.

Backend Development Python Software Development SQL Apache Airflow Apache Hadoop Data Analysis ElasticSearch Git Hadoop Airflow Data analysis Data engineering Elasticsearch REST API RESTful APIs

Posted 2024-11-10

Apply

🔥 Senior Health Informatics Data Analyst

Posted 2024-11-09

📍 United States

🧭 Full-Time

💸 140000 - 160000 USD per year

🔍 Digital health

5+ years of experience with healthcare products and data, particularly claims and patient data.
5+ years as a healthcare data analyst or data engineer.
Proficiency in SQL, Python, and shell scripting for automation and ETL.
Experience with dbt code and data warehouse infrastructure.
Excellent communication skills across various contexts.
Strong understanding of business processes and healthcare quality measures.

Perform data analysis to identify patient cohorts and assess program impact.
Develop visualizations to demonstrate Wellth program impact and ROI.
Collaborate with customer data teams for data ingestion and ROI accuracy.
Create transformations from customer formats to Wellth standards.
Assist internal teams with analyses and report generation.
Lead data discussions with customers and cross-functional improvement initiatives.

Leadership Python SQL Apache Airflow Data Analysis ETL Airflow Data analysis Data engineering Spark

Posted 2024-11-09

Apply

🔥 Data Engineer

Posted 2024-11-09

📍 United States

🔍 Healthcare

🏢 Company: OneImaging

B.S. / M.S in Computer Science, Engineering, Applied Math & Statistics, or a related computational field.
3+ years of experience in backend-focused software engineering/data engineering or 6+ years of experience.
Proven experience in backend development with strong proficiency in Node.js (Express), MongoDB, Mongoose, and Postgres.
Familiarity with scaling approaches for FTP and delivering assets to the API server.
Hands-on experience with AWS services including S3, Route 53, etc.
Experience with ETL processes using Apache Airflow or similar.
Knowledge of integrating databases with BI tools and data visualization frameworks.
Understanding of event-driven architecture using Kafka, SQS, or RabbitMQ.
Capability in data extraction from scanned documents using OCR.
Strong understanding of data security and compliance best practices.
Ability to work independently and collaboratively.

Implement and manage ETL scripts and processes for data ingestion, validation, transformation, database updates, and reporting results using Apache Airflow.
Develop and maintain scalable API adjacent services using Node.js (Express), MongoDB, Mongoose, and Postgres.
Make recommendations to optimize data models & schemas in MongoDB and Postgres.
Manage database migrations and helper scripts to ensure smooth updates.
Optimize query and controller performance.
Collaborate on developing event-driven communication between services using Kafka, SQS, or RabbitMQ.
Support database integration with BI tools and data visualization frameworks.
Maintain and optimize API services for frontend interactions.
Create documentation for key platform areas.

AWS Backend Development Leadership Node.js Apache Airflow ETL Kafka Kubernetes MongoDB Rabbitmq Airflow Data engineering Postgres Communication Skills

Posted 2024-11-09

Apply

🔥 Senior Health Informatics Data Analyst

Posted 2024-11-09

📍 United States

🧭 Full-Time

💸 140000 - 160000 USD per year

🔍 Digital health

🏢 Company: Wellth

5+ years of experience with healthcare products and data.
5+ years as a healthcare data analyst or healthcare data engineer.
Proficient in SQL, Python, and shell scripts for analysis and ETL tasks.
Experienced in executing and maintaining dbt code.
Proficient in analyzing database schemas and data dictionaries.
Knowledge of data warehouse solutions like Redshift and BigQuery.
Excellent communication skills for customer and internal interactions.
Ability to work independently and understand business processes and ROI.

Perform data analysis to propose target patient cohorts, identify trends, and assess the Wellth program’s impact and value.
Develop visualizations that demonstrate the impact and ROI of the Wellth program.
Partner with customer data teams for data ingestion and ROI analysis.
Create and validate data transformations.
Assist with internal analysis and Quarterly Business Review reports.
Lead customer-facing data discussions.
Manage cross-functional data analysis projects.
Collaborate with internal teams to improve data infrastructure.

Leadership Python SQL Apache Airflow Data Analysis ETL Airflow Data analysis Data engineering Spark

Posted 2024-11-09

Apply

🔥 AWS Cloud Architect (100% teleworking)

Posted 2024-11-09

📍 Spain

🔍 Consulting, Technology

2-4 years of experience designing and implementing solutions on AWS, Databricks, and Terraform.
Solutions Architect certifications (Associate and Professional) are highly valued.
Fluent in English (C1).

Designing scalable, efficient, and cost-effective solutions following AWS best practices.
Collaboration with other product teams for requirements gathering and translation to a cloud architecture.
Implementation of automated AWS solutions with Terraform, including VPCs, IAM, EC2, S3, Lambda, API Gateway, DMS, EKS, ECS, and MWAA Airflow.
Validation of secure designs, monitoring vulnerabilities, and adhering to company security guidelines.
Proper governance of the data platform, including cost monitoring, resource tagging, resource usage, access control through IAM, and data access control.

AWS Apache Airflow AWS EKS Cloud Computing Airflow Amazon Web Services Terraform

Posted 2024-11-09

Apply

🔥 Data Architect (The Data Infrastructure Visionary)

Posted 2024-11-07

📍 United States

🔍 Data Architecture and Management

🏢 Company: Unreal Gigs

Bachelor’s or Master’s degree in Computer Science, Data Management, Information Technology, or a related field.
5+ years of experience in data architecture or a related field.
Extensive experience in designing and implementing data architectures, including data modeling and data warehousing.
Proficiency with cloud data services such as AWS, GCP, or Azure.
Strong understanding of ETL processes and data integration tools like Apache NiFi or Talend.
Expertise in data governance frameworks and data security policies.
Certifications in data architecture or cloud platforms are highly desirable.

Architect and implement a scalable, high-performance data infrastructure to support business analytics and operational reporting.
Create data governance frameworks and enforce data security policies.
Work closely with leadership and data teams to align architecture with strategic goals.
Design and manage data warehousing solutions for efficient storage and retrieval.
Develop and oversee ETL pipelines for accurate data integration.
Enable advanced analytics by building architecture for machine learning.
Create comprehensive documentation for data architectures and best practices.

AWS SQL Apache Airflow Apache Hadoop ETL GCP Hadoop Machine Learning Snowflake Strategy Airflow Azure Data science Spark Collaboration

Posted 2024-11-07

Apply

Shown 10 out of 67

Apache Airflow Jobs

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities