Spark Jobs

Find remote positions requiring Spark skills. Browse through opportunities where you can utilize your expertise and grow your career.

410 jobs found. to receive daily emails with new job openings that match your preferences.

410 jobs found.

Set alerts to receive daily emails with new job openings that match your preferences.

Apply

🔥 Staff Machine Learning Engineer, Ads Optimization

Posted 2024-11-21

📍 United States, Canada

🧭 Full-Time

💸 230000 - 322000 USD per year

🔍 Advertising technology

7+ years of contributing high-quality code to production systems that operate at scale.
5+ years of experience building control systems, PID controllers, multi-armed bandits, reinforcement learning algorithms, or bid/pricing optimization systems.
Experience leading large engineering teams and collaborating with cross-functional partners.
Experience designing optimization algorithms in an ad serving platform and/or other marketplaces.
Significant experience in one or more general-purpose programming languages like Java, Python, Go, Scala, C++, or similar.
Familiarity with data processing frameworks like Spark, Flink, Kafka, Druid, etc.
Experience with a cloud service provider like AWS or GCP.
Knowledge of tools like Kubernetes, Drone, CircleCI, Spinnaker, Argo, Airflow, Docker, Thrift.
Experience with datastores such as ElasticSearch/Amazon OpenSearch, Redis, Postgres, Cassandra, BigQuery.
Experience with machine learning modeling frameworks like TensorFlow or PyTorch.

Building Reddit-scale optimizations to improve advertiser outcomes using cutting-edge techniques in the industry.
Leverage live auction data and model predictions to adjust campaign bids in real time.
Incorporate knowledge of the Reddit ads marketplace into budget pacing algorithms powered by control & reinforcement learning systems.
Lead the team on designing new bid & budget optimization products and algorithms as well as conducting rigorous A/B experiments to evaluate the business impact.
Actively participate and work with other leads to set the long-term direction for the team, plan and oversee engineering designs and project execution.

AWS Docker Python ElasticSearch GCP Java Kafka Kubernetes Machine Learning PyTorch C++Airflow Algorithms Cassandra Elasticsearch Go Postgres Redis Spark Tensorflow

Posted 2024-11-21

Apply

🔥 Staff Data Engineer, Ads

Posted 2024-11-21

📍 US

🧭 Full-Time

💸 206700 - 289400 USD per year

🔍 Social media / Online community

MS or PhD in a quantitative discipline: engineering, statistics, operations research, computer science, informatics, applied mathematics, economics, etc.
7+ years of experience with large-scale ETL systems, building clean, maintainable, object-oriented code (Python preferred).
Strong programming proficiency in Python, SQL, Spark, Scala.
Experience with data modeling, ETL concepts, and manipulating large structured and unstructured data.
Experience with data workflows (e.g., Airflow) and data visualization tools (e.g., Looker, Tableau).
Deep understanding of technical and functional designs for relational and MPP databases.
Proven track record of collaboration and excellent communication skills.
Experience in mentoring junior data scientists and analytics engineers.

Act as the analytics engineering lead within Ads DS team and contribute to data science data quality and automation initiatives.
Ensure high-quality data through ETLs, reporting dashboards, and data aggregations for business tracking and ML model development.
Develop and maintain robust data pipelines and workflows for data ingestion, processing, and transformation.
Create user-friendly tools for internal use across Data Science and cross-functional teams.
Lead efforts to build a data-driven culture by enabling data self-service.
Provide mentorship and coaching to data analysts and act as a thought partner for data teams.

Leadership Python SQL Data Analysis ETL Tableau Strategy Airflow Data analysis Data engineering Data science Spark Communication Skills Collaboration Mentoring Coaching

Posted 2024-11-21

Apply

🔥 Data Engineer (Apache Spark)

Posted 2024-11-21

📍 Poland

🔍 Healthcare

🏢 Company: Sunscrapers sp. z o.o.

At least 3 years of professional experience as a data engineer.
Undergraduate or graduate degree in Computer Science, Engineering, Mathematics, or similar.
Excellent command in spoken and written English, at least C1.
Strong professional experience with Apache Spark.
Hands-on experience managing production spark clusters in Databricks.
Experience in CI/CD of data jobs in Spark.
Great analytical skills, attention to detail, and creative problem-solving skills.
Great customer service and troubleshooting skills.

Design and manage batch data pipelines, including file ingestion, transformation, and Delta Lake/table management.
Implement scalable architectures for batch and streaming workflows.
Leverage Microsoft equivalents of BigQuery for efficient querying and data storage.

Spark Analytical Skills CI/CD Customer service Attention to detail

Posted 2024-11-21

Apply

🔥 Head of Data Engineering

Posted 2024-11-21

📍 Mexico, Gibraltar, Colombia, USA, Brazil, Argentina

🧭 Full-Time

🔍 Cryptocurrency

🏢 Company: Bitso

4+ years of professional experience working with analytics, ETLs, and data systems as an individual contributor.
3+ years of experience in engineering management at tech companies.
Expertise in defining and implementing data architectures, including ETL/ELT pipelines, data lakes, data warehouses, and real-time data processing systems.
Expertise with cloud platforms (AWS preferred), data engineering tools (Databricks, Spark, Kafka), and SQL/NoSQL databases.
Expertise translating business requirements into technical solutions and data architecture.
Expertise with orchestration tools (e.g. AWS step functions, Databricks workflows, or Dagster).
Proven experience in building data migration services or implementing change data capture (CDC) processes.
Experience with CI/CD tools (Github actions).
Experience with CDP platforms and handling behavioral data (e.g. Segment, Amplitude, AVO).
Experience in infrastructure as code technologies (e.g. terraform) and serverless for data engineering tasks.

Lead the Data Engineering team and Data Governance lead on daily tasks with technical expertise and mentoring.
Prioritize workload, set clear goals and drive accountability to ensure the team delivers exceptional data products in a timely manner.
Mentor and coach all the Data Engineering division; fostering their professional development and an innovation culture.
Partner with Data Science divisions to drive data products that solve business problems.
Engage with stakeholders to define roadmaps according to Bitso’s priorities.
Recruit and retain top talent.
Define and drive Bitso’s data strategy in partnership with the SVP of Data Science.

AWS Leadership SQL Business Intelligence ETL Kafka Strategy Data engineering Data science Serverless Nosql Spark Collaboration CI/CD Mentoring DevOps Terraform

Posted 2024-11-21

Apply

🔥 Senior Product Security Engineer

Posted 2024-11-21

📍 United States

🧭 Full-Time

💸 188000 - 230000 USD per year

🔍 Mental health care technology

5+ years of experience in security and/or software engineering roles.
Demonstrated history of working on security-related projects.
Strong cross-functional experience with team collaboration.
Technical depth in building secure platforms and products.
Ability to tackle ambiguous problems in a fast-paced environment.
Focus on innovation in security and privacy technologies.
Results-driven and motivated by the mission to increase access to quality mental health care.

Partner with Product and Engineering for secure new product launches.
Engage in implementation efforts, security reviews, product design decisions, and auditing vulnerabilities.
Develop automated tooling for product security capabilities.
Define application guardrails for secure development practices.
Assist in ongoing security operations, including incident response and vulnerability management.

AWS Python Kafka TypeScript FastAPI Postgres Product design Redis React Spark

Posted 2024-11-21

Apply

🔥 SaaS E-Commerce Customer Onboarding Specialist (Remote)

Posted 2024-11-21

📍 US

🧭 Full-Time

💸 40000 - 60000 USD per year

🔍 SaaS E-Commerce

2+ years in SaaS onboarding.
2+ years in eCommerce, preferably in managing listings on marketplaces like eBay, Amazon, Walmart, or training sellers.
Intermediate Excel proficiency, including vlookup and handling large datasets.
Strong communication skills, both written and spoken.
Experience with remote training or person-to-person instruction.

Break projects into manageable tasks, track progress, and communicate with customers and team members.
Teach customers how to configure and use the platform effectively, support software-related inquiries, and ensure data accuracy.
Act as the primary point of contact during onboarding, addressing client questions and providing guidance.
Collect client feedback to inform product improvements and ensure client needs are met.
Complete all account documentation with thorough tracking for seamless account management.

Project Management Shopify Spark Communication Skills Attention to detail Organizational skills Time Management Documentation Compliance

Posted 2024-11-21

Apply

🔥 Software Engineer(L3), Backend

Posted 2024-11-21

📍 India

🔍 Communications

4+ years of experience writing production-grade code in a modern programming language.
Strong theoretical fundamentals and hands-on experience working with data and streaming technologies.
Highly effective collaborator who works well with teammates and product partners.
Well-versed in concurrent programming.
Solid grasp of Linux systems and networking concepts.
Experience maintaining and operating always-on cloud services.
Excellent written and verbal communications skills.

Build multi-tenant query engines that power the leading customer data platform (CDP).
Scale data pipelines and compute clusters to match growing customer demand.
Develop highly performant solutions that unlock differentiated CDP capabilities at scale.
Maintain a high bar of operational excellence for systems and services.

AWS Backend Development Software Development Amazon Web Services Go Golang Spark Communication Skills Collaboration Problem Solving Linux Written communication

Posted 2024-11-21

Apply

🔥 Associate Lead - Data & AI

Posted 2024-11-21

📍 India

🧭 Full-Time

🔍 Data & AI

🏢 Company: ProArch

Bachelor’s degree in Computer Science, Engineering, or related field (Master’s preferred).
8+ years of experience in AI, Data Engineering, and full-stack development.
Ability to work with ambiguity and deliver consultative solutions.
Familiarity with Agile methodologies (Scrum, Kanban).
Excellent communication and interpersonal skills.

Build and maintain a list of innovative ideas and evaluate them for feasibility.
Partner with presales teams to create tailored solutions for clients.
Provide technical expertise to resolve delivery challenges.
Conduct workshops to educate sales and marketing teams.
Share insights from sales calls with solution teams.

AWS Docker GraphQL Leadership Node.js PostgreSQL Python SQL Agile Blockchain Django Flask GCP IoT Java Jenkins Kafka Kubernetes Machine Learning MongoDB PyTorch SCRUM Snowflake Spring Spring Boot Vue.Js Azure Data engineering .NET Angular Serverless React Spark Tensorflow Vue.js CI/CD Agile methodologies DevOps Microservices

Posted 2024-11-21

Apply

🔥 Senior Data Engineer

Posted 2024-11-21

📍 Belgium, Spain

🔍 Hospitality industry

🏢 Company: Lighthouse

5+ years of professional experience using Python, Java, or Scala for data processing (Python preferred)
Experience with writing data processing pipelines and with cloud platforms like AWS, GCP, or Azure
Experience with data pipeline orchestration tools like Apache Airflow (preferred), Dagster or Prefect
Deep understanding of data warehousing strategies
Experience with transformation tools like dbt to manage data transformation in your data pipelines
Some experience in managing infrastructure with IaC tools like Terraform
Stay updated with industry trends, emerging technologies, and best practices in data engineering
Improve, manage, and teach standards for code maintainability and performance in code submitted and reviewed
Ship large features independently, generate architecture recommendations with the ability to implement them
Strong communicator that can describe complex topics in a simple way to a variety of technical and non-technical stakeholders.

Design and develop scalable, reliable data pipelines using the Google Cloud stack.
Ingest, process, and store structured and unstructured data from various sources into our data-lakes and data warehouses.
Optimise data pipelines for cost, performance and scalability.
Implement and maintain data governance frameworks, ensuring data accuracy, consistency, and compliance.
Monitor and troubleshoot data pipeline issues, implementing proactive measures for reliability and performance.
Mentor and provide technical guidance to other engineers working with data.
Partner with Product, Engineering & Data Science teams to operationalise new solutions.

Python Apache Airflow GCP Java Kafka Kubernetes Airflow Data engineering Grafana Prometheus Spark CI/CD Terraform Documentation Compliance

Posted 2024-11-21

Apply

🔥 Implementation Consultant

Posted 2024-11-21

📍 United States

🧭 Full-Time

💸 50000 - 60000 USD per year

🔍 Workforce Development Software

🏢 Company: Career Edge

Bachelor's degree from an accredited institution, or equivalent experience.
Strong analytical skills to translate client requirements into actionable development stories.
Excellent verbal and written communication skills for professional interactions with clients and internal teams.
Experience with Jira or similar project management tools for tracking development and managing workflows.
Strong problem-solving skills and the ability to work under pressure with a positive, client-focused attitude.
Ability to prioritize tasks in a fast-paced environment, balancing multiple competing demands with attention to detail.
Enthusiastic, self-motivated, and adaptable with a strong initiative to drive projects forward.

Engage directly with clients to gather, analyze, and document requirements, developing detailed specifications.
Assess client requests for feasibility and collaborate with the product team regarding impacts on the product roadmap.
Create detailed Jira stories and tickets with technical specifications, overseeing the development process and managing testing phases.
Coordinate with clients for user acceptance testing, managing expectations and addressing feedback.
Provide client training on new features, developing clear documentation for guidance.
Manage project timelines and communicate delays to maintain project momentum.
Build strong, professional relationships with clients and act as a trusted advisor.

Project Management Jira Product Development Spark Communication Skills Analytical Skills Attention to detail Written communication Documentation

Posted 2024-11-21

Apply

Shown 10 out of 410

Spark Jobs

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities