Senior Big Data Engineer (Scala)
New
V
VarwiseAdtech
Warszawa, Kraków, Wrocław, Gdynia, Poznań, PL, Must be able to work with team members in NYCFull-TimeSenior
Salary10,000 - 10,800 USD per month
Apply NowOpens the employer's application page
Job Details
- Languages
- En C2
- Experience
- 8+ years
- Required Skills
- AWSPythonETLGCPJavaAirflowSparkLinuxScalaDatabricks
Requirements
- 8+ years of professional software engineering experience with a focus on data engineering in big data environments.
- Expert-level proficiency in Python and at least one other high-level programming language (Java, Scala, C++, C#).
- Proven experience building and optimizing large-scale data pipelines using Apache Spark.
- Hands-on experience developing and deploying data solutions in a major cloud platform (AWS, GCP, or Azure).
- Experience working with AI, LLMs, Agents, and Generative AI technologies.
- Experience with Scala.
- Experience with big data visualization and analytics using OLAP tools.
- Familiarity with big data tools and frameworks such as MLFlow, dbt, Kafka, Airflow, or Databricks.
- Experience with large-scale data management formats like Parquet, Delta Lake, or Iceberg.
- Relevant technology experience in the advertising industry.
Responsibilities
- Design, develop, and maintain highly scalable data pipelines, ETL processes, and data models using Python, Spark, and cloud platforms (AWS/GCP).
- Lead the integration of AI and LLMs into data products by collaborating with data scientists to productionize models and agentic workflows.
- Utilize AI-driven software development tools like GitHub Copilot to enhance productivity and code quality.
- Maintain the data lake by building searchable data sets for business use.
- Scale, troubleshoot, and optimize distributed data processing systems.
- Ensure 24/7 reliability of data pipelines.
- Lead technical discussions to improve development tools, processes, and project outcomes.
View Full Description & ApplyYou'll be redirected to the employer's site
