Senior Software Engineer of Data Platform and Services
L
LivePersonConversational AI
Berlin, Germany; Remote-First Model: Be flexible, work flexibly and from anywhere - no defined working hours or boundaries with offices in Mannheim and BerlinFull-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Experience
- 6–8 years
- Required Skills
- PythonHadoopKubernetesAirflowDatabricks
Requirements
- 6–8 years of software engineering experience in distributed systems and large-scale data platforms.
- Demonstrated experience driving legacy-to-modern platform migrations (e.g., Hadoop/Impala to Databricks).
- Strong expertise in the Databricks ecosystem: Jobs, Delta Lake, and cluster configuration.
- Deep proficiency with Python for ETL scripting and designing production Airflow DAGs.
- Skilled in porting complex SQL from Impala/Hive to Spark SQL.
- Experience with cloud-native data storage, specifically Google Cloud Storage (GCS).
- Familiarity with Kubernetes (GKE) and operating data workloads in containerized environments.
- Proven experience with data integration patterns and event-driven systems.
- Strong understanding of distributed data processing, batch vs. real-time trade-offs, and observability.
- Proficiency in 'vibe coding' and AI-assisted development (e.g., prompt engineering, Claude, Codex).
Responsibilities
- Own and drive the migration of legacy Hadoop, Spark, and Impala data pipelines to Databricks.
- Leverage AI coding assistants (e.g., Claude, Codex) to port complex SQL and translate legacy scripts.
- Design, build, and maintain scalable Airflow DAGs and Databricks Jobs.
- Implement validation strategies to ensure data parity and quality during the migration process.
- Enhance platform reliability by contributing to SLOs/SLAs and participating in on-call rotations.
- Apply data-driven design principles for conversational analytics and business reporting.
View Full Description & ApplyYou'll be redirected to the employer's site
