Senior Data Engineer
New
E
EdgeConneXData Center Infrastructure
Bulgaria, Georgia, Lithuania, Mexico, Moldova, Poland, Ukraine, Minimum overlap with EST time zone - 4 -5 hoursFull-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Experience
- 7+ years
- Required Skills
- PythonSQLJavaKubernetesAmazon Web ServicesApache KafkaClickhouseTerraform
Requirements
- 7+ years of data or software engineering experience, including production streaming systems.
- Deep hands-on experience with Apache Kafka and Apache Flink.
- Strong proficiency in Java, Python, and SQL.
- Experience with lakehouse technologies such as Apache Iceberg, Parquet, and Avro.
- Production experience with analytical serving databases like ClickHouse, Druid, or Pinot.
- Experience with relational databases including SQL Server or PostgreSQL.
- Solid Kubernetes skills including EKS, Helm, and GitOps with Argo CD.
- Proficiency with Terraform and GitHub Actions.
- In-depth knowledge of AWS services: MSK, EKS, EMR Serverless, MWAA, S3, Athena, and IAM.
- Familiarity with ML fundamentals, LLMs, and vector search.
- Ability to work independently in ambiguous environments.
- Strong communication skills for technical and non-technical stakeholders.
Responsibilities
- Design, build, and operate real-time streaming pipelines using Apache Kafka and Apache Flink for high-throughput telemetry data.
- Manage streaming data contracts and schema evolution using Avro and a schema registry.
- Build and maintain analytical serving layers using columnar OLAP databases like ClickHouse.
- Develop and operate batch and scheduled workflows with Apache Airflow.
- Build and run the data lakehouse using Apache Iceberg, PySpark, and Amazon Athena.
- Deploy containerized workloads on Amazon EKS using Argo CD, Helm, and GitOps.
- Manage infrastructure as code with Terraform and CI/CD pipelines with GitHub Actions.
- Monitor production data platforms using Prometheus and Grafana and participate in incident response.
- Contribute to cloud migration work, porting pipelines from legacy Azure stacks to AWS.
- Write production-grade code in Java, Python, and SQL with automated testing.
View Full Description & ApplyYou'll be redirected to the employer's site
