Market Data Engineer
New
SpainFull-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Languages
- English
- Experience
- 5+ years
- Required Skills
- PythonSQLKubernetesAirflowSparkTerraform
Requirements
- 5+ years building production-grade data systems.
- Hands-on experience with Apache Iceberg, Delta, or Hudi.
- Familiarity with Apache Arrow.
- Experience with market data protocols including ITCH, FIX/FAST, and multicast UDP.
- Experience with network packet capture (pcap).
- Expert-level Python (Polars/PySpark).
- Advanced SQL skills including complex aggregations and window functions.
- Experience with Airflow and Apache Spark.
- Solid fundamentals in Linux, Docker, and Kubernetes/EKS.
- Experience with infrastructure-as-code (Terraform) and GitOps (ArgoCD).
- English fluency for documentation and collaboration.
Responsibilities
- Own the full capture path from wire to lake: decode and normalize raw exchange feeds and vendor sources into a unified canonical model.
- Build batch and stream pipelines using Airflow and Spark for tick and reference data.
- Own L2/L3 order-book reconstruction with gap handling.
- Design and maintain the Iceberg-over-S3 lakehouse, including partitioning, schema evolution, and compaction.
- Build internal libraries for schema management, data contracts, and lineage.
- Develop shared access services for Research and trading teams.
- Embed monitoring, alerting, and CI/CD pipelines across capture layers on Kubernetes.
- Maintain data-quality dashboards and incident runbooks.
- Collaborate with Quant Research, Data Science, and DevOps to translate requirements into platform capabilities.
View Full Description & ApplyYou'll be redirected to the employer's site
