Senior Market Data Engineer
Company:
Location:Worldwide
Languages:English
Seniority level:Senior, 6+ years
Experience:6+ years
Skills:
AWSDockerPythonSQLApache AirflowC++AlgorithmsApache KafkaData engineeringData StructuresGogRPCREST APICI/CDLinuxData visualizationData modelingData management
Requirements:
6+ years of experience building and maintaining production-grade data systems. Proven expertise in architecting and launching data lakes from scratch. Expert-level Python development skills. Hands-on experience with modern orchestration tools (Airflow). Hands-on experience with streaming platforms (Kafka). Advanced SQL skills including complex aggregations, window functions, query optimization, and indexing. Experience designing high-throughput APIs (REST/gRPC) and data access libraries. Solid fundamentals in Linux, containerization (Docker). Solid fundamentals in cloud object storage solutions (AWS S3, GCS). Strong knowledge of handling diverse data formats. Experience optimizing storage strategies such as partitioning, compression, and cost management. English at C1 level.
Responsibilities:
Architect batch + stream pipelines for market data. Provide reusable SDKs in Python and Go. Implement and tune S3, column-oriented and time-series data storage. Own partitioning, compression, TTL, versioning and cost optimization. Develop internal libraries for schema management, data contracts, validation and lineage. Contribute to shared libraries and services. Embed monitoring, alerting, SLAs, SLOs and CI/CD. Champion automated testing, data quality dashboards and incident runbooks. Partner with Data Science, Quant Research, Backend and DevOps.
