Commercial experience of financial instruments and markets (equities, futures, options, forex, etc.), particularly understanding how historical data is used for algorithmic trading.
Familiarity with market data formats (e.g., MDP, ITCH, FIX, SWIFT, proprietary exchange APIs) and market data providers.
Strong programming skills in Python (Go/Rust is a nice to have)
Familiarity with ETL (Extract, Transform, Load) processes (or other data pipeline architecture) and tools to clean, normalize, and validate large datasets.
Commercial experience in building and maintaining large-scale time series or historical market data in the financial services industry.
Strong SQL proficiency: aggregations, joins, subqueries, window functions (first, last, candle, histogram), indexes, query planning, and optimization.
Strong problem-solving skills and attention to detail, particularly in ensuring data quality and reliability.
Bachelor’s degree in Computer Science, Engineering, or related field.
Responsibilities:
Design, develop, and maintain systems for the acquisition, storage, and retrieval of historical market data from multiple financial exchanges, brokers, and market data vendors
Ensure the integrity and accuracy of historical market data, including implementing data validation, cleansing, and normalization processes.
Build and optimize data storage solutions, ensuring they are scalable, high-performance, and capable of managing large volumes of time-series data.
Develop systems for data versioning and reconciliation to ensure that changes in exchange formats or corrections to past data are properly handled.
Implement robust integrations with various market data providers, exchanges, and proprietary data sources to continuously collect and store historical data.
Build internal tools to provide easy access to historical data for research and analysis, ensuring performance, ease of use, and data integrity
Work closely with quantitative researchers and traders to understand their data requirements and optimize the systems for data retrieval and analysis for backtesting and strategy development.
Develop scalable solutions to handle growing volumes of historical market data, including ensuring efficient queries and data retrieval for research and backtesting needs.
Work on optimizing data storage solutions, balancing cost-efficiency with performance, and ensuring that large datasets are managed effectively.
Ensure historical market data systems comply with regulatory requirements and assist in data retention, integrity, and reporting audits.