- Design, build, and maintain scalable data infrastructure to support analytics and reporting across the organization.
- Develop and operate ETL pipelines to ingest, transform, and deliver large-scale datasets.
- Work with distributed data processing frameworks such as Spark, Hive, or similar MPP architectures.
- Use SQL and data modeling techniques to structure and optimize datasets for analytics use cases.
- Process and analyze large volumes of structured and semi-structured data using tools such as Spark and Presto.
- Write production-quality code using Python, Java, Scala, or Go.
- Ensure data reliability and availability, operating and monitoring hundreds of ETL pipelines with strict SLAs.
- Investigate and resolve complex data issues, including root-cause analysis of pipeline failures or data inconsistencies.
- Partner closely with Data Analysts and cross-functional stakeholders to provide reliable datasets and guide them in using data effectively.
- Troubleshoot data issues in dashboarding tools (e.g., Tableau, Power BI, MicroStrategy) and propose solutions.
PythonSQLETL+8 more