Senior Data Engineer, Data Transformation Platform
New
J
JobgetherStructured finance
Workplace type: Remote; based in the United States.Full-TimeSenior
Salary160,000 - 175,000 USD per year
Apply NowOpens the employer's application page
Job Details
- Experience
- At least 7 years of experience designing, building, and operating production data pipelines and data platforms on a major cloud provider.
- Required Skills
- PythonSQLGCPSparkData modelingBigQuerydbtDatabricks
Requirements
- Have at least 7 years of experience designing, building, and operating production data pipelines and data platforms on a major cloud provider.
- Demonstrate strong data modeling and architecture capabilities, including designing maintainable, understandable, and extensible pipelines and schemas.
- Have advanced proficiency in Python and SQL.
- Have hands-on experience with dbt and Apache Spark.
- Have practical experience using Databricks for data pipelines and analytics, including workspace administration.
- Have experience managing deployment processes and orchestration for complex production data pipelines.
- Be able to optimize large-scale data processing workloads for performance and cost efficiency.
- Be familiar with cloud data services such as BigQuery, Cloud Composer, and Dataproc, or comparable solutions.
- Be able to troubleshoot production issues and identify their underlying causes.
- Be able to work independently, prioritize effectively, and collaborate with technical and non-technical stakeholders.
- Experience with Scala or Java is an advantage.
- Interest in structured finance and financial markets is expected; prior financial industry experience is preferred but not required.
Responsibilities
- Design, develop, and maintain production data pipelines using Python, SQL, and dbt across Google Cloud Platform and Databricks.
- Own data architecture, pipeline design, and data modeling for scalable, reliable, and maintainable schemas and transformation workflows.
- Profile, troubleshoot, and optimize large-scale data processing jobs for performance, efficiency, and infrastructure costs.
- Lead roadmap initiatives, including developing pipelines, migrating legacy workflows, and modernizing the data platform.
- Monitor production pipelines, troubleshoot issues, perform root-cause analysis, and participate in an on-call rotation.
- Conduct code reviews, mentor engineers and analysts, and promote engineering quality and maintainability.
- Improve development practices and team tooling, including adoption of AI-assisted development tools such as Claude Code.
- Collaborate with analysts, product teams, and engineering groups to translate business and data requirements into data transformations.
- Evaluate technologies and identify opportunities to improve platform architecture, workflows, and operational performance.
View Full Description & ApplyYou'll be redirected to the employer's site
