Principal Data Engineer (Data Practice Lead)
New
Y
YLDData engineering consultancy
Location: LondonFull-TimePrincipal
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Required Skills
- PythonSQLApache AirflowCI/CDData modelingdbt
Requirements
- Have a track record of technical leadership, such as leading a team or function.
- Bring strong hands-on data engineering experience and be able to contribute to deep technical reviews.
- Use SQL for engineering, including reviewing queries that perform at scale and understanding query planning, engine quirks, and materialisation trade-offs.
- Have built and maintained production data systems in typed, testable Python.
- Make and explain data modelling trade-offs across dimensional, Data Vault, normalised, and denormalised designs.
- Design idempotent, incremental, dependency-aware transformations and think in DAGs.
- Evaluate batch, streaming, and micro-batch pipeline trade-offs.
- Apply data testing and observability practices, including alerting, incident response, and root cause analysis.
- Version, test, and deploy data pipelines using CI/CD, environment promotion, rollback strategies, and infrastructure as code.
- Have implemented data governance practices such as lineage, cataloguing, sensitive data classification, or access control.
- Have experience managing warehouse spend, storage strategies, or job efficiency.
- Bring platform experience across orchestration, data warehouses, ingestion, and transformation; examples include Airflow, Dagster, Snowflake, BigQuery, Databricks, Fivetran, Airbyte, dbt, and Spark.
- Have experience with agentic coding workflows and AI engineering constructs such as RAG, semantic search, evaluations, and guardrails.
- Bring client or stakeholder-facing experience and be comfortable presenting to senior stakeholders and negotiating realistic commitments.
- Be willing to travel regularly to client sites, workshops, and industry events.
Responsibilities
- Build and maintain data pipelines, transformation layers, models, and supporting infrastructure.
- Adapt data engineering approaches to legacy warehouse migrations, greenfield lakehouses, and transformation layers that need improvement.
- Guide the wider team and unblock difficult technical problems.
- Define and evolve internal standards for data testing, documentation, project structure, and code review.
- Mentor and grow data engineers across the company.
- Join client and prospect meetings as the technical authority on data.
- Shape proposals and pitches by translating client problems into a credible data approach and scope.
- Act as an escalation point for technical decisions on live data engagements.
- Travel to client sites for workshops and discovery sessions, and represent YLD at external data events.
- Grow the data practice’s capability, reputation, and headcount, and support hiring and sales.
View Full Description & ApplyYou'll be redirected to the employer's site
