Back to list

Vacancy description

Principal Data Engineer (Data Practice Lead) in London, United Kingdom

YLD

London, United Kingdom

YLD is a software engineering and product design consultancy, founded in 2013 based in London, Lisbon and Porto.

We work with engineering and product leaders who know what good looks like and are frustrated by the gap between strategy and execution. We embed senior engineers and designers inside client teams, working in their codebase, their tooling, their delivery rhythm. AI runs through how our teams work, and we help clients adopt it into their own workflows too.

About the role:

We're building out data as a first-class practice at YLD, and we need someone in London who can be the credible technical voice in front of clients and prospects, not just describing what we could do, but backing it with real delivery experience.

This is a senior role, reporting to our Head of Engineering. You'll sit alongside our commercial and client services leadership in meetings where technical credibility matters, help shape and win new engagements, and stay close enough to client work to guide it and step in yourself when an engagement needs you. Alongside, we expect you to build our external reputation in data through events, writing, and the wider community.

About you:

You've been a strong data engineer or data lead for years, and you're comfortable in front of a client as well as in a pull request. You know what good data work looks like well enough to spot it, or its absence, in the first ten minutes of a client conversation. You've probably led a team before. Consultancy background is a strong plus, but not essential if you've got real client or stakeholder exposure elsewhere.

Your day-to-day responsibilities will look like this:

Hands-on delivery (core, from day one)

  • Doing hands-on data engineering work directly, pipelines, transformation layers, models, and the infrastructure around them, not just guiding others
  • Adapting to different client contexts: legacy warehouse migrations, greenfield lakehouses, transformation layers that need rescuing
  • Guiding the wider team and unblocking hard technical problems as your remit grows beyond your own delivery
  • Defining and evolving our internal standards for data work: testing, documentation, project structure, code review
  • Mentoring and growing data engineers across the company

Client and commercial (core, from day one)

  • Joining client and prospect meetings as the technical authority on data, alongside our client services and commercial leads
  • Shaping proposals and pitches, translating client problems into a credible data approach and a defensible scope
  • Acting as the escalation point on live data engagements when technical judgement calls need weight behind them
  • Travelling to client sites for workshops, discovery sessions, and in-person relationship building, and representing YLD at external data events

Practice, talent, and external profile (growth, over the first year)

  • Owning the growth of YLD's data practice: capability, reputation, and headcount over time
  • Contributing to attracting and hiring strong data engineers into YLD
  • Representing YLD at data events and in the wider community as your standing in the market builds
  • Supporting our sales team so that data becomes something we can proactively sell, not just respond to

You'll have the following skills and experience:

Leadership and commercial

  • Consulting or agency experience is a strong plus, but real client or stakeholder-facing exposure elsewhere is acceptable
  • A track record of technical leadership: leading a team or function
  • Comfortable presenting to senior client stakeholders and holding your own under commercial pressure
  • Cross-functional fluency: you translate engineering constraints into business terms and negotiate realistic commitments
  • Appetite to build a public presence in the data community over time, even if you don't have one yet

Technical

Not theoretical, evidenced by things you've actually shipped, and deep enough to set the bar for a team and answer a sceptical client:

  • SQL as engineering: read and review SQL that performs at scale, and understand query planning, engine quirks, and how materialisation choices affect cost and performance well enough to guide a team's decisions
  • Python for data: built and maintained production data systems in typed, testable Python, not just notebooks, and can set that standard for others
  • Data modelling: made deliberate choices between dimensional, Data Vault, normalised and denormalised designs, and can explain the trade-offs in flexibility, query performance, and maintainability to both engineers and clients
  • Transformation architecture: design transformations that are idempotent, incremental, and dependency-aware. You think in DAGs, not scripts
  • Pipeline design: weigh batch, streaming, and micro-batch trade-offs against latency, complexity, cost, and reprocessability, and pick the right approach for the problem
  • Data testing: know what to catch at build time (schema contracts, assertions, transformation logic) versus defer to observability, and can make that call for a team
  • Data observability: treat data reliability like site reliability, with measurable indicators, alerting, incident response, and root cause analysis
  • CI/CD for data: version, test, and deploy pipelines like software, with environment promotion, rollback strategies, and infrastructure as code
  • Data governance: implemented lineage, cataloguing, sensitive data classification, or access control, and can hold a client to account on making data auditable and secure
  • Cost-conscious: optimised warehouse spend, storage strategies, or job efficiency, and treat compute as a resource to manage, not ignore
  • Platform fluency: worked across orchestration (Airflow, Dagster), warehouses (Snowflake, BigQuery, Databricks), ingestion (Fivetran, Airbyte, custom), and transformation (dbt, Spark)
  • AI Native: experienced in agentic coding workflows (context, harness, loop, and graph engineering), and able to embed AI Engineering constructs into pipelines (RAG, semantic search, evals, guardrails, etc)

You should apply if:

  • Confident and credible in front of clients and prospects, without needing a script, and comfortable holding a firm technical or commercial line when a client or prospect pushes back
  • Still able to hold up in a deep technical review today, architecture, data modelling, pipeline design, not just talk about decisions you made a few years ago
  • Self-motivated, and want to own building a practice, not just execute a workstream someone else has defined, and comfortable without a fully defined remit from day one
  • Able to work well with commercial and client services colleagues who aren't technical themselves, translating for them without talking down
  • Genuinely energised by variety: different clients, different stacks, different problems, rather than depth in one domain
  • Happy travelling regularly, client sites, workshops, industry events, wherever the work and the relationship-building actually happens

Benefits you’ll receive:

  • Company Private Health care (currently provided by Vitality)
  • Enhanced fully paid maternity and paternity leave for up to 6 months
  • Company's Pension Scheme
  • 25 days annual holiday (excluding Public Holidays)
  • £2000 annual allowance for Training/Conferences
  • £300 annual allowance for additional hardware
  • Mental Health support
  • Bonus (depending on Company performance and results)
  • Company laptop
  • Generous referral schemes

Our typical Recruitment Process looks like this:

  • Screening call with the Talent team (45-60 mins)
  • Introductory conversation with our Head of Engineering (30 mins)
  • Technical conversation with our Head of Engineering & Data Specialist (60 mins)
  • Final Interview with our leadership team (60–90 mins)

We live and breathe our values, and know you will too:

  • Growing every day
  • Including everyone
  • Relationships built on honesty and ethics
  • Inspiring solutions
  • Winning together

We’re an equal-opportunity employer and value diversity in all its forms. We do not discriminate based on race, religion, colour, national origin, gender, sexual orientation, pregnancy or maternity, age, marital status, or disability. We also offer a remote-first working environment, with flexible working and work–life balance as standard for all employees.

 

Similar vacancies

More like this role

  • On-site
... the Data Engineering team within the Global Data Office — supporting recruitment, performance management, skills growth and the adoption of engineering best practices. - Work closely with Data Office leadership to define engineering strategy, support delivery planning, and build a high-performing, collaborative engineering ...

Manchester, United Kingdom

  • On-site
  • Full-time
As a Data Consultant you will work with customers and partners, guiding them through planning, preparing and delivery of complex data transformation projects based on varying customer needs. You will build and maintain strong stakeholder relationship with C-level executives to shape their cloud and data transformation roadmap. ...

Bristol, United Kingdom

  • Hybrid
... AutoGen and similar technologies Create scalable AI workflows using Databricks, MLflow, Azure AI Foundry and cloud-native services Build and maintain robust data pipelines using modern data engineering practices and streaming technologies Deploy AI applications and services using Azure OpenAI, Azure AI Foundry, Databricks ...

London, United Kingdom

  • Remote
  • Hybrid
  • On-site
  • Full-time
... won Best Workplaces UK multiple years in a row. Join us on our journey. Let’s create tomorrow, together, today. About the role and team: We're looking for a Principal Data Engineer to join our Data Practice and lead the technical direction of our engineering capability. You'll work across a portfolio of clients spanning ...

United Kingdom

CreateFuture

Lead Data Engineer

  • Remote
  • Hybrid
  • On-site
  • Full-time
... on our journey. Let’s create tomorrow, together, today. About the role and team: We are looking for a Lead Data Engineer to own the technical direction of a data engagement, working with Microsoft Fabric, Python, and PySpark across our data team. You'll lead a team of data engineers, setting the standards for how pipelines ...

United Kingdom

  • Remote
  • Full-time
  • IT

$5.6 USD

... career sites. About the Role Fivetran is building data pipelines to power the modern data stack for thousands of companies. We’re seeking a Principal Software Engineer to contribute to the technical architecture, design and development of Fivetran’s Managed Data Lake product offering. As part of a senior engineering team, ...

United Kingdom

£140k per annum

... business is looking for a technically strong, delivery-focused professional who understands the demands of live data centre environments and has the experience to lead complex BMS, EPMS, PLC and SCADA delivery programmes. This is a high-profile position combining technical leadership, project delivery and engineering management, ...

London, United Kingdom

Our Future Health

Data Engineer (NHS Data)

  • Remote
  • Hybrid
  • On-site
  • Full-time
  • Contract

£62k per annum

... this role is defined by your ability to build pipelines that are reliable, compliant, and secure when handling sensitive health data. You'll work closely with Engineering Managers, Lead Data Engineers, researchers, and domain experts to deliver data products that make a real-world impact. We're looking for a Data Engineer ...

London, United Kingdom

  • Hybrid
  • Full-time
... information. Whether you're an experienced Data Architect, Solution Architect, Enterprise Architect, Data Engineer or Analytics Consultant looking to specialise in data architecture, this is an opportunity to work on some of the UK's most complex and rewarding programmes. As part of our Secure Government Practice, you'll work ...

Bristol, United Kingdom

  • Remote
  • Hybrid
  • Contract
... other. You will also work daily with our engineers, architects and product managers. This is a senior individual-contributor role. It carries no team. It is principal-level because the work has to move engineers, product managers and the data team without authority over any of them. What you’ll do Understand clients and ...

London, United Kingdom