Back to list

Vacancy description

Principal Data Engineer (Data Practice Lead) in London, United Kingdom

YLD

London, United Kingdom

YLD is a software engineering and product design consultancy, founded in 2013 based in London, Lisbon and Porto.

We work with engineering and product leaders who know what good looks like and are frustrated by the gap between strategy and execution. We embed senior engineers and designers inside client teams, working in their codebase, their tooling, their delivery rhythm. AI runs through how our teams work, and we help clients adopt it into their own workflows too.

About the role:

We're building out data as a first-class practice at YLD, and we need someone in London who can be the credible technical voice in front of clients and prospects, not just describing what we could do, but backing it with real delivery experience.

This is a senior role, reporting to our Head of Engineering. You'll sit alongside our commercial and client services leadership in meetings where technical credibility matters, help shape and win new engagements, and stay close enough to client work to guide it and step in yourself when an engagement needs you. Alongside, we expect you to build our external reputation in data through events, writing, and the wider community.

About you:

You've been a strong data engineer or data lead for years, and you're comfortable in front of a client as well as in a pull request. You know what good data work looks like well enough to spot it, or its absence, in the first ten minutes of a client conversation. You've probably led a team before. Consultancy background is a strong plus, but not essential if you've got real client or stakeholder exposure elsewhere.

Your day-to-day responsibilities will look like this:

Hands-on delivery (core, from day one)

  • Doing hands-on data engineering work directly, pipelines, transformation layers, models, and the infrastructure around them, not just guiding others
  • Adapting to different client contexts: legacy warehouse migrations, greenfield lakehouses, transformation layers that need rescuing
  • Guiding the wider team and unblocking hard technical problems as your remit grows beyond your own delivery
  • Defining and evolving our internal standards for data work: testing, documentation, project structure, code review
  • Mentoring and growing data engineers across the company

Client and commercial (core, from day one)

  • Joining client and prospect meetings as the technical authority on data, alongside our client services and commercial leads
  • Shaping proposals and pitches, translating client problems into a credible data approach and a defensible scope
  • Acting as the escalation point on live data engagements when technical judgement calls need weight behind them
  • Travelling to client sites for workshops, discovery sessions, and in-person relationship building, and representing YLD at external data events

Practice, talent, and external profile (growth, over the first year)

  • Owning the growth of YLD's data practice: capability, reputation, and headcount over time
  • Contributing to attracting and hiring strong data engineers into YLD
  • Representing YLD at data events and in the wider community as your standing in the market builds
  • Supporting our sales team so that data becomes something we can proactively sell, not just respond to

You'll have the following skills and experience:

Leadership and commercial

  • Consulting or agency experience is a strong plus, but real client or stakeholder-facing exposure elsewhere is acceptable
  • A track record of technical leadership: leading a team or function
  • Comfortable presenting to senior client stakeholders and holding your own under commercial pressure
  • Cross-functional fluency: you translate engineering constraints into business terms and negotiate realistic commitments
  • Appetite to build a public presence in the data community over time, even if you don't have one yet

Technical

Not theoretical, evidenced by things you've actually shipped, and deep enough to set the bar for a team and answer a sceptical client:

  • SQL as engineering: read and review SQL that performs at scale, and understand query planning, engine quirks, and how materialisation choices affect cost and performance well enough to guide a team's decisions
  • Python for data: built and maintained production data systems in typed, testable Python, not just notebooks, and can set that standard for others
  • Data modelling: made deliberate choices between dimensional, Data Vault, normalised and denormalised designs, and can explain the trade-offs in flexibility, query performance, and maintainability to both engineers and clients
  • Transformation architecture: design transformations that are idempotent, incremental, and dependency-aware. You think in DAGs, not scripts
  • Pipeline design: weigh batch, streaming, and micro-batch trade-offs against latency, complexity, cost, and reprocessability, and pick the right approach for the problem
  • Data testing: know what to catch at build time (schema contracts, assertions, transformation logic) versus defer to observability, and can make that call for a team
  • Data observability: treat data reliability like site reliability, with measurable indicators, alerting, incident response, and root cause analysis
  • CI/CD for data: version, test, and deploy pipelines like software, with environment promotion, rollback strategies, and infrastructure as code
  • Data governance: implemented lineage, cataloguing, sensitive data classification, or access control, and can hold a client to account on making data auditable and secure
  • Cost-conscious: optimised warehouse spend, storage strategies, or job efficiency, and treat compute as a resource to manage, not ignore
  • Platform fluency: worked across orchestration (Airflow, Dagster), warehouses (Snowflake, BigQuery, Databricks), ingestion (Fivetran, Airbyte, custom), and transformation (dbt, Spark)
  • AI Native: experienced in agentic coding workflows (context, harness, loop, and graph engineering), and able to embed AI Engineering constructs into pipelines (RAG, semantic search, evals, guardrails, etc)

You should apply if:

  • Confident and credible in front of clients and prospects, without needing a script, and comfortable holding a firm technical or commercial line when a client or prospect pushes back
  • Still able to hold up in a deep technical review today, architecture, data modelling, pipeline design, not just talk about decisions you made a few years ago
  • Self-motivated, and want to own building a practice, not just execute a workstream someone else has defined, and comfortable without a fully defined remit from day one
  • Able to work well with commercial and client services colleagues who aren't technical themselves, translating for them without talking down
  • Genuinely energised by variety: different clients, different stacks, different problems, rather than depth in one domain
  • Happy travelling regularly, client sites, workshops, industry events, wherever the work and the relationship-building actually happens

Benefits you’ll receive:

  • Company Private Health care (currently provided by Vitality)
  • Enhanced fully paid maternity and paternity leave for up to 6 months
  • Company's Pension Scheme
  • 25 days annual holiday (excluding Public Holidays)
  • £2000 annual allowance for Training/Conferences
  • £300 annual allowance for additional hardware
  • Mental Health support
  • Bonus (depending on Company performance and results)
  • Company laptop
  • Generous referral schemes

Our typical Recruitment Process looks like this:

  • Screening call with the Talent team (45-60 mins)
  • Introductory conversation with our Head of Engineering (30 mins)
  • Technical conversation with our Head of Engineering & Data Specialist (60 mins)
  • Final Interview with our leadership team (60–90 mins)

We live and breathe our values, and know you will too:

  • Growing every day
  • Including everyone
  • Relationships built on honesty and ethics
  • Inspiring solutions
  • Winning together

We’re an equal-opportunity employer and value diversity in all its forms. We do not discriminate based on race, religion, colour, national origin, gender, sexual orientation, pregnancy or maternity, age, marital status, or disability. We also offer a remote-first working environment, with flexible working and work–life balance as standard for all employees.

 

Similar vacancies

More like this role

  • Hybrid
... AutoGen and similar technologies Create scalable AI workflows using Databricks, MLflow, Azure AI Foundry and cloud-native services Build and maintain robust data pipelines using modern data engineering practices and streaming technologies Deploy AI applications and services using Azure OpenAI, Azure AI Foundry, Databricks ...

London, United Kingdom

  • Remote
  • Hybrid
  • On-site
  • Full-time
... won Best Workplaces UK multiple years in a row. Join us on our journey. Let’s create tomorrow, together, today. About the role and team: We're looking for a Principal Data Engineer to join our Data Practice and lead the technical direction of our engineering capability. You'll work across a portfolio of clients spanning ...

United Kingdom

  • Full-time

£80k - £90k per annum

Company: DATATECH Job Type: Permanent, Full Time Salary: £80000 - £90000/annum [{"name": "permanent_employment", "text": "Permanent", "categoryName": "Type of employment"}]

London, United Kingdom

  • Hybrid
  • On-site
  • Full-time
... compliance best practices. - Experience working with insurance data (not essential, but preferred). This role is an excellent opportunity for an experienced Data Engineer who wants to take on a leadership role, influence best practices, and drive high-impact data initiatives in a modern Azure-based data ecosystem. What ...

London, United Kingdom

Tenth Revolution Group

Lead Data Engineer

  • Remote

£110k per annum

Lead Data Engineer Remote UK - £110,000 + Share Options + Private Medical Insurance A high-growth, venture-backed AI technology company is looking for a Lead Data Engineer to play a pivotal role in shaping the future of its data, analytics and AI capabilities. This is an opportunity to join an innovative organisation delivering ...

Bampton, United Kingdom

  • Full-time
  • Contract
  • IT

£45.77k per annum

... sound judgement and a collaborative approach. Transformation Lead - Research Data Management & FAIR Practice Research is increasingly dependent on high-quality data, digital methods and reproducible workflows. We are looking for a motivated and collaborative Transformation Lead - Research Data Management & FAIR Practice to ...

Aberdeen, United Kingdom

  • Remote
  • Contract
... used well. - Mentor scientists and data scientists on the physics of the data and on rigorous experimental practice. External scientific partners (about 20%) - Lead the scientific relationship with CROs, contractors and academic groups: write technical statements of work, set data and code standards, and review what comes ...

London, United Kingdom

$165k - $209k per annum

... solutions. Be the go-to technical expert on data systems. - Engineering practices: Set engineering standards through code review, design review, and example. Mentor engineers and scientists informally and share knowledge. Qualifications: - Bachelor's degree in Computer Science, Data Engineering, Computational Biology, Proteomics, ...

Cambridge, United Kingdom

CreateFuture

Lead Data Engineer

  • Remote
  • Hybrid
  • On-site
  • Full-time
... on our journey. Let’s create tomorrow, together, today. About the role and team: We are looking for a Lead Data Engineer to own the technical direction of a data engagement, working with Microsoft Fabric, Python, and PySpark across our data team. You'll lead a team of data engineers, setting the standards for how pipelines ...

United Kingdom

  • Remote
  • Hybrid
  • Full-time
  • Data Analytics

£71.73k per annum

Principal Data Analyst CAPTION: - Company: Government Recruitment Service - Salary: from £71,734 a year - Hours: Full-time - Location: Glasgow, G2 8HS - Working pattern: Hybrid - 2 days remote - Job type: Permanent - Posting date: 14 Oct 2026 - Closing date: 30 Oct 2026 Apply for this job Summary The Public Sector Reform ...

Glasgow, United Kingdom

  • Remote
  • Full-time
  • IT

$5.6 USD

... career sites. About the Role Fivetran is building data pipelines to power the modern data stack for thousands of companies. We’re seeking a Principal Software Engineer to contribute to the technical architecture, design and development of Fivetran’s Managed Data Lake product offering. As part of a senior engineering team, ...

United Kingdom