Lead Data Engineer in United Kingdom

The work we do has an impact on millions of lives, and you can be a part of it. We help protect our customers against life’s uncertainties. Regardless of where you work within the company, you’ll be helping provide protection and peace of mind when our customers need it most. Protective is looking for a Lead Data Engineer to set the technical direction for a delivery pod building data products on Voyager, our Databricks lakehouse on Azure. You will lead the design of data products through the full medallion architecture — Bronze ingestion, Silver conformance, and Gold consumption — and be accountable for whether those products hold up for the consumers who depend on them. This is a hands-on technical leadership role, not a people-management or project-management role. You will still write and review production code daily. What you own is how the pod’s data products are designed, modeled, contracted, and tested, and the standard the pod holds itself to. The Product Owner owns the backlog and the Scrum Master owns the sprint; you own the engineering. On Voyager, the medallion layers are named Raw, Prep, and Prod. They map directly to Bronze, Silver, and Gold and are used interchangeably in this description. \n Key Responsibilities Design and modeling • Lead the design of data products end to end: what gets ingested, how it is cleansed and conformed, how it is modeled, and what the Gold layer looks like to the people querying it. • Own dimensional design — grain, natural and surrogate keys, Type 2 history, facts, bridges, and conformed dimensions shared across the pod’s products.

• Set the pod’s position on where logic belongs: what is cleaned in Silver, what is business logic in Gold, and what is a consumer’s own concern. • Keep models as simple as the questions require, and push back on designs that will not hold. • Partner with ML engineering where the pod’s Gold layer is the training or feature source for a model, so those datasets are contracted, versioned, and reproducible like any other consumer-facing product. Data contracts and consumer compatibility • Own the pod’s ODCS data contracts as real interfaces: named owners, named consumers, enforceable quality rules, freshness and update expectations, and an explicit breaking-change policy. • Make the compatibility call on every proposed contract change, and drive consumer notification when a change is genuinely breaking. • Represent the pod’s contracts in cross-domain conversations, where one pod’s Gold layer is another team’s dependency. Standards, quality, and operations • Set and hold the pod’s engineering standards for Python, SQL, dbt, testing, model structure, naming, and repository conventions — consistent with the platform’s paved paths and Azure DevOps CI gates rather than in competition with them. • Lead code review. Be the reviewer who catches the modeling mistake, the missing test, and the change that will break a consumer, and who explains why so the pod learns it. • Ensure quality rules are enforced in tests and asset checks rather than asserted in documentation, and that pipeline health is observable without someone going to look — freshness, volume, latency, and cost instrumented, and alerting set against the SLAs and SLOs the pod’s contracts commit to. • Own the pod’s operational posture for its own pipelines: failure diagnosis, data-issue triage, backfills, cost and performance tuning, on-call coverage and escalation, root-cause analysis, and runbooks someone other than the author can execute. • Keep the pod’s delivery inside the control expectations of a regulated carrier: change management through pull request and pipeline, segregation of duties between authoring and deploying, least-privilege access, and audit evidence that falls out of CI/CD rather than being assembled for an auditor. • Develop reusable frameworks, templates, and patterns that raise the pod’s consistency and delivery speed.