Were looking for strong hands-on experience with Databricks, AWS, Python, PySpark and SQL.
We need experience with Databricks notebooks, scheduled jobs, cluster configuration, driver and executor logs, Spark UI, and performance troubleshooting.
Were looking for experience with Spark SQL, Delta Lake, Parquet, Hive metastore tables, schemas, partitions, and the relationship between table metadata and underlying files.
We need hands-on AWS experience, particularly with S3, IAM and CloudWatch, plus an understanding of role-based access, KMS encryption, and diagnosing data access failures.
Were looking for a working knowledge of Git, code reviews, CI/CD, and controlled production deployments, including testing, rollback and release validation.
Experience with AWS Glue, Lambda, Step Functions, Linux, shell scripts, Terraform, GitLab or Jenkins would be advantageous.
Responsibilities:
We need you to run and maintain production data pipelines on Databricks and AWS.
Youll keep scheduled data processing reliable, resolve incidents, apply fixes, and improve data service performance.
Youll provide a combination of L2/L3 production support and data engineering.
Youll support and maintain production data pipelines, including incident investigation, safe recovery, root cause analysis and permanent remediation.
Youll own incidents from investigation through recovery and closure, diagnosing issues in Python, PySpark, SQL, Databricks jobs and AWS integrations.
Youll provide clear progress updates and escalate issues in a timely manner.
Technologies:
AWS
AWS Glue
Lambda
CI/CD
CloudWatch
Databricks
Git
GitLab
Hive
IAM
Support
Jenkins
Linux
Python
PySpark
SQL
Spark
Terraform
UX UI Design
Cloud
More:
Were hiring a Data Engineer for a permanent, full-time role on a large-scale Government project as part of a leading global IT transformation. The position is based in Leeds with a hybrid model, primarily working from home and typically travelling to our Leeds office once or twice a month. The expected start is ASAP in October or November 2026. We offer a salary of £60,000–£65,000, depending on experience, remote/hybrid working, a pension, 22 days holiday plus bank holidays (rising to 25 days), death-in-service cover and a healthcare plan.
Salary: £? - ? per year Requirements: - Proven data engineering experience delivering end-to-end data solutions - Strong SQL and Python development skills - Hands-on experience with GCP, BigQuery, or similar cloud data platforms - Knowledge of data modelling, governance, metadata, and lineage - Understanding of operational ...
... Design, architect, implement, and support key datasets and data warehouses for both static and real time data. - Engineer analytics dashboards and tooling for data retrieval. - Perform analysis to assess and enhance quality and meaning of data. - Leverage automated tools to extract data from primary and secondary sources. ...
... deployment of data solutions. - Develop and maintain technical documentation relating to data architecture, pipelines and processes. - Collaborate and pair-code with data engineers and data scientists. Experience and Skills Required - Minimum 1 year of experience in a Data Engineering or related role. - Strong programming skills ...
... hubs, we offer a monthly co-working stipend. ABOUT THE ROLE We’re looking for an experienced Data Engineer to join our Data team. You will build and scale our data pipelines, establish best practices, ensure data quality, and enable self-service data access across teams. – Own and streamline data pipelines from ingestion ...
... your technical expertise has a real impact on global operations and growth. The Role We are seeking a skilled and proactive Data Engineer to join our IT and Data & Analytics team. You will be responsible for designing, building, and maintaining data pipelines and integrations. You’ll also unify data from third-party platforms ...
... efficient and robust data engineering projects within both on-premises and cloud-based data stacks that add business value and comply with architectural and data security requirements. - Building, maintaining and supporting data engineering pipelines, with resolution of incidents. - End-to-end delivery of data engineering ...
... best use of their data. In building data platforms and pipelines, our data engineers create the foundation for diverse data & analytics solutions, including data science and AI. They build data lakes and warehouses, create the processes to extract or access operational data, and transform siloed datasets into integrated ...
... Mathematics, Engineering, or a related field, or equivalent practical experience. - Minimum 2 years of experience as a Data Engineer, Backend Engineer, or Software Engineer working with data-intensive systems and data pipelines. - Solid software engineering skills in Python, with experience writing reliable, maintainable data processing ...
... operating in both our legacy and future environments, ultimately transforming OEGEN into an efficient, data-driven organization. About the role We’re looking for a Data Engineer who can own the journey of data from source systems to decision-ready outputs. You’ll build and maintain pipelines, model data, stand up dashboards, ...
... availableThe RoleAs a Data Engineer, you will focus on designing, developing, and optimising data ingestion, transformation, and serving layers across Azure Databricks, Delta Lake, and Databricks SQL. Your work will enable reliable data products and insights for stakeholders, and you'll collaborate with data architects, ...
... commercial experience with Databricks - Desirable: exposure to large-scale data processing and cloud-based data platforms - Desirable: experience working with event data, tracking data or other complex datasets - Desirable: a relevant degree or postgraduate qualification in Computer Science, Data Engineering, Mathematics or a ...