Data Engineer - Databricks / AI

JACOBS
Manchester
4 days ago
Create job alert
Overview

At Jacobs, we're challenging today to reinvent tomorrow by solving the world's most critical problems for thriving cities, resilient environments, mission-critical outcomes, operational advancement, scientific discovery and cutting-edge manufacturing, turning abstract ideas into realities that transform the world for good.

Your impact

Enjoy designing elegant data systems, shipping production-grade code, and seeing your work make a measurable difference, then this role is for you!

Our team build data platforms and AI solutions that power critical infrastructure, transform operations, and move entire industries.

In this role you will work with a broad set of clients on high-scale problems, with the backing of a global organisation, investing heavily in Azure, Databricks, and applied AI. Working primarily on Azure + Databricks (Spark, Delta Lake, Unity Catalog) - Ship modern ELT/ETL, streaming and batch data products, and ML/AI pipelines, operating at serious scale across water, transport, energy, and more. Join a collaborative, engineering-led culture with real investment in platforms & tooling.

Stack
  • Azure: ADLS Gen2, Event Hubs, ADF/Azure Data Factory or Synapse pipelines, Functions, Key Vault, VNets
  • Databricks: Spark, Delta Lake, Unity Catalog, Workflows, MLflow (experiments, model registry)
  • Languages: Python (PySpark), SQL (Delta SQL), optional Scala
  • Engineering: Git, pull requests, code review, unit/integration tests, dbx, notebooks as code
  • Platform & Ops: Azure DevOps/GitHub, CI/CD, Terraform or Bicep, monitoring/alerting
Responsibilities
  • Design & build robust data platforms and pipelines on Azure and Databricks (batch + streaming) using Python/SQL, Spark, Delta Lake, and Data Lakehouse patterns.
  • Develop AI-enabling foundations: feature stores, ML-ready datasets, and automated model-serving pathways (MLflow, model registries, CI/CD).
  • Own quality & reliability: testing (dbx/pytest), observability (metrics, logging, lineage), and cost/performance optimisation.
  • Harden for enterprise: security-by-design, access patterns with Unity Catalog, data governance, and reproducible environments.
  • Automate the boring stuff: IaC (Terraform/Bicep), CI/CD (Azure DevOps/GitHub Actions), and templated project scaffolding.
  • Partner with clients: translate business problems into technical plans, run workshops, and present trade-offs with clarity.
  • Ship value continuously: iterate, review, and release frequently; measure outcomes, not just outputs.
Qualifications
  • Utilising SQL and Python for building reliable data pipelines.
  • Hands-on with Spark (preferably Databricks) and modern data modelling (e.g., Kimball/Inmon/Data Vault, lakehouse).
  • Experience running on a cloud data platform (ideally Azure). Sound software delivery practices: Git, CI/CD, testing, Agile ways of working.
  • Streaming/event-driven designs (Event Hubs, Kafka, Structured Streaming).
  • MPP/Data Warehouses (Synapse, Snowflake, Redshift) and NoSQL (Cosmos DB).
  • ML enablement: feature engineering at scale, MLflow, basic model lifecycle know-how.
  • Infrastructure-as-code (Terraform/Bicep) and platform hardening.

Don’t meet every single bullet? We’d still love to hear from you. We hire for mindset and potential as much as current skills!

Culture & inclusion

Joining Jacobs not only connects you locally but globally. Our values stand on a foundation of safety, integrity, inclusion and belonging. We put people at the heart of our business, and we truly believe that by supporting one another through our culture of caring, we all succeed. We value positive mental health and a sense of belonging for all employees.

With safety and flexibility always top of mind, we’ve gone beyond traditional ways of working so you have the support, means and space to maximize your potential. You’ll uncover flexible working arrangements, benefits, and opportunities, from well-being benefits to our global giving and volunteering program, to exploring new and inventive ways to help our clients make the world a better place. No matter what drives you, you’ll discover how you can cultivate, nurture, and achieve your goals – all at a single global company. Find out more about life at Jacobs.

We aim to embed inclusion and belonging in everything we do. We know that if we are inclusive, we’re more connected and creative. We are committed to building vibrant communities within Jacobs, including through our Jacobs Employee Networks, Communities of Practice and our Find Your Community initiatives, allowing every employee to find connection, purpose, and belonging. Find out more about our Jacobs Employee Networks here.

Jacobs partners with VERCIDA to help us attract and retain talent from a wide range of backgrounds. For greater online accessibility please visit https://www.vercida.com/uk/employers/jacobs to view and access our roles.

As a disability confident employer, we will interview disabled candidates who best meet the criteria. We welcome applications from candidates who are seeking flexible working and from those who may not meet all the listed requirements for a role.

We value collaboration and believe that in-person interactions are crucial for both our culture and client delivery. We empower employees with our hybrid working policy, allowing them to split their work week between Jacobs offices/projects and remote locations enabling them to deliver their best work.

Your application experience is important to us, and we’re keen to adapt to make every interaction even better. If you require further support or reasonable adjustments with regards to the recruitment process (for example, you require the application form in a different format), please contact the team via Careers Support.


#J-18808-Ljbffr

Related Jobs

View all jobs

Data Engineer

Data Engineer

Data Engineer

Data Engineer

Data Engineer

Data Engineer

Subscribe to Future Tech Insights for the latest jobs & insights, direct to your inbox.

By subscribing, you agree to our privacy policy and terms of service.

Industry Insights

Discover insightful articles, industry insights, expert tips, and curated resources.

The Skills Gap in Data Science Jobs: What Universities Aren’t Teaching

Data science has become one of the most visible and sought-after careers in the UK technology market. From financial services and retail to healthcare, media, government and sport, organisations increasingly rely on data scientists to extract insight, guide decisions and build predictive models. Universities have responded quickly. Degrees in data science, analytics and artificial intelligence have expanded rapidly, and many computer science courses now include data-focused pathways. And yet, despite the volume of graduates entering the market, employers across the UK consistently report the same problem: Many data science candidates are not job-ready. Vacancies remain open. Hiring processes drag on. Candidates with impressive academic backgrounds fail interviews or struggle once hired. The issue is not intelligence or effort. It is a persistent skills gap between university education and real-world data science roles. This article explores that gap in depth: what universities teach well, what they often miss, why the gap exists, what employers actually want, and how jobseekers can bridge the divide to build successful careers in data science.

Data Science Jobs for Career Switchers in Their 30s, 40s & 50s (UK Reality Check)

Thinking about switching into data science in your 30s, 40s or 50s? You’re far from alone. Across the UK, businesses are investing in data science talent to turn data into insight, support better decisions and unlock competitive advantage. But with all the hype about machine learning, Python, AI and data unicorns, it can be hard to separate real opportunities from noise. This article gives you a practical, UK-focused reality check on data science careers for mid-life career switchers — what roles really exist, what skills employers really hire for, how long retraining typically takes, what UK recruiters actually look for and how to craft a compelling career pivot story. Whether you come from finance, marketing, operations, research, project management or another field entirely, there are meaningful pathways into data science — and age itself is not the barrier many people fear.

How to Write a Data Science Job Ad That Attracts the Right People

Data science plays a critical role in how organisations across the UK make decisions, build products and gain competitive advantage. From forecasting and personalisation to risk modelling and experimentation, data scientists help translate data into insight and action. Yet many employers struggle to attract the right data science candidates. Job adverts often generate high volumes of applications, but few applicants have the mix of analytical skill, business understanding and communication ability the role actually requires. At the same time, experienced data scientists skip over adverts that feel vague, inflated or misaligned with real data science work. In most cases, the issue is not a lack of talent — it is the quality and clarity of the job advert. Data scientists are analytical, sceptical of hype and highly selective. A poorly written job ad signals unclear expectations and immature data practices. A well-written one signals credibility, focus and serious intent. This guide explains how to write a data science job ad that attracts the right people, improves applicant quality and positions your organisation as a strong data employer.