Lead Data Engineer

Birmingham
3 weeks ago
Create job alert

Lead Data Engineer - Databricks

Salary: £70-£95k (Dependant on Experience)

Location: Can be based anywhere in the UK (Largely remote, must be willing to travel weekly)

Are you an experienced Data Engineer with a passion for leading teams and delivering high-quality solutions?

We are looking for a Lead Data Engineer to join our dynamic consultancy customer.

This role is perfect for someone with a strong technical background in Databricks and Azure Analytics, combined with excellent leadership skills.

Key Responsibilities:

  • Databricks Expertise: Implement and optimize workloads in Databricks, with a full understanding of features, access controls, security, and networking. Experience with Unity Catalog is essential.

  • Team Leadership: Lead and mentor a team of internal and client engineers, setting high standards in coding, performance, and solution design.

  • Solution Architecture: Design and implement scalable data solutions using Azure Data Factory, Storage, Key Vault, Databricks, and/or Fabric Engineering.

  • Customer Engagement: Communicate complex technical solutions to non-technical stakeholders with clarity and confidence.

  • Technical Community Presence: Stay active in the data engineering community by contributing to events, blogs, or open-source initiatives.

  • On-Site Collaboration: Willingness to travel to client sites one day per week (expenses covered).

    Required Skills & Experience:

  • Expert-level proficiency in Python and Apache Spark.

  • Proven experience in Databricks, with a strong understanding of Unity Catalog, ingestion methods, and CI/CD.

  • Strong hands-on experience with Azure Data Factory, Key Vault, Storage, networking concepts, and Databricks (Microsoft Fabric a plus!)

  • Leadership experience with the ability to shape best practices and develop engineering teams.

  • Excellent communication and stakeholder management skills, able to bridge the gap between technical and non-technical audiences.

  • Ability to lead and run projects, and engage with key business stakeholders externally and internally.

    Join a team that values innovation, collaboration, and professional growth.

    Apply now to take the next step in your career

Related Jobs

View all jobs

Lead Data Engineer

Lead Data Engineer

Lead Data Engineer

Lead Data Engineer | London, UK

Lead Data Engineer, Subscriber Solutions

Lead Data Engineer (Data Infrastructure)

Get the latest insights and jobs direct. Sign up for our newsletter.

By subscribing you agree to our privacy policy and terms of service.

Industry Insights

Discover insightful articles, industry insights, expert tips, and curated resources.

Tips for Staying Inspired: How Data Science Pros Fuel Creativity and Innovation

Data science sits at the dynamic intersection of statistics, computer science, and domain expertise, driving powerful innovations in industries ranging from healthcare to finance, and from retail to robotics. Yet, the daily reality for many data scientists can be a far cry from starry-eyed talk of AI and machine learning transformations. Instead, it often involves endless data wrangling, model tuning, and scrutiny over metrics. Maintaining a sense of creativity in this environment can be an uphill battle. So, how do successful data scientists continue to dream big and innovate, even when dealing with the nitty-gritty of data pipelines, debugging code, or explaining results to stakeholders? Below, we outline ten practical strategies to help data analysts, machine learning engineers, and research scientists stay inspired and push their ideas further. Whether you’re just starting out or looking to reinvigorate a long-standing career, these pointers can help you find fresh sparks of motivation.

Top 10 Data Science Career Myths Debunked: Key Facts for Aspiring Professionals

Data science has become one of the most sought-after fields in the tech world, promising attractive salaries, cutting-edge projects, and the opportunity to shape decision-making in virtually every industry. From e-commerce recommendation engines to AI-powered medical diagnostics, data scientists are the force behind innovations that drive productivity and improve people’s lives. Yet, despite the demand and glamour often associated with this discipline, data science is also shrouded in misconceptions. Some believe you need a PhD in mathematics or statistics; others assume data science is exclusively about machine learning or coding. At DataScience-Jobs.co.uk, we’ve encountered a wide array of myths that can discourage talented individuals or mislead those exploring a data science career. This article aims to bust the top 10 data science career myths—providing clarity on what data scientists actually do and illuminating the true diversity and inclusiveness of this exciting field. Whether you’re a recent graduate, a professional looking to pivot, or simply curious about data science, read on to discover the reality behind the myths.

Global vs. Local: Comparing the UK Data Science Job Market to International Landscapes

How to evaluate salaries, opportunities, and work culture in data science across the UK, the US, Europe, and Asia Data science has proven to be more than a passing trend; it is now a foundational pillar of modern decision-making in virtually every industry—from healthcare and finance to retail and entertainment. As the volume of data grows exponentially, organisations urgently need professionals who can transform raw information into actionable insights. This high demand has sparked a wave of new opportunities for data scientists worldwide. In this article, we’ll compare the UK data science job market to those in the United States, Europe, and Asia. We’ll explore hiring trends, salary benchmarks, and cultural nuances to help you decide whether to focus your career locally or consider opportunities overseas or in fully remote roles. Whether you’re a fresh graduate looking for your first data science position, an experienced data professional pivoting from analytics, or a software engineer eager to break into machine learning, understanding the global data science landscape can be a game-changer. By the end of this overview, you’ll be better equipped to navigate the expanding world of data science—knowing which skills and certifications matter most, how salaries differ between regions, and what to expect from distinct work cultures. Let’s dive in.