Head of Data Engineering

Cornwallis Elt
London
3 days ago
Create job alert

Head of Data Engineering – Databricks, Azure, Data Strategy, Data Governance, Leadership, Management, Azure, Python, ETL, Data Pipelines, Private Markets


A global Private Equity firm are seeking an experienced Head of Data Engineering as part of a critical data transformation and modernisation programme, moving towards a data-first approach. They are currently in the process of replacing a legacy data-warehouse setup with a platform as a service model using Databricks hosted on Azure, which this role will be responsible for leading, taking line management responsibilities for their Data Engineering team.


You will take ownership for progressing their Databricks setup, from its current PoC/incubator phase through to production, ensuring a high level of optimisation and scalability to operate at a global scale.

This will also involve implementing a semantic layer within the platform for effective data management and organisation, as well as the development of associated data pipelines and real-time reporting & visualization capabilities.

Once the platform is in place, the business will then look to apply Data Science techniques with the aim of building a best-in-class data function that works in partnership with the Investment Team and actively provides deep, actionable insights to enable business growth.


The ideal candidate will demonstrate:

  • A technical background in data engineering with experience of technologies including Python, Spark, Databricks and Azure cloud.
  • Previous experience in managing the end-to-end build of an Azure Databricks platform
  • Passionate about all aspects of data (data governance, data quality management etc.) and the positive impact ‘good data’ can bring to a business
  • Demonstrable experience in setting up and managing high-performing technology teams, establishing and driving best practises and being able to still engage at a technical level
  • Able to credibly and confidently engage with the business at all levels
  • A background in Financial Services (Private Markets, Investment Banking, Investment Management) or similarly regulated indsutry such as Gambling/Online Gaming is critical


If you are a data enthusiast with experience in leading high-performing teams to develop modern data platforms, then this is a genuinely exciting time to join a growing business transitioning to a data-first approach.

Related Jobs

View all jobs

Head of Data Engineering | London, UK | Hybrid (Basé à London)

Head of Data Engineering - Product & Plan for Better

Head Of Data Engineering (Basé à London)

Head of Data Engineering - Product & Plan for Better (Basé à London)

Head of Data Engineering | London, UK | Hybrid (Basé à London)

Head Of Data Engineering (Basé à London)

Get the latest insights and jobs direct. Sign up for our newsletter.

By subscribing you agree to our privacy policy and terms of service.

Industry Insights

Discover insightful articles, industry insights, expert tips, and curated resources.

Jobs

Quantum-Enhanced AI in Data Science: Embracing the Next Frontier

Data science has undergone a staggering transformation in the past decade, evolving from a niche academic discipline into a linchpin of modern industry. Across every sector—finance, healthcare, retail, manufacturing—data scientists have become indispensable, leveraging statistical methods and machine learning to turn raw information into actionable insights. Yet as datasets grow ever larger and machine learning models become more computationally expensive, there are genuine questions about how far current methods can be pushed. Enter quantum computing, a nascent but promising technology grounded in the counterintuitive principles of quantum mechanics. Often dismissed just a few years ago as purely experimental, quantum computing is quickly gaining traction as prototypes evolve into cloud-accessible machines. When paired with artificial intelligence—particularly in the realm of data science—the results could be game-changing. From faster model training and complex optimisation to entirely new forms of data analysis, quantum-enhanced AI stands poised to disrupt established practices and create new opportunities. In this article, we will: Explore how data science has reached its current limits in certain areas, and why classical hardware might no longer suffice. Provide an accessible overview of quantum computing concepts and how they differ from classical systems. Examine the potential of quantum-enhanced AI to solve key data science challenges, from data wrangling to advanced machine learning. Highlight real-world applications, emerging job roles, and the skills you need to thrive in this new landscape. Offer actionable steps for data professionals eager to stay ahead of the curve in a rapidly evolving field. Whether you’re a practising data scientist, a student weighing up your future specialisations, or an executive curious about the next technological leap, read on. The quantum era may be closer than you think, and it promises to radically transform the very fabric of data science.

Jobs

Data Science Jobs at Newly Funded UK Start-ups: Q3 2025 Investment Tracker

Data science has become an indispensable cornerstone of modern business, driving decisions across finance, healthcare, e-commerce, manufacturing, and beyond. As organisations scramble to capitalise on the insights their data can offer, data scientists and machine learning (ML) experts find themselves in ever-higher demand. In the UK, which has cultivated a robust ecosystem of tech innovation and academic excellence, data-driven start-ups continue to blossom—fuelled by venture capital, government grants, and a vibrant talent pool. In this Q3 2025 Investment Tracker, we delve into the newly funded UK start-ups making waves in data science. Beyond celebrating their funding milestones, we’ll explore the job opportunities these investments have created for aspiring and seasoned data scientists alike. Whether you’re interested in advanced analytics, NLP (Natural Language Processing), computer vision, or MLOps, these start-ups might just offer the career leap you’ve been waiting for.

Jobs

Portfolio Projects That Get You Hired for Data Science Jobs (With Real GitHub Examples)

Data science is at the forefront of innovation, enabling organisations to turn vast amounts of data into actionable insights. Whether it’s building predictive models, performing exploratory analyses, or designing end-to-end machine learning solutions, data scientists are in high demand across every sector. But how can you stand out in a crowded job market? Alongside a solid CV, a well-curated data science portfolio often makes the difference between getting an interview and getting overlooked. In this comprehensive guide, we’ll explore: Why a data science portfolio is essential for job seekers. Selecting projects that align with your target data science roles. Real GitHub examples showcasing best practices. Actionable project ideas you can build right now. Best ways to present your projects and ensure recruiters can find them easily. By the end, you’ll be equipped to craft a compelling portfolio that proves your skills in a tangible way. And when you’re ready for your next career move, remember to upload your CV on DataScience-Jobs.co.uk so that your newly showcased work can be discovered by employers looking for exactly what you have to offer.

🍪 We use cookies to enhance your browsing experience on our website. By continuing to use this site, you consent to the use of cookies as described in our Cookie Policy. You can review our Cookie Policy for more information.