National AI Awards 2025Discover AI's trailblazers! Join us to celebrate innovation and nominate industry leaders.

Nominate & Attend

Data Engineer

Pioneer Search
City of London
1 week ago
Create job alert

Data Engineer

Azure | Databricks | PySpark | Hybrid Cloud | Fabric

Location:London (Hybrid)
Salary:£85,000 + Bonus + Benefits
Type:Permanent

A Data Engineeris required for a fast-evolving (re)insurance business at the heart of the Lloyd's market, currently undergoing a major data transformation. With a strong foundation in the industry and a clear vision for the future, they're investing heavily in their data capabilities to build a best-in-class platform that supports smarter, faster decision-making across the business.

As part of this journey, they're looking for aData Engineerto join their growing team. This is a hands-on role focused on building scalable data pipelines and enhancing a modern Lakehouse architecture usingDatabricks,PySpark, andAzure. The environment is currentlyhybrid cloud and on-prem, with a strategic move towardsMicrosoft Fabric-so experience across both is highly valued.

What you'll be doing:

  • Building and maintaining robust data pipelines usingDatabricks,PySpark, andAzure Data Factory.
  • Enhance and maintain a Lakehouse architecture using Medallion principles
  • Working across bothcloud and on-prem environments, supporting the transition toMicrosoft Fabric.
  • Collaborating with stakeholders across Underwriting, Actuarial, and Finance to deliver high-impact data solutions.
  • Support DevOps practices and CI/CD workflows in Azure

Technical Requirements:

  • Strong hands-on experience with:
    • Databricks(including pipeline development and orchestration)
    • PySparkandPythonfor data transformation and processing
    • Azure Data Factoryfor data integration
  • Solid understanding of:
    • Medallion architectureand Lakehouse design principles
    • Hybrid cloud/on-prem environments
    • Microsoft Azureecosystem
  • Exposure to or interest in:
    • Microsoft Fabricand its evolving role in enterprise data platforms
    • Azure DevOpsfor CI/CD and deployment
    • T-SQLand dimensional modelling (Kimball methodology)
    • Experience inFinancial ServicesorLloyd's marketis a plus

Apply now or get in touch to find out more -

Related Jobs

View all jobs

Data Engineer

Data Engineer

Data Engineer

Data Engineer

Data Engineer

Data Engineer

National AI Awards 2025

Subscribe to Future Tech Insights for the latest jobs & insights, direct to your inbox.

By subscribing, you agree to our privacy policy and terms of service.

Industry Insights

Discover insightful articles, industry insights, expert tips, and curated resources.

LinkedIn Profile Checklist for Data Science Jobs: 10 Tweaks to Elevate Recruiter Engagement

Data science recruiters often sift through dozens of profiles to find candidates skilled in Python, machine learning, statistical modelling and data visualisation—sometimes before roles even open. A generic LinkedIn profile won’t suffice in this data-driven era. This step-by-step LinkedIn for data science jobs checklist outlines ten targeted tweaks to elevate recruiter engagement. Whether you’re an aspiring junior data scientist, a specialist in MLOps, or a seasoned analytics leader, these optimisations will sharpen your profile’s search relevance and demonstrate your analytical impact.

Part-Time Study Routes That Lead to Data Science Jobs: Evening Courses, Bootcamps & Online Masters

Data science sits at the intersection of statistics, programming and domain expertise—unearthing insights that drive business decisions, product innovation and research breakthroughs. In the UK, organisations from fintech and healthcare to retail and public sector are investing heavily in data-driven strategies, fuelling unprecedented demand for data scientists, machine learning engineers and analytics consultants. According to recent projections, data science roles will grow by over 40% in the next five years, offering lucrative salaries and varied career paths. Yet many professionals hesitate to leave their current jobs or pause personal commitments for full-time study. The good news? A vibrant ecosystem of part-time learning routes—Evening Courses, Intensive Bootcamps and Flexible Online Master’s Programmes—empowers you to learn data science while working. This comprehensive guide explores every pathway: foundational CPD units and short courses, hands-on bootcamps, accredited online MScs, plus funding options, planning strategies and a real-world case study. Whether you’re an analyst looking to formalise your skills, a software developer pivoting into data or a manager seeking to harness data-driven decision-making, you’ll find the right route to fit your schedule, budget and career goals.

The Ultimate Assessment-Centre Survival Guide for Data Science Jobs in the UK

Assessment centres for data science positions in the UK are designed to replicate the multifaceted challenges of real-world analytics teams. Employers combine psychometric assessments, coding tests, statistical reasoning exercises, group case studies and behavioural interviews to see how you interpret data, build models, communicate insights and collaborate under pressure. Whether you’re specialising in predictive modelling, NLP or computer vision, this guide provides a step-by-step roadmap to excel at every stage and secure your next data science role.