Data Engineering Lead

Cornwallis Elt
London
4 days ago
Create job alert

Job Description

Lead Data Engineer– Python, SQL, dbt, AWS, Dremio, Data Mesh, ETL, Data Pipelines, Data Warehousing, DWH, S3, Glue, Data Lake, Automation, London

A Lead Data Engineer is sought after by a leading workplace pensions provider to run their expanding data engineering team in their London City office. The team has recently built out a next-gen data lake hosted in AWS which makes use of Dremio, Data Mesh architecture, and a wide range of AWS tools (S3, Glue, Step Functions, etc.).

With the infrastructure of this new Data Lake in place, there is now a focus on enhancing the data stored within (via monitoring, cleaning, and a requirement to design, build, and implement robust ETL pipelines for effective integration with the wider business).

As a Lead Engineer, you will also be responsible for overseeing a small team of Data Engineers, setting and driving best practices and standards, ensuring project delivery is kept on track, and regularly engaging with stakeholders up to C-suite for feedback/updates. This will also involve working directly alongside the Head of Data Platforms to ensure ongoing engineering work is aligned to overall architecture and roadmap, as well as future project planning.

If you demonstrate the following skillset, please do apply!

  1. 7/8 years’ experience in Python programming, particularly around data analysis, automation, and the building of data pipelines (as well as an understanding of data warehousing concepts)
  2. Strong SQL skills, especially in relation to ETL processes and data management (querying, cleaning, storing)
  3. Experience working with and deploying to AWS cloud (having worked with any/all of S3, Glue, Lambda, Step Functions, etc.)
  4. Usage of PowerBI for data visualization
  5. Experience of both Dremio and Data Mesh Architecture
  6. Prior experience of leading software engineering teams (including line management)
  7. Clear, confident communications with previous experience of working directly with business users

#J-18808-Ljbffr

Related Jobs

View all jobs

Data Engineering Lead

Data Engineering Lead

Data Engineering Lead

Data Engineering Lead - AWS & Snowflake

Engineering Lead / Integration Lead

QuantFund Data Engineering Lead

Get the latest insights and jobs direct. Sign up for our newsletter.

By subscribing you agree to our privacy policy and terms of service.

Industry Insights

Discover insightful articles, industry insights, expert tips, and curated resources.

Careers

Global vs. Local: Comparing the UK Data Science Job Market to International Landscapes

How to evaluate salaries, opportunities, and work culture in data science across the UK, the US, Europe, and Asia Data science has proven to be more than a passing trend; it is now a foundational pillar of modern decision-making in virtually every industry—from healthcare and finance to retail and entertainment. As the volume of data grows exponentially, organisations urgently need professionals who can transform raw information into actionable insights. This high demand has sparked a wave of new opportunities for data scientists worldwide. In this article, we’ll compare the UK data science job market to those in the United States, Europe, and Asia. We’ll explore hiring trends, salary benchmarks, and cultural nuances to help you decide whether to focus your career locally or consider opportunities overseas or in fully remote roles. Whether you’re a fresh graduate looking for your first data science position, an experienced data professional pivoting from analytics, or a software engineer eager to break into machine learning, understanding the global data science landscape can be a game-changer. By the end of this overview, you’ll be better equipped to navigate the expanding world of data science—knowing which skills and certifications matter most, how salaries differ between regions, and what to expect from distinct work cultures. Let’s dive in.

Careers

Data Science Leadership for Managers: Strategies to Motivate, Mentor, and Set Realistic Goals in Data-Driven Teams

Data science has become a linchpin in modern business, transforming oceans of raw data into actionable insights that guide strategy, product development, and personalised customer experiences. With this surge in data-centric operations, the need for effective data science leadership has never been more critical. Guiding a team of data scientists, analysts, and machine learning engineers requires not only technical acumen but also the ability to foster collaboration, champion ethical practices, and align complex modelling efforts with overarching business goals. This article provides practical guidance for managers and aspiring leaders aiming to excel in data-driven environments. By exploring strategies to motivate data science professionals, develop mentoring frameworks, and set achievable milestones, you will be better prepared to steer your team towards meaningful, evidence-based outcomes.

Jobs Careers

10 Essential Books to Read to Nail Your Data Science Career in the UK

Data science continues to be one of the most exciting and rapidly evolving fields in tech. With industries across the UK—ranging from finance and healthcare to e-commerce and government—embracing data-driven decision-making, the demand for skilled data scientists has soared. Whether you're a recent graduate looking for your first role or a professional aiming to advance your career, staying updated through books is crucial. In this article, we explore ten essential books every data science job seeker in the UK should read. Each book provides valuable insights into core concepts, practical applications, and industry-standard tools, helping you build skills employers are actively looking for.

🍪 We use cookies to enhance your browsing experience on our website. By continuing to use this site, you consent to the use of cookies as described in our Cookie Policy. You can review our Cookie Policy for more information.