Senior Data Engineer - Darwin

DLG is evolving. Across every facet of our business, our teams are embracing new opportunities and putting customers at the heart of everything they do. By joining them, you’ll have the opportunity to not just be recognised for your skills but encouraged to build upon them and empowered to do your absolute best.

At Darwin Insurance, a brand within DLG, you’ll help build the next generation of our low‑latency, cloud‑native data platforms — the systems that power real‑time pricing, machine learning workflows, and operational intelligence. You’ll own the design of resilient ETL/ELT pipelines, distributed data processing, and scalable AWS infrastructure, collaborating closely with a tight‑knit team of engineers, architects, and product leaders who care deeply about doing things the right way.

We engineer like a startup: Clean Code, TDD, CI/CD, Infrastructure‑as‑Code, full ownership, and a bias for shipping well‑designed, well‑tested systems. You’ll have a genuine voice in architectural direction, mentor strong engineers, and help raise the technical bar across the team.

This role is ideal for someone who wants meaningful impact — someone who’s excited by the blend of startup autonomy and big‑company reach, and who wants to shape the future of Darwin’s data platform as we scale our ambitions.

• 
  
• Lead collaborations with cross‑functional teams to understand business requirements and communicate complex data concepts clearly to non‑technical stakeholders, facilitating informed decision‑making across the organization.
  
• Design robust data transformation solutions using ETL and ELT methodologies and maintain comprehensive documentation for all data pipelines and integration procedures to ensure clarity and effective management.
  
• Proactively monitor data systems for performance optimisation and implement data quality checks to maintain high standards of accuracy and reliability in data processing.
  
• Develop and enforce security protocols to protect data assets and ensure all data handling complies with legal and regulatory standards, supporting robust data governance.
  
• Stay abreast of advancements in data engineering technologies and practices, incorporating innovative tools and methods to enhance the organization’s data capabilities.
  
• Mentor junior data engineers, fostering an environment of growth and continuous learning, while leading by example in technical expertise and basic project management.
  
• Work closely with data architects and other engineers to refine data storage and architecture, ensuring the infrastructure supports scalable solutions and future technological developments.
  

• 
  
• Significant years of experience, gained in a professional services setting, in designing and implementing complex data transformations using both ETL and ELT methodologies to support scalable data architecture in high‑demand environments.
  
• Excellent understanding of distributed data processing, ideally working with Apache Spark.
  
• Extensive experience (5+ years) with advanced SQL techniques and database management, including proficiency with database systems like PostgreSQL, MySQL, and Amazon Redshift, optimising database performance and integrity in large‑scale environments.
  
• In‑depth knowledge of AWS cloud services including Glue, S3, KMS, Lambda, and Kinesis, demonstrating capability in deploying and managing scalable, secure cloud infrastructure.
  
• Advanced proficiency in Python programming, with demonstrated experience in developing, automating, and enhancing data processing tasks and analytical applications.
  
• Strong command over Unix/Linux operating environments, with the ability to manage and optimise system performance for data applications.
  
• Solid understanding and practical experience in Agile development processes, skilled in translating strategic goals into executable stories, with a focus on incremental and iterative delivery.
  
• Strong advocate for Software Craftsmanship and Clean Code, with a commitment to maintaining high standards in code quality and system reliability.
  
• Demonstrated experience and passion for Test‑Driven Development (TDD), ensuring robust and reliable software solutions through comprehensive testing practices.
  
• Excellent communication skills, capable of effectively articulating technical concepts to both technical and non‑technical stakeholders, facilitating clear understanding and cooperative problem‑solving.
  
• Clear understanding of DevOps practices and Infrastructure as Code, with hands‑on experience in automating and optimising infrastructure deployment and management.
  
• Over 5 years of experience managing clustered systems, ensuring high availability and performance scalability in distributed computing environments.
  
• Expert knowledge of various database technologies, including SQL and NoSQL, and their appropriate use cases and trade‑offs, guiding optimal database selection and use.
  

• 
  
• 9% employer‑contributed pension
  
• 50% off home, motor and pet insurance plus Green Flag breakdown cover
  
• Additional optional Health and Dental insurance
  
• Up to 10% AIP Bonus
  
• EV car scheme allowing all colleagues to lease a brand‑new electric or plug‑in hybrid car in a tax‑efficient way.
  
• Generous holidays
  
• Buy‑as‑you‑earn share scheme
  
• Employee discounts and cashback
  
• Plus, many more.
  

We want everyone to get the most out of their time at DLG. Which is why we’ve looked beyond the financial rewards and created an offer that takes your whole life into account. Supporting our people to work at their best – whatever that looks like – and offering real choice, flexibility, and a greater work‑life balance that means our people have time to focus on the things that matter most to them. Our benefits are about more than just the money you earn. They’re about recognising who you are and the life you live.

Direct Line Group is an equal opportunity employer, and we think diversity of background and thinking is a big strength in our people. We're delighted to feature as one of the UK's Top 50 Inclusive Employers and are committed to making our business an inclusive place to work, where everyone can be themselves and succeed in their careers. We know you're more than a CV, and the things that make you, you, are what bring potential to our business. We recognise and embrace people that work in different ways so if you need any adjustments to our recruitment process, please speak to the recruitment team who will be happy to support you.

Location: London, England, United Kingdom.