Lead Data Engineer

The Team: The Data Engineering team is positioned in the Delivery business unit of the IT function within the Data & Automation department sitting alongside the Report Engineering, Data Governance, Data Migrations and Automation teams.

Data Engineering is a perfect blend of Data Engineers and QA Engineers that are renowned for their forward thinking and trail blazing. Exceptionally talented, supportive and personable, it’s an area that’s challenging, provides excellent growth opportunities and rewarding.

Role Expectations:

• Leading by example, through hands-on development and review of a quality Python and SQL codebase.

• Contributing towards a productive team culture through the adoption of automated CI/CD and testing industry practices.

• Inputting into planning and estimating work / delivery roadmap.

• Keeping up to date with Databricks platform developments and maintaining tech currency of internal systems.

• Ensuring accurate and adequate documentation, and system resilience is maintained.

• Working with cross-functional teams to contribute towards TalkTalk data strategy.

• Identifying and implementing cost optimization ideas.

• Coaching and mentoring a distributed team of data engineers across multiple delivery projects.

• Establishing data governance frameworks for secure handling of delivery information.

• Identifying opportunities for improvement across all data engineering projects.

• Managing relationships with stakeholders across operations, technology, and business teams to understand requirements and ensure.

Must Have:

• 8+ years’ experience in data engineering and 1+ experience as Lead Data Engineer

• Proven expertise with Microsoft Azure

• Strong proficiency in Databricks with hands-on Scala (Spark) and PySpark.

• Expertise in Databricks for large-scale data engineering and analytics workloads. Design and implement scalable data architectures using Databricks Unity Catalog, Delta Lake, and Apache Spark

• Implement advanced Delta Lake features including time travel, vacuum operations, and Z-ordering.

• Develop custom solutions using Databricks APIs and integrate with external systems

• Solid understanding of SQL and relational database concepts

• Experience with CI/CD, Git and modern DevOps practices for data solutions

• Strong problem-solving, communication and client-facing collaboration skills

• Exposure to machine learning or data science workflows is a plus but not required

• Hands-on with Azure DevOps pipelines (YAML, agents, service connections).

• Strong problem-solving and troubleshooting skills.

• Excellent communication skills with the ability to collaborate with cross-functional teams to understand requirements, data solutions, data models and mapping documents.

• Experience with data migration projects and legacy system modernization.

• Desirable: certification in Databricks Data Engineer and/or Azure Data Engineer are a plus.

Be Yourself. Make an Impact. Join Us.

As a recognised Top 50 Inclusive Employer in the UK, we believe that diversity fuels innovation and success. We’re committed to building a workplace that reflects the communities and customers we serve. At TalkTalk, inclusion is part of our DNA – we’re all 100% human, and we’ve created a culture where you can truly be yourself.

We’re not your traditional 9-5. We’re a dynamic, flexible workplace, and we’re excited to hear how you like to work. Whether you thrive in collaboration, focus better at home, or prefer a bit of both – let’s make it work.

What We Offer

• Flexible hybrid working – with a minimum of 50% office presence to support teamwork and connection

• Collaborative office spaces designed for creative thinking and innovation

• Free on-site parking at our offices

• Generous holiday package – 25 days annual leave, 3 wellbeing days, and your birthday off (plus the option to buy up to 10 more days!)

• Private healthcare for all employees

• Competitive pension scheme and performance-related bonus opportunities

• Free broadband for all employees

• Life event gifts – celebrating milestones like marriages and births

• Inclusive employee networks – open to all, supporting peer connection and thought-provoking conversations

• Salary sacrifice scheme – save on dental, gym, and more

• Big retail and leisure discounts

• 3 paid volunteering days a year – because making a difference matters to us too