Job Title: Technical Lead - Data Engineering
Location: London/Hybrid- Onsite 2-3 days per week, with flexibility to work remotely once up and running
Salary/Rate: Up to £580 per day INSIDE IR35
Start Date: As soon as possible
Job Type: 12 month contract initially + extensions

Company Introduction

Coltech is partnered with a prestigious consultancy client who are looking for an experienced Technical Lead in Data Engineering to lead the development of the data Lakehouse platform

Job Responsibilities

• Lead the design, development, and implementation of data engineering solutions for our data Lakehouse platform.
  
  
• Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions.
  
  
• Provide technical leadership, mentoring, and coaching to the data engineering team.
  
  
• Ensure best practices in data engineering, including data modelling, ETL processes, and data quality.
  
  
• Optimize big data workloads in Spark and other big data technologies.
  
  
• Manage CI/CD pipelines using Azure DevOps or Git.
  
  
• Develop and maintain event-driven pipelines using .NET and other relevant technologies.
  
  
• Implement and manage Databricks Unity Catalog and other data governance tools.
  
  
• Deliver projects in the streaming data world using Kafka, KSQL DB, and similar technologies.
  
  
• Utilize reporting tools such as Power BI and Qlik for data visualization and reporting.
  
  Required Skills/Experience:
  
  
• Extensive experience with Databricks, Apache Spark, and data lakes.
  
  
• Proficiency in Python and SQL.
  
  
• Experience with CI/CD processes using Azure DevOps or Git.
  
  
• Knowledge of streaming data technologies such as Kafka and KSQL DB.
  
  
• Familiarity with reporting tools like Power BI and Qlik.
  
  
• Software engineering experience with .NET for event-driven pipelines and automation testing.
  
  
• Experience with Databricks Unity Catalog.
  
  
• Strong skills in data modelling, including Snowflake modelling, star schema modelling, and other techniques.
  
  
• Ability to optimize big data workloads in Spark.
  
  Desired Skills/Experience:
  
  
• Experience in delivering projects in the streaming data world.
  
  
• Understanding of data governance and data quality best practices.
  
  
• Familiarity with automation testing frameworks.
  
  
• Experience working in Energy trading related projects.
  
  Apply now for immediate consideration