National AI Awards 2025Discover AI's trailblazers! Join us to celebrate innovation and nominate industry leaders.

Nominate & Attend

Senior Data Engineer

Insight Global
Sheffield
2 weeks ago
Applications closed

Related Jobs

View all jobs

Senior Data Engineer

Senior Data Engineer

Senior Data Engineer

Senior Data Engineer

Senior Data Engineer

Senior Data Engineer

Insight Global is seeking a Azure Data Engineers to join a prestigious energy client based in London. The successful candidates will be responsible for designing, implementing, and managing live data streaming pipelines for the client’s Energy Trading team. This role involves designing data pipelines for Crude and Products trading data from on-premise DB2 databases to an Azure Cloud environment.


This is an Inside IR35 contract role and the individual must sit out of the UK.


Key Responsibilities:

  • Pipeline Management: Design, implement, and manage live data streaming pipelines using Azure Databricks to ensure seamless data flow and real-time processing.
  • Process Evaluation: Assess and optimize on-premise to cloud data exchange processes for accuracy, efficiency, and scalability.
  • Code Review and Debugging: Conduct thorough code reviews and debugging sessions, providing guidance and mentorship to junior data engineers to ensure high-quality code and best practices.
  • Problem Solving: Develop innovative solutions to address computing and cost challenges, leveraging advanced technologies and methodologies.


Must Haves:

  • Data Engineering experience
  • Deep expertise with Azure Databricks (Delta Live Tables, Data Streaming , Unity Catalogue etc.)
  • Prove experience designing high volume, live data streaming solutions using Azure DLT (Delta Live Tables)
  • Expert with Apache Spark and PySpark (ability to review quality of code and debug issues)
  • Background in Data warehousing (SAP Hana, BI/BW, Oracle etc.)
  • Proficient with SQL
  • Proven ability to problem solve and think outside the box


Plusses:

  • Background in Trading domain, specifically energy commodities trading
National AI Awards 2025

Subscribe to Future Tech Insights for the latest jobs & insights, direct to your inbox.

By subscribing, you agree to our privacy policy and terms of service.

Industry Insights

Discover insightful articles, industry insights, expert tips, and curated resources.

LinkedIn Profile Checklist for Data Science Jobs: 10 Tweaks to Elevate Recruiter Engagement

Data science recruiters often sift through dozens of profiles to find candidates skilled in Python, machine learning, statistical modelling and data visualisation—sometimes before roles even open. A generic LinkedIn profile won’t suffice in this data-driven era. This step-by-step LinkedIn for data science jobs checklist outlines ten targeted tweaks to elevate recruiter engagement. Whether you’re an aspiring junior data scientist, a specialist in MLOps, or a seasoned analytics leader, these optimisations will sharpen your profile’s search relevance and demonstrate your analytical impact.

Part-Time Study Routes That Lead to Data Science Jobs: Evening Courses, Bootcamps & Online Masters

Data science sits at the intersection of statistics, programming and domain expertise—unearthing insights that drive business decisions, product innovation and research breakthroughs. In the UK, organisations from fintech and healthcare to retail and public sector are investing heavily in data-driven strategies, fuelling unprecedented demand for data scientists, machine learning engineers and analytics consultants. According to recent projections, data science roles will grow by over 40% in the next five years, offering lucrative salaries and varied career paths. Yet many professionals hesitate to leave their current jobs or pause personal commitments for full-time study. The good news? A vibrant ecosystem of part-time learning routes—Evening Courses, Intensive Bootcamps and Flexible Online Master’s Programmes—empowers you to learn data science while working. This comprehensive guide explores every pathway: foundational CPD units and short courses, hands-on bootcamps, accredited online MScs, plus funding options, planning strategies and a real-world case study. Whether you’re an analyst looking to formalise your skills, a software developer pivoting into data or a manager seeking to harness data-driven decision-making, you’ll find the right route to fit your schedule, budget and career goals.

The Ultimate Assessment-Centre Survival Guide for Data Science Jobs in the UK

Assessment centres for data science positions in the UK are designed to replicate the multifaceted challenges of real-world analytics teams. Employers combine psychometric assessments, coding tests, statistical reasoning exercises, group case studies and behavioural interviews to see how you interpret data, build models, communicate insights and collaborate under pressure. Whether you’re specialising in predictive modelling, NLP or computer vision, this guide provides a step-by-step roadmap to excel at every stage and secure your next data science role.