National AI Awards 2025Discover AI's trailblazers! Join us to celebrate innovation and nominate industry leaders.

Nominate & Attend

Pyspark Data Engineer

iO Associates - UK/EU
Newcastle upon Tyne
2 weeks ago
Applications closed

Related Jobs

View all jobs

Data Engineer

Azure Data Engineer

Data Engineer

Data Engineer

Senior Business Intelligence Developer

Azure Data Engineer/Architect - Customer Facing Role - Outside IR35 Contract

PySpark Data Engineer | up to £450/day Inside | Remote with occasional London travel

We are seeking a PySpark Data Engineer to support the development of a modern, scalable data lake for a new strategic programme. This is a greenfield initiative to replace fragmented legacy reporting solutions, offering the opportunity to shape a long-term, high-impact platform from the ground up.

Key Responsibilities:
* Design, build, and maintain scalable data pipelines using PySpark 3/4 and Python 3.
* Contribute to the creation of a unified data lake following medallion architecture principles.
* Leverage Databricks and Delta Lake (Parquet format) for efficient, reliable data processing.
* Apply BDD testing practices using Python Behave and ensure code quality with Python Coverage.
* Collaborate with cross-functional teams and participate in Agile delivery workflows.
* Manage configurations and workflows using YAML, Git, and Azure DevOps.

Required Skills & Experience:
* Proven expertise in PySpark 3/4 and Python 3 for large-scale data engineering.
* Hands-on experience with Databricks, Delta Lake, and medallion architecture.
* Familiarity with Python Behave for Behaviour Driven Development.
* Strong understanding of YAML, code quality tools (e.g. Python Coverage), and CI/CD pipelines.
* Knowledge of Azure DevOps and Git best practices.
*Active SC clearance is essential- applicants without this cannot be considered.

Contract Details:
* 6-month initial contract with long-term extension potential (multi-year programme).
* Inside IR35.

This is an excellent opportunity to join a high-profile programme at its inception and help build a critical data platform from the ground up. If you are a mission-driven engineer with a passion for scalable data solutions and secure environments, we'd love to hear from you.

National AI Awards 2025

Subscribe to Future Tech Insights for the latest jobs & insights, direct to your inbox.

By subscribing, you agree to our privacy policy and terms of service.

Industry Insights

Discover insightful articles, industry insights, expert tips, and curated resources.

Data Science Jobs Salary Calculator 2025: Find Out What You Should Earn in the UK

Why last year’s pay survey is already out of date for UK data scientists “Am I being paid enough?” Every data professional eventually asks that question—often after a teammate reveals a hefty counter‑offer, a recruiter shares a six‑figure opening, or a headline trumpets the latest multimillion‑pound AI investment. Yet salary guides published even twelve months ago belong in a museum. Generative‑AI hype re‑priced Machine‑Learning Engineer roles, LLM fine‑tuning turned Prompt Engineering into a genuine career path, & fresh regulation forced companies to hire Responsible‑AI Officers on senior‑scientist money. To cut through the noise, DataScience‑Jobs.co.uk distilled a transparent, three‑factor formula. Insert your role, your region, & your seniority, and you’ll get a realistic 2025 salary benchmark—no stale averages, no vague ranges. This article walks you through the formula, examines the forces pushing data‑science pay ever higher, and offers five concrete actions to boost your market value within ninety days.

How to Present Data Science Solutions to Non-Technical Audiences: A Public Speaking Guide for Job Seekers

The ability to communicate clearly is now just as important as knowing how to build a predictive model or fine-tune a neural network. In fact, many UK data science job interviews are now designed to test your ability to explain your work to non-technical audiences—not just your technical competence. Whether you’re applying for your first data science role or moving into a lead or consultancy position, this guide will show you how to structure your presentation, simplify technical content, design effective visuals, and confidently answer stakeholder questions.

Data Science Jobs UK 2025: 50 Companies Hiring Now

Bookmark this guide—refreshed every quarter—so you always know who’s really expanding their data‑science teams. Budgets for predictive analytics, GenAI pilots & real‑time decision engines keep climbing in 2025. The UK’s National AI Strategy, tax relief for R&D & a sharp rise in cloud adoption mean employers need applied scientists, ML engineers, experiment designers, causal‑inference specialists & analytics leaders—right now. Below you’ll find 50 organisations that have advertised UK‑based data‑science vacancies or announced head‑count growth during the past eight weeks. They’re grouped into five quick‑scan categories so you can jump straight to the kind of employer—& culture—that suits you. For every company you’ll see: Main UK hub Example live or recent vacancy Why it’s worth a look (tech stack, mission, culture) Search any employer on DataScience‑Jobs.co.uk to view current ads, or set up a free alert so fresh openings land straight in your inbox.