Vacancy for Web Archiving Data Analyst and Crawl Engineer at The National Archives (UK)

Digital Preservation Coalition
City of London
1 month ago
Create job alert
Overview

Vacancy for Web Archiving Data Analyst and Crawl Engineer at The National Archives (UK)

21 October 2025

London, England

Full-Time

About the role

The National Archives (UK) is looking for an enthusiastic and skilled Data Analyst with experience in web crawling, scraping, or analysis, to support workflows, help understand more about collections, and grow web archiving capability.

Web archives are fascinating. At The National Archives, we deliver three public web archive services. They are vast collections of government websites and social media. The scale of these collections, the variety of users’ needs, and the complexity of the data make them challenging and fertile ground for innovation. This role is fundamental to our mission to improve our collection processes, ensuring the highest quality and fidelity, understanding our collection, and conveying these insights to a range of people.

Responsibilities

As The National Archives’ Web Archiving Data Analyst and Crawl Engineer, you will bring your expertise and in-depth knowledge to develop and shape key aspects of our web archiving services and therefore you will be a key member of the team as we evolve our services. We work with suppliers who deliver us many technical services, but we are increasing our in-house capability and expertise. These workflows are now important parts of our service and you will own them and develop them, including through finding ways to improve efficiency and resilience. You will embrace challenge and look for opportunities to do things differently.

Working closely with the Senior Data Engineer and our Web Archivists, you will help deepen our understanding of our web archiving and social media collections and use these insights to help tell the story of government online. This includes engaging with experts within The National Archives as well as with external organisations across the digital preservation community and other government departments, by sharing your knowledge with others and raising the profile of our work.

You will be passionate about data and technology. You will thrive in an environment which values and supports continuous learning and self-development.

Web archiving is an exciting, specialist, varied and rapidly evolving field that is a lot of fun to be involved in. Building and maintaining excellent web archiving services calls on a range of skills: problem solving, creativity, developing new techniques for capturing and replaying content, as well as supporting research, and managing stakeholders and projects.

You will support others’ research by delivering development that will help users explore our services “as data”. You will also contribute to the team’s tools and processes, ensuring that we can go about our work as efficiently and effectively as possible.


#J-18808-Ljbffr

Related Jobs

View all jobs

Data Engineer | Wolves FC

Vacancy for Data Engineer at The UK National Archives

Vacancy for Data Analyst (Developer) at The National Archives UK

Associate Data Scientist (HEO)

Data Analyst

Data Analyst

Subscribe to Future Tech Insights for the latest jobs & insights, direct to your inbox.

By subscribing, you agree to our privacy policy and terms of service.

Industry Insights

Discover insightful articles, industry insights, expert tips, and curated resources.

How Many Data Science Tools Do You Need to Know to Get a Data Science Job?

If you’re trying to break into data science — or progress your career — it can feel like you are drowning in names: Python, R, TensorFlow, PyTorch, SQL, Spark, AWS, Scikit-learn, Jupyter, Tableau, Power BI…the list just keeps going. With every job advert listing a different combination of tools, many applicants fall into a trap: they try to learn everything. The result? Long tool lists that sound impressive — but little depth to back them up. Here’s the straight-talk version most hiring managers won’t explicitly tell you: 👉 You don’t need to know every data science tool to get hired. 👉 You need to know the right ones — deeply — and know how to use them to solve real problems. Tools matter, but only in service of outcomes. So how many data science tools do you actually need to know to get a job? For most job seekers, the answer is not “27” — it’s more like 8–12, thoughtfully chosen and well understood. This guide explains what employers really value, which tools are core, which are role-specific, and how to focus your toolbox so your CV and interviews shine.

What Hiring Managers Look for First in Data Science Job Applications (UK Guide)

If you’re applying for data science roles in the UK, it’s crucial to understand what hiring managers focus on before they dive into your full CV. In competitive markets, recruiters and hiring managers often make their first decisions in the first 10–20 seconds of scanning an application — and in data science, there are specific signals they look for first. Data science isn’t just about coding or statistics — it’s about producing insights, shipping models, collaborating with teams, and solving real business problems. This guide helps you understand exactly what hiring managers look for first in data science applications — and how to structure your CV, portfolio and cover letter so you leap to the top of the shortlist.

The Skills Gap in Data Science Jobs: What Universities Aren’t Teaching

Data science has become one of the most visible and sought-after careers in the UK technology market. From financial services and retail to healthcare, media, government and sport, organisations increasingly rely on data scientists to extract insight, guide decisions and build predictive models. Universities have responded quickly. Degrees in data science, analytics and artificial intelligence have expanded rapidly, and many computer science courses now include data-focused pathways. And yet, despite the volume of graduates entering the market, employers across the UK consistently report the same problem: Many data science candidates are not job-ready. Vacancies remain open. Hiring processes drag on. Candidates with impressive academic backgrounds fail interviews or struggle once hired. The issue is not intelligence or effort. It is a persistent skills gap between university education and real-world data science roles. This article explores that gap in depth: what universities teach well, what they often miss, why the gap exists, what employers actually want, and how jobseekers can bridge the divide to build successful careers in data science.