Description
Role:Data Architect
Location:London (Hybrid Working)
Department:Data Services
Contract:Permanent
Join NatCen’s digital transformation and shape the future of reporting architecture.
We are seeking an experiencedData Platform Architectto design and lead the implementation of a scalable, secure, and modern data platform usingDatabricks. This platform will underpin both ourBusiness Intelligence (BI)reporting andresearch dataenvironments, which will operate distinctly but be supported by a shared infrastructure.
You will be the technical lead responsible for establishing the architectural foundations of our data capabilities — ensuring that the platform is robust, flexible, and future-proof to support both real-time and longitudinal data needs across the organisation. You will have full architectural autonomy to shape our platform.
Key Responsibilities
- Architect and deliver a data platform using Databricks and Azure supporting:
- BI and operational reporting
- Research data workflows (e.g. surveys, longitudinal studies, modelling)
- Data science and AI developments
- Design formulti-tenancy and workload isolationbetween BI and research environments.BI and operational reporting
- Establish data architecture patterns for bronze/silver/gold layers, metadata management, and schema evolution.
- Build and maintain CI/CD workflows, job orchestration (e.g. Databricks Workflows or Airflow), and cluster management policies.
- Leadperformance tuning, cost optimisation, and observabilityacross the platform.
- Develop and maintain platform documentation and reusable architecture assets.
- Collaborate with business stakeholders and technical SMEs toalign solutions with strategic goals.
- Driveinnovationby staying current with emerging trends in cloud, data, and AI technologies.
- Provide technical guidance and upskilling for analysts, researchers, and data professionals across the organisation — ensuring they understand how to access, use, and trust the data platform.
- Develop and deliver training materials, workshops, and onboarding sessions for platform users with varying levels of technical experience.
- Act as a champion for platform adoption, promoting best practices, self-service, and data literacy across teams.
Skills, Knowledge and Expertise
Essential:
3+ years’ experiencearchitecting and deliveringdata platforms on Databricks(3+ years minimum).
Strong experience with cloud-based data solutions, particularly in Azure and Databricks ecosystems.
- Proven ability to design and deliverenterprise-scale data warehousesin regulated or complex environments.
- Expertise in ETL/ELT, and reporting system architectures.
- Strong technical skills in SQL, Python, PySpark, Apache Spark.
- Hands-on background as a data engineer or platform engineer — you can design and build.
- Excellent communication and relationship-building skills across technical and non-technical audiences.
- Demonstrated success in leading cross-functional delivery teams and influencing architectural decisions.
Nice to Have:
- Experience working in or with non-profits, research institutions, or public sector organisations.
- Familiarity with tools like Power BI, dbt, Airflow, or MLflow
- Understanding of data cataloguing, stewardship, and quality frameworks
- Exposure to survey data, longitudinal studies, or research ethics/data protection.
Benefits
As well as a competitive salary, an excellent working environment (including a home/office hybrid working environment), you will be working for the largest independent social research organisation in the UK. We are proud of the benefits we offer our employees which include:
- 25 days holiday (plus 8 bank holidays) and flexible working
- Generous company pension scheme
- Life insurance and health cash plan
- Cycle to Work scheme and season ticket loan
- Career development, professional subscriptions support and mentoring support from industry-leading experts.
At The National Centre for Social Research we believe that social research has the power to make life better. By really understanding the complexity of people’s lives and what they think about the issues that affect them, we give the public a powerful and influential role in shaping services that can make a difference to everyone. And as an independent, not-for-profit organisation we’re able to focus our time and energy on meeting our clients’ needs and delivering social research that works for society.
