Senior Data Engineer

Location: Croydon | Sheffield | Liverpool | Manchester

Salary: £46,062 (£50,182 London) plus skills allowance of up to £11,338

Close Date: Thursday 19th March

Home Office Digital designs, builds and develops services for the rest of the department and for government. Every year our systems support up to 3 million visa applications, checks on 100 million border crossings, up to 8 million passport applications and deliver 140 million police checks on people, vehicles and property.

The End User Compute and Collaboration (EUC&C) team develops and delivers a range of Microsoft 365 solutions, including Teams, SharePoint, OneDrive, Power Platform, and Office applications. These tools support collaboration and productivity across the organisation.

We are establishing a new data capability to drive the adoption of data driven insights across the organisation’s product and service landscape. Working on end-to-end solutions in both AWS and Azure, using languages including Python, SQL, and PowerShell and orchestration platforms such as Glue and Fabric, you will work with technical leads to implement the design, development, and maintenance of robust, scalable data pipelines. You’ll work closely with senior data colleagues and key stakeholders to deliver EUC&C’s data vision and will have the opportunity to directly influence this to ensure alignment with organisational standards and objectives.

You’ll get to work with some of the largest and most varied datasets around, and benefit from a wealth of continuous professional development resources and career opportunities. You’ll play a key role in delivering joined-up, intelligent services that unlock the value from data and deliver better outcomes for the UK.

Senior Data Engineers design and lead the implementation of complex data flows to connect operational systems, data for analytics and BI systems. As a Senior Data Engineer, you will identify, build, modify and manage data flows between complex systems using the appropriate solution.

You will be working with the Data Science community, who set the scope of data engineering work through user research and data pipelines. Through sharing work with Product Owners, you will identify strategies for how the service is best able to support business needs.

You will work with fellow members of the Data Engineering community to set the direction of the service technology and data architecture. You will also mentor more junior members of the team - promoting challenge, collaborating and encouraging an agile approach to working.

Main Responsibilities

• Analysing problems and experimenting with possible solutions to find the underlying causes of issues or discrepancies to identify problems and to assist in the development of innovative solutions.
• Analysing and reporting test activities and results, supporting data engineers and stakeholders on data analysis issues.
• Designing and implementing a data streaming service, including the development of new data models and ETL processes.
• Ensuring the successful delivery of completed data loads for customers, Data Analysts and Data Scientists, troubleshooting where required.
• Designing, building and testing data products and solutions that are complex or large scale, through full development, test and deployment lifecycles.
• Applying concepts and principles of data modelling and producing relevant and varied data models across multiple subject areas and providing guidance on how to use them.
• The delivery of projects in data analysis, solution design and end user reporting.
• IT programming and automation using tools such as such as Github, GitLab, Azure DevOps, or other source control software.
• Using tools such as industry standard ETL tools, network databases and scheduling and orchestration toolsincluding Azure Automation, Azure Data Factory, Azure DevOps, AWS Step Functions, Kubernetes.
• Using Cloud Data technologies, solutions and future Cloud Data Strategies such as Azure Synapse, Microsoft Fabric, Azure SQL, ADLS2/Onelake, AWS Lambda, AWS Glue, Athena, Redshift and APIs such as REST, GraphQL.
• Effectively managing and communicating with non-technical and senior stakeholders about performance and analysis.

Why work for us...

Find out more information at: Benefits - Home Office Careers, but some of the primary ones are:

• A Civil Service Pension with employer contribution rates of at least 28.97%.
• In-year reward scheme for one-off or sustained exceptional personal or team achievements.
• 25 days annual leave on appointment, rising with service.
• 8 days of public holidays, plus 1 additional privilege day.
• Where business needs allow, some roles may be suitable for a combination of office and home-based working. This is a non-contractual arrangement where all employees will be expected to spend a minimum of 60% of their working time in an office.

Click on apply now to be redirected to our application portal and the full job advert

#J-18808-Ljbffr