Senior Data Engineer

Our client is currently recruiting a Senior Data Engineer. Based in Aberdeen, the role is on a permanent (staff) basis and offers a hybrid working arrangement. 

ROLE

Reporting to the Analytics and Integration Lead, the Senior Data Engineer will play a pivotal role in shaping the future of the Company's data analytics capabilities.

RESPONSIBILITIES:

Provision of Pipeline engineering support across all Development Projects within the Company's portfolio.

RESPONSIBILITIES

Supporting Data Platform Program: Collaborate with cross-functional teams to maintain and enhance our modern data platform, leveraging your expertise in Synapse and data engineering techniques.
Stay up to date with innovation: Understand best practice of data engineering and its application, and stay up to date with emerging technologies in the data space
Analyse, Model and Organise Data: Work with a range of stakeholders and business users to understand the use and utility of datasets and systems to then analyse, model and organise data from their respective source data systems into the medallion data lake for further use in reporting.
Ensure data quality and data reliability: Drive improvements in data quality assessments, and ensure that data is processed effectively, efficiently, robustly and timely. Implementing data validation and cleansing processes to improve data management.
Maintaining Data Governance: Ensuring that data governance policies and procedures are followed, and that data lineage and cataloguing is maintained for data discoverability
Bringing New Data Projects to Life: Take the lead in initiating, designing, and executing data projects, ensuring their entire lifecycle is managed effectively.
Deliver Business Insights: Support and deliver business insight, intelligence and predictive analytics reporting which creates real business value
Team Growth: Participate in the growth of our data engineering team by actively contributing to recruitment and mentoring, as this position is focused on expansion rather than backfill.
Performance Monitoring: Optimise and tune pipelines and data processing to increase and improve performance and efficiency.
Performance Management: Looking at wider trends across the data processing infrastructure to identify improvements. Establishing and implementing monitoring and logging solutions across the data platform to improve visibility and management of the data platform
Project Scoping and Management: Defining project scopes and timelines for delivery of a variety of projects, in collaboration with the Digital Technology Partners and the Data and Analytics Lead.
Data Governance: Working with the Data and Analytics Lead to improve Data Governance policies and procedures and its enforcement across the data estate
Code Management: Acting as a trusted advisor and authority in the development of code artifacts, code deployment and code management
Documentation Management: Acting as a trusted authority in the maintenance of comprehensive documentation across the data platform
Leadership and Collaboration: Providing technical guidance and mentorship to the team and leading by example across the activities within the Data Programme.
REQUIREMENTS:

Appropriate professional qualification or equivalent experience
Proven competency working with data warehousing, ETL/ELT, integration tools and business intelligence solutions that will help to deliver the data and analytics strategy.
Skills with Synapse (notebooks and data flows) is essential for Senior Data Engineer.
Working knowledge of PySpark is essential for Senior Data Engineer
Capability working with Azure Data Lake Storage is essential for Senior Data Engineer
Working knowledge of medallion data lakehouse architecture is essential for Senior Data Engineer
Exceptional problem-solving skills to tackle complex data engineering challenges.
Strong background in data analytics, with a focus on data transformation and modelling.
Competency of Master Data Management principles and implementations is essential for Senior Data Engineer
Oil & Gas industry exposure is not required; however, a passion for working with data is a must.
Familiarity with Power BI is desirable but not essential.
Ability to establish, document and communicate data engineering applications and pipelines with both peers and stakeholders