Senior Data Engineer

Role Overview:

We are seeking a highly skilled and experienced Senior Azure Data Engineer to join our team. In this role, you will lead the design, development, and maintenance of scalable data solutions in Microsoft Azure. You will work closely with cross-functional teams to transform complex data into actionable insights, driving business decisions and innovation.

This is a remote working position with only occasional visits to the office, which is based in Burton-on-Trent, Staffordshire.

Key Responsibilities:

Data Architecture and Design:

• Design, develop, and implement end-to-end data pipelines in Azure.
• Architect data solutions using Azure Data Lake, Azure Synapse Analytics, and Azure Databricks.
• Ensure solutions are scalable, efficient, and aligned with business objectives.

ETL/ELT Processes:

• Build and optimize ETL/ELT pipelines using Azure Data Factory (ADF).
• Develop data ingestion processes for structured and unstructured data sources.

Data Modelling and Warehousing:

• Design and implement enterprise data models and data warehousing solutions.
• Optimize data models for performance and scalability in Azure environments.

Data Governance and Security:

• Implement data governance best practices, ensuring data quality and consistency.
• Enforce security measures, such as role-based access and encryption, in compliance with organizational and regulatory standards.

Collaboration and Leadership:

• Lead and mentor data engineers.
• Collaborate with stakeholders, data specialist, and BI teams to gather requirements and deliver data solutions.

Monitoring and Optimization:

• Monitor and troubleshoot Azure data services to ensure high availability and performance.
• Optimize costs and performance for Azure services.

Skills, Experience & Competencies

Required Skills:

• Extensive experience in data engineering with Microsoft Azure.
• Proficiency in Azure services such as Azure Data Factory, Azure Data Lake, Azure Databricks, Azure Synapse Analytics, and Azure SQL Database.
• Strong experience with ETL/ELT pipelines, data modelling, and data integration.
• Proficiency in SQL and programming languages like Python, Scala, or PowerShell.
• Knowledge of data governance, security, and compliance frameworks.

Preferred Skills:

• Experience with machine learning or advanced analytics on Azure platforms.
• Familiarity with DevOps practices, CI/CD pipelines, and Infrastructure as Code (IaC) using tools like Terraform or Azure DevOps.
• Knowledge of big data frameworks like Spark and Hadoop.

Education and Experience:

• 5+ years of experience in data engineering, with 3+ years working specifically on Azure.

Certifications such as Microsoft Certified: Azure Data Engineer Associate or Azure Solutions Architect are highly desirable.