Global Data Engineer

Responsibilities

of the role:
Data Pipeline Development:Build and maintain batch and streaming pipelines using Azure Data Factory and Azure Databricks.Data Categorisation & Enrichment:Structure unprocessed datasets through tagging, standardisation, and feature engineering.Automation & Scripting:Use Python to automate ingestion, transformation, and validation processes.ML Readiness:Work closely with data scientists to shape training datasets, applying sound feature selection techniques.Data Validation & Quality Assurance:Ensure accuracy and consistency across data pipelines with structured QA checks.Collaboration:Partner with analysts, product teams, and engineering stakeholders to deliver usable and trusted data products.Documentation & Stewardship:Document processes clearly and contribute to internal knowledge sharing and dataernance.Platform Scaling:Monitor and tune infrastructure for cost-efficiency, performance, and reliability as data volumes grow.On-Call support: Participate in an on-call rota system to provide support for the production environment, ensuring timely resolution of incidents and maintaining system stability outside of standard working hours.
Requirements

What you will need:

The ideal candidate will be proactive and willing to develop and implement innovative solutions, capable of the following:

Rmended:
2+ years of professional experience in a data engineering or similar role. Proficiency inPython, including use of libraries for data processing (, pandas, pySpark). Experience working withAzure-based data services, particularlyAzure Databricks, Data Factory, and Blob Storage. Demonstrable knowledge of data pipeline orchestration and optimisation. Understanding of SQL for data extraction and transformation. Familiarity with source control, deployment workflows, and working in Agile teams. Strongmunication and documentation skills, including translating technical work to non-technical stakeholders.
Preferred:
Exposure to machine learning workflows or model preparation tasks. Experience working in a financial, payments, or regulated data environment. Understanding of monitoring tools and logging best practices (, Azure Monitor, Log Analytics). Awareness of cost optimisation and scalable design patterns in the cloud. Job ID 788405236A