Senior Data Engineer

Senior Data Engineer

Location: London, UK (3 days in the office)

SC Cleared: Required

Essential Skills & Experience:

• 10+ years of experience in data engineering, with at least 3+ years of hands-on experience with Azure Databricks.
  
  
• Strong proficiency in Python and Spark (PySpark) or Scala.
  
  
• Deep understanding of data warehousing principles, data modelling techniques, and data integration patterns.
  
  
• Extensive experience with Azure data services, including Azure Data Factory, Azure Blob Storage, and Azure SQL Database.
  
  
• Experience working with large datasets and complex data pipelines.
  
  
• Experience with data architecture design and data pipeline optimization.
  
  
• Proven expertise with Databricks, including hands-on implementation experience and certifications.
  
  
• Experience with SQL and NoSQL databases.
  
  
• Experience with data quality and data governance processes.
  
  
• Experience with version control systems (e.g., Git).
  
  
• Experience with Agile development methodologies.
  
  
• Excellent communication, interpersonal, and problem-solving skills.
  
  
• Experience with streaming data technologies (e.g., Kafka, Azure Event Hubs).
  
  
• Experience with data visualisation tools (e.g., Tableau, Power BI).
  
  
• Experience with DevOps tools and practices (e.g., Azure DevOps, Jenkins, Docker, Kubernetes).
  
  
• Experience working in a financial services or economic data environment.
  
  
• Azure certifications related to data engineering (e.g., Azure Data Engineer Associate)