Senior Data Engineer

Join Barclays as a Senior Data Engineer, where you'll design and implement scalable, secure, and high-performance data pipelines across Data Products. You’ll also benefit from being part of a vast professional network, collaborating with industry mentors and experts.

To be successful as a Senior Data Engineer, you should have:

• 
  
• Advanced Apache Spark expertise (API design and performance optimization) combined with ample software engineering skills in Java, Scala, and Python, including unit testing best practices.
  
• Experience with data storage and serialization formats (Parquet, Avro) and open table formats/catalogs (Hive Metastore, AWS Glue, Unity Catalog, Iceberg).
  
• Proficient in packaging and build tooling (sbt, Nexus) and version control using GitLab.
  
• Ample DevOps and automation skills, including GitOps, CI/CD pipelines in GitLab, Bash scripting, and Linux environments.
  

• 
  
• Experience with the Databricks platform and AWS infrastructure/architecture design.
  
• Proficient in Infrastructure as Code (Terraform) for cloud provisioning and environment management.
  
• Experience with streaming and queuing technologies, including Apache Kafka, AWS SNS, AWS SQS, and MQ.
  
• Framework development experience with a focus on scalable and maintainable solutions.
  

You may be assessed on the key critical skills relevant for success in the role, such as risk and controls, change and transformation, business acumen, strategic thinking, digital and technology, as well as job-specific technical skills.

This role will be based in either our Glasgow or London office.

To build and maintain the systems that collect, store, process, and analyse data, such as data pipelines, data warehouses and data lakes to ensure that all data is accurate, accessible, and secure.

• 
  
• Build and maintenance of data architecture pipelines that enable the transfer and processing of durable, complete and consistent data.
  
• Design and implementation of data warehouses and data lakes that manage the appropriate data volumes and velocity and adhere to the required security measures.
  
• Development of processing and analysis algorithms fit for the intended data complexity and volumes.
  
• Collaboration with data scientists to build and deploy machine learning models.
  

• 
  
• Contribute or set strategy, drive requirements and make recommendations for change. Plan resources, budgets, and policies; manage and maintain policies and processes; deliver continuous improvements and raise breaches of policies and procedures.
  
• If managing a team, define jobs and responsibilities, plan for the department’s future needs and operations, counsel employees on performance and contribute to employee pay decisions and changes. Lead a number of specialists to influence the operations of a department, in alignment with strategic and tactical priorities, while balancing short- and long-term goals and ensuring that budgets and schedules meet corporate requirements.
  
• If the position has leadership responsibilities, demonstrate a clear set of leadership behaviours to create an environment for colleagues to thrive and deliver consistently excellent standards. The four LEAD behaviours are: Listen and be authentic, Energise and inspire, Align across the enterprise, Develop others.
  
• As an individual contributor, serve as a subject matter expert within own discipline and guide technical direction. Lead collaborative, multi-year assignments and guide team members through structured assignments, identify the need for inclusion of other specialisations to complete assignments. Train, guide and coach less experienced specialists and provide information affecting long-term profits, organisational risks and strategic decisions.
  
• Advise key stakeholders, including functional leadership teams and senior management on functional and cross-functional areas of impact and alignment.
  
• Manage and mitigate risks through assessment, in support of the control and governance agenda.
  
• Demonstrate leadership and accountability for managing risk and strengthening controls in relation to the work your team does.
  
• Demonstrate comprehensive understanding of the organization’s functions to contribute to achieving the goals of the business.
  
• Collaborate with other areas of work, for business-aligned support areas to keep up to speed with business activity and strategies.
  
• Create solutions based on sophisticated analytical thought, comparing and selecting complex alternatives. Perform in-depth analysis with interpretative thinking to define problems and develop innovative solutions.
  
• Adopt and include outcomes of extensive research in problem-solving processes.
  
• Seek out, build and maintain trusting relationships and partnerships with internal and external stakeholders to accomplish key business objectives, using influencing and negotiating skills to achieve outcomes.
  

All colleagues will be expected to demonstrate the Barclays Values of Respect, Integrity, Service, Excellence and Stewardship – our moral compass, helping us do what we believe is right. They will also be expected to demonstrate the Barclays Mindset – to Empower, Challenge and Drive – the operating manual for how we behave.