Senior Data Engineer

Optima Partners
Edinburgh
3 days ago
Create job alert
Senior Data EngineerWho we are

We are an advanced data and business consultancy headquartered in Edinburgh, UK. We are a practitioner-led organisation that collaborates with top consumer brands to drive transformation and foster customer-centricity through our expertise in customer strategy, innovative design, and advanced data science and engineering.

We work with leading consumer brands to tackle and overcome complex business and customer problems to drive transformation and champion customer-centric agendas. We are proud to include some of the leading UK and global brands among our current clients such as Lloyds Banking Group, NatWest Group, Bank of Ireland, Nationwide, Aviva, Biogen, Eon Next, OVO, Virgin Media O2, BT, HMD Global, Centrica, and GSK. We are obsessive about delivering value for our clients and work in a collaborative, engaged, and creative way with our colleagues and clients. We strive to support the transition of knowledge and capability into strategic teams.

The opportunity

We have an exciting new opportunity for a Senior Data Engineer at Optima Partners. In this role, you will be instrumental in designing, building, and optimising scalable data pipelines, architectures, and workflows for our diverse client base. You’ll work within dynamic, multi-disciplinary teams to implement advanced data solutions while championing best practices and innovative trends in data engineering. This is a valuable opportunity for an experienced data professional to lead projects and drive technical excellence across a variety of commercial sectors.

At Optima, you’ll collaborate with experts across fields, gaining exposure to varied business challenges in an inclusive environment that fosters continuous learning, knowledge sharing, and clear career development.

What you will be doing
  • Data Pipeline Development: Design and implement robust, scalable, and efficient data pipelines to collect, transform, and integrate data from various sources.
  • Data Architecture: Develop and optimise data architectures, including data warehouses, data lakes, and other storage solutions.
  • ETL/ELT Processes: Create and maintain reliable ETL/ELT workflows to ensure data quality and accessibility.
  • Collaborate: Partner with data scientists, analysts, and client stakeholders to understand requirements and deliver impactful solutions.
  • Performance Optimisation: Monitor and enhance the performance of data systems to ensure minimal downtime and rapid query responses.
  • Automation and Tools: Identify automation opportunities and recommend tools to improve data engineering workflows.
  • Documentation: Maintain detailed technical documentation for all solutions and processes.
  • Leadership: Mentor junior engineers and lead technical initiatives within cross-functional teams.
What skills we would like you to have
  • Programming: Proficiency in Python, Java, Scala, or similar languages for data processing.
  • Big Data Technologies: Hands-on experience with tools such as Databricks, Apache Spark, and Hadoop.
  • Cloud Platforms: Familiarity with AWS, Azure, GCP, or other cloud ecosystems for data engineering tasks.
  • Database Management: Expertise in relational databases (e.g. Postgres, SQL Server).
  • Data Integration Tools: Knowledge of platforms like Airflow, Apache NiFi, or Talend.
  • Data Storage and Modelling: Experience with data warehousing tools (e.g. Snowflake, Redshift, BigQuery) and schema design.
  • Version Control and CI/CD: Familiarity with Git, Docker, and CI/CD pipelines for deployment.
  • Experience: 5+ years in data engineering or a related role, with a proven track record of delivering advanced data solutions in consulting or client-facing environments.
  • Methodologies: Experience with Agile or Scrum practices is beneficial.
Personal Qualities
  • Strong problem-solving and analytical skills.
  • Excellent communication abilities to explain complex technical concepts to non-technical stakeholders.
  • A collaborative, adaptable, and team-oriented mindset.


#J-18808-Ljbffr

Related Jobs

View all jobs

Senior Data Engineer

Senior Data Engineer

Senior Data Engineer

Senior Data Engineer

Senior Data Engineer

Senior Data Engineer

Subscribe to Future Tech Insights for the latest jobs & insights, direct to your inbox.

By subscribing, you agree to our privacy policy and terms of service.

Industry Insights

Discover insightful articles, industry insights, expert tips, and curated resources.

How to Write a Data Science Job Ad That Attracts the Right People

Data science plays a critical role in how organisations across the UK make decisions, build products and gain competitive advantage. From forecasting and personalisation to risk modelling and experimentation, data scientists help translate data into insight and action. Yet many employers struggle to attract the right data science candidates. Job adverts often generate high volumes of applications, but few applicants have the mix of analytical skill, business understanding and communication ability the role actually requires. At the same time, experienced data scientists skip over adverts that feel vague, inflated or misaligned with real data science work. In most cases, the issue is not a lack of talent — it is the quality and clarity of the job advert. Data scientists are analytical, sceptical of hype and highly selective. A poorly written job ad signals unclear expectations and immature data practices. A well-written one signals credibility, focus and serious intent. This guide explains how to write a data science job ad that attracts the right people, improves applicant quality and positions your organisation as a strong data employer.

Maths for Data Science Jobs: The Only Topics You Actually Need (& How to Learn Them)

If you are applying for data science jobs in the UK, the maths can feel like a moving target. Job descriptions say “strong statistical knowledge” or “solid ML fundamentals” but they rarely tell you which topics you will actually use day to day. Here’s the truth: most UK data science roles do not require advanced pure maths. What they do require is confidence with a tight set of practical topics that come up repeatedly in modelling, experimentation, forecasting, evaluation, stakeholder comms & decision-making. This guide focuses on the only maths most data scientists keep using: Statistics for decision making (confidence intervals, hypothesis tests, power, uncertainty) Probability for real-world data (base rates, noise, sampling, Bayesian intuition) Linear algebra essentials (vectors, matrices, projections, PCA intuition) Calculus & gradients (enough to understand optimisation & backprop) Optimisation & model evaluation (loss functions, cross-validation, metrics, thresholds) You’ll also get a 6-week plan, portfolio projects & a resources section you can follow without getting pulled into unnecessary theory.

Neurodiversity in Data Science Careers: Turning Different Thinking into a Superpower

Data science is all about turning messy, real-world information into decisions, products & insights. It sits at the crossroads of maths, coding, business & communication – which means it needs people who see patterns, ask unusual questions & challenge assumptions. That makes data science a natural fit for many neurodivergent people, including those with ADHD, autism & dyslexia. If you’re neurodivergent & thinking about a data science career, you might have heard comments like “you’re too distracted for complex analysis”, “too literal for stakeholder work” or “too disorganised for large projects”. In reality, the same traits that can make traditional environments difficult often line up beautifully with data science work. This guide is written for data science job seekers in the UK. We’ll explore: What neurodiversity means in a data science context How ADHD, autism & dyslexia strengths map to common data science roles Practical workplace adjustments you can request under UK law How to talk about your neurodivergence in applications & interviews By the end, you’ll have a clearer sense of where you might thrive in data science – & how to turn “different thinking” into a real career advantage.