Senior Data Engineer

Simple Machines
City of London
3 months ago
Applications closed

Related Jobs

View all jobs

Senior Data Engineer

Senior Data Engineer

Senior Data Engineer

Senior Data Engineer

Senior Data Engineer

Senior Data Engineer

Senior Data Engineer


Simple Machines is a leading independent boutique technology firm with a global presence, including teams in London, Sydney, San Francisco, and New Zealand. We specialise in creating technology solutions at the intersection of data, AI, machine learning, data engineering, and software engineering. Our mission is to help enterprises, technology companies, and governments better connect with and understand their organisations, their people, their customers, and citizens. We are a team of creative engineers and technologists dedicated to unleashing the potential of data in new and impactful ways. We design and build bespoke data platforms and unique software products, create and deploy intelligent systems, and bring engineering expertise to life by transforming data into actionable insights and tangible outcomes.


We engineer data to life™.


Role

The Senior Data Engineer at Simple Machines is a dynamic, hands-on role focused on building real-time data pipelines and implementing data mesh architectures to enhance client data interactions. This position blends deep technical expertise in modern data engineering methods with a client-facing consulting approach, enabling clients to effectively manage and utilise their data. Within a team of top-tier engineers, the role involves developing greenfield data solutions that deliver tangible business outcomes across various environments.


Technical Responsibilities

  • Developing Data Solutions: Implement and enhance data-driven solutions integrating with clients' systems using state-of-the-art tools such as Databricks, Snowflake, Google Cloud, and AWS. Embrace modern data architecture philosophies including data products, data contracts, and data mesh to ensure a decentralized and consumer-oriented approach to data management.
  • Data Pipeline Development: Develop and optimise high-performance, batch and real-time data pipelines employing advanced streaming technologies like Kafka, and Flink. Utilise workflow orchestration tools such as Dataflow and Airflow.
  • Database and Storage Optimization: Optimize and manage a broad array of database technologies, from traditional relational databases (e.g., PostgreSQL, MySQL) to modern NoSQL solutions (e.g., MongoDB, Cassandra). Focus on strategies that enhance data accessibility, integrity, and performance.
  • Big Data Processing & Analytics: Utilise big data frameworks such as Apache Spark and Apache Flink to address challenges associated with large-scale data processing and analysis. These technologies are crucial for managing vast datasets and performing complex data transformations and aggregations.
  • Cloud Data Management: Implement and oversee cloud-specific data services including AWS Redshift, S3, Google BigQuery, and Google Cloud Storage. Leverage cloud architectures to improve data sharing and interoperability across different business units.
  • Security and Compliance: Ensure all data practices comply with security policies and regulations, embedding security by design in the data infrastructure. Incorporate tools and methodologies recommended for data security and compliance, ensuring robust protection and governance of data assets.

Consulting Responsibilities

  • Client Advisory: Provide expert advice to clients on optimal data practices that align with their business requirements and project goals.
  • Training and Empowerment: Educate client teams on the latest technologies and data management strategies, enabling them to efficiently utilise and maintain the solutions we have developed.
  • Professional Development: Keep up with the latest industry trends and technological advancements, continually upgrading skills and achieving certifications in the technologies Simple Machines implements across its client base.

Ideal Skills and Experience

  • Core Data Engineering Tools & Technologies: Demonstrates proficiency in SQL and Spark, and familiarity with platforms such as Databricks and Snowflake. Well-versed in various storage technologies including AWS S3, Google Cloud BigQuery, Cassandra, MongoDB, Neo4J, and HDFS. Adept in pipeline orchestration tools like AWS Glue, Apache Airflow, and dbt, as well as streaming technologies like Kafka, AWS Kinesis, Google Cloud Pub/Sub, and Azure Event Hubs.
  • Data Storage Expertise: Knowledgeable in data warehousing technologies like BigQuery, Snowflake, and Databricks, proficient in managing various data storage formats including Parquet, Delta, ORC, Avro, and JSON to optimise data storage and retrieval.
  • Building and Managing Large-scale Data Systems: Experienced in developing and overseeing large-scale data pipelines and data-intensive applications within production environments.
  • Data Modelling Expertise: Proficient in data modelling, understanding the implications and trade-offs of various methodologies and approaches.
  • Infrastructure Configuration for Data Systems: Competent in setting up data system infrastructures, favouring infrastructure-as-code practices using tools such as Terraform and Pulumi.
  • Programming Languages: Proficient in Python and SQL, with additional experience in programming languages like Java, Scala, GoLang, and Rust considered advantageous.
  • CI/CD Implementation: Knowledgeable about continuous integration and continuous deployment practices using tools like GitHub Actions and ArgoCD, enhancing software development and quality assurance.
  • Testing Tools and Frameworks: Experienced with data quality and testing frameworks such as DBT, Great Expectations, and Soda, ensuring the reliability of complex data systems.
  • Commercial Application of Data Engineering Expertise: Demonstrated experience in applying data engineering skills across various industries and organisations in a commercial context.
  • Agile Delivery and Project Management: Skilled in agile, scrum, and kanban project delivery methods, ensuring efficient and effective solution development.
  • Consulting and Advisory Skills: Experienced in a consultancy or professional services setting, offering expert advice and crafting customised solutions that address client needs. Effective in engaging stakeholders and translating business requirements into practical data engineering strategies.

Professional Experience and Qualifications

  • Professional Experience: At least 5+ years of data engineering or equivalent experience in a commercial, enterprise, or start-up environment. Consulting experience within a technology consultancy or professional services firm is highly beneficial.
  • Educational Background: Degree or equivalent experience in computer science or a related field.


#J-18808-Ljbffr

Subscribe to Future Tech Insights for the latest jobs & insights, direct to your inbox.

By subscribing, you agree to our privacy policy and terms of service.

Industry Insights

Discover insightful articles, industry insights, expert tips, and curated resources.

Careers

How Many Data Science Tools Do You Need to Know to Get a Data Science Job?

If you’re trying to break into data science — or progress your career — it can feel like you are drowning in names: Python, R, TensorFlow, PyTorch, SQL, Spark, AWS, Scikit-learn, Jupyter, Tableau, Power BI…the list just keeps going. With every job advert listing a different combination of tools, many applicants fall into a trap: they try to learn everything. The result? Long tool lists that sound impressive — but little depth to back them up. Here’s the straight-talk version most hiring managers won’t explicitly tell you: 👉 You don’t need to know every data science tool to get hired. 👉 You need to know the right ones — deeply — and know how to use them to solve real problems. Tools matter, but only in service of outcomes. So how many data science tools do you actually need to know to get a job? For most job seekers, the answer is not “27” — it’s more like 8–12, thoughtfully chosen and well understood. This guide explains what employers really value, which tools are core, which are role-specific, and how to focus your toolbox so your CV and interviews shine.

Jobs

What Hiring Managers Look for First in Data Science Job Applications (UK Guide)

If you’re applying for data science roles in the UK, it’s crucial to understand what hiring managers focus on before they dive into your full CV. In competitive markets, recruiters and hiring managers often make their first decisions in the first 10–20 seconds of scanning an application — and in data science, there are specific signals they look for first. Data science isn’t just about coding or statistics — it’s about producing insights, shipping models, collaborating with teams, and solving real business problems. This guide helps you understand exactly what hiring managers look for first in data science applications — and how to structure your CV, portfolio and cover letter so you leap to the top of the shortlist.

Education

The Skills Gap in Data Science Jobs: What Universities Aren’t Teaching

Data science has become one of the most visible and sought-after careers in the UK technology market. From financial services and retail to healthcare, media, government and sport, organisations increasingly rely on data scientists to extract insight, guide decisions and build predictive models. Universities have responded quickly. Degrees in data science, analytics and artificial intelligence have expanded rapidly, and many computer science courses now include data-focused pathways. And yet, despite the volume of graduates entering the market, employers across the UK consistently report the same problem: Many data science candidates are not job-ready. Vacancies remain open. Hiring processes drag on. Candidates with impressive academic backgrounds fail interviews or struggle once hired. The issue is not intelligence or effort. It is a persistent skills gap between university education and real-world data science roles. This article explores that gap in depth: what universities teach well, what they often miss, why the gap exists, what employers actually want, and how jobseekers can bridge the divide to build successful careers in data science.

🍪 We use cookies to enhance your browsing experience on our website. By continuing to use this site, you consent to the use of cookies as described in our Cookie Policy. You can review our Cookie Policy for more information.