Senior Data Engineer (UK)

Atreides
City of London
3 months ago
Applications closed

Related Jobs

View all jobs

Senior Data Engineer

Senior Data Engineer

Senior Data Engineer

Senior Data Engineer

Senior Data Engineer

Senior Data Engineer

About Us

ATREIDES is a leader in advanced geospatial technologies, big data solutions, and data analytics. We specialize in providing cutting-edge software platforms that enable our clients to process, analyze, and visualize large-scale geospatial and sensor data for critical decision-making. Our mission is to deliver innovative, scalable, and high-performance platforms that help organizations unlock the full potential of their data for various industries, including defense, national security, environmental monitoring, and urban planning.

We are seeking a highly skilled Senior Data Engineer with a strong focus on big data, data analytics, and geospatial data intelligence to join our engineering team. In this role, you will help design and develop robust, scalable platforms that handle vast amounts of geospatial data and provide actionable insights through data analytics.

Job Overview

As a Senior Data Engineer at ATREIDES, you will be responsible for developing and optimizing our core software platform, with an emphasis on integrating and processing big data, performing advanced data analytics, and enabling geospatial intelligence features.

You will work with cross-functional teams, including data engineers, geospatial analysts, and product managers, to create a high-performance platform that processes massive datasets, integrates complex geospatial data, and offers real-time or near-real-time insights.

You will have the opportunity to work on cutting-edge technologies, including distributed computing, cloud infrastructure, and machine learning, to deliver a world-class platform that powers critical data-driven applications.

Key Responsibilities
  • Platform Development & Optimization: Design, develop, and optimize the core software platform to handle large-scale geospatial datasets, integrate big data sources, and support advanced data analytics. Ensure high availability, reliability, and performance of platform components.
  • Big Data Architecture: Build and maintain big data architectures and data pipelines to efficiently process large volumes of geospatial and sensor data. Leverage technologies such as Hadoop, Apache Spark, and Kafka to ensure scalability, fault tolerance, and speed.
  • Geospatial Data Integration: Develop systems that integrate geospatial data from a variety of sources (e.g., satellite imagery, remote sensing, IoT sensors, and GIS data) and process this data for use in data analytics applications. Ensure geospatial data is processed, stored, and queried efficiently to meet operational and analytical needs.
  • Advanced Data Analytics: Implement data analytics capabilities on the platform that enable processing, analysis, and visualization of geospatial and sensor data. Develop algorithms and tools for geospatial analysis, pattern recognition, anomaly detection, and predictive modeling using machine learning techniques.
  • Real-Time Data Processing: Build real-time or near-real-time data processing systems to deliver actionable insights to end-users. Optimize data flows and streaming analytics to ensure fast, low-latency decision-making capabilities.
  • Cloud & Distributed Systems: Work with cloud platforms (AWS, GCP, Azure) to deploy and scale big data systems. Utilize containerization (e.g., Docker, Kubernetes) and cloud-native services to ensure flexible and scalable platform infrastructure.
  • Data Security & Compliance: Ensure that platform components adhere to data privacy, security, and compliance regulations. Implement best practices for data encryption, access controls, and secure data processing to protect sensitive geospatial and sensor data.
  • Collaboration & Cross-Functional Work: Collaborate with data scientists, geospatial analysts, and product managers to identify requirements and build platform features that align with business objectives. Participate in technical discussions and decision-making around platform architecture.
  • Continuous Improvement: Continuously monitor platform performance, troubleshoot issues, and implement optimizations to improve scalability, efficiency, and user experience. Stay up-to-date with the latest trends in big data technologies, geospatial intelligence, and data analytics to drive innovation.
Required Qualifications
  • Education: Bachelor’s or Master’s degree in Computer Science, Engineering, Geospatial Intelligence, Data Science, or a related field, or equivalent experience.
  • Experience:
    • 5+ years of experience in software engineering, with a focus on building and optimizing large-scale platforms for big data, data analytics, or geospatial data.
    • Strong background in developing big data applications, data pipelines, and distributed systems.
    • Proven experience working with geospatial data, including GIS, satellite imagery, and remote sensing data, and integrating it into data-driven applications.
    • Familiarity with geospatial data formats (e.g., GeoJSON, Shapefiles, KML) and tools (e.g., PostGIS, GDAL, GeoServer).
Technical Skills
  • Expertise in big data frameworks and technologies (e.g., Hadoop, Spark, Kafka, Flink) for processing large datasets.
  • Proficiency in programming languages such as Python, Java, or Scala, with a focus on big data frameworks and APIs.
  • Experience with cloud services and technologies (AWS, Azure, GCP) for big data processing and platform deployment.
  • Strong knowledge of data warehousing, data lakes, and data pipeline design for large-scale data integration and storage.
  • Familiarity with machine learning and AI techniques for data analytics (e.g., classification, regression, clustering, anomaly detection).
  • Experience with containerization and orchestration tools (e.g., Docker, Kubernetes) for deploying scalable applications.
Geospatial Intelligence & Data Analysis
  • Strong understanding of geospatial concepts and techniques (e.g., spatial queries, coordinate systems, projections). Experience with geospatial analytics tools such as ArcGIS, QGIS, or similar platforms is a plus.
  • Data Modeling & Querying: Experience with SQL and NoSQL databases, including spatial databases (PostGIS, MongoDB, Elasticsearch), for storing and querying geospatial and big data.
Preferred Qualifications
  • Degree or higher in Computer Science, Geospatial Intelligence, Data Science, or related field.
  • Experience with data visualization tools and libraries (e.g., Tableau, D3.js, Mapbox, Leaflet) for displaying geospatial insights and analytics.
  • Familiarity with real-time stream processing frameworks (e.g., Apache Flink, Kafka Streams).
  • Experience with geospatial data processing libraries (e.g., GDAL, Shapely, Fiona).
  • Background in defense, national security, or environmental monitoring applications is a plus.
Compensation and Benefits
  • Competitive salary
  • Comprehensive health, dental, and vision insurance plans
  • Flexible hybrid work environment
  • Additional benefits like flexible hours, work travel opportunities, competitive vacation time and parental leave

We look forward to hearing from you if you are passionate, curious and ready to "work smart and get things done".


#J-18808-Ljbffr

Subscribe to Future Tech Insights for the latest jobs & insights, direct to your inbox.

By subscribing, you agree to our privacy policy and terms of service.

Industry Insights

Discover insightful articles, industry insights, expert tips, and curated resources.

Careers

How Many Data Science Tools Do You Need to Know to Get a Data Science Job?

If you’re trying to break into data science — or progress your career — it can feel like you are drowning in names: Python, R, TensorFlow, PyTorch, SQL, Spark, AWS, Scikit-learn, Jupyter, Tableau, Power BI…the list just keeps going. With every job advert listing a different combination of tools, many applicants fall into a trap: they try to learn everything. The result? Long tool lists that sound impressive — but little depth to back them up. Here’s the straight-talk version most hiring managers won’t explicitly tell you: 👉 You don’t need to know every data science tool to get hired. 👉 You need to know the right ones — deeply — and know how to use them to solve real problems. Tools matter, but only in service of outcomes. So how many data science tools do you actually need to know to get a job? For most job seekers, the answer is not “27” — it’s more like 8–12, thoughtfully chosen and well understood. This guide explains what employers really value, which tools are core, which are role-specific, and how to focus your toolbox so your CV and interviews shine.

Jobs

What Hiring Managers Look for First in Data Science Job Applications (UK Guide)

If you’re applying for data science roles in the UK, it’s crucial to understand what hiring managers focus on before they dive into your full CV. In competitive markets, recruiters and hiring managers often make their first decisions in the first 10–20 seconds of scanning an application — and in data science, there are specific signals they look for first. Data science isn’t just about coding or statistics — it’s about producing insights, shipping models, collaborating with teams, and solving real business problems. This guide helps you understand exactly what hiring managers look for first in data science applications — and how to structure your CV, portfolio and cover letter so you leap to the top of the shortlist.

Education

The Skills Gap in Data Science Jobs: What Universities Aren’t Teaching

Data science has become one of the most visible and sought-after careers in the UK technology market. From financial services and retail to healthcare, media, government and sport, organisations increasingly rely on data scientists to extract insight, guide decisions and build predictive models. Universities have responded quickly. Degrees in data science, analytics and artificial intelligence have expanded rapidly, and many computer science courses now include data-focused pathways. And yet, despite the volume of graduates entering the market, employers across the UK consistently report the same problem: Many data science candidates are not job-ready. Vacancies remain open. Hiring processes drag on. Candidates with impressive academic backgrounds fail interviews or struggle once hired. The issue is not intelligence or effort. It is a persistent skills gap between university education and real-world data science roles. This article explores that gap in depth: what universities teach well, what they often miss, why the gap exists, what employers actually want, and how jobseekers can bridge the divide to build successful careers in data science.

🍪 We use cookies to enhance your browsing experience on our website. By continuing to use this site, you consent to the use of cookies as described in our Cookie Policy. You can review our Cookie Policy for more information.