Senior Data Engineer (UK)

About Us:

ATREIDES is a leader in advanced geospatial technologies, big data solutions, and data analytics. We specialize in providing cutting-edge software platforms that enable our clients to process, analyze, and visualize large-scale geospatial and sensor data for critical decision-making. Our mission is to deliver innovative, scalable, and high-performance platforms that help organizations unlock the full potential of their data for various industries, including defense, national security, environmental monitoring, and urban planning.

We are seeking a highly skilled Senior Data Engineer with a strong focus on big data, data analytics, and geospatial data intelligence to join our engineering team. In this role, you will help design and develop robust, scalable platforms that handle vast amounts of geospatial data and provide actionable insights through data analytics.

Job Overview:

As a Senior Data Engineer at ATREIDES, you will be responsible for developing and optimizing our core software platform, with an emphasis on integrating and processing big data, performing advanced data analytics, and enabling geospatial intelligence features.

You will work with cross-functional teams, including data engineers, geospatial analysts, and product managers, to create a high-performance platform that processes massive datasets, integrates complex geospatial data, and offers real-time or near-real-time insights.

You will have the opportunity to work on cutting-edge technologies, including distributed computing, cloud infrastructure, and machine learning, to deliver a world-class platform that powers critical data-driven applications.

Key Responsibilities:

• Platform Development & Optimization: Design, develop, and optimize the core software platform to handle large-scale geospatial datasets, integrate big data sources, and support advanced data analytics. Ensure high availability, reliability, and performance of platform components.
• Big Data Architecture: Build and maintain big data architectures and data pipelines to efficiently process large volumes of geospatial and sensor data. Leverage technologies such as Hadoop, Apache Spark, and Kafka to ensure scalability, fault tolerance, and speed.
• Geospatial Data Integration: Develop systems that integrate geospatial data from a variety of sources (e.g., satellite imagery, remote sensing, IoT sensors, and GIS data) and process this data for use in data analytics applications. Ensure geospatial data is processed, stored, and queried efficiently to meet operational and analytical needs.
• Advanced Data Analytics: Implement data analytics capabilities on the platform that enable processing, analysis, and visualization of geospatial and sensor data. Develop algorithms and tools for geospatial analysis, pattern recognition, anomaly detection, and predictive modeling using machine learning techniques.
• Real-Time Data Processing: Build real-time or near-real-time data processing systems to deliver actionable insights to end-users. Optimize data flows and streaming analytics to ensure fast, low-latency decision-making capabilities.
• Cloud & Distributed Systems: Work with cloud platforms (AWS, GCP, Azure) to deploy and scale big data systems. Utilize containerization (e.g., Docker, Kubernetes) and cloud-native services to ensure flexible and scalable platform infrastructure.
• Data Security & Compliance: Ensure that platform components adhere to data privacy, security, and compliance regulations. Implement best practices for data encryption, access controls, and secure data processing to protect sensitive geospatial and sensor data.
• Collaboration & Cross-Functional Work: Collaborate with data scientists, geospatial analysts, and product managers to identify requirements and build platform features that align with business objectives. Participate in technical discussions and decision-making around platform architecture.
• Continuous Improvement: Continuously monitor platform performance, troubleshoot issues, and implement optimizations to improve scalability, efficiency, and user experience. Stay up-to-date with the latest trends in big data technologies, geospatial intelligence, and data analytics to drive innovation.

Required Qualifications:

• Education: Bachelor’s or Master’s degree in Computer Science, Engineering, Geospatial Intelligence, Data Science, or a related field, or equivalent experience.
• Experience:
  • 5+ years of experience in software engineering, with a focus on building and optimizing large-scale platforms for big data, data analytics, or geospatial data.
  • Strong background in developing big data applications, data pipelines, and distributed systems.
  • Proven experience working with geospatial data, including GIS, satellite imagery, and remote sensing data, and integrating it into data-driven applications.
  • Familiarity with geospatial data formats (e.g., GeoJSON, Shapefiles, KML) and tools (e.g., PostGIS, GDAL, GeoServer).
• Technical Skills:
  • Expertise in big data frameworks and technologies (e.g., Hadoop, Spark, Kafka, Flink) for processing large datasets.
  • Proficiency in programming languages such as Python, Java, or Scala, with a focus on big data frameworks and APIs.
  • Experience with cloud services and technologies (AWS, Azure, GCP) for big data processing and platform deployment.
  • Strong knowledge of data warehousing, data lakes, and data pipeline design for large-scale data integration and storage.
  • Familiarity with machine learning and AI techniques for data analytics (e.g., classification, regression, clustering, anomaly detection).
  • Experience with containerization and orchestration tools (e.g., Docker, Kubernetes) for deploying scalable applications.
• Geospatial Intelligence & Data Analysis: Strong understanding of geospatial concepts and techniques (e.g., spatial queries, coordinate systems, projections). Experience with geospatial analytics tools such as ArcGIS, QGIS, or similar platforms is a plus.
• Data Modeling & Querying: Experience with SQL and NoSQL databases, including spatial databases (PostGIS, MongoDB, Elasticsearch), for storing and querying geospatial and big data.

Preferred Qualifications:

• Degree or higher in Computer Science, Geospatial Intelligence, Data Science, or related field.
• Experience with data visualization tools and libraries (e.g., Tableau, D3.js, Mapbox, Leaflet) for displaying geospatial insights and analytics.
• Familiarity with real-time stream processing frameworks (e.g., Apache Flink, Kafka Streams).
• Experience with geospatial data processing libraries (e.g., GDAL, Shapely, Fiona).
• Background in defense, national security, or environmental monitoring applications is a plus.

Compensation and Benefits:

● Competitive salary

● Comprehensive health, dental, and vision insurance plans

● Flexible hybrid work environment

● Additional benefits like flexible hours, work travel opportunities, competitive vacation time and parental leave

While meeting all of these criteria would be ideal, we understand that some candidates may meet most, but not all. If you're passionate, curious and ready to "work smart and get things done," we'd love to hear from you.

#J-18808-Ljbffr