Data Engineer

VBeyond Corporation
Milton Keynes
1 month ago
Applications closed

Related Jobs

View all jobs

Data Engineer

Data Engineer

Data Engineer

Data Engineer - UK Perm - London Hrbrid

Data Engineer, DE55

Data Engineer - Databricks - £60,000

Job Description:

As aData EngineerwithIcebergexperience, you will play a crucial role in the design, development, and maintenance of our data infrastructure. Your work will empower data-driven decision-making and contribute to the success of our data-driven initiatives.


Key Responsibilities:

  • Data Integration: Develop and maintain data pipelines to extract, transform, and load (ETL) data from various sources into AWS data stores for both batch and streaming data ingestion.
  • AWS Expertise: Utilize your expertise in AWS services such as Amazon EMR , S3, AWS Glue, Amazon Redshift, AWS Lambda, and more to build and optimize data solutions.
  • Data Modeling: Design and implement data models to support analytical and reporting needs, ensuring data accuracy and performance.
  • Data Quality: Implement data quality and data governance best practices to maintain data integrity.
  • Performance Optimization: Identify and resolve performance bottlenecks in data pipelines and storage solutions to ensure optimal performance.
  • Documentation: Create and maintain comprehensive documentation for data pipelines, architecture, and best practices.
  • Collaboration: Collaborate with cross-functional teams, including data scientists and analysts, to understand data requirements and deliver high-quality data solutions.
  • Automation: Implement automation processes and best practices to streamline data workflows and reduce manual interventions.
  • Experience working with bigdata ACID file formats to build delta lake, particularly with Iceberg file formats and loading methods of Iceberg.
  • Good knowledge on Iceberg functionalities to use the delta features to identify the changed records, optimization and housekeeping on Iceberg tables in the data lake.


Must have:AWS, ETL, EMR, GLUE, Spark/Scala, Java, Python,


Good to have:Cloudera – Spark, Hive, Impala, HDFS, Informatica PowerCenter, Informatica DQ/DG, Snowflake Erwin

Get the latest insights and jobs direct. Sign up for our newsletter.

By subscribing you agree to our privacy policy and terms of service.

Industry Insights

Discover insightful articles, industry insights, expert tips, and curated resources.

Jobs Careers

10 Essential Books to Read to Nail Your Data Science Career in the UK

Data science continues to be one of the most exciting and rapidly evolving fields in tech. With industries across the UK—ranging from finance and healthcare to e-commerce and government—embracing data-driven decision-making, the demand for skilled data scientists has soared. Whether you're a recent graduate looking for your first role or a professional aiming to advance your career, staying updated through books is crucial. In this article, we explore ten essential books every data science job seeker in the UK should read. Each book provides valuable insights into core concepts, practical applications, and industry-standard tools, helping you build skills employers are actively looking for.

Careers

Navigating Data Science Career Fairs Like a Pro: Preparing Your Pitch, Questions to Ask, and Follow-Up Strategies to Stand Out

Data science has taken centre stage in the modern workplace. Organisations rely on data-driven insights to shape everything from product innovation and customer experience to operational efficiency and strategic planning. As a result, there is a growing need for skilled data scientists who can analyse large volumes of data, build predictive models, communicate findings effectively, and collaborate cross-functionally. If you are looking to accelerate your data science career—or even land your first role—attending data science career fairs can be a game-changer. Unlike traditional online applications, face-to-face interactions let you showcase your personality, passion, and communication skills in addition to your technical expertise. However, to stand out in a busy environment, you need a clear strategy: from polishing your personal pitch and asking thoughtful questions to following up with a memorable message. In this article, we’ll guide you through every step of making a strong impression at data science career fairs in the UK and beyond.

Jobs

Common Pitfalls Data Science Job Seekers Face and How to Avoid Them

Data science has become a linchpin for decision-making and innovation across countless industries, from finance and healthcare to tech and retail. The demand for data scientists in the UK continues to climb, with businesses seeking professionals who can interpret complex datasets, build predictive models, and communicate actionable insights. Despite this high demand, the job market can be extremely competitive—and many applicants unknowingly fall into avoidable traps. Whether you’re an aspiring data scientist fresh out of university, a professional transitioning from a quantitative role, or a seasoned analyst looking to expand your skill set, it’s crucial to navigate your job search effectively. In this article, we explore the most common pitfalls data science job seekers face and provide pragmatic advice to help you stand out. By refining your CV, portfolio, interview strategies, and communication skills, you can significantly increase your chances of landing a rewarding data science role. If you’re looking for your next data science job in the UK, don’t forget to explore the listings at Data Science Jobs. Read on to discover how to avoid critical mistakes and position yourself for success.

🍪 We use cookies to enhance your browsing experience on our website. By continuing to use this site, you consent to the use of cookies as described in our Cookie Policy. You can review our Cookie Policy for more information.