SC Data Engineer - AWS
IR35: Outside
Rate: £400-450 / day (negotiable)
Clearance: SC
Start: ASAP
Duration: 6 months (extensions expected)
Location: Remote with occasional travel to Bristol / London
Job Brief: We are seeking a skilled and motivated Data Engineer with expertise in AWS to join our dynamic team. The ideal candidate will be responsible for designing, building, and maintaining scalable data pipelines that process vast amounts of data across different platforms. They will leverage AWS technologies to ensure seamless integration and optimization of data flows, ensuring high availability, security, and efficiency.
Skills & Qualifications:
* Strong experience with AWS services such as S3, Redshift, EC2, Lambda, Glue, Athena, and EMR.
* Proficiency in programming languages such as Python, Java, or Scala for data engineering tasks.
* Experience with data warehousing, ETL (Extract, Transform, Load) processes, and data integration.
* In-depth knowledge of cloud-based architectures and best practices for deploying data pipelines in AWS.
* Expertise in designing and implementing scalable and efficient data pipelines for both batch and real-time processing.
* Hands-on experience with data transformation and data processing frameworks (e.g., Apache Spark, Apache Kafka).
* Solid understanding of relational databases (e.g., MySQL, PostgreSQL) and NoSQL databases (e.g., DynamoDB, MongoDB).
* Familiarity with data security, governance, and compliance standards, particularly in cloud environments.
* Strong problem-solving skills and attention to detail.
* Excellent communication skills for collaboration with cross-functional teams.
* Ability to work in an agile development environment and manage multiple priorities.
Preferred Qualifications:
* Experience with containerization technologies like Docker and Kubernetes.
* Knowledge of data lakes, serverless architectures, and microservices.
* Familiarity with DevOps practices and CI/CD pipelines for automated deployment of data solutions.
* Certifications such as AWS Certified Data Analytics - Specialty or AWS Certified Solutions Architect are a plus.
Responsibilities:
* Design, develop, and optimize data pipelines on AWS to ingest, process, and transform data.
* Collaborate with data scientists, analysts, and business teams to understand requirements and deliver data solutions.
* Implement and manage efficient data storage solutions using AWS technologies.
* Ensure data quality, security, and compliance across all data engineering processes.
* Continuously monitor and improve data systems to ensure scalability and performance.
* Contribute to the development and implementation of best practices and standards for data engineering.
APPLY NOW!
Due to the nature and urgency of this post, candidates holding or who have held high level security clearance in the past are most welcome to apply. Please note successful applicants will be required to be security cleared prior to appointment which can take up to a minimum 10 weeks.
