Senior Data Engineer

Senior Data Engineer

UST are currently recruiting for a Senior Data Engineer with proficiency in developing data pipelines including coding and testing for ingesting wrangling transforming and joining data from various sources.

About the role

The ideal candidate should be adept in ETL tools like Informatica Glue Databricks and DataProc with strong coding skills in Python PySpark and SQL.

You will have expertise in data warehousing solutions such as Snowflake BigQuery Lakehouse and Delta Lake is essential including the ability to calculate processing costs and address performance issues. A solid understanding of DevOps and infrastructure needs is also required.

Job Description:

• Work as part of a self-organised Ab Initio engineering team to design and develop software features that are that are delightful and vital to the lives of credit card consumers across all walks of life
• Work to achieve the goals and vision articulated by Product Management and your management chain
• Your focus is on a major portion of existing or new team software (e.g., significant component, set of features, mid-size application or service)
• Drive engineering best practices (e.g., Operational Excellence, Security, Quality, etc.) and set standards, across your team and wider engineering teams
• Innovate within your team, initiative area and contribute within your technical domain. Requirements for role:
• You will have leadership experience in delivering throughout the dev life cycle
• A complete understanding of the capabilities of the latest technologies related to Ab Initio
• Able to resolve questions from the team related to requirements from architects and product managers
• Able to quickly come up with a solution to a requirement and identify gaps in requirements to fully define a solution
• Able to do hand on programming in the GDE and unix scripting languages
• You are an advocate for quality and can lead the team leveraging approaches such as Pair Programming. You will be able to find potential savings in effort spent on delivering and find ways to continually increase delivery throughput and eradicate waste
• You will be able to review code others have developed and feedback any inconsistencies with the standards (written or best practices)
• You will be able to estimate the delivery timescales and justify these estimates
• You will be aware of the Jenkins pipeline principles
• Knowledge of Agile practices Experience that would be advantageous:
• You will be aware of AWS interactions and file storage
• You will have a core understanding of Lambda programming
• Solving real world problems and being comfortable working in a complex regulated environment

Skills

Ab Initio,Data Engineering,Agile