IT Data Engineer

About The Role

The IT Data Engineer works within the IT department to undertake analysis, design software and data solutions and solutions aimed at improving our business processes, decision making capabilities and analytic workflows.

This position will be part of a small team tasked with producing high quality, innovative solutions to enhance our core business systems and create significant intellectual property that will differentiate us in our sector and markets. The position holder will also be instrumental in helping establish leading software and service delivery processes and steer the team in the direction of DevOps, continuous integration, continuous delivery and delivery lifecycle automation.

This is a role for a sector specialist who has experience working as an analyst programmer in a fast-paced environment and also for someone who is friendly, approachable and proactive in bringing new ideas to the table.

Responsibilities

Business Analysis

• Perform business analysis in support of business change, project delivery or software / database development
• Evaluate business processes, discover and document requirements, uncover areas for improvement
• Ensure solutions meet business and technology needs and requirements
• Document and demonstrate solutions by developing documentation, flowcharts, layouts, diagrams, charts
• Perform unit and system acceptance testing against (non) functional requirements
• Ensure that training services and documentation are in place to educate staff on how to use new software or technology effectively

Software Delivery

• Analyse requirements, design and deliver software and data solutions to meet our goals and objectives as standalone changes or as part of a broader programme of work
• Review, analyse and test other team members code and solutions where appropriate
• Contribute to the definition and maintenance of our target state solution architecture, development frameworks and delivery methods
• Be accountable for the quality of software development and deployment by delivering modular, efficient and testable code
• Develop software verification plans and quality assurance procedures

Delivery Lifecycle

• Own and maintain the delivery lifecycle for assigned development work
• Manage 3rd party/external development teams acting as a team lead/scrum master where necessary
• Review, communicate and work to resolve any impediments or technical challenges encountered whilst developing solutions
• Contribute to continuous improvement of our software and service delivery by identifying areas of improvement opportunity and taking the initiative to manage the change
• Management or significant contribution to defining and implementing delivery methods such as DevOps, CI/CD, Test Automation, use of backlogs and sprint planning
• Champion Service and Support considerations and service transition activity in all development activity

Performance & Quality

• Continually update technical knowledge and skills by attending in-house and external courses, reading manuals and accessing new applications
• Work with internal and third-party technology teams to ensure development activity is conducted to the expected level of quality and timeliness, providing constructive feedback and coaching where appropriate
• Effectively manage the balance of time spent developing new solutions versus providing support for incident and problem management
• Be an ambassador for IT, working across the business to provide effective communication on IT software and service delivery and build relationships with other teams to ensure effective dialogue between departments

Requirements

Qualities and Skills

Core:

• Data Engineering: A strong background in data engineering, with thorough understanding of concepts like ETL (Extract, Transform, Load), data cleaning, data structures, and data warehousing.
• SQL Database Experience: Proficiency in SQL databases with the ability to write complex queries and procedures. Demonstrable experience working with Azure SQL Database and Snowflake are essential to this role.
• Azure Knowledge: Comprehensive understanding of the Azure platform, including knowledge about its architecture, services, and security measures. Hands-on experience with Azure data services like Azure SQL Database, Azure Data Factory, Azure Data Lake, and Azure Synapse Analytics are essential.
• Programming Skills: Proficiency in programming languages such as Python, Java, or C#, which are commonly used for data manipulation and automation.
• Data Modelling: The ability to design and implement effective database models to store and retrieve company data.
• DevOps Practices: Understanding of DevOps principles and CI/CD pipelines, and experience with tools like Azure DevOps or GitHub.
• Communication Skills: Ability to communicate effectively with both technical and non-technical stakeholders, understanding their needs and translating them into data solutions.

Desired:

• BI Tools: Experience with BI tools such as Qlik, Tableau or Power BI, for creating reports and data visualizations.
• Stakeholder Relationship Management: Experience working with different stakeholders, understanding their needs and communicating effectively. Proven ability to maintain strong stakeholder relationships.
• Big Data Technologies: Familiarity with big data technologies like Apache Spark, Hadoop. Even though they might not be needed directly, the understanding could help in broader data architecture discussions.
• Unit Testing Frameworks: Experience with unit testing frameworks in languages like Python, Java, or C#. Ability to write and run tests to ensure the integrity of data processes and outputs.
• Azure Advanced Analytics: Knowledge of Azure's more advanced analytics tools like Azure Analysis Services could be beneficial.
• Data Lake and Stream Analytics: Even if not a core part of the role, understanding of Azure Data Lake Storage and Azure Stream Analytics could be beneficial in certain scenarios.
• Data Warehousing: Experience with designing, developing, and maintaining data warehousing systems, even beyond Azure, could be beneficial.
• Certifications: Possessing Azure Data Engineer Associate certification (DP-200, DP-201) or any other related professional certification can validate your skills and give you an edge.
• Project Management: Experience with project management methodologies like Agile, Scrum, or Kanban, which can be useful in a team setting.

This description reflects the core activities of the role but is not intended to be all-inclusive and other duties within the group/department may be required in addition to changes in the emphasis of duties as required from time to time. There is a requirement for the post holder to recognise this and adopt a flexible approach to work. Job descriptions will be reviewed regularly and where necessary revised in accordance with organisational needs. Any major changes will be discussed with the post holder.