Data Engineer

Job Description:

Our client, an esteemed wealth management firm, is seeking an experienced Data Engineer to join their Newcastle team on a permanent basis.

The successful candidate will work closely with developers and collaborate with the Analytics and Data Science teams to ensure the business has access to reliable, high-quality data for informed decision-making.

Essential Skills/Experience:

• Experience with Azure technologies, including Fabric, Azure Data Factory, Data Lake, Data Warehouse, and Blob Storage
• Strong experience in Spark and PySpark for large-scale data processing and transformation
• Familiarity with ETL/ELT processes and best practices for data integration and transformation
• Strong troubleshooting skills for resolving issues in data pipelines, SQL procedures, and Spark-based transformations
• Proficient in T-SQL, with the ability to write, optimize, and troubleshoot complex queries and procedures
• Solid understanding of OLTP and OLAP data structures, with the ability to translate transactional data into analytical models (facts and dimensions)
• Ability to analyze and understand complex data relationships in OLTP systems and engineer efficient OLAP solutions
• Capable of explaining complex technical concepts in simple terms to non-technical stakeholders
• Demonstrates a proactive approach to improving existing pipelines, workflows, and data models for better performance and scalability
• Excellent communication skills, with the ability to work closely with developers, analytics teams, and data scientists to understand requirements and deliver solutions
• Focused on delivering measurable outcomes, such as optimized data pipelines, reliable OLAP models, and actionable data sets
• Thrives in a dynamic environment, adapting to new tools, technologies, and business needs
• Understands the importance of data accuracy, consistency, and governance
• Keeps up to date with advancements in data engineering and cloud technologies
• Detail-oriented with a focus on delivering high-quality, accurate, and reliable data models
• Takes ownership of assigned tasks and ensures timely and high-quality delivery
• A team player who can collaborate across departments to achieve shared goals
• Ensures all engineered data aligns with organizational standards and complies with security and privacy policies

Core Responsibilities:

• Continuously enhance existing MS Fabric data pipelines to improve performance, scalability, and reliability while minimizing processing times and resource usage
• Engineer new data pipelines to integrate additional data sources, both internal and external, into the existing data infrastructure, adhering to best practices in ETL/ELT processes
• Translate complex OLTP data structures into well-optimized OLAP models (facts and dimensions) that meet the needs of the Analytics and Insights team, ensuring data is accurate, reliable, and ready for use in MI Power BI dashboards
• Work collaboratively with developers to migrate legacy reporting data into MS Fabric, ensuring a seamless transition with minimal disruption to ongoing operations
• Partner with the Analytics team to understand their requirements, translating these into engineered data sets and summary tables that support their reporting and insights needs
• Collaborate with the Data Science team to create summary and aggregated data sets, enabling advanced analyses and predictive modeling
• Design and deliver new data warehouse models tailored to meet specific business and analytics requirements, improving the accessibility and usability of data across teams
• Fulfill ad hoc data requests efficiently, leveraging MS Fabric data and ensuring high-quality outputs that meet stakeholder expectations
• Monitor, maintain, and optimize daily SQL procedures to ensure timely and accurate data processing for reporting and analysis purposes

Benefits:

• A highly competitive salary
• Wider Benefits package

Core-Asset Consulting is an equal opportunities recruiter and we welcome applications from everyone irrespective of age, disability, gender, gender identity or expression, race, colour, ethnic or national origin, sexual orientation, religion or belief, marital/civil partner status or pregnancy.

Job reference: 15966

To apply for this vacancy applicants must be eligible to work in the UK in accordance with the Immigration, Asylum and Nationality Act 2006.

#J-18808-Ljbffr