Senior Data Engineer

About OpenCorporates

OpenCorporatesis acertified B Corp, revolutionizing access to company data. It has built the world's largest open database of companies, trusted by regulators, financial institutions, investigative journalists, and businesses for its unparalleled breadth of data, covering over 200 million companies across 140+ jurisdictions, including the entirety of the USA.

Legal entities are at the heart of the modern world – and as we move into a world of digitalized commerce and AI-generated uncertainty, OpenCorporates will provide the trusted foundational data the world needs.

About the Senior Data Engineer role

Data is our product. Our data supports a plethora of client solutions from KYC to analytics and data management. Hundreds of organizations rely on our data at scale to power their critical operations, including the US government, the OECD, Oracle, and many others.

As the Senior Data Engineer, you will be instrumental in building and optimizing our data pipelines, ensuring scalability, efficiency, and reliability. You will need to possess a deep understanding of Snowflake, AWS data tooling, data warehousing, and best practices in cloud-based data platforms.

You will be responsible for:

• Designing, developing, and maintaining robust data ingestion pipelines using AWS services (S3, Lambda, ECR, ECS, DynamoDB).
• Developing and optimizing data transformation pipelines using a combination of SQL and Python.
• Working extensively with Snowflake as a data platform, ensuring efficient and scalable structures and patterns.
• Implementing best practices for data engineering, including performance tuning, security, and data governance.
• Collaborating with data analysts, scientists, and other stakeholders to deliver high-quality data solutions.
• Monitoring, troubleshooting, and optimizing data pipelines for performance and reliability.
• Ensuring data quality, consistency, and integrity across multiple data sources.

We would like you to demonstrate solid experience in:

• Data warehousing concepts.
• Expert-level proficiency in SQL and Python, focusing on simplicity and maintainability.
• Hands-on experience with AWS services, specifically S3, Lambda, ECR, ECS, and DynamoDB.
• Deep understanding of data modeling, including dimensional modeling and best practices for cloud-based data platforms.
• Strong problem-solving skills with the ability to optimize queries and improve data performance.
• Experience in handling large-scale datasets and implementing scalable data solutions.

Bonus points if you have 1 or 2 of the following:

• Experience with utilizing AI in the data engineering space.
• Experience with Terraform for infrastructure as code.
• Knowledge of CI/CD practices for automating deployment processes.
• Familiarity with Data Vault modeling for data warehousing.

Benefits:

• Flexible work culture supporting a strong work-life balance.
• Remote-first approach with opportunities for in-person collaboration.
• 28 days of annual leave per year, plus bank holidays & an extra day for every year of service.
• Flexibility to work outside of the UK for up to 4 weeks in a rolling 12-month period.
• Regular company socials.
• Discretionary time off for special life moments.
• Private healthcare with Aviva, life assurance, plus an annual budget for your wellbeing.
• Personal development budgets and learning days.
• Competitive package, including share options and a bonus scheme.
• Cycle to work scheme.
• £500 home office set-up budget.

Diversity Matters

Don’t meet every single requirement? At OpenCorporates, we're assembling a diverse innovative team that defies our industry's norms. Think this role could suit you? We encourage you to apply even if your past experience doesn’t align perfectly with every qualification. We welcome applicants with a curious growth mindset.

#J-18808-Ljbffr