Senior data engineer

Entity: Technology Job Family Group: IT&S Group Job Description: Our purpose is to deliver energy to the world, today and tomorrow. For over 100 years, bp has focused on discovering, developing, and producing oil and gas in the nations where we operate. We are one of the few companies globally that can provide governments and customers with an integrated energy offering. Delivering our strategy sustainably is fundamental to achieving our ambition to be a net zero company by 2050 or sooner! About the Role : We are seeking a Senior Data Engineer to join our growing data engineering team. In this role, you will be responsible for designing, building, and optimizing scalable data pipelines and infrastructure that power analytics, machine learning, and business intelligence across the organization. You will work closely with data scientists, analysts, and platform engineers to ensure robust and efficient data delivery. This is an excellent opportunity for someone who is passionate about data, cloud technologies, and scalable systems, and wants to have a direct impact on data-driven decision-making. Key Responsibilities: Design, build, and maintain scalable and reliable ETL/ELT data pipelines using Python, PySpark, and SQL. Develop and manage data workflows and orchestration using tools such as Airflow or similar. Optimize data processes for performance, scalability, and cost-efficiency, particularly in cloud environments. Collaborate with data scientists, analysts, and product teams to understand data needs and deliver high-quality data sets. Implement and uphold data quality, data governance, and monitoring standards. Work with cloud infrastructure (primarily AWS) to build and manage data lakes, data warehouses, and distributed data processing systems. Ensure data security and compliance best practices are followed across data systems and processes. Required Qualifications: 5 years of experience in data engineering or a related field. Proficiency in Python and PySpark for data processing and automation. Strong command of SQL for data querying, transformation, and performance tuning. Deep experience with cloud platforms , preferably AWS (e.g., S3, Glue, Redshift, Athena, EMR, Lambda). Experience with Azure or GCP is a plus. Experience building and managing data lakes and data warehouses. Strong understanding of distributed systems and big data processing. Experience with data workflow orchestration tools such as Airflow, dbt , or similar. Solid understanding of data modeling, data quality, and data governance concepts. Familiarity with CI/CD pipelines and infrastructure as code (e.g., Terraform or CloudFormation) is a plus. Excellent problem-solving, communication, and collaboration skills. Preferred Qualifications: Bachelors or masters degree in computer science, Engineering, Data Science, or related field. Experience with streaming data technologies (e.g., Kafka, Kinesis). Familiarity with data cataloging and metadata management tools. Exposure to machine learning pipelines or MLOps is a bonus. At bp, we provide the following environment and benefits to you: A company culture where we respect our diverse and unified teams, where we are proud of our achievements and where fun and the attitude of giving back to our environment are highly valued Possibility to join our social communities and networks Learning opportunities and other development opportunities to craft your career path Life and health insurance, medical care package And many other benefits. We are an equal opportunity employer and value diversity at our company. We do not discriminate based on race, religion, colour, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status. We will ensure that individuals with disabilities are provided reasonable accommodation to participate in the job application or interview process, perform crucial job functions, and receive other benefits and privileges of employment. Dont hesitate to get in touch with us to request any accommodations. Travel Requirement No travel is expected with this role Relocation Assistance: This role is not eligible for relocation Remote Type: This position is a hybrid of office/remote working Skills: Commercial Acumen, Communication, Data Analysis, Data cleansing and transformation, Data domain knowledge, Data Integration, Data Management, Data Manipulation, Data Sourcing, Data strategy and governance, Data Structures and Algorithms (Inactive), Data visualization and interpretation, Digital Security, Extract, transform and load, Group Problem Solving Legal Disclaimer: We are an equal opportunity employer and value diversity at our company. We do not discriminate on the basis of race, religion, color, national origin, sex, gender, gender expression, sexual orientation, age, marital status, socioeconomic status, neurodiversity/neurocognitive functioning, veteran status or disability status.Individuals with an accessibility need may request an adjustment/accommodation related to bps recruiting process (e.g., accessing the job application, completing required assessments, participating in telephone screenings or interviews, etc.). If you would like to request an adjustment/accommodation related to the recruitment process,please contact us. If you are selected for a position and depending upon your role, your employment may be contingent upon adherence to local policy. This may include pre-placement drug screening, medical review of physical fitness for the role, and background checks.