Job description

Data Engineer
Hybrid Working, 1 day per week in Tunbridge Wells OR Peterborough, rest from home

We are currently looking for a Data Engineer with experience in Big Data technologies to join a large insurance group within the Data Solutions Team.

You you will be responsible for collecting, storing, processing, and analysing huge sets of data in an Azure Cloud based Data Lake. Working alongside other Data Engineers and Data Architects your primary focus will be choosing optimal solutions to design, develop, implement, monitor & maintain key deliverables to support the businesses growing need for data consumption. You will also be responsible for integrating them with the architecture used across the company.
Enabling BI and Insight Analysts the ability to easily interrogate and navigate data in a meaningful way and helping them to create actionable insight for the business.

If you have 'Big Data' experience and able to conduct Enterprise Data modelling for Analytics, and are passionate about data, this could be the perfect role for you.

Key tasks will include:

• Build ingestion pipelines in the cloud based Data Lake to accommodate multiple data sources in line with the Groups Data Architects principles. This will need to be able to support real time streaming and batch ingestion.
• Partner with the Data Scientists, Architects, software developers, and business experts to understand how data needs to be translated, modelled and presented for consumption. Striving for One version of truth.
• Interrogate and model data from multiple sources, translating into an Enterprise useable logical data model fit for Analyst consumption.

Suitable candidates will have:

• Big Data experience (Microsoft Azure, Hadoop, Google BigQuery etc)
• Knowledge of Azure Data Factory, Data Lake, DataBricks and Synapse Analytics
• Working Knowledge of Power BI and providing optimised data sets that can be fully utilised by the product.
• Able to translate business requirements into detailed technical design
• Experience with integration of data from multiple data sources
• Machine Learning, AI and Automation techniques for Data Management & Analytics
• Awareness of Data Management best practice, including Data Lifecycle Management across Core IT & Big Data ecosystems as well as Data Privacy & Security constraints