Job description

Azure Data Engineer

Tunbridge Wells, Kent and Remote (2 days per week in office in Tunbridge Wells

Gerrard White is curently looking a Data Engineer to join a client within the insurance sector with collecting, storing, processing, and analysing huge sets of data in a cloud based data lake. The primary focus will be choosing optimal solutions to design, develop, implement, monitor & maintain key deliverables to support the businesses growing need for data consumption. You will also be responsible for integrating them with the architecture used across the company.

Key tasks will include:

• Build ingestion pipelines in the cloud based Data Lake to accommodate multiple data sources in line with the Groups Data Architects principles. This will need to be able to support real time streaming and batch ingestion.
• Interrogate and model data from multiple sources, translating into an Enterprise useable logical data model fit for Analyst consumption.
• Through data and understanding of business processes, advise on Analytical uses in Insight
• Provide guidance and translation to Analysts on how to interpret and navigate the data for specific use case.
• Help the Data Architect to create an overview of the Data Lineage (from data flows, data transformations inside applications to Analytical output)
• Providing estimates of work received and ratifying these estimates based on experience/facts.
• Provide clear documentation of the business rules embedded in the modelled data ensuring full transparency of data transformations
• Identify and champion process improvements including data content, quality, standardization, automating manual processes and optimizing data delivery. Apply Machine Learning and Automation techniques to processes within Lake helping to detect Quality issues with data.

Suitable Candidate:

• Experience in one of the following: Python, Spark, R , SQL, .net, Java, C++
• Big Data experience (Microsoft Azure, Hadoop, Google BigQuery etc)
• Azure Data Factory, Data Lake, DataBricks and Synapse Analytics
• Working Knowledge of Power BI and providing optimised data sets that can be fully utilised by the product.
• Able to translate business requirements into detailed technical design
• Experience with integration of data from multiple data sources
• Machine Learning, AI and Automation techniques for Data Management & Analytics
• Awareness of Data Management best practice, including Data Lifecycle Management across Core IT & Big Data ecosystems as well as Data Privacy & Security constraints