Job description
Gerrard White is currently seeking a Data Engineer to create, maintain and manage software components, data stores and pipelines that support advanced data analytics for better use and wider consumption of the Company's and its Clients' data. The Data Engineer will work with members of a team to analyse business requirements, design and deliver solutions with a focus on cloud data technologies.
The Data Engineer will be responsible for understanding the Company's and its Clients' requirements so that the appropriate solutions can be de designed and delivered. They are responsible for articulating and documenting those requirements and developing solutions that meet those requirements.
This will include:
- Work with members the Company's and its Clients' teams to assist in the definition and development of processes and technologies that support secure and efficient data ETL, and provide software solutions for advancing the use of data analytics.
- Design, write and test code modules that add value to data for reporting and other uses.
- Integrate applications with other data solutions to improve functionality.
- Develop solutions that connect to a variety of on- and off- premise Company and Client data sources, ensuring data is cleansed, mapped, transformed and otherwise optimised for storage and to meet requirements for business and technical use cases.
Suitable candidates will have;
- Microsoft Certified Data Analyst qualification is desirable
- Azure Certification is desirable
- A minimum of five years' experience in Data Engineering using Python or SQL or other mainstream language in a data analysis environment
- A background in software engineering and software development lifecycle is desirable
- Experience of working with large data sets in an enterprise environment
- Working with business and technical users to understand, document and agree requirements
- Logical and physical model design and implementation
- Create and populate staging areas and data pipelines to perform a range of transformations
- Experience in Azure data components is desirable, including
- Azure Data Lake (Gen 1 and 2)
- Azure Analysis Services
- Azure SQL Data Warehouse (Synapse Analytics)
- Azure Data Factory
- Databricks or similar Spark environment
- Tools and techniques for ensuring code and data quality, security, validation and recovery
- Experience of testing tools and performance monitoring is desirable
- Data reporting and visualisation using Power BI is desirable
- Understanding of the Insurance industry is desirable
Skills
- Strong analytical skills, with the ability to adopt a logical approach to solving problems
- Data exploration, cleansing and preparation
- Expert Python or SQL coding skills and the ability to apply these to data engineering and analytics
- Strong data modelling skills
- Good SQL scripting skills are desirable
- Good time management and organisation skills
- Identification and management of risk
- Ability to create and maintain sets of reference architecture documentation
Knowledge
- Microsoft Azure cloud data architectures
- Azure Databricks / Spark knowledge is desirable
- Microsoft Power BI knowledge is desirable
- Knowledge of Data Warehousing principles is desirable
- Knowledge of a structured yet agile approach to delivery
- Knowledge of analysis, design and modelling techniques
- Knowledge of system and use case modelling and requirements gathering techniques
- Knowledge of Architectural patterns
