Beschreibung
The IT team enables data scientists in various business functions by providing required tools, technology and data.
The team builds a solution, which on one hand allows data scientists to analyse data and build advanced analytics models (Analytics Lab) and on the other hand to operationalize data pipelines to execute data sourcing, data integration, data enrichment with analytical results and integration with the downstream systems in an automatic manner (Analytics Factory).
Key Responsibilities:
The role of Big Data Engineer for this contract will primarily focus on data sourcing and data integration tasks.
Here is an overview of the role's main responsibilities:
- Collaborate closely with data scientists in the business teams to understand data and functional requirements.
- Design and build data pipelines to ingest, integrate, standardise, clean and publish data.
- Integrate analytical models developed by the data scientists into end-to-end data pipelines
The department:
The Analytics IT team consists of five people with another 3-5 IT colleagues from other teams involved into the day-to-day activities of this function.
The role will support at least 1-2 business teams with 2-4 data scientists in each.
The team works in an agile way with the colleagues closely collaborating with each other. There is a good mix of skills and personalities within the Analytics IT team as well as within the teams of data scientists. Nice, supportive and stimulating environment to work in.
Essentials Skills and Qualifications:
- Demonstrated work experience as a Data Engineer using Hadoop Big Data technologies.
- Understanding of Datamodelling and creating data structures on Hadoop.
- Experience with building data pipelines on Hadoop, ideally Cloudera distribution, using Spark, Scala, Python and similar languages.
- Very good knowledge of SQL, Impala and Hive databases.
- Software life cycle experience, ie build, deployment, testing, release and maintenance.
Candidate Value Proposition:
This role is an opportunity for the candidates to work on a relatively new analytics team using modern technologies and supporting real use cases for the stakeholders.
In case of mutual interest of a longer-term relationship, there may be an opportunity for internalisation after the initial contract period.