Beschreibung
Source Group are currently working alongside a leading Financial Institution in Europe as they look to recruit a number of Freelancers to join their Data and Analytics Team.You will join the Data and Analytics team who are responsible for developing, maintaining, and supporting the data and analytics platforms and systems. The team is tackling a range of complex software and data challenges, including big data, data warehousing, advanced analytics, business intelligence and data governance.
Responsibilities include:
- Design, implementation and maintenance of an enterprise installation of Kafka
- Participating in the creation and execution of data governance, including message design, schema validation, and versioning
- Working with developers from other business unit technical teams to assist them in implementing functional solutions using Kafka
- Supporting the establishment of a platform SLA, including defining non-functional requirements
- Working with networks, data center, and infrastructure teams to optimize hardware solutions for the installation of Kafka
Core skills and experience:
- Profound experience in designing, implementing and maintaining Apache Kafka as part for Cloudera Data Platform distribution
- Data pipeline and workflow management tools: Airflow, RunDeck,Nifi etc.
- Stream-processing systems: Kafka, Spark-Streaming, etc.
- Experience in designing and developing high-volume mission critical transactional data integration solutions
- Knowledge of message queuing, stream processing and highly scalable 'big data' data stores
- Building processes supporting data transformation, data structures, metadata, dependency and workload management
- Proven experience of designing middleware message governance, topic subscription and management, including schema validation and version compatibility
- Agile methodologies like Scrum
- Knowledge of Service-oriented architecture and experience in API creation and management technologies (REST, SOAP etc)