Beschreibung
Tasks+General: Focus on implementation rather than architectural or conceptual design
+Ingestion/Standardization of various source systems such as SAP or other systems (e.g., parking) into the data platform (Focus)
+Build ingestion pipelines batch: Azure Data Factory, stream: Databricks) and processing modules (Python, PySpark)
+Building data products from ingested/standardized data, require data modelling and data processing steps. And eventually Power BI for pure visualization purposes.
+Migrate batch to stream ingestion of already built data pipelines.
+Automation Improvements such as Integration Testing for Data Pipelines in Azure Data Factory, Pipelines for Rebuilding +Standardized Datasets
+Other improvements: Rollout Logging Concept to Data Processing Notebooks
Requirements
+API Design (important: because little seniority in the team in this topic)
+DevOps & Automation (important: because little seniority in the team in this topic)
+Software Engineering Practices such as Clean Code, Design Principles, Refactoring, Security, etc. (important: because little seniority in the team in this topic)
+Python (important)
+Bringing Code into Production (important)
+Distributed processing with Spark (Nice to have)
+Data Modelling techniques (Nice to have)
+Cloud Knowledge
Coding:
+API - Python: FastApi, Pydantic, SQLModel
+Processing: - PySpark - Python - SQL
+Monitoring - Kusto
+Infrastructure as Code - Terraform - Automation Scripts - PowerShell
Miscellaneous
+Start: asap
+Duration: until 31.12.2024+
+Location: ZH und remote, onsite 1 day fixed per week
Have we piqued your interest?
Mr. Maykel Lienhard, Senior IT Recruiter, is looking forward to receiving your dossier at