Data Lakehouse with Azure Databricks
Energieversorgungsunternehmen
(Energie, Wasser und Umwelt)
Schlagwörter
Microsoft Azure
Bash Shell
Information Engineering
Devops
Python
Windows Powershell
SQL
Pyspark
Terraform
Docker
Skills
Bash Scripting, Data Lakehouse, Data Mesh, Delta Lake, Docker, Microsoft Azure, Microsoft Azure DevOps, Packer, PowerShell, PySpark, Python, SQL, Terraform
Projekthistorie
07/2023
-
bis jetzt
Conception and realization of Data Lakehouse with Azure Databricks and Power BI
- Configuration of Databricks workspace
- Setup of CI/CD pipelines for Databricks
- Realization of ETL processes with SCD2
- Setup Auto Loader for event based triggering
- Data exploration to design and validate the data model
- Conception and realization of Delta Live Tables
- Realization of Power BI datasets and reports
05/2023
-
12/2023
Data Lakehouse with Azure Databricks
Automobil
(Automobil und Fahrzeugbau)
Conception and realization of Data Lakehouse with Azure Databricks and Power BI
- Business requirements engineering to validate the scope and define the KPIs
- Data exploration to design and validate the data model
- Support for offsite development team
02/2023
-
09/2023
Data Lakehouse with Azure Synapse
IT-Consulting
(Internet und Informationstechnologie)
Realization of Data Lakehouse with Azure Synapse and Power BI
- Setup an Azure Synapse landscape as Data Lakehouse
- Setup CI/CD pipelines for Azure Synapse (develop and production)
- Realization of ETL processes with SCD2
- Conception and realization of SQL views
- Realization of Power BI datasets and reports
03/2019
-
04/2023
Advanced Analytics Platform on Azure
Digitaldienstleister für VNB und MSB
(Energie, Wasser und Umwelt)
- Set-up and operation of a multi-client and multi-stage advanced analytics platform on Azure
- Azure Databricks & Azure ML
- Azure App Services
- Azure Data Factory & Azure Synapse
- Azure Application Gateway
- Azure SQL Database & Azure Storage Account
- Set-up and operation of CI/CD pipelines for DevOps and ML Ops on Azure DevOps
- Code security with bandit, safety and SonarCube within build pipeline
- CI/CD flow for Azure Data Factory & Azure Synapse
- Infrastructure as Code with Terraform
07/2022
-
01/2023
Data Lakehouse with Azure Synapse
Digitaldienstleister für VNB und MSB
(Energie, Wasser und Umwelt)
Proof of concept and realization for Data Lakehouse with Delta Lake and Azure Synapse
- Data wrangling with Data Flow & Python notebook
- Serverless SQL pool as source for Power BI datasets
- Delta load processes with Delta tables
01/2020
-
12/2022
ETL Processes with Azure Data Factory & Azure Synapse
Digitaldienstleister für VNB und MSB
(Energie, Wasser und Umwelt)
Conception and realization of ETL processes with Azure Data Factory and Synapse
- Data wrangling with Data Flow & Python notebook
- Self-hosted integration runtime on Windows VMs
- Serverless SQL pool as source for Power BI datasets
Zertifikate
Azure Administrator Associate
Microsoft
2022
Reisebereitschaft
Nur Remote verfügbar
Sonstige Angaben
My focus is on Data Infrastructure Engineering and Data Engineering.
My portfolio ranges from Cloud Infrastructure (Azure, Terraform) to CI/CD Pipelines and Orchestration to Data Lakehouse (Databricks, Synapse, Python, SQL).
exali IT-Haftpflicht-Siegel (Sondertarif für Freelancermap-Mitglieder)
Das original exali IT-Haftpflicht-Siegel bestätigt dem Auftraggeber, dass die betreffende Person oder Firma eine aktuell gültige branchenspezifische Berufs- bzw. Betriebshaftpflichtversicherung abgeschlossen hat. Diese Versicherung wurde zum Sondertarif für Freelancermap-Mitglieder abgeschlossen.
Versicherungsbeginn:
01.02.2023
Versicherungsende:
01.02.2026
