16.10.2026 aktualisiert
100 % verfügbar
Data Engineer | Databricks | Azure | PySpark | ETL/ELT | AI
Aigaleo, Griechenland
Nur Remote
Master degree of Advanced Information Systems, University of Piraeus (October 2022 - July 2024)Skills
Data ArchitectureETLData TransformationData WarehousingDatenbankentwicklungAzure Data LakeAzure Data FactoryApache SparkPySparkAzure Synapse AnalyticsDaten-PipelineDatabricks
Databricks & Apache Spark
Extensive hands-on experience with Databricks for building and optimizing data pipelines, writing PySpark code, managing CI/CD migrations, and working with Delta Tables across medallion architecture layers (Bronze, Silver, Gold).
ETL/ELT Pipeline Development
Design and implementation of robust ETL/ELT pipelines using Azure Data Factory and Databricks, enabling efficient ingestion and transformation of large-scale data from various sources including REST and SOAP APIs.
Python & PySpark
Development of scalable data processing solutions using Python and PySpark, applied in cloud environments such as IBM Cloud and Azure Synapse Analytics for large-scale data transformation workflows.
Azure Data Factory (ADF)
Engineering and optimization of ETL/ELT pipelines in Azure Data Factory, including parameterization and tuning of Azure DevOps pipelines for efficient data workflows.
Azure Synapse Analytics
Development and maintenance of data pipelines in Azure Synapse Analytics, ensuring consistency between development and production environments and supporting efficient data processing and transformation.
CI/CD & Azure DevOps
Daily interaction with Azure DevOps for version control, CI/CD workflows, shared repository collaboration using pull/push requests, rebases, merges, and conflict resolution.
SQL
Writing and optimizing SQL code within Databricks and Azure environments to manage data transformations and ensure data quality across pipelines.
Data Governance & Unity Catalog
Implementation and maintenance of Unity Catalog for centralized governance and data lineage across medallion architecture layers, ensuring reliable and compliant data management.
Qlik Tools (Qlik Replicate & Qlik Compose)
Specialization in ETL processes and migration of small and large tables using Qlik Replicate and Qlik Compose to Delta Lake in Databricks.
IBM Cloud
Design and implementation of cloud data platform infrastructure on IBM Cloud, including scalable data processing pipelines using PySpark and tools such as IBM Cloud Pak and IBM Watson.data.
Extensive hands-on experience with Databricks for building and optimizing data pipelines, writing PySpark code, managing CI/CD migrations, and working with Delta Tables across medallion architecture layers (Bronze, Silver, Gold).
ETL/ELT Pipeline Development
Design and implementation of robust ETL/ELT pipelines using Azure Data Factory and Databricks, enabling efficient ingestion and transformation of large-scale data from various sources including REST and SOAP APIs.
Python & PySpark
Development of scalable data processing solutions using Python and PySpark, applied in cloud environments such as IBM Cloud and Azure Synapse Analytics for large-scale data transformation workflows.
Azure Data Factory (ADF)
Engineering and optimization of ETL/ELT pipelines in Azure Data Factory, including parameterization and tuning of Azure DevOps pipelines for efficient data workflows.
Azure Synapse Analytics
Development and maintenance of data pipelines in Azure Synapse Analytics, ensuring consistency between development and production environments and supporting efficient data processing and transformation.
CI/CD & Azure DevOps
Daily interaction with Azure DevOps for version control, CI/CD workflows, shared repository collaboration using pull/push requests, rebases, merges, and conflict resolution.
SQL
Writing and optimizing SQL code within Databricks and Azure environments to manage data transformations and ensure data quality across pipelines.
Data Governance & Unity Catalog
Implementation and maintenance of Unity Catalog for centralized governance and data lineage across medallion architecture layers, ensuring reliable and compliant data management.
Qlik Tools (Qlik Replicate & Qlik Compose)
Specialization in ETL processes and migration of small and large tables using Qlik Replicate and Qlik Compose to Delta Lake in Databricks.
IBM Cloud
Design and implementation of cloud data platform infrastructure on IBM Cloud, including scalable data processing pipelines using PySpark and tools such as IBM Cloud Pak and IBM Watson.data.
Erfahrung
Berufserfahrung5 - 10 Jahre
Freelancer seit2025
Sprachen
Englishgut
Projekthistorie
Designed and implemented pipelines and codebase on IBM Cloud, establishing foundational infrastructure where none previously existed. Developed robust, scalable data processing pipelines using PySpark to support large-scale data transformation. Took full ownership of the solution architecture and implementation, from initial design through deployment, ensuring reliable and maintainable data workflows within a cloud-based environment. Daily Tools: IBM Cloud Pak, IBM Watson.data.
Working on project migrations, including CI/CD Databricks pipeline migrations from legacy Databricks CLI to the latest version. Daily interaction with Azure DevOps, collaborating on shared repositories using pull/push requests, rebases, merges, and conflict resolution. Daily Tools: Databricks, Databricks CLI, Azure DevOps.
Engineered and optimized ETL/ELT pipelines in Databricks and Azure Data Factory (ADF), enabling efficient ingestion and transformation of large-scale data. Implemented and maintained Unity Catalog for centralized governance and data lineage across medallion architecture layers (Bronze, Silver, Gold). Designed and maintained Slowly Changing Dimensions (SCD) Type 1 and Type 2 tables. Tuned and parameterized Azure DevOps pipelines. Daily Tools: Databricks, Azure Data Factory, Azure DevOps.
Zertifikate
Databricks Generative AI Engineer Associate
Databricks2026
Databricks Certified Data Engineer Professional
Databricks2026
Databricks Associate Developer for Apache Spark
Databricks2026
Microsoft Fabric Data Engineer Associate
Microsoft2025
Qlik Replicate Certification
Qlik2024
Databricks Certified Data Engineer Associate
Databricks2024