08.10.2026 aktualisiert

**** ******** ****
100 % verfügbar

Data Engineer & Cloud Architect

Berlin, Deutschland
Weltweit
Dipl.-Ing. (M.Sc. equiv.) Theoretical Electrical Engineering, TU Ilmenau
Berlin, Deutschland
Weltweit
Dipl.-Ing. (M.Sc. equiv.) Theoretical Electrical Engineering, TU Ilmenau

Profilanlagen

Christian_Richter_CV_DE.pdf
Christian_Richter_CV_EN.pdf

Über mich

Freelance data engineer and cloud architect based in Berlin, freelancing since 2010 with 25+ projects. Data platforms, cloud infrastructure (AWS, GCP, Kubernetes) and AI integration (LLM, MCP). Typical engagement: design the platform, build it with the client team and hand it over.

Skills

Data MeshArgo CDGitOpsHelmdbtVertex AIDelta LakeSAP CloudJavaKünstliche IntelligenzApache AirflowAmazon Web ServicesAmazon S3Data AnalysisChemieAutomobilindustrieMicrosoft AzureTelekommunikationCloud ComputingApache ImpalaContinuous DeliveryContinuous IntegrationData ArchitectureData GovernanceDateninfrastrukturDevOpsDisaster RecoveryGitHubApache HiveSpring FrameworkPythonPostgreSQLLiquibaseMarktforschungMongoDBOpenShiftOpenStackRedisMicrosoft Power BIRequirements EngineeringEinzelhandelSQLStreamingGoogle CloudLarge Language ModelsSpring BootKonsumgüterGenerative AIAWS LambdaInfrastructure as Code (IaC)JupyterGitCloudformationKotlinpandasData LakePySparkKubernetesAvroAWS GlueEchtzeitdatenDSGVOTerraformAmazon Elastic Mapreduce (EMR)DockerAmazon RedshiftDatabricksMicroservices
Data Engineer & Cloud Architect
Data Platforms · Cloud Infrastructure (AWS, GCP, Kubernetes) · AI Integration (LLM, MCP)

Independent data engineer and cloud architect with more than 25 years of experience in software and data engineering, freelancing since 2010. Delivered 25+ projects for clients in automotive, chemicals, retail, market research, media and telecommunications, taking data platforms from architecture and requirements through to production operation. Typical engagement: design the platform, build it with the client team and hand it over. Data models, integration patterns and governance are considered from the start.

Focus areas
  • Architecture and delivery of data warehouses, data lakes and lakehouse platforms (AWS, GCP, Kubernetes), from concept to production
  • Design and operation of ETL/ELT, streaming and ML pipelines (Spark, Kafka, Airflow, dbt) at terabyte scale
  • AI-assisted data processing: document extraction, semantic search and workflow automation with LLM tool integration (MCP), including a self-built MCP server in production
  • Cloud-native infrastructure as code (Terraform, Helm, GitOps), including high-availability and disaster-recovery concepts
  • GDPR-compliant data management including data residency, data modeling, and requirements and process analysis
  • Technical leadership of distributed teams, architecture reviews and knowledge transfer

Core skills
  • Core stack: Python, SQL, Spark, Kafka, Airflow, dbt, Terraform, Kubernetes, AWS, GCP, MCP
  • Data engineering: Spark/PySpark, Databricks, Delta Lake, Hudi, dbt, Airflow, AWS Glue, Apache NiFi, Pandas, Polars, Power BI
  • Streaming: Kafka, Kafka Connect, Kafka Streams, Strimzi, Amazon Kinesis, Apache Pekko
  • Databases: PostgreSQL, OpenSearch, MongoDB, DynamoDB, DuckDB, TimescaleDB, MySQL/MariaDB, Oracle, Hive, Impala
  • Cloud: AWS (S3, EMR, Glue, Athena, Lake Formation, Lambda, Kinesis, RDS, IAM, CloudFormation, CDK), Google Cloud (GKE, Vertex AI, Document AI), OpenStack
  • Containers: Kubernetes, OpenShift, Docker, Helm, Kustomize, Argo CD, Prometheus, Grafana
  • IaC & DevOps: Terraform, Terragrunt, CDKTF, Ansible, GitOps, GitLab CI, GitHub, Jenkins, Open Policy Agent
  • AI & LLM: Vertex AI, Document AI, Model Context Protocol (MCP), llama.cpp, embeddings and vector search, Claude Code
  • Programming: Python, Java, Kotlin, SQL, Scala, Bash; Spring Boot, Flask, Liquibase, Flyway

Additional information
  • Own product: Shibui Finance (since Sep 2025), public MCP server that makes 60+ years of US stock market data queryable in plain English from Claude; listed in the official MCP Registry; semantic layer and tested SQL query patterns, 3–5% error rate
  • Co-inventor of European patent EP1377924 B1 (Fraunhofer IDMT)
  • Open source: caddy-jsonrpc-matcher (Caddy module for rate limiting of MCP tool calls); Kafka and Redis export modules for Schedoscope (Otto Group)
  • Languages: German (native), English (fluent)

Work model: remote or on-site, hybrid preferred (based in Berlin)

Erfahrung

Berufserfahrungmehr als 10 Jahre
Freelancer seit2010

Sprachen

DeutschMutterspracheEnglischverhandlungssicher

Projekthistorie

Data Architect

Energy utility (confidential)

Energie, Wasser und Umwelt

Cloud-native data platform on Kubernetes
  • Create architecture diagrams and documentation (backup/restore, parameterization/validation)
  • Define and implement standards for Kubernetes-native services
  • Document architecture decisions as ADRs
  • Design and document standardized ETL pipelines
  • Evaluate high-availability deployment scenarios for PostgreSQL and Kafka brokers against system requirements
  • Design disaster-recovery processes for data infrastructure components
Technologies: Kubernetes, Terraform, Helm, Strimzi, Kafka, PostgreSQL Operator, MongoDB, Argo CD, GitLab, Apicurio, Velero, GitHub Copilot

Data Engineer

Rapid Innovation GmbH

Transport und Logistik

10-50 Mitarbeiter

AI-assisted data extraction and ETL for an SCM platform
  • Helped build the master data set for a supply chain management application from 8 data sources
  • Designed and implemented ~60 ETL pipelines on Google Cloud
  • Designed and implemented LLM-based workflows for document data extraction and analysis (Document AI, Vertex AI), processing ~120k documents
  • Analyzed and combined the data sources to improve data quality and coverage
  • Built a dashboard to monitor data quality
  • Set up CI/CD pipelines, monitoring and alerting for production ETL processes
Technologies: Google Cloud, GKE, Document AI, Vertex AI, Kubernetes, Terraform, Kafka, Kafka Connect, Kotlin, Pekko, dbt, SQL, GitHub, Claude Code

Data Architect & Engineer

Covestro AG

Industrie und Maschinenbau

>10.000 Mitarbeiter

Data warehouse for research and production data
  • Advised on the data model, ETL pipelines and cloud architecture
  • Designed and implemented multiple ETL pipelines providing access to research and production data
  • Integrated SAP PLM to extract and process laboratory research data
  • Integrated the manufacturing execution system (MES) for data extraction and processing
  • Built an OpenSearch index for semantic search (vector search / k-NN)
  • Transferred knowledge to internal staff
Technologies: AWS, CloudFormation, SAP PLM, SAP MES, OpenSearch, Docker, Java, Spring, Flyway, SQL

Data Architect & Engineer

GfK SE

Internet und Informationstechnologie

5000-10.000 Mitarbeiter

Migration of an on-premises data warehouse to AWS
  • Evaluated Databricks/Delta Lake against AWS managed services/Hudi as the foundation of a data lake
  • Designed and built the data warehouse on AWS Lake Formation and the Glue Data Catalog (~1.5 TB data lake)
  • Migrated 100+ ETL jobs (Spark, Hive, Pig) from Cloudera to AWS Glue and Amazon EMR, with shorter runtimes through elastic scaling
  • Replaced Oozie with Airflow for workflow orchestration
  • Designed and implemented several GitLab CI/CD pipelines
  • Trained internal staff and transferred knowledge
Technologies: AWS (Glue, EMR, Lake Formation), Airflow, Databricks, Terraform, Python, Java, SQL, Cloudera, Hadoop, Hive, Pig, Oozie, GitLab

Data Architect & Lead Engineer

RTL Group

Medien und Verlage

1000-5000 Mitarbeiter

Cloud-based data warehouse for user data analytics
  • Designed and built the cloud infrastructure for the data warehouse
  • Introduced a decentralized data mesh architecture, including governance guidelines
  • Documented architecture decisions as ADRs and presented them to the architecture board
  • Integrated Airflow as the workflow engine and the Spark on Kubernetes operator as the ETL runtime
  • Built and led a 6-person team developing ETL processes for 4 data sources; implemented several ETL jobs in PySpark
  • Implemented an event streaming pipeline for real-time analytics, including a real-time Braze integration with under 1 second latency
  • Prepared data for downstream analysis in Power BI and advised on the overall data architecture
Technologies: AWS, Kinesis, Kubernetes, Istio, Docker, Kafka, Kafka Streams, Braze, Airflow, Python, PySpark, Pandas, Avro, Terraform, Kustomize, Power BI

Founder & Managing Director

AltusInsight GmbH

Internet und Informationstechnologie

< 10 Mitarbeiter

Own company: product design and implementation
  • Founded and ran the company for eight years; responsible for product design and implementation
  • Built LambdaNow, a cloud-based platform for fully automated deployment of Apache Hadoop clusters on AWS and OpenStack
Technologies: Apache Hadoop, AWS, OpenStack

Data Engineer

Volkswagen AG

Automobil und Fahrzeugbau

>10.000 Mitarbeiter

Cloud-based data warehouse for vehicle data; evaluation of a data science environment
  • Extended a prototype for large-scale data processing and brought it into production, improving runtime by a factor of 10
  • Built and launched a CI/CD pipeline; set up project structure and release management
  • Designed, implemented and deployed a backend API, including a Helm chart
  • Built ETL pipelines for vehicle data validation, ingestion and indexing of millions of documents in OpenSearch
  • Implemented feature extraction from vehicle data
  • Evaluated Kubeflow and Databricks/Delta Lake
Technologies: AWS (Lambda, IAM), Kubernetes, Kubeflow, Docker, Helm, Databricks, Terraform, Python, OpenSearch, Logstash, Kibana

Cloud & Data Engineer

D. Swarovski KG

Konsumgüter und Handel

>10.000 Mitarbeiter

Sensor data processing platform and extension of a data science environment
  • Advised on the design and tooling of a Kubernetes-based infrastructure for sensor data processing
  • Designed and built the infrastructure on Kubernetes (Kafka cluster, Spark, ZooKeeper)
  • Designed and built the ETL pipeline for data ingestion
  • Designed and built a CI/CD pipeline with Bamboo and Kubernetes
Technologies: AWS, Kubernetes, Helm, CloudFormation, Kafka, Kafka Connect, PySpark, InfluxDB, Java, Python, Docker, Bamboo

Data Architect

aixigo AG

Banken und Finanzdienstleistungen

50-250 Mitarbeiter

Design and implementation of a microservice architecture
  • Aligned and coordinated several teams on technology choices
  • Co-designed core elements: (de)serializers, data pipeline design, error and message handling, database design
  • Introduced Kafka as the central message bus for microservices
  • Introduced Liquibase for database schema management
  • Provided functional and technical ownership of one microservice
Technologies: Java, Kafka, Docker, Liquibase, microservices

Data Architect & Engineer

D. Swarovski KG

Konsumgüter und Handel

>10.000 Mitarbeiter

Cloud-based data warehouse and data science environment
  • Designed a dynamically scalable data warehouse and its data model and storage layout
  • Implemented the infrastructure as code with CloudFormation and infrastructure components on Kubernetes
  • Built ETL pipelines for data ingestion
Technologies: AWS, CloudFormation, Kubernetes, Docker, Spark, R, Apache NiFi, Python, Jupyter

Lead Data Engineer

GfK SE

Internet und Informationstechnologie

5000-10.000 Mitarbeiter

Big data warehouse on AWS
  • Led the project technically and designed the AWS cloud infrastructure
  • Implemented data pipelines, the data warehouse and workflow management
  • Handled data preparation and process management
Technologies: AWS, Spark, SparkR, Cloudera, Hadoop, Hive, Avro, Python, R, Jupyter, Bamboo, Terraform

Requirements Engineer

Open Grid Europe GmbH

Energie, Wasser und Umwelt

1000-5000 Mitarbeiter

Evaluation of big data vendors
  • Gathered and documented technical and business requirements for building and operating a Hadoop-based data warehouse
  • Requested vendor proposals and prepared the information for decision-making
  • Implemented a data ingestion prototype
Technologies: Hortonworks, Cloudera, SAP Cloud, Apache NiFi, Parquet, AWS, Microsoft Azure

Data Engineer

Helix Leisure Pte Ltd (Singapore)

Konsumgüter und Handel

250-500 Mitarbeiter

Architecture review and real-time streaming layer
  • Reviewed the existing architecture and data model; ran a workshop on data management and Lambda architecture
  • Designed and implemented a real-time layer with Spark Streaming and integrated it with the batch layer
Technologies: Hadoop, Spark, Scala, Protocol Buffers, MapReduce, JCascalog, Redshift, AWS, CloudFormation

Data Engineer (Trainer)

University Hospital Basel

Öffentlicher Dienst

5000-10.000 Mitarbeiter

Workshop on big data technologies
  • Delivered a three-day workshop introducing the big data/Hadoop ecosystem, with hands-on exercises on AWS
Technologies: Hadoop, Spark, MapReduce, Hive, Pig, ORC, Avro, R, AWS

DevOps Engineer

GfK SE

Internet und Informationstechnologie

5000-10.000 Mitarbeiter

Continuous delivery pipeline for a data-driven application on AWS
  • Planned and built a big data infrastructure on AWS and a continuous deployment pipeline
  • Led the client's internal team technically
Technologies: AWS, Hadoop, Spark, Terraform, Vagrant, Bamboo, Git, InfluxDB

Data Engineer

Otto GmbH & Co. KG

Konsumgüter und Handel

>10.000 Mitarbeiter

ETL pipelines for a Hadoop-based data warehouse
  • Built a Hive export module and Kafka and Redis export modules for the open-source project Schedoscope
  • Implemented an algorithm for clickstream analysis
Technologies: Hadoop, Hive, HCatalog, Spark, Kafka, Redis, Avro, Scala, Schedoscope

Data Engineer

RadioOpt GmbH

Internet und Informationstechnologie

10-50 Mitarbeiter

Big data warehouse for OLAP workloads
  • Planned and built the cluster infrastructure, benchmarked input formats for performance, and prepared and ran load tests
Technologies: Hadoop, Impala, Hive, AWS

Cloud & Data Engineer

GfK SE

Internet und Informationstechnologie

5000-10.000 Mitarbeiter

Big data architecture for telecommunications data
  • Planned and built the network (AWS VPC) and a 100 TB Hadoop cluster, including deployment and monitoring
  • Implemented a data ingestion framework storing ~300 GB of data per day
Technologies: Cloudera, Hadoop, Hive, Flume, Java, Spring, Puppet, Ubuntu, AWS

Lead Data Engineer

Technicolor SA

Konsumgüter und Handel

>10.000 Mitarbeiter

Batch and real-time processing of machine data
  • Implemented a distributed, fault-tolerant, high-throughput messaging and analytics system for machine data (Lambda architecture)
  • Planned the deployment environment, evaluated data ingestion and processing technologies, and led the team technically
Technologies: Hadoop, Samza, Spark, Kafka, Java, AWS, CloudFormation

Lead Data Engineer

Ubisoft / Blue Byte GmbH

Medien und Verlage

>10.000 Mitarbeiter

Hadoop-based data warehouse for game analytics
  • Planned and built the data warehouse, including a distributed, fault-tolerant, high-throughput messaging system
  • Led and coordinated a distributed team across Germany, China and Canada
Technologies: Hadoop, MapReduce, Kafka, Hive, Java, Linux

DevOps Engineer

Deutsche Telekom AG

Telekommunikation

>10.000 Mitarbeiter

Big data infrastructure in virtualized environments
  • Built a deployment process for on-demand Hadoop clusters on OpenStack
  • Prototyped several algorithms in MapReduce
Technologies: Hadoop, OpenStack, Chef, Java, Linux

Data Engineer

exactag GmbH

Marketing, PR und Design

10-50 Mitarbeiter

Design and rollout of a Hadoop cluster
  • Advised on the cluster design, selected the hardware, set up the deployment process and rolled out the cluster
  • Ported existing statistics routines to MapReduce
Technologies: Cloudera, Hadoop, Hive, Pig, Python, Java, Maven, Puppet, Debian

Data Engineer

etracker GmbH

Marketing, PR und Design

10-50 Mitarbeiter

Re-implementation of an analytics tool as a MapReduce application
  • Ported an existing implementation to MapReduce using the Hadoop Streaming API
  • Installed and configured a Hadoop cluster with monitoring and set up the deployment process
Technologies: Cloudera, Hadoop, HBase, Java, Maven, Ganglia, Chef, PHP, Debian

Backend Developer

Aupeo GmbH

Internet und Informationstechnologie

< 10 Mitarbeiter

Payment provider integration
  • Integrated a payment provider into the existing backend
  • Mapped, converted and imported data, including text matching against the existing database
Technologies: Ruby on Rails, OAuth, MySQL, Git, Debian

Backend Developer

OpenLimit SignCubes GmbH

Internet und Informationstechnologie

10-50 Mitarbeiter

Signature component for an email client
  • Integrated a digital signature component into KMail, including debug environment setup and testing
Technologies: C++, Qt, KDE, Ubuntu

Backend Developer

etracker GmbH

Marketing, PR und Design

10-50 Mitarbeiter

Refactoring and production rollout of a C++ analytics tool
  • Set up a build environment for C++ projects
  • Refactored the prototype
  • Adapted and extended the software for production (logging, error handling, unit tests)
  • Set up a deployment process
  • Set up a build server (continuous integration)
Technologies: C++, MySQL C/C++ API, Doxygen, Hudson, Ubuntu/Debian Linux

Portfolio

Kontaktanfrage

Einloggen & anfragen.

Das Kontaktformular ist nur für eingeloggte Nutzer verfügbar.

RegistrierenAnmelden