Data Engineer | ETL/ELT | Python, Stripe, Xero, PostgreSQL | Financial

Python | SQL | Apache Spark | Kafka | Airflow | dbt | Snowflake | BigQuery | AWS | GCP | Azure | ETL/ELT | Web Scraping | Web Crawling

? Senior Data Engineer

✅ 8+ Years of Experience

✅ Data Platforms & Pipelines

✅ ETL/ELT & Data Architecture

✅ Cloud Data Warehouses & Data Lakes

✅ Batch & Real-Time Processing

✅ Web Scraping & Web Crawling

✅ Data Modeling & Analytics

✅ Production-Ready, Scalable Systems

Hi, I'm a Senior Data Engineer with 8+ years of experience building reliable data platforms, pipelines, and automation systems.

I help businesses collect data from APIs, databases, SaaS platforms, and websites, then clean, transform, store, and deliver it in a useful format.

I work across the full data lifecycle, from extraction and ingestion to transformation, orchestration, validation, monitoring, and reporting.

My focus is always on building systems that are reliable, maintainable, and ready to scale.

?️ Data Platforms & Architecture

I design and build data warehouses, data lakes, lakehouses, and hybrid data platforms using technologies such as Snowflake, BigQuery, Redshift, Azure Synapse, PostgreSQL, AWS, GCP, and Azure.

⚙️ ETL/ELT & Data Pipelines

I build and improve production data pipelines using Python, SQL, Airflow, dbt, Spark, AWS Glue, Prefect, and Dagster.

This includes incremental loading, transformation, data validation, error handling, retries, monitoring, orchestration, and performance optimisation.

?️ Web Scraping & Crawling

I build scalable scraping and crawling systems for e-commerce, directories, SaaS platforms, marketplaces, and other public web sources.

I work with BeautifulSoup, Requests, Scrapy, Selenium, Playwright, Puppeteer, Axios, Cheerio, and Regex, including dynamic websites and browser automation.

Data can be delivered as CSV, JSON, Excel, SQL, APIs, or directly into your database/data pipeline.

⚡ Real-Time & Large-Scale Data

For high-volume or real-time workloads, I work with Kafka, Kinesis, Spark Streaming, Flink, Hadoop, and Hive.

☁️ Cloud & DevOps

I work across AWS, GCP, and Azure, including cloud warehouses, object storage, data lakes, serverless processing, CI/CD, and infrastructure automation.

I also use Docker, Kubernetes, Terraform, and GitHub Actions.

? Data Modeling & Analytics

I design fact/dimension models, star and snowflake schemas, normalized and denormalized structures, and analytics-ready datasets.

I also work with Looker, Tableau, Power BI, and Superset.

? Technical Stack

Languages: Python, SQL, Java, Scala, JavaScript/Node.js, Bash

Data: Spark, Kafka, Airflow, dbt, Flink, Hadoop, Hive

Warehouses/DBs: Snowflake, BigQuery, Redshift, Synapse, PostgreSQL, MySQL, Oracle, MongoDB, Cassandra, Redis

Cloud: AWS, GCP, Azure

Scraping: Scrapy, Playwright, Selenium, Puppeteer, BeautifulSoup, Requests

DevOps: Docker, Kubernetes, Terraform, GitHub Actions

Storage: S3, Parquet, Avro, Delta Lake, Iceberg, Hudi