Projects

Data Projects

Browse my data work tagged with "big-data".

NYC 311 PySpark ETL Pipeline cover image

NYC 311 PySpark ETL Pipeline

Data

Four-stage PySpark ETL pipeline on 40M+ NYC 311 service requests. Identical pandas pipeline for direct comparison. Benchmarked at 100k, 1M, and 10M rows — showing the exact crossover where Spark's parallelism overcomes its initialisation overhead.

No projects found

No projects match the current filter.