DatabricksPython
Benchmarking PySpark shuffle: what the metrics actually tell you
Building a benchmarking utility for shuffle and network transfer metrics in Databricks clusters.
20 Feb 2026 · 7 min read
Tagged
Databricks
3 posts
DatabricksSpark
Production Implementation of Machine Learning models
How we built an end-to-end pipeline using Spark and H2O Sparkling Water to get machine learning models from source control into production with minimal friction.
10 Oct 2015 · 2 min read
DatabricksSpark
How Apache Spark Works (Short Summary)
A concise overview of why Apache Spark was created, how RDDs enable in-memory processing for iterative and interactive workloads, and its key programming abstractions.
8 Aug 2013 · 2 min read