Scaling computation

Hamilton enables a variety of tools for allowing you to scale your data processing by integrating with third-party libraries.

Specifically, we have four examples that show how to scale Hamilton both by parallelizing transformations (ray and dask) and running on larger, distributed datasets (pandas on spark, pyspark map UDFs).

1. Integrating hamilton with pandas on spark.

2. Integrating hamilton with ray.

3. Integrating hamilton with dask.

4. Integrating hamilton with pyspark.