Posts by Nathan Ballou

Process Hundreds of GB of Data in the Cloud with Polars

Local machines can struggle to process large datasets due to memory and network limitations. Coiled Functions provide a cloud-based solution that allows for efficient and cost-effective handling of such extensive datasets, overcoming the constraints of local hardware for complex data processing tasks. Incorporating libraries like Polars can further enhance this approach, leveraging optimized computation capabilities to process data more quickly and efficiently.

Code snippet of using the coiled.function decorator to run a query with Polars on a large VM in the cloud

Read more ...