The system normally runs workloads on local GPUs. We wanted the option to use cloud capacity when the local machines are busy, or when we need to run more work at once.

As I mapped out the flow, I ran into a practical constraint. Some datasets are hundreds of gigabytes, and they still live on local storage.

Compute has to wait for its input

If we choose cloud compute first, the data still needs to get there. By the time the transfer finishes, the capacity we planned around may no longer be available.

That changed the order of the work. I focused on the storage side first.

Prepare, then allocate

I put storage behind adapters, added S3-compatible staging, and built the upload and verification flow. The point was to know that the data was actually ready.

Once the data is staged in the right region, we can turn to finding compute for it. The transfer becomes an explicit part of the workflow, with a result we can check.

The wider system is still in progress. This part was a useful reminder to trace what a workload needs before deciding where it should run.