Presentation: From S3 to GPU in One Copy: Rethinking Data Loading for ML Training

Onur Satici: Before I get into anything, I want to show you this video. Not specifically like what's playing in the video, but how we produce this video. Because this is not backed by a regular MP4 file in some main disk, but this is actually a visualization of a file scan. What I mean by this is we have a Vortex file, Vortex being a columnar file format. It has three columns, one per color. Each column would then have a row per frame of this video. Each frame would include around 8 million pixels, because it's a 4K video. Then we then process this data, which is three columns, around 8 million pixels per frame, 60 frames per second, so around 13 gigabits per second, from S3, through the network card, over the CPU, and to the GPU. The GPU is seeing this data this fast, but we want to see what's happening in the GPU, we want to visualize that. We can't downlink that much data to our laptops, so because it's already in the GPU, I'm using the GPU itself to re-encode it into H.264, wrap it into TCP buffers, send it over my SSH tunnel,…