Rendering is different because the assets are very interdependent and the tooling is very POSIX specific. Rendering a scene (especially with ray tracing) can use thousands of assets from the movie even if they aren't directly on screen. Artists draw and save files in tools like Autodesk Maya and renderers read files from disk and work on them while in memory.
For some assets and tooling FUSE mounted s3 can work but generally FUSE and other userspace mounters I've seen slow down artists and rendering measurably.
Think of rendering assets more like code files and git with trunk based development. You want all of your artists to use the latest assets which are daily being updated. All of the assets should be co-located and you don't want geo-replication because of latency. Even if your artists are located all over the world you'll want them to store the saved assets in one place. Where the rendering happens.
There will be different assets that are hot as the movie progresses. But you're more likely to try to keep the latest version of all assets hot rather than all versions of specific assets hot.
Most studios use HPC style environments. NFS + big compute servers connected with high bandwidth.
Interesting. Is Kubernetes ever used as the orchestrator in HPC workloads, or rather are even containers used there? Also what are some good resources to get better at kubernetes? Currently I am mostly playing with managed k8s like digital ocean's and thinking about transitioning to k3s based bare-metal solution. So, I would really like to learn what's the right way of doing this.
There are some HPC environments that use Kubernetes but they likely use custom schedulers optimized for batch workloads (e.g. https://github.com/volcano-sh/volcano).
"containers" are often used but not always docker containers. HPC environments I've seen will often use container primitives (e.g. cgroups, namespaces).
For some assets and tooling FUSE mounted s3 can work but generally FUSE and other userspace mounters I've seen slow down artists and rendering measurably.
Think of rendering assets more like code files and git with trunk based development. You want all of your artists to use the latest assets which are daily being updated. All of the assets should be co-located and you don't want geo-replication because of latency. Even if your artists are located all over the world you'll want them to store the saved assets in one place. Where the rendering happens.
There will be different assets that are hot as the movie progresses. But you're more likely to try to keep the latest version of all assets hot rather than all versions of specific assets hot.
Most studios use HPC style environments. NFS + big compute servers connected with high bandwidth.