Mistaken understanding of how GPUs are used in AI.

GPUs can be used for training but there is no requirement for the model to be executed on GPU. For example, a network can simply be created for incrementing a counter which could be what a "choose the best sort" network ends up doing.

I assume the idea is to do some profiling and throw in some machine learning to select the perfect mix at compile time.

This is doable, but the risk is that looking at the numbers might take longer than just sorting them, even with a sub optimal algorithm.

I guess someone will just have to try this.

Here is what a human did:

It begins with quicksort, it switches to heapsort when the recursion depth exceeds a level based on (the logarithm of) the number of elements being sorted and it switches to insertion sort when the number of elements is below some threshold.

I think machines can do this too.

If you need to sort bit and very often, then I imagine profiling it with your load and optimizing automagically every night could bring benefits.