My guess is that this is a bit of a throwaway experiment before they actually spend millions on training a larger model based on the technology.
Even if there is no improvement in terms of quality, the speed alone will make it usable for a lot of downstream tasks.
It feels like ChatGPT3.5 moment to me.
My guess is that this is a bit of a throwaway experiment before they actually spend millions on training a larger model based on the technology.