Bligh’s open boat voyage[1] after the mutiny on the Bounty has them beat at 6400km, but the Shackleton expedition certainly had to deal with much more adverse conditions.

https://en.m.wikipedia.org/wiki/Mutiny_on_the_Bounty#Bligh's...