It's because units up to hours are of a fixed size, but days in most places are only 24h for ~363/365 days of the year, with some being 23h and some being 25h.

(This is ignoring leap seconds, since the trend is to smear those rather than surface them to userspace.)