Eh... it's a lot more than that. It's the specific packages themselves, and how those packages are built, and how the OS is structured, which versions of libraries are used, how certain common tools are named (think python vs python3). The philosophy of different distros can be radically different. Startup environments can be completely different. Defaults and config in different places. Packages are patched differently. Libraries named differently (think dynamic linking to specific library versions). Init vs. systemd. Wayland vs. XWindows. Some distros even eliminate the distinction between /bin and /sbin. If RHEL, for example, adopted apt, a RHEL pkg likely won't work on Debian. That's why Snap, Flatpak and AppImage are what they are: completely self-contained or statically linked with no dependence on system libraries or image trees.