I like MacPorts over Homebrew because MacPorts lives in its own directory. Homebrew can hose your system binaries and that was enough for me to never use it.
Homebrew, like, for example, CPAN, runs reasonably arbitrary code at package-install time. Much of it is sandboxed, or reviewed by the maintainers, but scripts/configure programs/etc. can spray files, or remove files, anywhere they want on your system. The same is true with ill-behaved "configure; make; make install" deployment systems.
This is contrasted with something like dnf/yum, which has registries of assets that are supposed to be installed. I'm not positive, but I suspect something like that would be a way higher maintenance burden for OSX (for maintainers of the package management system and packagers themselves, many of whom have shown little to no interest in packaging for OSX, hence Homebrew's patch system).
It also lives primarily in its own directory, unless you're installing casks (which usually "do exactly what the installer would do if I downloaded and installed it myself") or packages with special cases that manage external-to-homebrew things. For packagers, the API for "use Homebrew-managed/blessed directories for e.g. --prefix configure arguments" is pretty good, but people (and bad, old build scripts) can and do still mess around outside of where they're supposed to. "Hosing your system binaries" is a pretty darn rare occurrence unless you're explicitly configuring brew to put things where it wouldn't normally.
