We've also built and upgraded a few projects on Rails 4.1 recently and it has been very smooth.
On a side note: we avoid ActiveAdmin like the plague for the very reason that you specify here; our experience upgrading it between Rails versions has been terrible. We use our own simpler admin 'scaffolding' instead - https://github.com/SquareMill/generic_admin_controller/tree/...
It depends. For initial work, AA is a quick hack to get something out there fast. It's akin to django admin. Anything more complicated than batch spreadsheet CRUD will benefit from non-AA, custom admin UIs. One thing that we've encountered is we'd like Disqus style comments, but AA's are flat without an ability to delete them. Yeah we had to use a whole raft of crap to get AA working on 4.1rc including pinning sprockets and sass-rails version. Overall, AA has its place but there's never been, nor will there ever be, a perfect tool for all seasons.
On a side note: we avoid ActiveAdmin like the plague for the very reason that you specify here; our experience upgrading it between Rails versions has been terrible. We use our own simpler admin 'scaffolding' instead - https://github.com/SquareMill/generic_admin_controller/tree/...