Interesting article. So is the “unfriendliness” of Linux kernel developers towards ZFS that is mentioned strictly due to licensing issues or are there other points of contention?
In the context of Linus stance that the kernel must never, ever break userland, I find it strange that he has not slapped the change of a function to GPL-only callers as incorrect and reverted it.
You have a policy that you can't break users' programs, but borking their whole file-system is fine? Keep feeding end-users fears of open source petty feuds making people miserable. That works.
Because... That's not userspace, that's kernelspace. Linux's policy on kernelspace API breakage is famously, "we do what we like, and if you're not in-kernel, then you get to keep both pieces when it breaks".
If ZFS wanted to live in userspace (and get Linux's userspace API guarantees) then they could have made a FUSE filesystem instead.
ZFS-FUSE existed before ZoL did. The reason they moved to their SPL-based solution is because FUSE didn't want to make the required changes to support high performance file systems.
SPL now handles the guarantee to separate ZFS's userland components from Linux's constant internal kernelspace API churn. ZFS, internally, doesn't really know what a Linux is beyond a few little bits here and there.
> ZFS-FUSE existed before ZoL did. The reason they moved to their SPL-based solution is because FUSE didn't want to make the required changes to support high performance file systems.
These changes were made years later, but the developer of ZFS-FUSE disappeared, so they were never used.
The FUSE v3 API further builds on that, and today, it's definitely possible to build a high-performance filesystem using FUSE. In fact, there are many examples of this in the wild: GlusterFS, Ceph, NTFS-3G, and so on.
Today, there's absolutely _no_ reason ZFS would not sensibly work as high performance filesystem entirely from userspace. In fact, it still remains an open issue and a desirable target for ZoL: https://github.com/zfsonlinux/zfs/issues/8
Glad you brought this up. It feels, at least to me, like FUSE is only for...hmm...gimmick or science-fare projects. Maybe something useful on your development machine. Indeed, at a previous employer, several problems were addressed using a handful of FUSE filesystems, quite elegantly. Of course the caveat was "this ain't prod, duh, it's FUSE". But this doesn't NEED to be the case. Tangentially, microkernels pop up here often, and arguable, moving some filesystems to user space makes sense in a related vein.
I'll admit, I'm all talk and if it really mattered, pull requests are probably welcome.
End-user dont'care about splitting hairs. Iwas using Linux in the mid-90's and that kind of stance is what turned me off. The sound card would brek, the networking would break, configuring the modem would be hell after each upgrade.
The attitude that you must not break LibreOffice is not okey but breaking the file-system okay 'because it's a driver!' is just untenable.
Making people's computer fail will turn users away. It's not just a Linux thing. When Windows breaks stuff, all hell break loose on the Internet. The problem with OSS, is that its maintainers have no direct, immediate monetary incentive to amend. The long-term loss of confidence is hard to measure, but it's real.
Using internal kernel APIs is a pretty long way to being userland. Internal APIs are guaranteed to be broken, and for in-tree modules, the breakage is fixed by those, who break it. External modules have to do fixes themselves, thus motivating them to become in-tree.
Well you can read the thread on the mailing list but to put it bluntly - because they wanted to stop ZoL from working. They view non-GPL kernel modules as a violation of the GPL.
"My tolerance for ZFS is pretty non-existant" Greg Kroah-Hartman
"please switch to FreeBSD instead of advocating to violate the copyright and licensing rule on my and others work." Christoph Hellwig
For context, Hellwig is the guy that tried to sue vmware in Germany and had the case dismissed because the evidence submitted summed up to references of stuff on the internet and basically copy and pasted git output. It didn't even include details on which lines of code were allegedly used by vmware, or details on authorship in any of the code comparisons that were present.
Which is a shame, because it would have been nice to see if the shim usage pattern is actually a violation of the GPL - a lot of us would like a clear ruling there. I'm sympathetic to his viewpoint, but that whole ordeal was a bit of a "wtf?" because of how it was handled, and I can't imagine him being sympathetic to anything else in an even somewhat similar situation.
users demand it. If the kernel devs pushed too hard legally a lot of users would switch to BSD (probably freebsd, but there are other good candidates). Linux is slightly better for desktop users, but if you cannot use your graphics hardware for legal reasons BSD will still work and is the lesser evil.
So basically NVIDIA gets a monopoly on the ability to release non-GPL linux kernel modules so that Linux can have a better reputation among desktop users (which is only a small fraction of Linux use in the first place)?
I don't get it -- do they care more about what the users want, or about enforcing copyleft? If it's the latter, then they should be happy if those users who don't care about the GPL move to BSD. If it's the former then they should stop playing games and implement a fair policy for all non-GPL module authors.
People not associated with the leadership of the project shouldn't be making decisions about what licensing models are acceptable or not in the first place.
Because the kernel dev decided that interfacing with the kernel through the API would change the kernel too much to not be in compliance with the GPL if your license isnt.
Which is proooobably a lie. It's pretty hard to imagine how such a generic function as enabling and disabling the fpu could mean that you're combining the two works. But there's not much process to call someone on technically inappropriate use of _GPL. It's a mostly-political tool pretending to be a technical tool.
BTRFS is being positioned as an alternative to ZFS (amongst others) without the licensing issues (and less rational concerns like the current/past politics of it all), so perhaps there is a "why do we X it when Y is close" with a bit of extra NIH syndrome mixed in?
Oracle discontinued BTRFS development when they bought Sun, merged the ZFS and BTRFS team, and then fired anyone on the BTRFS team that didn't have ZFS experience.
BTRFS is basically on maintenance mode, and no one at Redhat/IBM or Ubuntu has any interest in keeping it alive. Anyone that requires HA services will not be using BTRFS.
Redhat ships with XFS as the default file system, Ubuntu ships with Ext4 as default with heavy install-time ZFS support for enterprise storage, and BTRFS is no longer being spoken of by anyone as a potential Ext5 candidate.
90% of this is wrong. Oracle bought Sun in 2009, and continued to make substantial contributions for years after that, and they still have developers actively contributing to Btrfs. The idea they merged the ZFS and Btrfs teams is absurd, the two file systems are nothing alike either in terms of design, on disk format, or code base.
Btrfs is on maintenance mode? Based on what evidence? There's 1k-2k commits per kernel cycle. There are dozens of developers actively working on it. Facebook is certainly using it in production for high availability in concert with Gluster FS, they're also using Btrfs almost exclusively with Tupperware, their containerization service.
Red Hat has quite a lot of device mapper, LVM, and XFS developers so it makes sense for them to work on storage solutions that continue to build on that. And Red Hat hasn't been significantly contributing to Btrfs for many years now, so their lack of interest isn't new and hasn't affected Btrfs development.
>The idea they merged the ZFS and Btrfs teams is absurd, the two file systems are nothing alike either in terms of design, on disk format, or code base.
You've never seen the same team working on two very separate code bases or products?
I have no idea if the btrfs and zfs teams at Oracle were merged, but I don't know that "they are two separate products" is actually a real argument that they weren't. Product teams working on separate things get merged all the time.
I've seen a tiny handful of developers juggle more than one file system at a time. They're that complicated. I'm familiar with most of the Btrfs developers and can't think of a single one who works on ZFS; the most active developers when asked about it quite a few years ago on the Btrfs development list said they were unfamiliar with ZFS and it wasn't used as a guide for Btrfs.
You can track developers through their git commits, and you'll see even when they change companies, they almost invariably keep working on the same filesystem they were before the move. Which is why the idea that Red Hat has no one working on Btrfs, means they want to see it go away is not what's going on at all. They have a lot of other developers already, who'd lose years becoming familiar with Btrfs (or ZFS for that matter). So you're going to see them build on what they already are familiar with, rather than moving laterally to a filesystem technology they're not familiar with.
Is it in maintenance mode? Chris Mason (the main btrfs developer) works for Facebook now and Facebook uses btrfs on "millions of servers" (source: https://code.fb.com/open-source/linux/).
Facebook's usage of it is very specific, but as you mentioned, Chris Mason (who used to also work at Oracle on their BTRFS team before Oracle dumped BTRFS) is there in-house to make sure it is smooth.
However, I'm pretty sure if I boiled the commit history down for the kernel, most of Facebook's commits would be Flashcache, and not BTRFS.
What I forgot, though, is SuSE supports it as their enterprise FS of choice, which is maybe more important than Facebook.
Yes. It's important to recognize that there's a big difference in one's willingness to adopt a technology if its creator is on staff. Chris Mason's full-time job is to make sure the thing he built works well for his employer.
If you work some place that doesn't have that kind of luxury, perhaps a little extra conservatism is warranted before you go deploying across "millions of servers".
btrfs has a problem with it's image as being unreliable (some well deserved). Redhat obsoleted it, because they use kernel 3.10 in RHEL7 and they were doing nothing but backporting, without having any btrfs expertise in house anyway. Btrfs continues to be developed by Oracle, Suse, Facebook, Fujitsu and the rest of the usual suspects.
RHEL is famous for using kernels that are ancient, even by the notoriously conservative and risk averse debian-stable standards.
I can't even imagine the hassle of backporting stuff from much newer kernels into 3.10 to make btrfs work.
If you want to use btrfs in production (not the best idea, IMHO, but orgs such as facebook do it) you absolutely need to be running a 4.9.x or better kernel.
Calling them ancient is a bit unfair, they regularly backport hardware support and select other features from newer kernels. Unlike other distributions, however, they have a stable kernel ABI for an entire release, so they can’t bring in changes that break white listed symbols.
Not that I've seen. It's there and works. I think a lot of that impression is driven by the fact that... let's be honest: filesystem features are stale, dinosaur stuff. All the cool kids are worrying about cloud scale storage backends.
At the end of the day btrfs and ZFS both are just ways to help single-system administrators do better and more reliable backups. I know that's not how they're positioned, but really that's all they do.
btrfs still has a lot of issues unfortunately. I tried using buildroot once and the entire filesystem kinda died with the infamous "out of space" error even though there was plenty free. Several RAID levels are also unsafe to use. After experiencing both, I trust ZFS a lot more.
I've only used BTRFS a handful of times and I've still had three filesystems die on me. Two became cripplingly slow, probably because of automatic snapshots (even though there was a small retention limit). One was fine until it suddenly got into a state where it can only be mounted read-only or mounting never finishes, despite a lot of parameters thrown at it trying to fix that. The first two were not terribly long ago, and the last one was 2018.
Agreed. However, Synology uses it in their NAS products along with md RAID. Also, SuSE ships it in all their products. OpenSuSE LEAP 15 which was released two weeks ago and has a transactional updates feature that uses btrfs snapshots.
Honestly, I've never really understood the appeal of trying to shoehorn RAID into the FS itself. A more traditional softraid has always felt "good enough" to me (especially in this day and age of affordable SSDs).
ZFS in particular seems to push more toward RAIDZ, whereas I'm a bit averse to striping in general nowadays for data recovery/reliability reasons (RAID1 + JBOD is dead-simple to recover, more flexible with heterogeneous disk sizes, and much easier to reason about IMO).
Not sure where you are getting the push from to use RAIDZ over mirroring. Definitely not from ZFS proper. It's all about what's the best for the situation. Most my things are ZFS mirrors. But given a choice between RAID 3/5/6 or RAIDZ, I don't have to think long.
I've had the opposite experience. I've been using btrfs for years with no issues (pro-tip: don't create 10,000 snapshots and then try to rebalance the fs)
Having snapshots has saved me occasionally and find-new is really useful.
Maybe, but with btrfs your data is definitely at risk. Don't ever use it for anything important. Backups are important but even with those, btrfs is a filesystem that should be uninvented.
