Given SUSE's habitual brown nosing of RedHat, I suspect that it will probably bin btrfs eventually.
Red Hat banishes Btrfs from RHEL
Red Hat has banished the Btrfs, the Oracle-created file system intended to help harden Linux's storage capabilities. The Deprecated Functionality List for Red Hat Enterprise Linux 7.4 explains the decision as follows: The Btrfs file system has been in Technology Preview state since the initial release of Red Hat Enterprise …
COMMENTS
-
-
Wednesday 16th August 2017 05:10 GMT thames
I doubt that Suse will want to spend the resources required to keep Btrfs going. There's no market advantage to having it. I cant' see anyone one else stepping in to take over either. Suse will deprecate Btrfs at some time appropriate to their release cycle, and then gradually phase it out.
Given the way that technology is going, I suspect that the future is going to involve file systems that were designed specifically for flash storage. Rotating disk will be used more like tape and have file systems to suit that role.
-
Wednesday 16th August 2017 11:40 GMT Doctor Syntax
"Given the way that technology is going, I suspect that the future is going to involve file systems that were designed specifically for flash storage."
I think that, in response to malware, we might have to start looking at storage in a new way. Rather than letting any old application write to whatever lump of storage to which the user has access it will need to ask a service to do the writing and the service will ensure that the application has the appropriate credentials.
-
Wednesday 16th August 2017 14:33 GMT Anonymous Coward
@doctor syntax
"Rather than letting any old application write to whatever lump of storage to which the user has access it will need to ask a service to do the writing "
I can't think of any malware or virus that got its way into a system via writing to the filesystem.
Anyway it would be hideously slow due to paging data around the kernel and because of this will have to have exceptions for programs to write direct (eg for RDBMS's) so making it full of potential security issues which will be yet another thing for admins to worry about.
-
-
Thursday 17th August 2017 08:25 GMT Anonymous Coward
Re: @doctor syntax
"And once it gets in it never does stuff like, let's say, overwrite all a user's files?"
And your point is what exactly? Have you never heard of file permissions. Anyway, the filesystem is not a security weakness, its a basic facility. Also on unix/linux it would be quite easy to intercept any filesystem calls using LD_PRELOAD when test running a binary, having another layer on top of the filesystem solves nothing. Even MS realised this when they dumped WinFS.
-
-
-
Wednesday 16th August 2017 18:28 GMT elip
But you've just described exactly what a file system does.
Whether you have a regular human user account with access to the data, or a service account/application token writing to the data store, the file system is responsible for the reads/writes/access enforcement. Why would we want even more abstraction?
-
Wednesday 31st January 2018 06:27 GMT Tom Samplonius
"I think that, in response to malware, we might have to start looking at storage in a new way. Rather than letting any old application write to whatever lump of storage to which the user has access it will need to ask a service to do the writing and the service will ensure that the application has the appropriate credentials."
Congratulations, you have just discovered SELinux. SELinux can enforce file access on a per application basis, plus network access, ports, etc.
-
-
Wednesday 16th August 2017 11:44 GMT TVU
"I doubt that Suse will want to spend the resources required to keep Btrfs going. There's no market advantage to having it. I cant' see anyone one else stepping in to take over either. Suse will deprecate Btrfs at some time appropriate to their release cycle, and then gradually phase it out".
However, they have in-house experts and I expect that they'll keep on working on Btrfs.
As for Red Hat, I get the distinct impression that they are still not happy with the development and stability of Btrfs and so they are switching to the more mature XFS which has been around for years. They will be developing XFS and their recent acquisition of Permabit is entirely in line with a long term XFS development strategy.
-
Wednesday 16th August 2017 19:50 GMT Alan Brown
"Given the way that technology is going, I suspect that the future is going to involve file systems that were designed specifically for flash storage."
That's already where things are headed
"Rotating disk will be used more like tape and have file systems to suit that role."
Which is a pretty good description of the area that ZFS fits in (Spinning media fronted by gobs of flash as cache, with checksumming and proactive correction at every level to combat the inevitable errors that creep in every 45TB or so a disk reads. Compression, dedupe and encryption are all optional). Think of it as a faster Hierarchical filesystem
Tape is still king in the archival format area though - and will be for a long time to come. Nothing beats it for cold storage.
-
Wednesday 16th August 2017 23:40 GMT pwl
@thames
"There's no market advantage to having it. I cant' see anyone one else stepping in to take over either. "
- every admin command on SLES including OS updates gets a snapshot before & after thanks to btrfs. any errors can be rolled back virtually instantaneously. downtime due to errors is significantly reduced. in the event of catastrophic failure, you can recover from a previous "good" on-disk version of the OS : no reinstall/rebuild needed. only SUSE offers this on enterprse linux
- redhat's contribution to OSS is always important, but the main contributors to Btrfs were always Facebook, Fujitsu, Intel , Oracle, SanDisk, Netgear, & SUSE ... none of those companies have announced dropping work on the project...
-
Thursday 17th August 2017 03:57 GMT dajoker
Btrfs Market Advantage
The market advantage is definitely there if you are interested in enterprise products, which is why it is a little weird that RedHat would not want to be in on that, but their decision to not include it is about one drop in the overall bucket of contributions to the Btrfs filesystem. Go check lkml and see who is involved and maybe the reasoning for dropping Btrfs becomes more-clear as they lack expertise and may want to cut costs rather than employing people who know it well, instead focusing on other technologies that they know better despite lost functionality/
In the meantime, I've saved more than a couple servers, and my own laptop, from needing to be reinstalled thanks to Btrfs. Whether it's a bad patch (even a bad kernel), a bad use of the great tools that mange the system (which automatically snapshot), or user error in some other way, reverting a snapshot is just awesome. Also I get a lot of pleasure from comparing before/after snapshots from non-RPM-packaged software to see what they really do to my system. Comparing in realtime, both before and after snapshots from the running system, is just wonderful. I've even seem clients using it for forensics, finding out what some miscreant did, or tried to do anyway, to cover up tracks, giving investigators exactly what they needed to prosecute all because the snapshot recorded changes made (creates, modifies, deletions) that were done to avoid incrimination; it's like somebody making a lot of all the things they want to do and handing it to the authorities.
Disclaimer: I worked at SUSE several years ago, and am now a consultant on, among other things, SUSE and RHEL server products.
-
Thursday 17th August 2017 19:32 GMT AdamWill
Optional
Note: I work for Red Hat, but not on the relevant teams, so I absolutely don't have the authoritative answer on this, it's just my impression.
"Go check lkml and see who is involved and maybe the reasoning for dropping Btrfs becomes more-clear as they lack expertise and may want to cut costs rather than employing people who know it well"
My impression is it's kind of the other way around. Generally we hire people to work on stuff we're interested in shipping. There was a period when there was quite a lot of belief within RH that btrfs was The Future, and IIRC, at this time, we actually did employ multiple folks to work on btrfs full-time. (This was the period when it was a running joke that btrfs was going to be the default in Fedora in the *next* release...every release...it kept getting proposed as a feature, then delayed).
Over the next several years, enthusiasm for btrfs just kinda generally declined internally, because it always seems to be perpetually not quite ready and running into inconvenient problems like eating people's data. (Again, this is not my main area of focus so this is only a vague impression I have, I'm not in a position to cite lots of data or state this authoritatively; I'm not going to get into a "who's right, SUSE or RH?" debate because I just don't have the expertise and I have nothing at all against SUSE), and in this time, most of the development resources we had for btrfs got reallocated elsewhere.
So it's not so much "we don't want to ship btrfs because we don't have anyone to work on it" - I mean, it's not like we don't have the money to hire some more storage engineers if we *wanted* to - it's rather more "we're not hiring people to work on btrfs because we decided we don't think it's the right horse to bet on any more".
-
-
Wednesday 13th July 2022 10:39 GMT cosmodrome
f2fs?
> Given the way that technology is going, I suspect that the future is going to involve file systems that were designed specifically for flash storage
F2FS has been around and stable for years. I've been using it on USB drives and SSD for quite a while, so have Samsung on their smartphones, AFAIK.
-
Wednesday 16th August 2017 23:32 GMT pwl
probably shouldn't feed the trolls...
...but RedHat tends to follow where SUSE quietly leads without fanfare, such as providing xfs in the standard bundle (rather than 000's extra per cpu), using corosync/pacemaker for HA (after a 10+ year lag), showing an interest in Ceph (2 years after SUSE signed an enterprse deal with inktank), and properly supporting other platforms like IBM z.
Btrfs isn't going away from SUSE - it's a shame RedHat won't contribute anymore, but since most of the involvement was from Fujitsu, SanDisk, Intel, Netgear, Facebook, and others as well as SUSE, it probably won't slow things down too much.
Meanwhile, if you want full-OS snapshot/rollback on your Linux, it looks like SUSE will be your only choice for the forseeable future...
-
Sunday 18th November 2018 02:18 GMT hrudy
Re: probably shouldn't feed the trolls...
Having switched recently to OpenSuse and SLES, Btrfs has quite a learning curve. Some of their features like subovols almost seem like a solution in search of a problem. Perhaps the btrfs community and Suse has done a poor job in clearly explaining what the advantages (and Liabilities) of btrfs are.
One enterprise user I talked to said that his metadata fills up on his root volume and locked up his system, I know that you can run janitor programs that balance the system. However, it seems that these systems are tuned to resolve esoteric problems like bitrot and system rollback after updates but create more issues for the Sys Admins then they solve. I know that Kernel 4.19 has improvements for btrfs. However, t unless you are running OpenSuse tumbleweed or unless Suse decides to bacport thes fixes for SLES 15 not going to help in the short term future.
-
-
Thursday 17th August 2017 03:57 GMT Anonymous Coward
Brown-nosing... what a crock
While it's both easy and popular to think the Linux world revolves around [your distribution of choice here] let's not somehow make vacuous claims about one distro vs. another without at least, you know, evidence. SUSE doesn't follow RHEL as is evident to anybody familiar with both products. Sure, they have a lot of common products, like Linux, and OpenStack-based things, etc., but the timelines based firmly in reality and documented online show a lot of non-following, at least by SUSE, when it comes to adopting technologies. The filesystem is definitely no exception; XFS was free and commonly -used by SLES long before RHEL finally made it available to everybody. BtrFS's primary contributor isn't RH, but SUSE, followed by several others, and then possibly RH gets in there with a few commits now and then.
In other news, the [pick some government] announced it would no-longer support Bitcoin for transactions with the government..
-
-
Wednesday 16th August 2017 05:36 GMT Anonymous Coward
After so many version of Fedora that promised brtfs as the default filesystem
Now they're binning it entirely? I find it hard to believe that lack of encryption was the only reason, especially since 1) every enterprise drive already supports encryption and 2) you can implement it using md.
I'll bet this has something to do with politics and Oracle.
-
Wednesday 16th August 2017 05:57 GMT Anonymous Coward
Re: After so many version of Fedora that promised brtfs as the default filesystem
Er, I think it was Google who aren't keen on it for its lack of encryption, not RedHat.
Stallman is going to blow a fuse if ZFS gets widely adopted.
This doesn't bode particularly well for the future of large scale GPL projects. Seems like more and more people are willing to mix code under different licenses even if they're incompatible. The allure of someone else's code can be too great to ignore. If the GPL is to remain influential, and not get routinely ignored, they need to get ZFS in Linux knocked on the head. If ZFS+Linux becomes the norm, unchallenged, it becomes harder to enforce GPL. Especially with court cases like Google vs Oracle which may yet pronounce on loosely similar copyright breaches being "fair use".
-
Wednesday 16th August 2017 06:44 GMT Teiwaz
Re: After so many version of Fedora that promised brtfs as the default filesystem
This doesn't bode particularly well for the future of large scale GPL projects.
- Any license issue that gets in the way will eventually get sorted, maybe not GPL, but something.
I can't actually fathom what was going on around Btrfs - lots of talk about it being the future default filesystem for 'Linux - yet it was left with Oracle?
I'm just waiting to for the next great 'Linux schism when it's announced Lennart Poettering is doing a new filesystem for Red Hat.
-
Wednesday 16th August 2017 10:49 GMT JSTY
Re: After so many version of Fedora that promised brtfs as the default filesystem
> I'm just waiting to for the next great 'Linux schism when it's announced Lennart Poettering is doing a new filesystem for Red Hat.
Don't worry, I'm sure it's on the SystemD roadmap
(Not even sure if the joke icon is appropriate here ...)
-
Wednesday 16th August 2017 07:39 GMT Anonymous Coward
@AC
Stallman is going to blow a fuse if ZFS gets widely adopted.
Being a huge fan of ZFS myself I'd pay to see that ;)
But I'm not sure I fully understand though. I mean, all because of a license? Because ZFS uses a license (CDDL) which people don't happen to like or anything? What happened to the free software philosophy? Because CDDL is just as much an open source license as the GPL is. Sometimes I can't help worry that certain people completely lose focus of the eventual goals.
Seems like more and more people are willing to mix code under different licenses even if they're incompatible.
Can you imagine that... People apparently like the freedom to use the (open) source as they want to use it. One important note though: it's the GPL which is usually incompatible, not the other licenses. Many open source licenses (I'm mostly familiar with the BSD, CDDL and Apache licenses) have no problem at all with mixing things together, as long as the license continues to get respected.
I think that's an important aspect here. GPL demands that everything gets redistributed under the GPL (all newly entered code) whereas the other licenses only demand that the original software simply continues to stay licensed under the same license it was given out with. Hardly an unfair demand to make I think, especially if you keep in mind that mixing that with another license is usually no problem.
-
Wednesday 16th August 2017 09:47 GMT Jon 37
Re: @AC
The issue isn't that "ZFS uses a license (CDDL) which people don't happen to like", the issue is that it is widely believed that distributing a binary that includes both GPL and CDDL parts is not allowed by the licenses, and therefore copyright infringement.
Only a court can eventually rule on this and give us a definite answer one way or another.
As for "What happened to the free software philosophy?", certainly the GNU developers were (and are) very careful not to infringe on anyone's copyrights, and if someone offers code freely available subject to certain conditions they would either follow the conditions or not use the code.
If you don't like that "GPL demands that everything gets redistributed under the GPL", then don't use GPL code. The *BSD distros and kernels are available as an alternative to Linux, and someone could port (maybe has ported?) CDDL-licensed ZFS to them. Alternatively, someone could look at the existing ZFS code and write a decent spec for ZFS, and someone else could implement it in new GPLed code for Linux.
-
Wednesday 16th August 2017 10:13 GMT Bronek Kozicki
Re: @AC
OpenZFS is natively available on FreeBSD. It is the same OpenZFS which is also available for Linux, under "ZFS on Linux" project, and which is included in Ubuntu since version 16.04. The parts of OpenZFS which are interacting with Linux kernel are doing so via modules called Solaris Porting Layer, which are all licensed under GPL (and not CDDL).
-
Wednesday 16th August 2017 11:46 GMT TVU
Re: @AC
"OpenZFS is natively available on FreeBSD. It is the same OpenZFS which is also available for Linux, under "ZFS on Linux" project, and which is included in Ubuntu since version 16.04. The parts of OpenZFS which are interacting with Linux kernel are doing so via modules called Solaris Porting Layer, which are all licensed under GPL (and not CDDL)".
^ Absolutely this. This is why it is unlikely that there will be any challenge and why it will be even less likely that any such challenge will succeed.
-
-
Wednesday 16th August 2017 10:32 GMT Anonymous Coward
Re: @AC
>> Because CDDL is just as much an open source license as the GPL is. Sometimes I can't help worry that certain people completely lose focus of the eventual goals.
Those people are called bureaucrats and lawyers. Their goals never change, they exist only to make more work for themselves, and they love arguments like this. Normal people would just say "It's public domain, go have fun".
All this fannying about over "my license is better than yours" reminds me of nothing more than the various "Palestinian" organisations in Life of Brian.
-
This post has been deleted by its author
-
-
Wednesday 16th August 2017 17:10 GMT Anonymous Coward
@AC - Re: After so many version of Fedora that promised brtfs as the default filesystem
I gave you a down vote wholeheartedly for your failure to see the separation between developers wish and lawyers will. And also for equating 4 lines of code in Google vs Oracle with the whole ZFS code.
You don't seem to understand much about licensing issues. Problem is not with ZFS adoption in Linux but with rights to distribute ZFS with Linux.
According to your logic even Microsoft's hand could be forced because people really want to use their technologies for free.
-
Wednesday 16th August 2017 19:55 GMT Alan Brown
Re: After so many version of Fedora that promised brtfs as the default filesystem
"If the GPL is to remain influential, and not get routinely ignored, they need to get ZFS in Linux knocked on the head. "
The incompatibilty between them is only if you _distribute_ the two together as binaries. There are a number of packages with this same issue and the response has been pragmatic for years - setup a script to pull in the required items from a separate source _after_ the OS has been installed.
Apart from the feature work, there's been a _lot_ of work put into ensuring that ZFS doesn't touch GPL symbols in the kernel and the reality is that the FS is mature, reliable plus blazingly fast for spinning media (and even faster if you run it on all-flash systems)
-
-
Wednesday 16th August 2017 16:42 GMT Oh Homer
Re: After so many version of Fedora that promised brtfs as the default filesystem
Personally I'm breathing a huge sigh of relief, as the ironically named btrfs is an utter clusterfsck, in my experience.
Sorry, but any filesystem that gets heavily fragmented by design is definitely not on my Christmas list, especially when this results in a performance degradation so crippling that I have to pull the plug, risking massive filesystem corruption in the process, then don't even have a fully functional (or even safe) fsck utility to recover afterwards.
I'm shocked that Red Hat took this long to finally ditch it.
ZFS has its own problems, of course, and I'm not even talking about the dreaded CDDL. For a start, it has a massive memory overhead. It also seems to be impossible to tell how much, if any, free space is available, due to the strange way that it views storage, and worse still there doesn't seem to be any way to reclaim "freed" space either.
Frankly I'm not sure what problem these highly dysfunctional filesystems are trying to solve, but whatever it is they don't seem to have succeeded, so I'll just stick with ext4. Thanks anyway.
-
Wednesday 16th August 2017 20:21 GMT Alan Brown
Re: After so many version of Fedora that promised brtfs as the default filesystem
"ZFS has its own problems, of course, and I'm not even talking about the dreaded CDDL. For a start, it has a massive memory overhead"
It's designed to be used on FILE servers, not your average desktop system. It also starts on the premise that "Disks are crap, cope with it and don't be a snowflake" - it doesn't demand you put in premium components and then have a tantrum when one goes down. It _expects_ things to break or give errors and heals automatically, on the fly when using commodity components (the only "critical" demand is ECC memory and you're not going to have that on your average desktop or laptop anyway). It plays to the strengths of hard drives and remediates their weaknesses using memory and flash drives to allow sustained simultaneous high IOP and high throughput activity.
That memory is cache, with even more cache on SSD and it will maximise cache whenever possible, in line with its mission of being a fileserver's filesystem (You can tune the memory usage down but why? The more memory you throw at it, the better it becomes!)
The advantage of doing it this way isn't obvious on your desktop or laptop (apart from the block checksumming, which is worthwhile by itself), but it shows up in spades when someone puts 200,000 files in one directory or you have thousands of users banging hard all over a 400TB directory tree containing 500 million files. This is why outfits like Lawrence Livermore laboratories have invested so much effort into it - and why I dropped £120k a couple of years back on a dedicated ZFS fileserver that has 256Gb of ram and 1TB of fast SSD cache
Yes, you could run ext4 on this system - If you don't mind periodic downtime for housekeeping (fsck), and add the expense of an expensive fast hardware raid controller, or the complexity of MD-raid (which only allows 2 stripes instead of ZFS's 3 - and that's a big deal when you run 100TB+ installations as we've lost RAID6 arrays before) plus LVM, plus myriad individual filesystems to herd. You'd find that the overall performance with the same amount of memory would be substantially lower and if you want to try and match ZFS you'll have to do a lot of fiddling with dm-cache, or put your writes at risk of power failure/crash. After all that, you'll still have it fall over when a user puts 250,000 files in one directory.
After spending years nursemaiding systems which suffered poor latency and got temperamental when users piled on the load it's a relief to have one which doesn't suddenly slow down 90%, pause for 4-5 minutes because a user did something stupid, (or become unstable), or be a major headache when a disk flipped a few bits (or died) - and for less money than comparable clustering "solutions" - which bitter experience has shown are not fit for purpose.
ZFS is the right tool for the job it's designed for. Putting it on a low-load, low-memory desktop or laptop is on par with using a bucketwheel excavator when you only wanted to shift a wheelbarrow load. It will do it and it can be tuned to do it relatively well, but that's not what it's intended for.
-
Thursday 17th August 2017 23:10 GMT Oh Homer
Re: "It's designed to be used on FILE servers"
Tell that to the masses who seem to think that ZFS is simply the better alternative to btrfs on the Linux desktop, for whom ZFS is the standard response to all criticism of btrfs, and who then set about defending the CDDL to allay fears that mass adoption might be stymied by licensing issues.
No, I don't get it either, which is why I like to constantly remind everyone about how unsuitable ZFS is for the Linux desktop.
-
-
-
Wednesday 16th August 2017 17:06 GMT John Sanders
Re: After so many version of Fedora that promised brtfs as the default filesystem
>> I'll bet this has something to do with politics and Oracle.
And lack of robust RAID5/6, or robust anything for that matter.
I was once in love with BTRFS, but time and time again I found myself going back to mdadm + lvm, or just pure lvm.
I wanted BTRFS to succeed, but how many years its been in beta?
In the same time ext4 has gained all sort of niceties (like built-in ACLs or native encryption) and at the current rate we'll end getting native snapshots or Raid functionality way before BTRFS is ready.
And yes I know Ted Tso (maintainer of ext4) said BTRFS is the future.
-
Thursday 17th August 2017 19:39 GMT AdamWill
Re: After so many version of Fedora that promised brtfs as the default filesystem
Few points:
1. If you look at the proposed / accepted 'Features' / 'Changes' for the last several Fedora releases (we changed the name from 'Features' to 'Changes' a few cycles back...) you'll notice that whole circus of 'we promise it's going to be the default next release!...no, wait, we're delaying it again' stopped happening several releases back; it hasn't actually been proposed as a feature/change for several releases. Which some savvy folks interpreted as something of a sign about RH's declining keenness on btrfs, and...well, I guess now it's not revealing any secrets to say they weren't wrong. :P
2. btrfs isn't being 'binned entirely' from *Fedora*; this announcement is specific to RHEL. Its status in Fedora for a long time has basically been "it's there, and it's approximately as supported as any other filesystem which is included and selectable from the installer but isn't the default", and that's still its status at present. Though I know the installer team has made noises about how their lives would be rather easier if they could kick it out of the installer again, I don't think that idea's live *right now*.
(Also FWIW, which is very little as I'm certainly not plugged into to all the internal channels on this, I don't *think* there's anything particularly political about this, it's purely the case that our storage folks have been kinda gradually losing confidence in btrfs being really good enough for our customers for some time.)
-
-
Wednesday 16th August 2017 06:28 GMT Anonymous Coward
ZFS is the right choice for a server system
ZFS contains a superset of btrfs features, plus a great number of nice bits for dealing with large and complex disk subsystems and NFS file serving. If your target is beefy enterprise servers (it's RHEL, after all!), deprecating btrfs in favour of ZFS seems to be an obvious choice.
On a client system, I'd choose btrfs over ZFS any day - it still has the COW, the snapnoshots, and the cloning, which accounts for 99% of what I need on a client. It also has a much lighter resource footprint, and is harder to misconfigure.
Sometimes, an apple is just an apple.
-
Wednesday 16th August 2017 08:20 GMT Anonymous Coward
Re: ZFS is the right choice for a server system
Actually no.
ZFS is NOT automatically "the right choice for a server system".
For a start. The very founding principle of ZFS (that many people forget) is that it was designed as, and continues to be maintained as a JBOD DAS file system.
If you are running an abstraction layer over your storage (such as a RAID controller, as many people do), then running ZFS on that is very much not recommended and WILL (not if), come back to kick you in the backside one day.
-