Don't panic, but Linux's Systemd can be pwned via an evil DNS query Systemd, the Linux world's favorite init monolith, can be potentially crashed or hijacked by malicious DNS servers. Patches are available to address the security flaw, and should be installed ASAP if you're affected. Looking up a hostname from a vulnerable Systemd-powered PC, handheld, gizmo or server can be enough to trigger …
Let me provide an analogy. You know, xpdf frequently had code execution vulnerabilities found in it and ultimately was removed from Gentoo in 2014 when another one resurfaced and became the last straw:
https://security.gentoo.org/glsa/201402-17
"Description: Multiple vulnerabilities have been discovered in Xpdf. Please review the CVE identifiers referenced below for details.
Resolution: Gentoo has discontinued support for Xpdf. We recommend that users unmerge Xpdf: # emerge --unmerge "app-text/xpdf"
After that, there's no more xpdf in Gentoo. They use mupfd instead. I hope systemd meets the same fate, the sooner the better. Flushing toilet water icon, please.
I use systemd on all the servers I manage, out of choice. I refuse to set up non-systemd server-setups any more, it is just so vastly more pleasant to work with than the alternatives.
So, for me personally, when systemd came along, it solved all the problems I ever had with system initialization. What most systemd critics consider "bloat", I consider necessary complexity to solve a complex problem generically. You can say what you want about Poettering, but he actually realized what the problems with system initialization were and provided a working solution. I could go on for hours, but this should be a good summary.
A lot of the pushback against systemd - merited or not - is because a lot of people in charge of little parts of the bazaar have seen their pet projects cast aside by the major distros and taken over by the systemd devs. In a world where street cred is a big force in motivating people to contribute to open source being maintainer of 'x' where 'x' is part of each and every linux distro out there and then to see 'x' taken over by systemd in a fairly rough manner without any kind of co-operation between the old maintainers and the new kids on the block there are bound to be a lot of ruffled feathers. But that's not technology, that's just ego.
I find amusing that no one here is asking why systemd-resolved was introduced, or what problem was it intended to solve, read this post:
https://lists.ubuntu.com/archives/ubuntu-devel/2016-May/039350.html
Everything that is wrong with systemd is down to it M$ style design, it doesn't belong on professional systems anymore than windows does.
Yes, you gain ease of use and fast setup but at the cost of security and stability with the obvious result that you open yourself up to security problems that occur because you let someone else make the decisions you thought weren't as important as getting up and running.
The pushback against systemd is because it takes what were independent systems and rolls them into a tightly coupled monolith. The independence of those prior systems was their greatest strength - the more independent those systems are, the less opportunity there is to bring down the entire OS by crashing one of those systems.
systemd can claim to be modular all it wants; the fact that you can take down the entire OS via the init with a malicious dns response is a fucking travesty in this day and age. It's the sort of thing that even Windows left behind at the turn of the century.
Can you explain to me your rationale for believing that "you can take down the entire OS via the init with a malicious dns response" when systemd-resolved is quite clearly a separate binary from that running as init and has also dropped its privileges and is running as a non-root user?:
# ps | grep init
1 root 7824 S {systemd} /sbin/init ldb
1108 root 2696 S grep init
# readlink -f /sbin/init
/usr/lib/systemd/systemd
# readlink -f /usr/lib/systemd/systemd-resolved
/usr/lib/systemd/systemd-resolved
# ps | grep systemd-resolved
359 systemd- 5816 S /usr/lib/systemd/systemd-resolved
1097 root 2696 S grep systemd-resolved
# cat /etc/passwd | grep systemd-resolve
systemd-resolve:x:231:231:systemd-resolve:/:/bin/nologin
systemd-resolve:x:231:
# cat /etc/group | grep systemd-resolve
systemd-resolve:x:231:
#
I find amusing that no one here is asking why systemd-resolved was introduced, or what problem was it intended to solve, read this post:
To paraphrase for anyone not wanting to read it: "DNS is probably too complex for you to manage, so we're adding an extra caching resolver to every machine, which simplifies certain desktop configurations".
If I'm going to be treated like a child, I expect milk and cookies at 3pm, and then a nap.
This post has been deleted by its author
I've always found it odd that systems would enable dns reverse lookups for all sorts of things where it provides no value. I don't trust DNS to give me a valid name if all I have is an IP address.
The whole thing of probing someone's network and have them look up your IP address where you then send TCP packets back to crash their name server or other application has been around since the early 1990s.
It also adds an unpredictably slow, often blocking process to whatever you're doing. There are situations where it's useful, but almost all of them are better handled in after-the-fact processing by non-privileged code.
However, there are other scenarios where this bug could trigger. For example, if you're using an untrusted network that can intercept your DNS queries.
I've always found it odd that systems would enable dns reverse lookups for all sorts of things where it provides no value. I don't trust DNS to give me a valid name if all I have is an IP address.
That's why you also look up the name to see it matches the same IP address, otherwise there's no point, in fact it's worse than no lookup at all if the name is logged instead of the IP.
The whole thing of probing someone's network and have them look up your IP address where you then send TCP packets back to crash their name server or other application has been around since the early 1990s.
That's a daft thing to say. If someones name server can be crashed this way, then it is at fault, and needs to be fixed. Nothing to do with DNS lookups, apart from maybe reducing "security by obscurity", which of course is a non-reason.
To people who think reverse DNS is a good idea, consider concepts like CNANE loops. Smart DNS servers will catch it but there are plenty of dumb DNS implementations out there. In IPv4 we could send a UDP request out and expect to get a UDP response back but now with IPv6, the packet sizes often exceed the MTU resulting in several packets. Once you get a large chunk of data back, someone at the other end might just be playing games with malformed DNS packets or even just broken DNS settings. What does you application do when you get back thousands names for a reverse lookup? What happens when each lookup results in a chain of CNAMES? What happens when the end of those chains result in hundreds of addresses that are all the same?
DNS isn't authoritative, it is informational. It is great when it doesn't lie to you. But you can't test for when it does.
ALL software has bugs, get over it.
It is precisely because all software has bugs that design decisions should be made to limit the impact of those bugs. It's like RAID, all disks are unreliable, eventually the industry twigged that the answer wasn't to make more reliable disks (even though that helped a bit) the answer was to design storage systems where disks dying wasn't a big problem.
When it comes to SW the trick is to make sure that you're not exposed to the risk of bugs unnecessarily. Design your SW so as to limit the impact of the inevitable bugs.
When it comes to SW the trick is to make sure that you're not exposed to the risk of bugs unnecessarily. Design your SW so as to limit the impact of the inevitable bugs.
It's unclear whether you think that the language should have protected the developer from risks, or that the software should have been composed of more coherent, smaller, and less tightly coupled components.
@ Tom 38
> It's unclear whether you think that the language should have protected the developer from risks, or that the software should have been composed of more coherent, smaller, and less tightly coupled components.
I think both. A traditional "init" is written in C but most of the functionality is actually in shell scripts which are invoked by "init". Now shell is messy, but at least it is not vulnerable to buffer overruns. In principle there is nothing stopping you from rewriting the shell scripts in some saner programming language (Python, Ruby, whatever rocks your boat).
Note that dividing the software in smaller blocks also means that you can use an appropriate programming language for each block.
And that's an argument in favour of a giant monolithic buggy piece of software?
Many things that systemd does are bad copies of stand-alone software with reduced functionality and/or bugs. If it were a stand-alone DNS daemon then you could easily change to another one.
Why would you even want to shove everything in the same process? If it goes down the system is hosed, if it's exploited then all functionality that systemd does is compromised.
>> If it were a stand-alone DNS daemon then you could easily change to another one.
Dude, you can just do that, systemd-resolved is a separate module.
systemd stop systemd-resolved
systemd disable systemd-resolved
systemd mask systemd-resolved
apt update
apt install dnsmasq
I guess this is the 21st century version of the peasants with pitchforks.
I cringe every time I hear someone try to deflect the storm of _VALID_ criticism that blows in when someone brings up systemd by dismissing the fact that the problem can be fixed/worked around/removed and or put back to the way it was when it was still working.
Yeah, it's replaceable, but you'd have to replace it EVERY TIME, SYSTEM by SYSTEM. You'd have to regression test ever major version upgrade to make sure it doesn't turn itself back on. You may not even have root access on the underlying system. Ever hear of cloud hosting?
There are real problems with systemd. It breaks things. If we have to fix something that wasn't broken before, THAT IS A PROBLEM. A problem that we didn't have before and we now have to fix.
@ rtfazeberdee
All software has bugs, therefore software like init, which runs as the very first process and with elevated permissions, should be kept as simple as possible.
In contrast, the systemd developers have included a ridiculous amount of functionality into the init process. (process 1). See http://suckless.org/sucks/systemd for an overview. Often for no other stated reason than to "keep the number of processes down".
A traditional init system wouldn't even contain networking code, let alone do reverse DNS lookups.
Consider that it is generally estimated that high-quality code contains one defect every 1000 lines. Consider the size of systemd, all of which is running as the privileged process 1. Weep.
"bad software has bugs" FTFY
Every person who has done any coding will have made a "hello world", the ones who progressed will have made certain that it worked before moving on, so clearly it is possible for all competent coders to write without bugs. Mathematically that is a lot of good code that proves that not all software has bugs
The excuse that the complexity of coding is greater than any other human endevour is also clearly false as computers are fundementally simple in operation. The computers never get confused only programmers.
The "all software has bugs" and "computers are so complex we should be excused for our mistakes" are Urban myths i.e both false but have been repeated so often that people actually believe them true. M$ I am looking at you
IMHO anyone who quotes either of them is either a shill and/or an incompetent.
The "all software has bugs" and "computers are so complex we should be excused for our mistakes" are Urban myths i.e both false but have been repeated so often that people actually believe them true. M$ I am looking at you
If you type with 99% accuracy that means on average you mis-type one character in every 100. With red squiggly underlines and the like you can pick up typos where the typed word is not found in the dictionary, but if you type "cant" where you meant "can't", there's a good chance you'll not pick up the typo.
Where that typo is a number in a program of a few million lines of code, each done by several people over the years, you will only find it if the typo causes a compile error or warning, or if it eventually causes a noticeable failure in the software. In some cases (eg Heartbleed or EternalBlue) those bugs can take a number of years to be noticed, because the flaw is not triggered in such a way as to cause a problem.
But there's other bugs as well, such as a condition the programmer never thinks to test for (no, you cannot test beyond your experience and beyond what is suggested to you), your code might work flawlessly on every possible situation, but someone has a single pixel in a photo that is #FFEE66 and something in your code reacts with that, and you tried a few hundred thousand colours in your testing but just not that one. It could be a slight delay in how one CPU processes commands, or a system configuration that fragments memory in some weird way, or... So many things you cannot possibly expect and test for.
When code gets beyond a few 10s of lines it becomes hard to spot typos and other bugs. Transpose a couple of digits, not have an obvious flaw at compile and testing, the bug will ship. You may never know it exists, or it may be found the day after shipping. That's why patching systems in professional-grade OS's (and software) work so beautifully, because we need to be able to fix software when someone finds something is wrong.
Theoretically it is not impossible to write perfect code in a complex program. In theory you could type every character in a couple of million lines without one typo. In reality, most humans struggle even to get to a mere 90% typing accuracy (that's one mistake in every 10 characters typed!).
That human's err is not unknown, since we know this then we can take actions to mitigate the chances of a typo being an issue.
It reminds of electronic projects from my youth and machine code programming onto an EPROM via entering the hex and manually writing it byte by byte, if you made one fat fingered mistake then you had to wipe it and start again. Whilst it was an absolute pain it was possible, with sufficent diligence and practice input errors disappeared.
Flash forwards a few decades and people are saying that it is not possible to avoid typos in a highlevel programming language which has lots of verification tools to help, from my experience if you cannot manage to avoid typos ending up in your finished code then clearly things have been made too easy for you to enter bad data. Perhaps if compilation error detection wiped your source you might learn to code without errors too.
The lessions I learned were to be certain of what your are entering and keep things small so you do not have to maintain that level of concentration for too long. Now programming is so much easier we seem to have lost the discipline and especially the attention to detail.
Bug free code is possible but why bother when enough people believe that it is not
om my experience if you cannot manage to avoid typos ending up in your finished code then clearly things have been made too easy for you to enter bad data
I referred to rates of typing mistakes in my post, and I stand by that. Sure, with experience like yours you can become much better than average at typing accuracy, but is that at the sacrifice of speed?
I don't know of any tool for entering bad data other than the keyboard BTW. Well, you could probably use some voice recog or mouse-driven interface but speed drops quite a bit there, keyboard is still the fastest way to enter words into a computer IME (am looking for decent voice stuff but little out there).
A compiler will pick up obvious errors, and may pick up a number of other errors that would still compile but there's something "not quite right", but there's a lot of stuff that can be syntactically perfect, make perfect sense to you when you read the code, but still not be doing what it's supposed to.
But you can guarantee that every number you've entered into code is right? You've never noticed that you'd accidentally transposed a couple of variables (or a variable and a constant)? Never forgotten to remove debugging stuff ( {ifdef debug..} ) wrapped around what should be a random number but you were using a constant so you could test the code? You're either one in several billion, or maybe your code isn't that complex.
BTW, I can spot a half a dozen potential grammar errors and few typos, and at least one word-choice error in your short message! :)
Getting rid of systemd won't stop buffer overruns.
No it won't be getting rid of buffer overruns, but something as critical to system operation as systemd simply should never be placed in a position where it can be exposed to this kind of attack, or any kind of remote attack. That's not an "I hate systemd" statement, that's a basic security for kindergarten level lesson.
Fixing these issues is going to be a giant game of wack'o'mole where there is simply no need to expose yourself to danger in the first place. This isn't just a bug it's a fundamental design flaw.
No, the reason to hate systemd is that Linux already has a bunch of DNS resolvers that are pretty secure. You can pick your favourite.
But systemd must use its own, special, freshly-written one. Because, systemd. So that decision obliges most Linux distros to host a single, new DNS resolver. That is broken.
Unfortunately, this has been the way with systemd for a while. New way of doing things becomes The One, True, Only Way before the code is actually fully finished and tested. And, if bugs are found, or pre-existing features are removed, it is your fault for trying to do whatever you used to do before.
So, yeah, hate. Just been embarrassed in front of a customer by the COMPLETELY UNNECESSARY change to the way that partitions (especially swap) are mounted. Oh, and did you know, that if your machine ceases to boot because of a systemd config error, you can't fix it in the recovery console?
Bah!
"the article makes it clear that use of the systemd resolver isn't compulsory. Yet. At least not if you use Debian."
this is true. my Debian 8 'beater' box [which I'm using to test changes/updates to a customer web site before implementing them on the actual site] has systemd on it, unfortunately, but isn't using resolved for anything significant (or at all, for that matter).
/etc/resolv.conf is a regular file with the expected text-based contents in it
I think you may be able to DISABLE systemd-resolved by making /etc/resolv.conf a static file, rather than a symlink to the /run/systemd/whatever file. Can anyone confirm that?
This post has been deleted by its author
> Unfortunately, this has been the way with systemd for a while.
> New way of doing things becomes The One, True, Only Way before the
> code is actually fully finished and tested. And, if bugs are found, or
> pre-existing features are removed, it is your fault for trying to do whatever you
> used to do before.
That's not unique to systemd, that's Linux in general. Every time I upgrade my Mythbuntu setup to a new release I have to learn at least one new way of doing something that was working perfectly well before. I think they like to call it progress. I just call it a pain in the arse but it seems unfair to single out systemd alone for that treatment.
> That's not unique to systemd, that's Linux in general. Every time I upgrade my
> Mythbuntu setup to a new release I have to learn at least one new way of doing
> something that was working perfectly well before.
That's Linux for you. Everything is about the shiny-shiny. If something is broken, don't fix it - write a new one! OSS was broke on linux... replace with alsa!
udev, devd, hal, not hal - there is no discipline in the design, it's all just thrown together. You should check out the BSDs for stable and sensible changes
Glad, but not surprised, to see that the average Reg reader is more resistant to the systemd Koolaid.
One extra point that hasn't been hammered home yet is all of the compatibility breaking changes it introduced. I can't think of a better example of change for changes sake (Not that the bootloader cleanup, that was badly needed).
Today I was wrestling with a VM that handles tftp services for updating switch and phone firmware. I typed ifconfig to confirm one of it's ip addresses. OOPS some self righteous @$$hat decided that systemd is so awesome that we should all type "ip addr show" now, because you know reasons. Want to tail -f a log file that's been in the same place since 1970? Guess what, the same genius couldn't even provide a symlink or command alias, so now off to google to figure out you need to use "journalctl" now, because they apparently haven't heard of "man" or "info" either. They have apparently not worked on many embedded systems either, where you wind up serial porting into a tiny distro that has had virtually everything stripped out of it but you find that the graybeard that set it up still left you vi so you can fix a config file on the fly.
What burns me isn't that people tried to push for a better boot process forward and ended up with systemd. It burns me that after all the problems it causes they still haven't fixed it, and we haven't replaced it. Systemd as it stands is an objective failure. Perhaps not catastrophic, but a failure.
It should be fixed, or it should go away.
The fact that the people who came up with it are thin skinned and and find it precious isn't a reason to put up with it. The fact that it seemed to solve THEIR problems does not mean they should ignore the problems they cause for a huge part of the community. Like the IPv6 fiasco, they have lost sight of their objectives, and failed to deliver an effective replacement. Like IPv6 they lost track the core problem they were supposed to solve(Boot process improvements instead of ipv4 address exhaustion) along the way, and started fixing things that weren't really broken (like banning NAT in IPv6, or bypassing the existing system DNS resolver in systemd, or screwing with logging system.
This should of gotten stopped in it's tracks before it got merged into the distros. In hindsight we'd ask...
What does this change improve? It slightly improves the boot process by introducing more deterministic boot behavior.
What are the drawbacks? Huge tracts of non-audited code that haven't been tested at scale, impacting huge sections of the OS that aren't strictly involved with the boot process. Scrapping backwards compatibility with previous version Linux, and current versions of BSD and OSX. Every programmer of every app will have to read, understand, and then assess and possibly implement changes to their application, even if it didn't suffer under init.
Well, gee that sounds like a terrible deal, lets not do that.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
