back to article Today the web was broken by countless hacked devices – your 60-second summary

Today a vast army of hijacked internet-connected devices – from security cameras and video recorders to home routers – turned on their owners and broke a big chunk of the web. Compromised machines, following orders from as-yet unknown masterminds, threw massive amounts of junk traffic at servers operated by US-based Dyn, which …

Page:

            1. Charles 9

              Re: Standards Bodies need notice

              ""We can't stop them all so we might as well do nothing"."

              In this case, it's accurate. It's not worth swatting one angry bee because there are a million more after you. You really DO need an "all or nothing" solution to it or the ones that slip by kill you.

              Problem is, sovereignty gets in the way. How can you regulate devices when they can just be shipped direct from companies who don't care?

              1. Stoneshop

                Re: Standards Bodies need notice

                Problem is, sovereignty gets in the way. How can you regulate devices when they can just be shipped direct from companies who don't care?

                When I try to buy a laser device from Alidealgoodbest, I get a notice that "due to regulations, we can't sell lasers over $smallnum mW to @countries", probably followed by "Kthxbye" if that laser is over $smallnum mW and I'm in one of @countries. So that part of international regulation enforcement works, more or less, and I don't see why it can't be extended to cruddy IoT stuff*. There's also your country's customs between China and you, and while your individual shipment may or may not get caught, a container full of uncertified idIoT tat is unlikely to reach $shadydealer.

                * once appropriate regulation is in force, which will quite likely take a while.

                1. DainB Bronze badge

                  Re: Standards Bodies need notice

                  You're trolling right ?

                  You need to ship laser device by mail. You don't need ship IoT device to any particular country to cause damage, same device in Venezuela will do as much damage as if it was in California.

                  1. Stoneshop
                    FAIL

                    Re: Standards Bodies need notice

                    You're trolling right ?

                    I will not buy this record, it is scratched.

                    Look, if you want to wallow in your opinion that any action is futile, go right ahead. I happen to disagree.

                    1. DainB Bronze badge

                      Re: Standards Bodies need notice

                      Well, you're either trolling or don't have any idea what you're talking about, it's up to you.

                      I don't agree with you on concept of Internet of trust where only allowed devices can access it as implications of that are too far fetched.

                      First of all, it's not possible to implement simply because there's no concept of global standards and what you're talking about would not work without it, Internet traffic does not care about your regulations.

                      Second - who and how decides which device should be banned ? Again see #1, banned in one country does not mean much in other.

                      TLDR There is solution but it's not even remotely close to what you're rallying for.

                      1. Stoneshop

                        Re: Standards Bodies need notice

                        I don't agree with you on concept of Internet of trust where only allowed devices can access it as implications of that are too far fetched.

                        Trust, to the extent that "this device is configured with reasonable protection against remote attacks, which includes [list of security 101 measures]". This needs to be done to mitigate proliferation of Mirai c.s., and is by no means the one single solution required.

                        TLDR There is solution but it's not even remotely close to what you're rallying for.

                        I haven't seen any details of YOUR plan yet. Care to provide some, instead of muttering defaitist boilerplate?

                      2. Doctor Syntax Silver badge

                        Re: Standards Bodies need notice

                        "There is solution but it's not even remotely close to what you're rallying for."

                        I haven't seen you suggest it.

                        1. Stoneshop
                          FAIL

                          Re: Standards Bodies need notice

                          I haven't seen you suggest it.

                          Oh, I found this:

                          The only solution for this particular issue is a protocol that can stop traffic towards victim at originating ISP level. Not that hard to do really.

                          Yeah, that totally doesn't require just about every* ISP on the planet to sign up for that, agreeing to some extension of a couple of very basic network protocols, upgrading their software and maybe even their equipment to accommodate that protocol, and figuring out a way to reliably determine which of those millions of network packets are actually malicious.

                          And never mind that, next to China being a major source of idIoT junk, there's also a lot of networking and telco gear manufactured there.

                          * If you can't get South American and Asian providers on board you'll have the same problem as with those countries not banning (and enforcing that ban) IoT stuff that essentially hollers "Pwn me!"

          1. JLV
            Boffin

            Re: Standards Bodies need notice

            >strict requirements in EU and USA

            Market loss. Take these 2 out and your trinkets become a lot less profitable. C.f. Cyanogen becoming non-viable due to an India lockout.

            Fixing 80% of this problem is probably 20% of effort. Later they can worry about subtler things than factory default passwords

          2. Dan 55 Silver badge

            Re: Standards Bodies need notice

            If manufacturers are forced to follow standards in the west, they may as well do so for the rest of the markets. It gets more expensive to maintain two forks.

            Once tat is updated and tat from uncooperative manufacturers recalled, the west can legitimately begin to bring political pressure on foreign governments and economic pressure on foreign backbones. They might mirror the same certification steps in their countries or trading blocs to avoid this.

    1. DainB Bronze badge

      Re: Standards Bodies need notice

      That's just silly. How would you test for not yet known httpd or OpenSSL vulnerability ?

      You can hold anyone to any standard you want but you can't make a company that sold million routers with exploitable vulnerability and went out of business year later to fix anything.

      1. Voland's right hand Silver badge

        Re: Standards Bodies need notice

        That's just silly. How would you test for not yet known httpd or OpenSSL vulnerability ?

        There is a precedent - you cannot sell a car unless you guarantee that you will accept it for recycling and unless you provide spare parts for X years. While the laws which combine to form these reqs are different in Eu and US the net effect is the same.

        In any case, most of the insecure crap is resold with "brand labels" like Belkin, Dlink, etc and those are not going anywhere. In fact, let's hope that this incident contributes towards the reduction of "outsourcing your incompetence and putting a brand label on it".

        1. DainB Bronze badge

          Re: Standards Bodies need notice

          I just wonder if you notice subtle difference between $30K car and $50 electronic device and how differently both industries regulated.

          The only solution for this particular issue is a protocol that can stop traffic towards victim at originating ISP level. Not that hard to do really.

          1. Uffish

            Re: Standards Bodies need notice

            I put something like €50 of diesel into my car when I fill it up. The diesel fuel meets several standards that protect me but also specifically protect the population at large (sulphur content etc). The sale of non-standard fuel is illegal.

            Either you enforce some sort of legislation that makes IoTs less vulnerable or you live with an internet with the quality of service of Southern Rail.

          2. Doctor Syntax Silver badge

            Re: Standards Bodies need notice

            "I just wonder if you notice subtle difference between $30K car and $50 electronic device and how differently both industries regulated."

            Your $50 electronic device should already be regulated as regards electrical safety.

      2. Stoneshop

        Re: Standards Bodies need notice

        You can hold anyone to any standard you want but you can't make a company that sold million routers with exploitable vulnerability and went out of business year later to fix anything.

        However, once the regulating bodies declare non-conforming* devices to be illegal and requiring them to be taken offline, the next step should be to legitimise ISPs using the Mirai code (and other means) to identify vulnerable devices. If end users don't respond to notifications that they're using uncertified crap, they need to be sandboxed or taken offline entirely.

        Drastic, yes, and needs law and regulation changes, as well as secure processes for upgrading certified devices, so it won't happen tomorrow, but to me it looks to be the only way to get rid of IoT shit that's vulnerable and can't/won't be upgraded.

        * certification includes having a way to patch in case new vulnerabilities are found.

        1. DainB Bronze badge

          Re: Standards Bodies need notice

          I'm not even sure if you're trolling or just unable to understand what you're proposing. Do you really want to live in communist utopia where government can control which device you can use to connect to Internet ?

          1. Stoneshop

            Re: Standards Bodies need notice

            Do you really want to live in communist utopia where government can control which device you can use to connect to Internet ?

            Proposals to certify idIoT devices are nothing new, and equivalent regulations concerning wireless comms have been around for eight decades. This is to try to reduce the number of devices that are actively disturbing a particular communication medium, so not at all unlike the FCC and other agencies clamping down on inappropriate radio airwave use with bans and fines for using devices that lack certification.

          2. Doctor Syntax Silver badge

            Re: Standards Bodies need notice

            "Do you really want to live in communist utopia where government can control which device you can use to connect to Internet ?"

            I didn't see that being suggested. It's not a matter of controlling which device, it's a matter of controlling the safety standards they meet. They'll already by subject to all sorts of safety requirements. For instance the telecoms network operators will already have specs as to what can be connected to ensure it doesn't put harmful voltages on the line or draw excess current. Or are your telecoms providers communist-run?

  1. benderama

    I don't think you can say "we weathered the storm" AND "our systems are coming back online" in the same speech. If you're knocked offline, you did not "weather" an attack.

    1. Adrian Midgley 1

      Heaving to is the analogy there.

      Not making progress, but not sinking or breaking, is weathering the storm.

      Once the storm abates progress resumes.

      #philology

      1. yoganmahew

        Re: Heaving to is the analogy there.

        Well, maybe. Beached on a sandbank and refloated, if you want to use nautical terminology.

        1. DropBear

          Re: Heaving to is the analogy there.

          No. All your convoys were sunk last month by U-boots, if you want to use nautical terminology. This month they seem to arrive mostly. The ones from last month are resting on the sea floor and didn't "weather" jack squat; they just went down.

  2. Arbeebee

    Home Router Traffic

    Although I doubt any of my devices decided to join in the fun, how would I know? Any particular ports on the router to monitor?

    1. Destroy All Monsters Silver badge
      Holmes

      Re: Home Router Traffic

      The graph of I/O bit-per-seconds?

      Unless it was manipulated too...

      I'm looking forward to an Advice Dog Meme flood on the subjet of IoT shit.

    2. CommodorePet

      Re: Home Router Traffic

      Mostly port 23 / Telnet. MIRAI looks for that port, tries a bunch of known hardcoded values, then it usually finds a busybox shell running on ARM. It can then run busybox commands to download additional scripts and apps that perform the DDOS at whatever target they desire.

      1. frank ly

        Re: Home Router Traffic

        The ShieldsUP! tester at grc.com tells me that I have perfect stealthing, for the common ports at least. Does anyone know of any flaws in this form of testing?

        If I tether my laptop to my mobile phone, to use an internet access path from outside my domestic ISP, is there a 'probing' application I can use to check my home IP address for leaks and vulnerabilties? (I promise I'll only use it on my own home IP address.)

        1. Peter Gathercole Silver badge

          Re: Home Router Traffic

          The problem with Shields Up! is that by default it only checks the reserved ports 0-1023.

          You can use it to do custom scans, but the standard check will not check to see whether uPNP has opened up ephemeral ports through your firewall, and once these are set up, it could allow CnC channels to any devices.

          But most edge-firewalls allow outbound connections to a co-ordination server anyway (it really would be a pain to have to configure individual ports on the firewall), and once a session is established, will allow return control requests (remember TCP/IP sessions are bidirectional) even without uPNP (never wondered how your network attached, print-from-anywhere printer works? Well, this is it).

          Of course, it is necessary to get a foothold in the network for uPNP or outbound requests to be made, but who knows what is baked into the firmware of these IoT devices from China? I tent to run a Linux firewall, and do a sweep of the ports currently in use at the firewall, but it's difficult.

          It's all a bit of a mess. I favour using the vulnerabilities themselves to run destructive code on the IoT devices to break them, but that is illegal in pretty much all jurisdictions.

        2. Anonymous Coward
          Anonymous Coward

          Re: Home Router Traffic

          The ShieldsUP! tester at grc.com tells me that I have perfect stealthing, for the common ports at least. Does anyone know of any flaws in this form of testing?

          I am guessing it is potentially possible that a device could keep its ports closed until it sees a certain number of attempts to gain access, or access to a particular sequence of ports, or could advertise to the command centre its ready to accept command probes at specific times.

          It might be fairly easy to create a system which could fool GRC and other probes into thinking things are okay when they are not.

          Virgin Media have been telling - some would say scamming - customers into believing they are open to SSDP vulnerabilities even though they won't explain how that is and GRC shows no holes, and they say GRC showing no holes proves one is safe!

    3. Mage Silver badge

      Re: Home Router Traffic

      The most evil feature added after Autorun (Win95a and earlier Amiga) was uPNP, especially on routers!

      It should be illegal to have uPNP on a router/firewall and have internet without a firewall. It's only a partial mitigation, but would stop most of current IoT compromise.

      There is no complete solution.

      1. Metrognome

        Re: Home Router Traffic

        Sorry, wrong. uPNP and zero conf are godsends in home environments. Most peeps would rather have things work out of the box than worry if their fridge is actively DDoS-ing some far flung and mostly unknown entity that does "something on the internet".

        I remember the bad old days when I had to physically call up to my parents' house to re-work whatever mess they found themselves in. Now, uPNP, ports open automatically, things communicate and join each other and to be honest, they have precious little to secure on their devices much like most people.

        Also, to all the standards-talkers, persuade China first, discuss afterwards.

        Sorry to rain on your parade but some real-world perspective would help a lot of people understand the true limitations of the options they offer.

        1. Doctor Syntax Silver badge

          Re: Home Router Traffic

          "Also, to all the standards-talkers, persuade China first, discuss afterwards."

          No, require stuff legally on sale and/or in use to meet standards and China will be persuaded.

          1. Charles 9

            Re: Home Router Traffic

            China ships directly to you, AROUND standards. How do you stop that without a bureaucratic nightmare?

        2. Peter Gathercole Silver badge

          Re: Home Router Traffic @Metrognome

          UPnP.

          Convenient, yes.

          Secure, hell no.

          One thing it allows is any internal device to knock inbound holes in your firewall, without your knowledge or approval.

          I appreciate that without it, some consumers would have to learn something, but the downside is that all the IoT devices that sit inside home networks and use UPnP can potentially become a participants in a DDoS attack like this.

          Do consumers worry about this? Well probably none of them understand what it is that caused the DYN DNS outage, and even less about whether their house was part of the cause.

          But should we? Definitely yes, if we want to maintain a functional and usable Internet!

          I run my firewall with UPnP disabled, so it works inside my network for device discovery, but the firewall can't be controlled, and there's not that much that either I or the other members of my family have noticed that doesn't work.

          1. Metrognome

            Re: Home Router Traffic @Metrognome

            I'm fully with you on that. I operate under the assumption that everything is indeed compromised and then work backwards to see if anything was worth securing or not. But in the case of elderly relatives and associated clueless people, uPNP just works and it's good enough for them and their needs.

            As for DDoS'ing some unknown place, they couldn't care less and there's no way they'd start fiddling with any settings outside the default ones for the sake of no one. Hell, they hardly even do this for their own safety and on their own mobile phones, what makes you think they'd start now caring about DDoS-ing?

  3. stg

    Anyone surprised?

    http://www.passwordbingo.com/the-password-blog/2016/10/11/how-long-before-we-are-being-fined-for-having-lame-passwords

  4. John Crisp

    Campaign required

    Maybe El Reg can start a campaign a la 'Daily Fail' and then claim the credit if this monumental trip to Cockup City ever gets fixed :-)

  5. Number6

    What would help is for ISPs to cooperate and shut down customers who are clearly participating in the attack. Route 'harmless' http requests to a default web page explaining why they've been taken offline and what to do about it.

    I know such a feature could be abused, but I'm sure there would be a way round that with checks and balances and a proper procedure (yeah, right...)

    1. Anonymous Coward
      Anonymous Coward

      Oh absolutely. I've often thought about similarish schemes where you essentially stop a user in their tracks to force them to upskill (I'm fantasising, clearly), or address something.

      The problem as ever will be no company having the balls to do this. Fearing the paying customer backlash.

      I'd hate to be the phone jockey on THAT tech support call.

      The customer is not always right. I think this proves that with ownership comes a wider responsibility. It's like guns. You have to show you're responsible as the chance of misuse is lethal.

      Here, the chance of misuse is perhaps high, and boy, the botmaster behind this will be smug tonight w*nking himself silly over the sheer spread of problems the DDoS caused.

      1. Doctor Syntax Silver badge

        "The problem as ever will be no company having the balls to do this."

        Turn that one round. As one of Nixon's henchmen said, when you have them by the balls their hearts and minds will follow.

        Require them to do this.

        1. Charles 9

          You Can't REQUIRE a sovereign nation like China to do anything without a treaty. That's part of the definition of sovereignty.

    2. anoco

      I don't know about shutting down customers, but the ISPs could block their traffic to the affected IP very easily I think. Since the number of major ISPs is relatively small it would only take a few of them to make the attack manageable on the receiving end.

      A "DNS Alert" (similar to US Amber Alert) type of a thing alerting the 100 biggest ISPs on the planet would defang the attackers in minutes. You could even add another D to the attack description. DefangedDDoS

      OR.... deflect all the traffic to North Korea and watch their 3 servers go up in smoke, just saying...

      1. Charles 9

        Except there would be collateral damage. Those targets also have LEGITIMATE business via the web. You'd be doing the DDoS's job for them using that, and the way the IoT botnet works, they use the same legitimate requests we do, so they're camouflaged as well. As for the ISPs, they don't see a lot of traffic individually, and the amount they emit wouldn't probably surpass traffic from a home server running, say, a home camera feed.

    3. Ken Hagan Gold badge

      Would it be straightforward to limit domestic users to (say) one DNS query per second? Would this help?

      It's a well-known port, so the traffic would be easy to identify and handle separately. Domestic users are the most likely to be running dodgy IoT devices and the average domestic router ought to be configured with its own DNS cache anyway, so the throttling might not even be detectable by Joe User. It obviously wouldn't help against DDos attacks aimed at other services, but DNS seems to be a popular target, perhaps because the consequences are so spectacularly widespread.

      1. Updraft102

        "Would it be straightforward to limit domestic users to (say) one DNS query per second? Would this help?"

        DNS resolution is needed for a lot more than just the URL you typed into the browser or clicked in Google. Each of the secondary domains that site calls have to be resolved too, and there can be dozens of them on a fairly typical site on the internet. Running uBlock Edge, I see a counter that shows the number of third-party domains a site has attempted to contact, and it has sometimes exceeded a hundred of them. It's absolutely nuts (and much of it is about tracking and analytics related to advertising), but that's the state of things now.

        Not only that, but it would only work when it is a DNS server being attacked. That's not always the case.

        1. Doctor Syntax Silver badge

          "DNS resolution is needed for a lot more than just the URL you typed into the browser or clicked in Google. Each of the secondary domains that site calls have to be resolved too, and there can be dozens of them on a fairly typical site on the internet."

          To say nothing of the tertiary and quaternary domains. OTOH if this forced sites to serve all their own crap this could be seen as a useful by-product

  6. maccy
    FAIL

    "The blame is not with dyn"

    No, but it isn't with cheapo IoT either. DDoS seems to be built into the structure of the internet. Any system that relies on millions of components being "nice" is a system that is doomed to fail.

    1. Warm Braw

      Doomed to fail

      Unfortunately, that's pretty much right.The Internet was designed assuming that the network was under central control and that the only threat came from the physical destruction of its infrastructure: the goal was simply for packets to be able to get through if there was a path of some sort or another to the destination.

      Today's Internet has no central control and the main threat comes not from the physical destruction of its infrastructure but the unwanted behaviour of systems attached to it.

      The "other" protocol that was briefly being touted for public networks (X.25) had provision ("the D bit") for end-to-end flow control at the network layer which is one of the things that could be used to mitigate problems of this kind. However, even where X.25 was widely deployed, I'm not aware of networks (reliably) using the D bit feature. That's largely because network flow control is a hard problem to solve and the queue-or-discard mechanism of the Internet works at least as well as anything else under "normal" circumstances.

      However, it's a problem that needs solving. There has to be a back-pressure mechanism that sends a "stop" to the ingress point since there is no practical means of ensuring that every piece of equipment in private hands is well behaved. That of itself is not a panacea - and is potentially a new route to DDoS by spoofing the back pressure - and, if you look at the IPv6 gestation period, unlikely to be with us any time soon. It's also not the only issue that needs attention - more privacy, anyone?

      1. Charles 9

        Re: Doomed to fail

        "However, it's a problem that needs solving. There has to be a back-pressure mechanism that sends a "stop" to the ingress point since there is no practical means of ensuring that every piece of equipment in private hands is well behaved. That of itself is not a panacea - and is potentially a new route to DDoS by spoofing the back pressure - and, if you look at the IPv6 gestation period, unlikely to be with us any time soon. It's also not the only issue that needs attention - more privacy, anyone?"

        Intractable problem. The ONLY reliable way to manage a network is to introduce ironclad attestation. But that instantly eliminates privacy. What's happening is that the wired world is reaching the "wishbone" point: a point in which the third option is disappearing from the strain exerted from both extremes (in this case, the Anarchy of the current Internet and the Police State of a Stateful Internet). The pressures mean ANY third option quickly slides into one or the other extreme, rapidly NOT becoming a third option. Eventually, the wishbone will break, meaning no third option is possible anymore because it'll IMMEDIATELY gravitate towards one or the other extreme (the "winner"). In which case, only three options will be left: Anarchy, Police State, or Walk Out?

  7. Anonymous Coward
    Anonymous Coward

    Blame people

    Blame people for not changing the passwords

    Or blame the iot manufacturers for hard coding passwords, which happens too frequently

    Blame the DNS services for not being prepared

    But most of all blame the Ooh Nice Shiny culture that allows the IoT to be big enough to be dangerous!

Page:

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like