back to article Sysadmin unplugged wrong server, ran away, hoped nobody noticed

Another working week beckons so once again let’s kick it off with a fresh instalment of Who, me? For those of you new to the column, it's The Register confessional for IT pros who broke things. This week we meet “Hayden”, who confessed that: “Back in 1992 I was sort of a snot nose kid just a few years out of college that knew …

  1. technoise

    This story doesn't make sense.

    If the two cables were intertwined so that both would be disconnected if just one was pulled, how come his own server didn't switch off as well, the first time he tried to disconnect?

    And I presume the "blinkenlichts" were just power LEDs? You would surely shut down the server before disconnecting the power?

    1. Anonymous Coward Silver badge

      Pulling on the cable is not the same as pulling the plug.

      In the developed world, power plugs are mounted at right-angles to the cable so that pulling the cable gently can identify the plug without disconnecting it; I understand that in other parts of the world the plug is in-line with the cable so that I gentle pull would disconnect it (and I assume this is where your misunderstanding comes from).

      1. Anonymous Coward
        Anonymous Coward

        In your "developed world" the article already says that *one* of the plugs did indeed come out.

      2. IGnatius T Foobar

        plugs

        In the developed world, power plugs are mounted at right-angles to the cable so that pulling the cable gently can identify the plug without disconnecting it

        I understand that your intent is to declare the superiority of the UK power plug over the US power plug, and I might even agree with you, but in the "developed world" data centers use C13-C14 and C19-C20 connectors regardless of the locale and voltage. And those are never mounted at right angles.

        1. Angus Ireland
          Boffin

          C13/C14 is never at right angles, eh?

          but in the "developed world" data centers use C13-C14 and C19-C20 connectors regardless of the locale and voltage. And those are never mounted at right angles.

          It might not be common in "developed world data centres", but it's certainly done.

          http://cpc.farnell.com/schurter/r47850100/rewireable-iec-socket-90deg/dp/CN02277

          http://cpc.farnell.com/schurter/4736-0000/c14-iec-plug-e-r-a-rewireable/dp/CN20025

          Similar sockets and plugs can be found for C19/C20 if desired...

          1. Danny 14

            Re: C13/C14 is never at right angles, eh?

            my server cabinets have right angle leads. it makes the cable runs MUCH neater as they all cable 45d downwards straight out of the PDU.

        2. Anonymous Coward
          Anonymous Coward

          Re: plugs

          That’s true now, but the story was in the 1990s. The servers were on the floor rather than in racks. They had single power cords (no redundant PSU). The plugs would have been BS1363 rather than IEC C14.

          I have many unhappy memories of grovelling under the floor to find the right cable to unplug.

      3. J27

        So the developed world is the UK (and former protectorates)? US, Austrailia, Japan, Canada, most of Europe are all 3rd world countries?

        1. jake Silver badge

          Yep. Only the UK is developed. Every fule no that.

          Look at their completely modern home wiring system, for example. Outside of China (that well known bastion of safe and secure wiring habits), other first world nations that use Ring Wiring include Uganda and Indonesia. I think we should all learn from this fine example and pay close attention to TheBritishWay. Everybody else on the planet is obviously wrong.

          1. Yet Another Anonymous coward Silver badge

            Re: Yep. Only the UK is developed. Every fule no that.

            Everybody else on the planet is obviously wrong.

            Especially those that drive on the wrong side of the road.

        2. andy bird

          and your point is?

        3. Anonymous Coward
          Anonymous Coward

          So the developed world is the UK (and former protectorates)? US, Austrailia, Japan, Canada, most of Europe are all 3rd world countries?

          By now, the US has indeed sunk to that level. Dictator in charge, poor people being exploited to the max, no attention to Human Rights and absoluetly no interest in protecting the environment.

          The rest still has some resilience, but the question is how long it will take before they realise that being dependent on the US is a Very Bad Thing indeed as it is heading as a country for the same problem as Jared Kushner with 5th Avenue 666: an inability to repay mounting debt..

        4. onefang

          "Austrailia, Japan, Canada, most of Europe are all 3rd world countries?"

          Dunno about Japan, Canada, or Europe, but Australia has 3rd world quality Internet.

      4. Diez66

        In the developed world?

        In Spain, and a lot of Europe, the plug and cable are in a straight line, tug the cable, pull the plug.

        Sweet.

        We get other things right, maybe?

        Friend of mine did accidentally hit a big red mushroom on the wall that shut a whole test and development lab down, we went down the beer hall and never spoke of it again., Well until today.

    2. Little Mouse

      "both would be disconnected if just one was pulled"

      Only if both power cables had an identical length of cable betwixt the entanglement and the sockets.

      If not, the one with the least slack would pop first.

      1. Anonymous Coward
        Anonymous Coward

        > "both would be disconnected if just one was pulled"

        > Only if both power cables had an identical length of

        > cable betwixt the entanglement and the sockets.

        > If not, the one with the least slack would pop first.

        He was quoting -- and questioning -- the article. It said "both would come out" yet only one did.

        Why does the OP deserve all these downvotes?

        1. Fungus Bob

          "Why does the OP deserve all these downvotes?"

          For not understanding Murphy's Law... (maybe)

    3. David Nash Silver badge

      "the easy approach was just tug on the cable at the point I knew it was my power cable and with another hand wait for a tug at each of the plugs in the outlet.”

      “After five or six tugs the plug moved."

      This means he tugged the cable to feel which one it was. It didn't disconnect at that point.

      " I was like ‘bingo’, and promptly pulled the plug.”

      This was when he disconnected the plug that moved slightly. turns out it was the wrong one.

      What's so hard to understand?

      1. Anonymous Coward
        Anonymous Coward

        It's awkwardly phrased and some with dyslexia I could not follow

    4. Mark 85

      Maybe I'm looking at this wrong, but I'd shut it down, then disconnect it at the boxen. No need to rip up the floors, knock other cables loose. I also label the cables and leave a note in extra large bold about what server goes there and where it currently is located.

      1. Huw D

        Indeed. If you're desperate to pull the plug on something, pull it from the back of the something.

  2. Anonymous Coward
    Anonymous Coward

    I frown gently at hiding boo-boos - honesty is better

    and I can understand the intertwining, its like the leads at the back of my desk at work.

    It could almost be described as being akin to platted hair, I assume as all the cables were originally stored in coils.

    In my experience rooms like that have been comms cupboards where often they become uncontrolled due to no engineer ever visiting twice and each one knowing a different "best" way of doing things.

    1. Anonymous Coward
      Anonymous Coward

      Re: its like the leads at the back of my desk at work.

      Every time I power down my computer at home for the approximately annual dust vacuuming/ upgrade/ etc, I carefully detangle the moderately large collection of power/ usb/ network/ etc cables behind it. However, despite the fact that nothing goes back there until the next time, they never seem to be as tidy as I left them!

      1. Rich 11

        Re: its like the leads at the back of my desk at work.

        Having spent most of Sunday afternoon doing exactly this, I wholeheartedly agree.

        I have a theory -- a theory which is mine -- that the resident house spiders keep themselves amused during the long winter evenings by weaving all the cables together. Let's face it, they're big enough and have enough legs free to do the job.

        1. Mooseman Silver badge

          Re: its like the leads at the back of my desk at work.

          " -- a theory which is mine -- "

          Is this anything like your theory about brontosauruses?

          1. Aladdin Sane

            Re: its like the leads at the back of my desk at work.

            I've got a theory that it's a demon.

            1. Alister

              Re: its like the leads at the back of my desk at work.

              According to Sir PTerry, it's due to malignancy - the local hostility of things towards non-things.

            2. Sir Awesome

              Re: its like the leads at the back of my desk at work.

              A dancing demon? Eh, something isn't right there.

              1. Anonymous Coward
                Anonymous Coward

                Re: A dancing demon? Eh, something isn't right there.

                Nevertheless, evidence of dance demons /is/ available online :-)

                https://www.youtube.com/watch?v=2-x2TmIRkMg

                (now with a real video. hurrah)

            3. Anonymous Coward
              Anonymous Coward

              Re: its like the leads at the back of my desk at work.

              A dancing demon, no, something isn't right here...

          2. KarMann Silver badge

            Re: its like the leads at the back of my desk at work.

            My theory, which is mine, is that all power leads are thick at one end, much, much thinner in the middle, and then thick again at the far end.

            1. Anonymous Coward
              Anonymous Coward

              My theory, which is mine

              I believe that theory is actually my theory, which is mine

          3. Stevie
            Pint

            Re: Is this anything like your theory about brontosauruses?

            Well done Mooseman. I too heard the high pitch cough.

            Let us share e-beers and play the juke-box to drown out the "disbelieve" claque who live in a make-believe world of bent plugs and too many volts volts and Eskimos.

            1. Danny 14

              Re: Is this anything like your theory about brontosauruses?

              we just use a ring label and trunk all the cables. dial up a number or feed a colour and clip clip done. saved my bacon many times with network and power cables.

              1. John Brown (no body) Silver badge

                Re: Is this anything like your theory about brontosauruses?

                "we just use a ring label and trunk all the cables. dial up a number or feed a colour and clip clip done. saved my bacon many times with network and power cables."

                That's great in a well organised and largish operation. But I suspect in this case, especially all those years ago, the room probably started out with one server and slowly grew, and whoever was in charge at the time knew every box and cable. Then grew some more and the "one guy" moved on or got promoted and by then it was too late to strip it all out and tidy things up as the business could no longer afford the downtime.

                One of the few solution is to start a new room, where only new kit goes and maybe older kit which has to be powered down for repairs or whatever so can be safely moved to the "tidy" room. Eventually you end up with the entangled room running the last three or four servers that no one understands, no one appears to use but which will cause a catastrophe if turned off.

        2. Chris King

          Re: its like the leads at the back of my desk at work.

          "I have a theory -- a theory which is mine -- that the resident house spiders keep themselves amused during the long winter evenings by weaving all the cables together. Let's face it, they're big enough and have enough legs free to do the job",

          That's fine, until one of them calls for a human sacrifice.

        3. Terry 6 Silver badge

          Re: its like the leads at the back of my desk at work.

          You don't need house spiders to explain this - or rather, they don't explain it, because wire tangling is an essential property of the universe. If physicists weren't too busy messing around with trivia such as dark matter, string theory and quantum entanglement they'd be getting to the bottom of this. Cable entanglement is far more fundamental to understanding the universe. The real fundamental rule is that cable entanglement (e) = total mass of cable (t) multiplied by the square of the time unobserved (c). i.e. e=mc squared

          1. Baldrickk

            Re: its like the leads at the back of my desk at work.

            You don't need house spiders to explain this - or rather, they don't explain it, because wire tangling is an essential property of the universe. If physicists weren't too busy messing around with trivia such as dark matter, string theory and quantum entanglement they'd be getting to the bottom of this. Cable entanglement is far more fundamental to understanding the universe.

            I believe that there is in fact an entire branch of mathematics that describes how cables get tangled.

            Or I at least remember my A-Level Maths teacher telling me that there is.

            The story goes that the guy who started it got fed up of his headphones getting tangled in his pocket, and set out to understand just how it happens, and needed to come up with some totally new methods for handling it.

          2. onefang
            Boffin

            Re: its like the leads at the back of my desk at work.

            "If physicists weren't too busy messing around with trivia such as dark matter, string theory and quantum entanglement they'd be getting to the bottom of this."

            But string theory and quantum entanglement are precisely why cables get mysteriously entangled. Dark matter probably is involved to, it's usually dark for long periods of time where the cables are.

        4. Alistair
          Windows

          Re: its like the leads at the back of my desk at work.

          @ Rich

          I was wondering if Anne ever got married.....

          1. JassMan

            Re: its like the leads at the back of my desk at work. @Alistair

            Maybe he was just plagiarising Anne Elk's theory. But then again, someone much more erudite than me said "imitation is the sincerest form of flattery." Maybe it was Colton.

            Why is there no "tongue in cheek" emoticon when you need one?

        5. Jamie Jones Silver badge

          Re: its like the leads at the back of my desk at work.

          Those evil spiders really hate Christmas tree lights too!

        6. Fungus Bob

          Re: its like the leads at the back of my desk at work.

          "I have a theory -- a theory which is mine --"

          As do I - poltergeists.

        7. Aus Tech

          Re: its like the leads at the back of my desk at work.

          Don't you know that power, Ethernet, USB and other cables all have a life of their own? They seem to love to get all tangled up, despite all the care that a person takes when connecting the computer and associated peripherals.

      2. This post has been deleted by its author

    2. Doctor Syntax Silver badge

      Re: I frown gently at hiding boo-boos - honesty is better

      "I can understand the intertwining"

      It's how they breed. We all know that next time you look there are more cables than were there last time.

      1. JimmyPage Silver badge
        Joke

        Re: We all know that next time you look there are more cables than were there last time.

        In which case some sort of cross-breeding programme with ballpoint pens might be an idea ?

        1. Oh Homer
          Black Helicopters

          Re: breeding ballpoint pens

          My pens are more like socks than cables. They're an endangered species that disappear as soon as you leave them anywhere out in the open.

          Seriously, forget tracking birds, the RSPB needs to track the mass migration of pens, and figure out why they only ever migrate in one direction. Does the global population of pens fly south for the winter, only to die in the attempt?

          Either that, or quantum physicists need to forget about all that string compactification nonsense (haven't they figured out yet that you can just wrap your string into a ball?), and instead concentrate on finding and analysing all the portals to other dimensions that swallow pens, socks and ships, amongst other things.

          They could start with my tumble dryer. It's highly suspect. While I'm not actually aware of any ships lost whilst navigating the supernatural abyss of my tumble dryer, it seems to have consumed pretty much everything else that dared to even get close to the event horizon of its interdimensional gateway, or what the manufacturer claims is merely a "door".

          Then the bods should settle their unrelenting gaze on my co-workers, who are unquestionably also interdimensional portals to an alternate universe populated entirely by what were formerly my pens, established by the first pen to ever achieve self awareness, and which is now on a crusade to spread its sentience to all pen-kind and liberate them from the Basildon Bonds of literary servitude.

          1. The Oncoming Scorn Silver badge
            Thumb Up

            Re: breeding ballpoint pens

            “Somewhere in the cosmos, he said, along with all the planets inhabited by humanoids, reptiloids, fishoids, walking treeoids and superintelligent shades of the color blue, there was also a planet entirely given over to ballpoint life forms. And it was to this planet that unattended ballpoints would make their way, slipping away quietly through wormholes in space to a world where they knew they could enjoy a uniquely ballpointoid lifestyle, responding to highly ballpoint-oriented stimuli, and generally leading the ballpoint equivalent of the good life.

            And as theories go this was all very fine and pleasant until Veet Voojagig suddenly claimed to have found this planet, and to have worked there for a while driving a limousine for a family of cheap green retractables, whereupon he was taken away, locked up, wrote a book and was finally sent into tax exile, which is the usual fate reserved for those who are determined to make fools of themselves in public.”

          2. Anonymous Coward
            Anonymous Coward

            Re: breeding ballpoint pens

            "My pens are more like socks than cables. They're an endangered species that disappear as soon as you leave them anywhere out in the open."

            My pens leave their caps behind like abandoned skins.

            1. Adrian 4

              Re: breeding ballpoint pens

              The caps are the symbols of their servitude, which they cast off joyfully as they gain their freedom and run naked to their new life.

          3. Allan George Dyer

            Re: breeding ballpoint pens

            @Oh Homer - It's not your tumble dryer that you should be investigating, try The Back Of The Sofa, it is a portal to dimensions that exceed even L-Space.

          4. John Brown (no body) Silver badge
            Coat

            Re: breeding ballpoint pens

            "Does the global population of pens fly south for the winter, only to die in the attempt?"

            That could explain the rise of plastic particles in the oceans.

            1. Alter Hase

              Re: breeding ballpoint pens

              In my experience, the fertile ballpoint pens fly away, leaving behind large numbers of infertile (dry) ballpoint pens.

            2. onefang

              Re: breeding ballpoint pens

              "That could explain the rise of plastic particles in the oceans."

              And where squids get their ink.

          5. onefang
            Coat

            Re: breeding ballpoint pens

            I'm sure clothes hangers fit in there somewhere.

            I'll get my coat, it's the one sitting in a crumpled heap on the floor for some odd reason. I'm sure I hung it up properly before.

  3. Anonymous Coward
    Anonymous Coward

    I don't get why he didn't just unplug the cable from the back of the server. Surely any self respecting geek/admin has plenty of spare cables to use? It's a lot easier than hauling the server *AND* cables back and forth!

    1. JulieM Silver badge

      You probably don't get it because you're battle-scarred yourself. Or you've seen other people getting out of situations you decided you never want to get in.

      Until you've seen how situations can go wrong, you think everything is always going to Just Work.

    2. Oengus

      My thoughts exactly. If you unplugged it at the server ,after shutting it down, you could take it out of the server room to your office, fix the problem, then wheel it back to the server room and plug it back in ready to power up... No need to lift any tiles. No chance of mixing up the cables. No need to find an empty power socket when returning the server.

      1. John 110
        Angel

        "...If you unplugged it at the server..."

        Aah, the benefit of 20 years of hindsight. I wish I'd had that in my 20's...

      2. John Brown (no body) Silver badge

        "No need to find an empty power socket when returning the server."

        Yes, there is. If you remove a server and leave it's connections in the room, by the time you get back at least some if not all of the cables will have been used for something else.

    3. Velv
      Headmaster

      Experience is what you get when you didn't get what you really wanted

      1. Anonymous Coward
        Anonymous Coward

        Experience is what you get when you didn't get what you really wanted

        Or as the (allegedly Belgian) proverb in one of my Christmas sweeties had it: "Experience is a comb that natures gives us once we're bald".

      2. Huw D

        WYSIWYDAF.

        What you see is what you didn't ask for.

    4. Anonymous Coward Silver badge

      There is a possibility that the fault was a frail power cable causing brown-outs... you need to verify the power cable too, or plan on replacing it which involves yanking it out anyway.

      1. Anonymous Coward
        Anonymous Coward

        [There is a possibility that the fault was a frail power cable causing brown-outs... you need to verify the power cable too, or plan on replacing it which involves yanking it out anyway.]

        You have also been around too long, in a place like the one described, you would just add a new one - and still avoid the problem. It adds the the chaos that is under the floor, and therefore does not exist.

        Although I do recall working in one place where the floor tiles would not quite reseat due to the volumes of cable underneath the network cabinet... but that's another story.

        1. Paul

          I haven't been to Telehouse in a long time, but in the original building there were areas where cables under the floor caused the floor tiles to bulge up!

        2. Anonymous Coward
          Anonymous Coward

          Underfloor cabling hides a multitude of sins

          I used to work for a Government department that migrated from Alpha servers to Windows and ended up with a huge computer room complete with raised floor that was almost empty. As part of removing the Decs, the sysadmin took it on himself to empty the underfloor area of all excess cables.

          After 2 days and 3 ute loads of used cables, we could once again see the concrete floor and power points under the raised floor. :)

    5. cdegroot

      Indeed. When I ran my own ISP, I found out the hard way that cable management wasn't one of my strengths, so I always unplugged everything at the server side in our racks and let it dangle. A replacement, new server, or whatever was bound to need these connectors anyway at some time in the future.

      1. Anonymous Coward
        Anonymous Coward

        "A replacement, new server, or whatever was bound to need these connectors anyway at some time in the future."

        You still need to identify the plug if it contains a fuse that could be anything from 1A to 13A.

        1. Danny 14

          actaully it usually goes: unplug, wheel away, fix, bring back, wonder where your cable has gone, find another and add it to the franken-mess. find random canle 6 months later.

        2. Chris G

          "You still need to identify the plug if it contains a fuse that could be anything from 1A to 13A."

          Only if you are in the developed world.

          1. Anonymous Coward
            Anonymous Coward

            "You still need to identify the plug if it contains a fuse that could be anything from 1A to 13A."

            Only if you are in the developed world.

            You'll find out pretty quickly if the fuse is too small!

            Though I'm sure it'll generate some downvotes, the idea of having a fuse in a cable is ridiculous. If a cable is in the US is UL listed, it will have the correct gauge for the length given the NEMA/IEC plug type, and it isn't permitted to for example have an 18 gauge cable that's 50 feet long just because you think you will only draw 1A across it.

          2. DiViDeD

            Re: anything from 1A to 13A

            Nah. In the developed world, the plug came with a fitted 13A fuse, regardless of intended load, and matching the fuse to the intended load went in the too hard basket at install time.

            After all, you never know when you might need to run a 3 bar electric fire from that radio alarm cable.

            1. albegadeep

              Re: anything from 1A to 13A

              In the developed world on the west side of the pond, we use radial circuits, with wire designed to take more current than the breaker's trip point. Cables/plugs from machine to wall (and outlets, for that matter) are also designed for full current, so they don't need a fuse - and are therefore interchangeable. If the individual equipment needs a fuse, the manufacturer puts one in the equipment.

              1. Phil O'Sophical Silver badge

                Re: anything from 1A to 13A

                In the developed world on the west side of the pond, we use radial circuits, with wire designed to take more current than the breaker's trip point. Cables/plugs from machine to wall (and outlets, for that matter) are also designed for full current, so they don't need a fuse - and are therefore interchangeable.

                We used to do that sort of point-to-point "star" config, way back in the 1950s. Then it was realised that increasing use of appliances meant that modern homes needed more than one socket per room, and putting 6 or 8 16A sockets in each room, with 3 or 4 20A cables back to a 20A breaker, was overkill. It also required a lot of cable and a large distribution board. No-one would (or could) ever draw the full 16A from each without blowing the main house fuse, and most would only be used for low loads like lighting or radios. Today, of course, that is even more true.

                The logical solution is a "bus" architecture: a single 30A distribution circuit per 1000 sq. ft (approx one per floor in a normal home) which covers the total load needs for that floor. You can have as many outlets on the bus as you need within that area. To avoid the need to put a heavy 16A-20A cable on light duty equipment like radios or lamps, each can instead be fitted with appropriate 3A or 5A cable, fused in the plug to protect it from fire. The appliance could, if necessary, have its own protection (TVs & computers do, table lamps usually not).

                Overall it's a much simpler solution, which requires fewer resources.

                1. albegadeep

                  Re: anything from 1A to 13A

                  "The logical solution is a "bus" architecture"

                  I think we're talking about pretty much the same thing, but maybe I wasn't clear on my original statement. Here's an example of what I mean: My garage has 14 outlets, each designed for 120V 20A. I'm likely to pull some serious current in total (electric heater, air compressor, and lathe all at once, for instance), so it's split in 4 circuits, each with its own 20A breaker. 2-conductor (plus ground) wiring rated for 20A starts at the breaker, goes to the first outlet, from there to the second, and so on, stopping at the last outlet on the circuit - "radial" as opposed to ring. Pull over 20A on any one outlet, and a breaker trips, but there's 80A available in total.

                  Most equipment (house or garage) has a built-in, hard-wired cable rated higher than what the equipment should ever pull - but no fuse. Lamps, radios, and other low-power devices often have smaller cables; I can see why a fuse would be a good idea, in this case.

                  In contrast, most computer equipment has a cable that is removable at both ends. These are typically larger than those for lamps and radios, but are a fairly standard cable size and (presumably) rating - any "computer cable" can be used in any application, because the protection is provided by the breaker, and the outlet and in-wall wiring can take the breaker's trip current. If the equipment itself needs protecting, the manufacturer builds in a fuse to the equipment, not the cable.

                  (Side note: most house outlets are 15A; 20A outlets have a slightly different pin configuration. 20A outlets will accept 15A plugs, but not the other way around.)

        3. Anonymous Coward
          Anonymous Coward

          You still need to identify the plug if it contains a fuse that could be anything from 1A to 13A.

          No you don't. That fuse is there to protect the cable, and should be sized appropriately for the cable it's in. The server will have it's own fuse or other protection sized appropriately for it.

          What you do need to do is ensure that the cable you are using is rated appropriately for the server.

    6. Anonymous Coward
      Joke

      > I don't get why he didn't just unplug the cable from the back of the server.

      Capt. Mainwaring: "I was wondering who'd be the first to spot that"

  4. Anonymous Coward
    Anonymous Coward

    X.25

    On a side note, I worked in telecom for a few years and worked with a guy that I'm pretty sure helped design X.25. He would speak for hours about how superior it was to anything else until you managed to silently sneak away, or you died from sheer boredom.

    1. Dingo_Nates

      Re: X.25

      Was it me - I worried the best seller ‘X25 Explained’, published in 1984, updated in’86 etc. I remember a job interview where I was asked for 4 differences between X25 1980 and 1984 versions. Needless to say the interviewer got bored with my answers...

      1. Dingo_Nates

        Re: X.25

        worried = wrote (thanks IOS)

        1. Anonymous Coward
          Anonymous Coward

          Re: X.25

          year will calm

      2. heyjimmy

        Re: X.25

        ...which I still have in my Bookshelf. It was my reference book on the details of X.25 when I was working on debugging our companies X.25 product. It sits next to my "OS/8 Handbook". And "An Introduction to Programming" which is a valid title as long as you are into PDP-8 assembly code

        1. Steve Davies 3 Silver badge

          Re: PDP-8 programming

          If you are still inclined that way, you can induldge in a bit of PDP-8 development.

          I have one of these on my desk at the moment.

          http://obsolescence.wixsite.com/obsolescence/pidp-8

          PDP-11 versions are on the way.

    2. Anonymous Coward
      Anonymous Coward

      Re: X.25

      We have a new woman working in our office. At first I was thinking "it'll be nice to actually work with a woman in IT for a change", but it turns out she's a massive fan of Solaris and will quite happily talk your ear off about it's merits and superiority compared to every OS ever conceived. It was during the first 30 minutes of rant that I realised that the women in IT are capable of being just as annoying as the men.

      That's equality I guess.

      1. imanidiot Silver badge

        Re: X.25

        Solaris? Really?? Isn't that obsolete?

        1. Anonymous Coward
          Anonymous Coward

          Re: X.25

          No because there’s still nothing to replace it, except AIX or HP/UX. Everything else is a massive retrograde step.

          Unfortunately it will soon be obsolete, blame Larry for its imminent demise.

          1. Anonymous Coward
            Anonymous Coward

            Re: X.25

            You had me right up until the "HP/UX" comment. HP-UX is an abomination that deserves to die a horrible death. Tru64 was much better, HPaq picked the wrong *IX to kill :(.

    3. Anonymous Coward
      Anonymous Coward

      Re: X.25

      guy that I'm pretty sure helped design X.25

      The protocol? That would be Louis Pouzin and the late Hubert Zimmerman, mid 1970s. I never knew Pouzin, but I worked for Zimmerman for a while much later on. Nice guy.

  5. Alan Brown Silver badge

    Are you sure they were servers?

    Most servers have (at least) two power supplies.

    something doesn't smell right.

    1. Aitor 1

      Re: Are you sure they were servers?

      Most big Sun servers of the era did.. but they WILL work on one. Albeit they will overheat and HDDs might get damaged.

      I have seen many a data center with plenty of them running on a single cable..

    2. Rich 11

      Re: Are you sure they were servers?

      Most servers have (at least) two power supplies

      I'm going to guess that you were still in primary school back in 1992. Except for minicomputers I can't remember any standalone servers having more than one power socket, which is one of the reasons we switched to fully rack-mounted kit 15 years ago.

    3. Mooseman Silver badge

      Re: Are you sure they were servers?

      Our Dell server ran happily on it's one power supply for years. Replacing it a couple of years ago with a shiny new machine caused mild panic when they realised they would need an extra power socket.

      1. Adrian Midgley 1

        Until they realised they could

        use an old plug and put both cables into one.

      2. David Nash Silver badge

        Re: Are you sure they were servers?

        Maybe it was a server by nature of the work it was doing, rather than the hardware. And it was a test server.

    4. phuzz Silver badge
      Alert

      Re: Are you sure they were servers?

      I once unplugged one of the six power cables going into an HP blade enclosure and the entire thing powered off. Thirty seconds later I was having to deal with an irate boss, but he did calm down and agree that they were specifically supposed to be redundant PSUs, and so perhaps it wasn't entirely my fault.

      We eventually traced it to a bad PSU that would only take about 20% of it's rated load before failing. Thanks HP.

      1. Anonymous South African Coward Bronze badge

        Re: Are you sure they were servers?

        Quite interesting, and what a way to find out about that "feature"...

  6. Anonymous South African Coward Bronze badge

    Most of our stuff in our server room have redundant power supplies, so power cables for one PSU go to the left, second PSU go to the right.

    And ziptied cables are an abomination, rather use velcro ties instead.

    1. Uffish

      Re: Cable Ties

      "And ziptied cables are an abomination..." How about modifying a beautifully, waxed-thread, laced bunch of cables for real frustration.

      1. Alan Brown Silver badge

        Re: Cable Ties

        "How about modifying a beautifully, waxed-thread, laced bunch of cables for real frustration."

        The real frustration was threading the sodding things. I usually bled profusely all over my telco lacings and was profoundly grateful for being able (allowed) to use zipties in the mid 80s. Velcro bindings came (much) later and there was great rejoicing (as long as people didn't cut them exactly to length for the original bundles, which they usually did)

      2. Chozo

        Re: Cable Ties

        I like doing cable lacing :)

        With so few having the skill anymore it earns me the odd bit of beer money enhancing custom PC builds and motorcycle looms.

        1. Anonymous Coward
          Anonymous Coward

          Re: Cable Ties

          cable lacing was a skill I learned way back when i was doing a HND in electronics and radio engineering... A well laced loom is a sight of beauty.

  7. lansalot

    citrix and scvmm

    Brought up a new SCVMM server and pushed the updated client out to a few VMs. No issues, no reboot needed. So pushed it out to 3 hosts, each hosting 7 citrix xenapp VMs (so approx 200 users were being served).

    The hosts didn't take too kindly to the new client, and promptly blue-screened. However, as the estate wasn't at its most stable at the time, users were conditioned to logging back on again when citrix disappeared. Not a single call came in. So I kept quiet, until now...

  8. Dr Paul Taylor

    Around time of this story, while I was working at South Ken Tech, a JCB sliced through the main power cable in the street. The power was not off for very long. However, this was before journaling filesystems, so the machines took an hour or more to do all their fscking to recover.

  9. OzBob

    Honesty is the best policy

    but maybe not right away. My second day at my first IT job (large government department, late 80s) I brought down the COBOL based mainframe for 2 days by pressing the wrong set of keys on a locked up job. I confessed to my then boss, and got let off with a warning.

    Roll forward 10 years and I am the 2ic of the support team and I mention this little mishap to my different boss. He says "Oh yes, I remember not sleeping for 36 hours while I fixed that. Good think you never did it again".

    1. YetAnotherLocksmith Silver badge

      Re: Honesty is the best policy

      And now you know why he ended up being your boss!

  10. Anonymous Coward
    Anonymous Coward

    After reading this column for a while I notice one thing. all these cock ups happen in regular industry where the worst thing can happen is a little bit of data loss. maybe important data or a few people dont get emails for a few hours, but lets face it, nobody is getting killed....

    I cut my teeth in IT support in a large laboratory complex that is a little south of Oxford, that up to a few years ago did not appear on maps. These days its almost entirely to how it was. No armed police on the gate, no fire station, no on-site hospital. Buildings didn't have names, just numbers. One or two had been concrete filled after someone made a boo boo..... The sort of place that if something went wrong then it could be contained on site and that includes anything details of what happened.

    It was one of those places that you just did not make mistakes. The paperwork alone to get access to the server room meant you could not just pretend it was not you, and even inside the server room were servers in even more secure cages that anyone doing any work on had to be accompanied by 2 members of staff.

    one building, you could only get into if someone in the building "sponsors" you. They had to accompany you everywhere you went in the building. You even made sure you did not have to visit the bathroom while in the building, because of the health monitoring system meant more than the usual paperwork when going for a number two. It was bagged and sent for analyses...

    The particular company I worked for was also responsible for a few other sites across the UK, One site on the cost in the north west had an alarm system where it would beep every 15 seconds to let you know it was working. If it stopped beeping then you were in trouble an alarm would sound and depending which colour light was flashing depended which emergency route to take. They were marked out on the floor in different colour lines. One particular alarm, it it sounded it guided you to a cabinet with respirators. Depending on what colour light was flashing you took a blue or a white tablet. If one particular alarm was to sound, the instructions were to follow the white line as fast as you possibly can move yourself and if someone you were with fell over, leave them....

    So, after working in places think this for a year or so, where a little mistake could cost lives, you just did not make mistakes. to this day, I still do the same things I learned back then like putting labels on each end of a power lead

    I would advise anyone to take a contract in a place like that to learn how not to screw up....

    AC for obvious reasons like OSA.....

    1. Anonymous Coward
      Anonymous Coward

      "So, after working in places think this for a year or so..."

      Not quite the same contrast , but having worked in

      -Education

      -Private industry (Atos)

      -NHS

      I see radically different standards being applied.

    2. ArrZarr Silver badge

      That sounds terrifying...were the threats biological, chemical, radioactive or magical?

      1. Stu J

        My bet is that it was Harwell...

        (radioactive)

        1. Loyal Commenter Silver badge

          My bet is that it was Harwell...

          My bet is Porton Down (chemical / biological) and Sellafield (nuclear).

          1. Anonymous Coward
            Anonymous Coward

            and Sellafield (nuclear).

            Sellafield was the fancy name they gave to the site so to dissociate with the nightmare the former name Windscale had associated with it. The power station was still called Windscale.

            going back way before I worked there, for about a year the the fast breeder reactor was leaking, but they failed to report it because of the bad publicity and shit storm because of the number of times the fast breeder had leaked previous. When I was there, the iconic white ball shape of the fast breeder was a dirty grubby thing covered in patches.... but the photos of it in the visitors centre and the adverts on the tv inviting people to the visitors centre all showed the identical, but decommissioned fast breeder at Dounreay.

            I did also do some work at Dounreay, going from building to building you had to call for a police escort. We got in trouble one day when we decided to make a dash for it from one door, to the next that was about 5 metres away.... we got caught on camera and nearly got thrown off site...

          2. Anonymous Coward
            Anonymous Coward

            My bet is that it was Harwell...

            My bet is Porton Down (chemical / biological)

            Aldermaston was scary - I went there regularly. AC obviously >>===>

          3. Loyal Commenter Silver badge

            ...oddly enough, I almost wrote Sellafield (ex Windscale) before deciding that the sentence would end up with so many parentheses in it that it would risk looking like a Lisp script.

        2. Anonymous Coward
          Anonymous Coward

          My bet is that it was Harwell...

          it was indeed.... as well as the sister site nearby at Culham. The site with the criticality alarm was Windscale.... There was one particular funny story regarding putting new computers into the reprocessing plant and the use of a lift.... but I still think that maybe covered by OSA....

      2. Andytug

        For magical threats..

        there would have to be eight different colours of emergency exit lights.....

        1. Ochib

          Re: For magical threats..

          You mean 7a or the one in between seven and nine. You know, two cubed. Four plus four. Twenty-four divided by three .The one you can't say out loud or else you'll get ate. What do you mean, the ichor god Bel-Shamharoth can't tell the difference between two homonyms like the past tense of the verb "to eat", and the number eight... OK, so they sound identical but have different meanings, that's what a homonym means. Fine, but is this really the time for a grammar lesson?

        2. Anonymous Coward
          Anonymous Coward

          Re: For magical threats..

          there were different combos of lights and sirens..... the one that you shat your pants for was the big loud one that was the same as an air raid warning,.... It was tested every day but you still shat yourself when it went off....

          There was a week long induction when you first went on site with a test you had to pass to say you understood the ridiculously complex warning systems...

          1. Danny 14

            Re: For magical threats..

            there were actually two in the north west. a lesser known site in Preston called springfields. I didn't work there but my brother did. He had a few horror stories as they had "stuff" go missing and needed to hide the fact. Awesome.

    3. Anonymous Coward
      Anonymous Coward

      Would that be the kind of place where when the site alarm sounds they lock the gates to keep everyone in (so as not to spread the glowy) ?

      1. Anonymous Coward
        Anonymous Coward

        site alarm sounds they lock the gates to keep everyone in..

        The gates are permanently locked....

    4. The Oncoming Scorn Silver badge
      Holmes

      The Apollo Fecal Containment System

      The physical act of defecating in a bag was difficult*.

      On Gemini flights, the defecating astronaut couldn’t give his companion too much distance from the bowel movement; the spacecraft was about the size of the front seat of a small car.

      On Apollo missions, the astronaut needing to move his bowels would float his way into one corner while the other two men would move as far away as possible.

      He’d typically strip completely nude, removing rings and everything. Water was limited on board so washing fecal matter from clothing was impossible.

      Then he’d stick the adhesive opening to his naked buttocks and use the facilities. The whole exercise from stripping down to redressing could take more than an hour.

      And it didn’t always go according to plan. Rogue waste terrorized Apollo 10 on their trip back from the Moon.

      LMP Gene Cernan: Where did that come from?

      CDR Tom Stafford: Give me a napkin quick. There's a turd floating through the air.

      CMP John Young: I didn't do it. It ain't one of mine.

      LMP Cernan: I don't think it's one of mine.

      CDR Stafford: Mine was a little more sticky than that. Throw that away.

      CMP Young: God almighty.

      All three: (Laughter)

      *I seem to recall a interview where the bag was then filled with a blue preservative, that was mashed in with the fresh contents (Possibly by another astronaut, hence the comment about really knowing who your friends were) to be examined back on Earth.

      1. Gene Cash Silver badge

        Re: The Apollo Fecal Containment System

        Legend has it that on the Apollo 8 mission around the Moon, Frank Borman didn't. For six days.

    5. AustinTX
      Mushroom

      Fast food testing kitchen? :o

    6. Anonymous Coward
      Anonymous Coward

      on a hot day......

      on a very hot day you would make excuses to go in the server room.

      you went through a security door and up a short flight of steps, the room was a raised floor with little holes in every other floor panel that cold air was pumped through....

      you just lay back on the floor for a few minutes to cool right down !!

      1. DiViDeD

        Re: on a hot day......

        Ah yes. BP had a huge server room (we are talking football field sized here) at one site, but most of the server kit was little Compaq Proliants siting in acres of space in a room that was designed for a previous generation of machines. All air conditioned, with airlock doors, it became the wheelie chair volleyball chamber of choice, come summer.

    7. Huw D

      "After reading this column for a while I notice one thing. all these cock ups happen in regular industry where the worst thing can happen is a little bit of data loss. maybe important data or a few people dont get emails for a few hours, but lets face it, nobody is getting killed...."

      Do you even BOFH, bro?

  11. Prst. V.Jeltz Silver badge

    "because of the health monitoring system meant more than the usual paperwork when going for a number two. It was bagged and sent for analyses..."

    Wow . Did you have to bring it in if the no2 occured at home?

    1. Anonymous Coward
      Anonymous Coward

      Wow . Did you have to bring it in if the no2 occured at home?

      we didn't, but I cant be sure what the procedure was for regular workers in that area....

  12. Anonymous South African Coward Bronze badge

    You even made sure you did not have to visit the bathroom while in the building, because of the health monitoring system meant more than the usual paperwork when going for a number two. It was bagged and sent for analyses...

    Now what if you had the runs from a lager doctored by your friendly BOFH?

    1. Anonymous Coward
      Anonymous Coward

      Now what if you had the runs from a lager doctored by your friendly BOFH?

      Then you DEFINITELY didn't go in the building - he probably put something in it that would get you fired when detected...

  13. Grant Fromage

    pluggery and who knows?

    Possibly the ultimate case of this was BBC world service and a gentleman who failed his probation period in the very early 80`s

    There were acres of audio jackfields and a daily schedule, this was sparse after 2200 and one would with a pencil go back in time and unplug the patches very carefully. The standard BBC radio and TV lines format wasn`t helpful, it was indexed only by time and you had to be careful to scan up to 24 hours before.

    The legendary George Harries told me he caught him on several occasions pulling a plug and when the phone rang putting it back, which was not an approved method of jackfield takedown,

    At that time before it had the modern label of diversity aunty tried to get minorities in and lost that properly trained/qualified is not discriminatory,

    On one day Sterling rang George at shift start and said " I`m going to be late I am in Kingston" Don`t worry lad we`ll see you lunchtime.

    " No, Kingston Jamaica i was seeing my gran".

  14. Anonymous Coward
    Anonymous Coward

    access request hitler

    At least a few more decades elapsed before someone thought of robust procedures for access to these areas.

    I especially like jumping through several levels of pomposity to enter an area a cleaner walks in an out of via a side door with a floor polisher multiple times between nipping out for a fag.

    ICBM silo cleaners probably have both launch codes incase one of the two military types loses theirs.

    1. The Oncoming Scorn Silver badge
      Stop

      Re: access request hitler

      Sterile areas for pill packing was a nightmare, if I could get away with handing a keyboard or mouse over the "sign in" desk I would.

      Donning the snood, paper overalls & foot coverings, transitioning from one "clean" area to a cleaner area required a extra set of foot coverings over the first, stepping back into the first area was fine, but stepping back into the cleaner area required a fresh set to be applied over the prior coverings.

      Sterility wise this was all very fine & splendid, but night shift somehow managed to smuggle in & build a home gym into a disused building on the building roof & use it, that was subsequently found on a random investigation\check.

      1. Anonymous Coward
        Anonymous Coward

        Re: access request hitler

        transitioning from one "clean" area to a cleaner area required a extra set of foot coverings

        at windscale we had mats that were near enough sticky tape on every doorway (they were made by 3M) to stop contamination being spread from one area to another. I believe the mats were sent for testing 4 times each day.

        1. J. Cook Silver badge

          Re: access request hitler

          [RedactedCo] has these in our data centers, although one of them is jet black because A) No one has bothered to peel it off and expose a ‘fresh’ pad; and B) there’s a ludacris amount of traffic in and out of there.

    2. AustinTX

      Re: access request hitler

      "I especially like jumping through several levels of pomposity to enter an area a cleaner walks in an out of via a side door with a floor polisher multiple times between nipping out for a fag."

      Having worked as a janitor in a number of sensitive areas as a lad, I can vouch for this.

  15. Tom 7

    I got off by owning up.

    A meeting was held to discover why a design change had failed and about 30 people were invited. While waiting for the meeting to start I was tidying up my desk and found a piece of paper in the cambrian layer and realised that it was all my fault - I'd mislaid this instruction to rotate a test component so all the 20 or so devices had this (probably pointless but that's another matter) test component in the same orientation so all the devices could be pre-tested on the same rig before going on to their own individual tests. Oh fucking hell I'm going to get fucking slaughtered here. I made it to the loo before going to the meeting and prepared to die.

    The chair started the meeting and I could feel the anger in the room and at the first opportunity in the proceedings fessed up and sat back waiting for the shit to hit and working out what job I could get next and nothing happened. There were some comments but somehow my confession had caused some kind of impedance mismatch and I spent two hours sitting there waiting while the meeting seemed to be trying to hunt down someone who could be blamed for what I'd confessed to. I went from a state of abject terror to one of abject confusion to one of abject amusement if such a state can exist. Which it can and after eleventeen pints that evening I went into work the next day expecting them to have worked out I should really die but I just spent four grand* fixing the problem and updating the procedures and no-one ever mentioned it again in my presence.

    *about the cost of the meeting coincidentally.

    1. John Brown (no body) Silver badge

      Re: I got off by owning up.

      "while the meeting seemed to be trying to hunt down someone who could be blamed for what I'd confessed to."

      Sometimes, doing the unexpected can have amusing results. Being honest and 'fessing up to a mistake seems to be one of the most unexpected things to ever happen in a modern business environment and can cause inter-dimensional instability. Please don't do it again!

  16. Slacker20012

    Or just unplug at the back of the server rather than at the wall.

  17. StuntMisanthrope

    X25, my Gunship's named X400.

    Back in the day, when men were men and boys were well, er, boys. #oldspice #passthesaltpepperhashandtabasco

    1. pɹɐʍoɔ snoɯʎuouɐ

      Re: X25, my Gunship's named X400.

      Back in the day,,,,,,,,

      men were real men, women were real women, and small furry creatures from Alpha Centauri were real small furry creatures from Alpha Centauri. And all dared to brave unknown terrors to do mighty deeds to boldly split infinitives that no man had split before......

  18. Asylum_visitor

    Always pull the power out of the back of the host ;)

  19. johnmc

    Hayden, Pfffft. Where is the COLO manager???

    Hayden may have been the agent, but the master mind was the COLO manager.

    I worked for the #1 telco in the US for 25 years. Bunch of data ops firms before that. This disaster was laid down by lax management.

    * Production is sited in one part of the center not hodpodged where s#$@% fits.

    * Cords are in a locker or the whole unit is in a locked data cab.

    * The only dink authorized to muck with infrastructure is the shift tech. They have strict procedures on HOW.

    * If it can be screwed down it SHALL be screwed down.

    I remember a telconference I attended. 4 VPs, 6 directors all of them screaming at each other for a system being down. 10min in I called the center manager offline and asked if I could talk with the shift operator. 10min after that with a bit of confessional it was determined that a V.35 cable had been knocked loose that had not been screwed down. Had it not been my habit to buy everyone beers whenever I was in their area we might have never known. And yes I lied about how it happened. It was not worth the witch hunt, battling the union rep and losing three good techs.

  20. Handle123456

    Why not unplug it at the other side?

    I mean usually you can disconnect the power cable at both sides, both from the wall socket and from the computer. I don't think I ever met a computer that had the cable hardwired. If he had problems untwisting the cable, he should have disconnected it from the computer, leave it there and use a spare cable in his office.

  21. pɹɐʍoɔ snoɯʎuouɐ
    Trollface

    Who was it....

    come on,,,, fess up.... who works at TSB?

  22. Anonymous Coward
    Anonymous Coward

    Working on a helpdesk....

    ... computer once under a desk that had a carpet because I couldn't be bothered to take it to my desk. I finally found out what happens to computers with static shocks. Had always read about it. Always touched the side of the chassis but never, thankfully, experienced what happened. Until that that. I was working on the graphics card. It was loose. Shift it a bit, heard a pop and the user told me the PC has just turned off. Turned it back on and the addon graphics card was now dead.

    I made up some technical bollocks and got them to order a new one.

    Oops.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like