Sign in / up

back to article Boss made dirt list of minions' mistakes, kept his own rampage off it

Welcome again to On-Call, The Register's Friday column in which we share readers' tech support morality tales. This week, meet “Omar”, who in the early 1990s worked as a developer for an engineering company powered by a pair of ageing Digital Equipment Corporation mainframes. The workplace had another odd feature: an IT …

COMMENTS

Post your comment
House rules Send corrections
Add to 'My topics' unstar *

Page:

  1. jake Silver badge

    Fragile. Very fragile.

    An IT manager who is cluefull enough to know not only which cabinet(s) contains the RAM, but also which card(s) to pull, actually going through with removing said RAM from a running machine? On the word/whim of a vendor?

    Most of these stories are iffy at best, but this? Pull the other one.

    49 12 Reply
    1. Admiral Grace Hopper
      Reply Icon

      Re: Fragile. Very fragile.

      Yes, I'm with you all the way, right up to the point where I compile my personal list of people for whom I have worked who still retain a little of the technical nous that got them to where they are now, but have had all the common sense removed by a few years of management. Much as I want to live in a world where this could not happen, I have done too often and for too long.

      47 2 Reply
      1. wolfetone Silver badge
        Reply Icon

        Re: Fragile. Very fragile.

        To be honest the UK have some idiot as health minister who has absolutely no clue in how to manage the NHS or background in healthcare, yet he finds himself able to dictate what the needs and how much it costs.

        So I'm not at all surprised at this story.

        37 15 Reply
        1. MrRimmerSIR!
          Reply Icon
          Facepalm

          Re: Fragile. Very fragile.

          I'm sure his replacement will have an iPad to do the difficult stuff.

          6 0 Reply
    2. Anonymous Coward
      Reply Icon
      Anonymous Coward

      Re: Fragile. Very fragile.

      When I was a total noob, an IT manager decided to prank me by claiming a machine had the new type of ram that you could hot-swap while booted. He didn't actually let me go through with it.

      19 0 Reply
      1. Alan Brown Silver badge
        Reply Icon

        Re: Fragile. Very fragile.

        Scarily, there _ARE_ some systems with hotswap ram. :(

        2 0 Reply

        1. This post has been deleted by its author

          1. jake Silver badge
            Reply Icon

            Re: Fragile. Very fragile.

            That's nice, guys. Perhaps you failed to notice we were talking about a sensible computer company called "DEC", which ceased to exist before VMWare came into being?

            0 5 Reply
    3. Mage Silver badge
      Reply Icon
      Facepalm

      Re: Fragile. Very fragile.

      Sadly it's possible.

      Noooo... I cried as the CEO pulled out a hot swap SCSI HDD on a RAID5 system running NT4.0. He wanted to demonstrate the ability of the system to cope with failures to a customer.

      Except (1) The SW doesn't expect that sort of failure. (2) Even then rebuilding RAID5 took hours.

      Pulling out the mains on one PC making up an NT4.0 Enterprise cluster is less disastrous time and data wise. (3) He didn't check to see if there was already any issues with the array, he could have lost all the data.

      Or the "technician" from the IT support company that was asked to move the server, and did so without shutting it down. One of the internal HDD wasn't well mounted so touched off PCB of neighbours. Two HDDs failed, whatever the reason.

      A little knowledge is dangerous.

      32 0 Reply
      1. Alan Brown Silver badge
        Reply Icon

        Re: Fragile. Very fragile.

        "Even then rebuilding RAID5 took hours."

        Hours being long enough for another drive to fail due to the added load.

        25 0 Reply
      2. JimboSmith Silver badge
        Reply Icon

        Re: Fragile. Very fragile.

        We had someone who was not versed in technology (at all) but still made an acting manager whilst a proper replacement was yet to arrive. The idea was that they just signed off on things like bills and assigning people projects doing nothing technical. This was only a week long posting although legend has it though that this person wasn't averse to having a go at attempting to fix things. They would observe what others were doing and then work on the assumption that they knew how to fix it. So one day someone came down with their laptop that was expecting to get a memory upgrade. Bloke who was assigned to do it was out looking at a faulty monitor and the rest of the office was empty. He wanted to speed things up, so knowing where the RAM to be used was and how to get into the laptop at the bottom to install it decides to have a go.

        He'd been told that this was an electrostatic sensitive piece of kit and had watched people wrapping a tethered wrist strap on before working. He located a strap but was unsure what it attached to (there was a dedicated earth point) and looked for something suitable to connect to. He knew it was needed to be earthed and some of you will have already guessed...................but for those who haven't he went for the nearest mains socket. This obviously has an earth on it and he knew that, so he was trying to figure out which one was earth to be able to use that. Someone came in at that point and stopped him before he fried himself. Certainly would have sped things up if by things you mean his entry to A&E or the undertakers.

        A little knowledge can be fatal.

        29 2 Reply
        1. Richard 126
          Reply Icon

          Re: Fragile. Very fragile.

          Maybe I have a personality disorder but if I saw my boss doing that I would have just turned around and gone to the tea room to make a cup of tea and let him get on with it.

          24 0 Reply
          1. Anonymous Coward
            Reply Icon
            Anonymous Coward

            Re: Fragile. Very fragile.

            --->Maybe I have a personality disorder but if I saw my boss doing that I would have just turned around and gone to the tea room to make a cup of tea and let him get on with it.

            @Richard 126

            Mate you're obviously still a learner.

            The CORRECT thing to do is find your phone, proceed to video it while at the same time saying things like "...thanks for the opportunity to learn from you and allowing me to video it to review later...."

            If you're going to play the game remember to be vindictive.

            8 0 Reply
          2. rmason
            Reply Icon

            Re: Fragile. Very fragile.

            @Richard 126

            My response would have been similar, with a side of "texting a colleague to get here QUICK".

            2 0 Reply
        2. Doctor Syntax Silver badge
          Reply Icon

          Re: Fragile. Very fragile.

          "Someone came in at that point and stopped him before he fried himself."

          There's always a spoilsport.

          26 0 Reply
          1. JimboSmith Silver badge
            Reply Icon

            Re: Fragile. Very fragile.

            There's always a spoilsport.

            He was a really nice bloke and I still got on well with him when he reached his permanent posting in another department. Besides the paperwork is horrendous when someone got injured requiring A&E. We'd never had anyone actually killed at work so I don't know if there was more or less for that but I would lean towards more.

            6 0 Reply
      3. big_D Silver badge
        Reply Icon
        Paris Hilton

        Re: Fragile. Very fragile.

        @Mage I've had a manager order site services to move a workstation/server (an old Burroughs running BTOS) from one table to another, because he needed the table for a new employee. Because I was working off-site at a customer and I was the only one capable of supporting the kit, he didn't know how to turn it off, so he just had them pick up all 6 modules in one go, whilst running and dump it on the "new" desk.

        At one site, we had a memory upgrade on a VAX 11/780. The Digital technician turned up. The admins had moved all of the jobs and users from the machine to the next one in the row. The technician was told that the machine was now shut down and he could power it off... He threw the power on the wrong machine and the machine with the extra load suddenly found itself doing a Wyle Coyote, hanging in mid air over a tall cliff with no power...

        Needless to say, one of the drives crapped out.

        A while back the technical department needed to pull new network cables into the server room. The NAS standing behind the rack was in the way, so they just slid it across the floor until they could do their work, instead of contacting IT and getting it moved properly.

        Likewise, one of the apprentices was told to turn off the electricity in the electrical engineering production hall, he turned off the power for the entire premises! Luckily the UPS cut in and the servers were fine, but the Quantum Superloader didn't like the transfer from mains to UPS and back and hung.

        As much as these things should never happen, there is always somebody who should know better who just needs to quickly do something and doesn't thing about the consequences, whether it be a manager, a qualified technician or a trainee.

        Paris: because even a qualified technician can leave her looking intelligent at times.

        29 0 Reply
    4. DonM.
      Reply Icon
      Facepalm

      Re: Fragile. Very fragile.

      Back in the previous century our shop was running a VAX 11/780. We had a real DEC field service fellow come out for some minor fault - this guy reached in the main cabinet to remove one of the logic boards from a powered up and running system. We were able to intervene before he got the board out. Needless to say, this fellow was blacklisted and his manager reamed ....

      22 0 Reply
      1. jake Silver badge
        Reply Icon

        Re: Fragile. Very fragile.

        To be fair, some DEC kit (NOT memory!) was hot-swappable, but this fact was mostly undocumented. We weren't supposed to do it in front of customers, even when it was safe.

        14 1 Reply
  2. Mark 85

    The first place I worked in IT had 4 VAX's and a large HP box in the computer room along with a strict rule that no food, or beverages were to be brought in there. I should mentioned that the building was "smoke-free". Except for the IT manager and CIO, who after lunch most days would bring their coffee and cigars to the computer room and use the VAX's for tables.

    Same company also had a strict "no alcohol" rule during working hours. But the wine cellar was next to the computer room in the basement and the board room had a wet bar with two taps for the barrels. Yes, they were always cold and ready for use.

    33 0 Reply
  3. John Smith 19 Gold badge
    Coat

    "Has your boss ever exempted themselves from rules imposed on others?"

    You'll get a much more manageable response if you turn that question on it's head.

    Another good one is "Has there ever been a good idea in your department that you boss did not have first?*"

    *And by definition only the boss can have good ideas. If it was that good he would have thought of it already.

    32 1 Reply
    1. Anonymous Coward
      Reply Icon
      Anonymous Coward

      Re: "Has your boss ever exempted themselves from rules imposed on others?"

      I had a great idea at an old job. We ran our software for clients on our own kit. One potential client didn't want to switch to us because they'd spent a shit-ton of money on their own (single) server and paid a firm to write a basic service. Our software was by far the best in the (niche) industry but the clients software did work, it just didn't help them make as much money ( without going into detail, in that industry, better software = much more profit ).

      I suggest that we offer to buy the server from them - it would pay itself back in the long run, and we'd get another server to add to our pool that we'd end up buying at a few months down the line anyway. "That's a good idea." I was told.

      A few days later my boss returned from a trip to the pub/bookies/wherever he always disappeared to and proclaimed "I've had a great idea....".

      ( it was a tiny firm, boss was the owner, I don't believe he nicked it on purpose, but still.... )

      20 0 Reply
    2. Arthur the cat Silver badge
      Reply Icon

      Re: "Has your boss ever exempted themselves from rules imposed on others?"

      In one company I worked for the best way to get something radical past management was to have a conversation with the MD down the pub along the lines of "I've had an idea but I'm not sure whether it's workable, so what do you think about ...". 2-3 days later the MD would announce this brand new initiative that he'd come up with, and the techies would all grin at whoever's turn it had been to speak to the MD.

      24 0 Reply
    3. Doctor Syntax Silver badge
      Reply Icon

      Re: "Has your boss ever exempted themselves from rules imposed on others?"

      I and my small team had adjacent desks. The conversation between us was fairly continuous - there may have been complaints from surrounding teams. One year I got a bad report for not holding meetings with my team. This was written by my line manager who never held such meetings with me and countersigned by his manager who also never held such meetings with me. Do as I say!

      19 1 Reply
    4. Anonymous Coward
      Reply Icon
      Anonymous Coward

      Re: "Has your boss ever exempted themselves from rules imposed on others?"

      Yah, worked at a place run by a guy like that when my employer was acquired. Got the hell out before it killed me. Best move I ever made.

      Anon because the Internet has a long memory.

      1 0 Reply
  4. Will Godfrey Silver badge
    Happy

    We turned it on its head

    At one electronics place place I worked we had a long sheet of fanfold headed with:

    "Cockups we're prepared to admit".

    There were all the usual things and most people initialed their entries, including one of the bosses with "Plugged 110V kit into 240V supply".

    What was revealing was the people who never made a single entry.

    51 0 Reply
    1. Mark York 3 Silver badge
      Reply Icon
      Mushroom

      Re: We turned it on its head

      Spent a entire morning rebuilding a PSU (not a PC one) for use on ships & on-shore positioning systems.

      Went for Friday Lunchtime liquid lunch with my colleagues, returned & powered it on with a bang, yes I had forgotten to switch it from 110V back to 240V for testing (See icon choice).

      On the plus side later that day I was told I was going to Nantes for two weeks on a training course, departing Sunday, so fixing it again wasn't my problem. :D

      34 0 Reply
      1. Anonymous Coward
        Reply Icon
        Anonymous Coward

        Re: We turned it on its head

        "[...] yes I had forgotten to switch it from 110V back to 240V for testing"

        A UK customer had a new mainframe delivered. Owing to very long production delays the supplier had upgraded it from the range's smallest UK model to its largest model. That model had to be imported from the USA. The customer was very pleased with their bargain after the long wait.

        The supplier's engineers commissioned it successfully with the UK to lower USA voltage autotransformer jury rigged.

        Overnight the customer's electricians commissioned the autotransformer into a permanent arrangement. On power up it was instantly clear that they had connected it backwards. Presumably giving the mainframe 4 times its expected voltage. Not sure how long they had to wait for another one to be shipped from the USA.

        33 0 Reply
      2. Mark York 3 Silver badge
        Reply Icon
        Pint

        Re: We turned it on its head

        I hasten to add (Some 12 hours later admittedly) the training course was unrelated to electronics & setting voltage switches correctly, but the operation & setup of Syledis off shore position systems.

        There was a lot of wine & beer involved that first lunchtime with upper management of Sercel (We had to take the tram back to town & again to return to site the next day).

        3 0 Reply
    2. Terry 6 Silver badge
      Reply Icon

      Re: We turned it on its head

      That sounds like a good place to work.*

      *Unless it's a fantasy. I've never heard of anywhere like that.

      8 0 Reply
    3. Alistair Dabbs
      Reply Icon

      Re: We turned it on its head

      I think what makes the "Plugged 110V kit into a 240V supply" cock-up such a memorable lesson is the loud bang and, if you are in a darkened room, accompanying flash.

      It's something you don't do a second time. Of course, if you do, you're probably better off working on a farm.

      23 1 Reply
      1. Phil O'Sophical Silver badge
        Reply Icon

        Re: We turned it on its head

        is the loud bang

        Not always, we had a DEC VT240 terminal (yes, it was along time ago) that worked fine like that for a month before quietly blowing a fuse. Even worked for a few more days after that when the user just replaced the fuse...

        Cue chorus of "they don't make 'em like they used to"...

        27 0 Reply
        1. defiler
          Reply Icon

          Ah - the loud bang

          Back in in the days when "multimedia" was the cool new kid, I worked for PC World. We had a consignment of monitors delivered set to 110V. Many customers called in with shaky voices from that episode.

          We ended up cutting a hole in the side of the boxes and flipping them across without opening them, but we did test one to see how big a bang it made.

          Big. That's accurate enough! :)

          28 0 Reply
          1. CrazyOldCatMan Silver badge
            Reply Icon

            Re: Ah - the loud bang

            I worked for PC World

            And are prepared to admit it? You are braver then me..

            23 0 Reply
            1. Spacedinvader
              Reply Icon
              Trollface

              Re: Ah - the loud bang

              So you also worked there?

              4 0 Reply
            2. defiler
              Reply Icon

              Re: Ah - the loud bang

              And are prepared to admit it? You are braver then me..

              I was young. Foolish. I needed the money.

              Also, I got to see when the videogames were going cheap. Got System Shock CD for half price, and then a staff discount. :)

              2 0 Reply

      2. This post has been deleted by its author

      3. PPK
        Reply Icon
        Pint

        Re: We turned it on its head

        Second time? (raises hand sheepishly) I did...

        Annual trade show in Las Vegas, mid 90s. Shipped my dev PC to use on the booth - wouldn't power up - oh yes, 110V (flips switch). Shipped it home, got it back to the lab, pop...

        Next year - rinse and repeat. The second time I was familiar with the fix - luckily it didn't go full magic smoke, but just fried a fuse mounted to the PSU PCB. Still, lesson finally learned, no Farmer Giles smock for me! (in my defence, shipment arrived back 2 weeks after the show by which time I'd forgotten about it. Yes, I could have changed it before shipping it home, but the amount of alcohol imbibed during said show made it difficult to remember...see icon).

        6 0 Reply
      4. Kiwi
        Reply Icon
        Pint

        Re: We turned it on its head

        It's something you don't do a second time. Of course, if you do, you're probably better off working on a farm.

        Ahem.. large animals that can do lots and lots of damage if annoyed/scared. Large powerful (if somewhat slow) machinery that can do lots and lots and lots and lots of damage if misused (I had so much fun in my youth!), lots of stuff that burns, lots of stuff that goes bang in rather pretty and rather noisy ways.

        Oh, and commonly several lots of 3-phase mains...

        Wanna rethink that? :)

        3 0 Reply
    4. JimboSmith Silver badge
      Reply Icon

      Re: We turned it on its head

      What was revealing was the people who never made a single entry.

      Hah we had a small team and the name of everyone went into a jiffy bag (we didn't have a suitable hat). Then when a cock-up occurred and blame wasn't easy to attribute a name was pulled out of the hat and if it was your name it was your fault. It didn't matter if you were having the day off when it happened still your fault and you fixed it. It worked well because it made everyone potentially responsible and other people knew it probably wasn't actually your fault despite the fact that you were fixing it. Not sure HR approved though.

      34 1 Reply
      1. NateGee
        Reply Icon

        Re: We turned it on its head

        I've had a franchisee of a unnamed but well known franchise who once who claimed that he never touched the voltage switch on the POS's PSU but upon a site visit could clearly see that he had. He then claimed that the switch was too easy to switch over (it definately wasn't and I demonstrated as such with both the unplugged knackered PC and the replacement) and refused to cough up the cost of replacing the PC. Once quick call to his Area Manager changed that.

        13 0 Reply
        1. Mark York 3 Silver badge
          Reply Icon
          Mushroom

          Re: We turned it on its head

          Told the story of the small business owner that flipped the voltage selector switch to see what would happen many a time now (See icon again).

          The bill for reinstalling & recovering his business accounts & data from a very very shell shocked hard drive & file allocation table was in the order of 10 times the bill & labour for replacing the PSU.

          6 0 Reply
    5. Anonymous Coward
      Reply Icon
      Anonymous Coward

      Re: We turned it on its head

      A meager dev with no DC visits experience I was sent to replace a HDD in an already failing array, I pulled one of the last good disks out... He wasn't happy but considering he'd lose his shit to colleagues AND customers and have a reputation for it across the sector around here, I was genuinely spared. It was lying on its side and I just didn't count the right "2nd from the top".. 50/50. or 30/70 with noise, heat and one hand on the phone to the boss (excuses excuses)

      5 0 Reply
    6. CrazyOldCatMan Silver badge
      Reply Icon

      Re: We turned it on its head

      Plugged 110V kit into 240V supply

      We had a whole shipment of Trigem 386 (or maybe 486 - it was a fair while ago and some bottles of wine have been had since..) machines delivered to us with the power supply switch set to 110v (this was the days before autosensing PSUs).

      We used to build the machines in tranches of 20 (unbox, cable up, flick bench power supply on). 20 almost simultaneous pops as 20 PSU units emitted the magic smoke. And 20 motherboards fried be oivervoltage leakage.

      At least the supplier admitted their mistake and replaced the machines at their expense. We did add a "check power supply settings" to the workflow though.

      15 0 Reply
    7. allthecoolshortnamesweretaken
      Reply Icon

      Re: Plugged 110V kit into 240V supply

      Sort of. I was using a "portable" PC, a knockoff of the Compac cooler box, 486, plasma VGA display, you know the one I mean. I was using it at three different sites. One day during packing/transport/unpacking the switch on the PSU must have slided from the "230V" setting to the "110V" setting. Loud bang & magic smoke... Only I was using that machine for my engineering thesis at the time. Took a couple of allnighters to get back on schedule... Been checking the settings of switches ever since.

      13 0 Reply
  5. eJ2095

    The GFX Card removed

    Did not take place at work, was for me friends brothers back in 2000s he was 15 at the time.

    Was explaining over MSN messagner how to remove gfx card to put another one in

    Then all of a sudden he vanished off line.

    then got a phone call saying his computer had gone off..

    Yup you can guess what happened there.. Did make me laugh, luckily it all still worked afterwards

    22 0 Reply
    1. Olivier2553
      Reply Icon

      Re: The GFX Card removed

      Removing a card from a life PC, I have done it at least once.

      During a debugging repair phase, where you swap cards until you find one that works: insert a card turn on the machine, it does not work, turn off the machine, remove the card rinse and repeat. At some stage, I forgot to turn off the machine. I am not sure what the result was, but if I removed it, the card must have been bad already.

      22 0 Reply
      1. Anonymous Coward
        Reply Icon
        Anonymous Coward

        Re: The GFX Card removed

        Did something similar with a new PC model. Powered it off and left the mains lead connected as a static ground. Plugging the new card in - the PC burst into life. Fortunately with no ill effects.

        It was the first time we had met a chassis with "standby" power and a "soft" mains switch. Later models had a "case open" microswitch to guard against that. Unfortunately it proved difficult to jam when you needed to observe the internals of a powered up machine.

        16 0 Reply
  6. Oengus
    Pint

    IBM PS/2

    I had a manager who received one of the first PS/2 PCs in our office. We had an agreement with IBM that they would send out someone to install all equipment (including PCs as they were very new to us). The manager the PS/2 was destined for decided that he could install the new box himself. He assembled the system (forgetting to plug in the mouse), fired up the PS/2 and was rewarded with the Windows desktop. When he went to do something he realised he had forgotten the mouse. This was one of the "new" PS/2 bus mice (previously all we had were serial mice). He went to the back of the running PS/2 and plugged the mouse into the appropriate PS/2 port. The PC immediately shut down. All efforts to revive the box failed.

    He rapidly re-boxed the PS/2 taking great care to put everything back where it should be and waited for the IBM tech to turn up to install the PS/2.

    When the IBM tech removed everything and set it up it failed to startup. The IBM tech declared the PS/2 DOA and arranged a new one. He later told us that it was a defective motherboard.

    By all accounts plugging a mouse or keyboard into a running PS/2 could cause a spike that would destroy the motherboard because of the way the PS/2 bus was connected to the system.

    To this day I still give that manager a hard time whenever he gets something new and decides to set it up himself (he is a friend of mine long retired)...

    45 0 Reply
    1. Gene Cash Silver badge
      Reply Icon

      Re: IBM PS/2

      We had a problem in our uni lab of PS/2 motherboards dying. All at the same seat, which wasn't used by the same person, so this time it didn't seem to be user error.

      Turns out someone had run a chair or something over the mouse cord and it was internally shorted. After a day or so, it would kill the motherboard.

      I noticed there was a 90 degree kink in the cable and replaced the mouse, and then a week later noticed our semi-daily run of dead motherboards had ceased.

      Remember the damned Model 80s that were so heavy they had a "two-man lift" warning sticker on them?

      13 0 Reply

Page:

POST COMMENT House rules

Not a member of The Register? Create a new account here.

Forgotten password ?

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like