back to article Sysadmin sank IBM mainframe by going one VM too deep

Welcome to another glimpse inside the dark-curtained (in)box that is "Who, me?" – El Reg's confessional column in which readers seek penance for sins of the past. This week, meet "George", who many years ago was one of two sysadmins in the office products division of a three-letter computer company. And was bored. Bored, bored …

Page:

        1. John 110
          Boffin

          As A scientist...

          As a scientist I resent that... wait no, as you were. I'll start again.

          As a scientist I resemble that remark!

          1. Stevie

            Re: As A scientist...

            A scientist eh? I have a question for you: WHERE'S MY FLYING CAR YOU USELESS LUMP?

            1. jake Silver badge

              Re: As A scientist...

              Expecting a scientist to deliver your flying car? Well, there's your problem. The professional you are actually looking for is called an Engineer.

      1. AndyMulhearn

        Scientists...

        Scientists, not as good at following instructions as you think

        Your mistake was not prefixing the instructions with an Abstract, adding a few references to random publications and putting in on arxiv.org for peer review. It may not have made any difference in the long run but they may have actually read it...

      2. Anonymous Coward
        Anonymous Coward

        "Scientists, not as good at following instructions as you think"

        Oh indeed ! I could name you dozens of examples where I've been on the other end of the phone to Senior Professors who couldn't find their way out of a paper bag when it comes to subjects outside their area of expertise (such as making the great scientific discovery that YES, the map and address of the event has already been published on the website on the event page ...... no, instead they ring me or my colleagues instead quite literally asking "Where is the event taking place?" ).

        1. Voland's right hand Silver badge

          Oh indeed ! I could name you dozens of examples where I've been on the other end of the phone to Senior Professors

          You obviously failed to read the orientation leaflet.

          PhD student: Knows Everything

          Associated Professor: Knows Everything in one narrow well defined area

          Senior Professor: Knows where the Associated Professor(s) and PhD Student(s) are at any given time.

          I have heard it from my dad (Senior Professor, Differential equations and Optimal Control).

    1. onefang

      "There's no such thing as a development machine, they're all "production" to somebody."

      Not true at all, the little test PC on my desk has always been a development / test box from the day I bought it, and likely always will be.

  1. This post has been deleted by its author

  2. Denarius

    BTW, welcome

    Rebecca

  3. jake Silver badge

    If you're in the bizz long enough, you'll crash a production machine.

    It goes along with the job description.

    However, I once found the bug that was being exploited by a group of developers to crash the system so they'd get home in time for Monday Night Football. I patched it Sunday night ... and was extremely unpopular at that shop for a couple months ... not that I cared, I was the consultant, we're not supposed to be popular :-)

    1. DJV Silver badge

      Re: If you're in the bizz long enough, you'll crash a production machine.

      Back in the 1980s I was working in a company that was using Unisys kit (actually rebadged Convergent Technology systems) running CTOS. These could be "clustered" together into small networks. Downstairs, the boss and admin staff used a system that consisted of a network consisting of a master computer, with the hard disks attached, and two or three networked slave machines. Upstairs, each developer had a single computer but one developer (let's call him Derek, mainly because that was his name) had quite a powerful system.

      When the downstairs master computer had a problem the boss decided that Derek's computer would have to act as the master computer for a while. Derek did warn them that, in the process of developing software, it was possible that we would often crash the computers, but the boss wasn't deterred. Derek had also discovered (via the CTOS API manuals) that there was a system call the forced a deliberate crash. He wrote a small program to call that API and every so often, when he was bored or whatever, would whisper to the rest of the developers (who were safely isolated on their own systems) something like, "Listen for the shouts downstairs." Then he'd run the program to crash the network resulting in cries of anguish below. He's then shout out something like, "Sorry, but I did warn you that we often cause crashes."

      It didn't take too long before a new spare computer was made available and Derek's system became disconnected from downstairs again.

    2. Anonymous Coward
      Anonymous Coward

      Re: If you're in the bizz long enough, you'll crash a production machine.

      I once took out a Beowulf cluster by changing my windows password. I mean, it was a /terrible/ setup, 80-odd PCs running windows XP, with regular user accounts - in this case muggins, hence the whole thing going south when my password changed - but it worked well enough most of the time.

      And I'm not even a sysadmin! It's inevitable that if you have the power and the admin rights to break almost anything, you'll break *something* eventually.

    3. Alan Brown Silver badge

      Re: If you're in the bizz long enough, you'll crash a production machine.

      "I patched it Sunday night"

      You could have kept quiet and watched the fireworks when they realised they couldn't buzz off early.

      1. jake Silver badge

        Re: If you're in the bizz long enough, you'll crash a production machine.

        Alan, I did just that. Complete with a message on the perp's screen to the effect of "Forget about it. Get back to work!". He hated me. He'd have hated me more if I hadn't talked his boss out of firing him for the stupid stunt.

    4. Anonymous Coward
      Anonymous Coward

      Re: If you're in the bizz long enough, you'll crash a production machine.

      I once brought a production MVS machine to a standstill by the simple expedient of using the paging storage for a large sort.

  4. HPCJohn

    VM/CMS

    Wow. That brings back memories.

    As a graduate student in high energy physics, I got my very own 3270 terminal, connected to the mainframe downstairs by twinax cables.

    Best keyboard I have ever had.

    Many joyful days running virtual machines under CMS. I remember the REIPL CMS command to reboot (Initial Program Load). It probably would all come back if I got a 3270 in front of me!

    In later days I got a FALCO terminal at CERN. VT220 emulation, plus a hotkey to Tektronix graphics emulation. I used a 3270 emulator on DEC VMS to access the mainframes there.

    1. Missing Semicolon Silver badge
      Happy

      Re: VM/CMS

      Get yourself an IBM Model M keyboard. And feel the "best keyboard" again!

      1. HPCJohn

        Re: VM/CMS

        Those keyboard did make a racket though when you were typing.

        1. Alan Brown Silver badge

          Re: VM/CMS

          "Those keyboard did make a racket though when you were typing."

          You can still get Model M keyboards. Or Cherry G80s with old school clicky switches. I'm typing on one now.

        2. Zarno

          Re: Those keyboard did make a racket...

          Can confirm.

          I have a Unicomp EnduraPro (USB cabled reproduction IBM M13 (the one with the tracpoint) in black with grey caps) for my work keyboard.

          It sounds like an ammo depot cooking off when I start typing in earnest.

          So far, the office mates haven't had any real issue with it. :)

          The original model M is someplace in the black hole of storage at the house...

      2. Amos1

        Re: VM/CMS

        Northgate Computer systems had the best keyboard I ever used. It was my first PC, a 386 with 1 MB of RAM and two, count them, TWO 65 MB RLL hard drives. It only cost me $3,495. I later upgraded it to 4 MB of RAM by replacing around thirty-two discrete integrated circuits so I could run DesqVIEW. I used that keyboard for years.

        1. 2Nick3
          Coat

          Re: VM/CMS

          "I later upgraded it to 4 MB of RAM by replacing around thirty-two discrete integrated circuits so I could run DesqVIEW."

          You can be honest here - it was to run Wing Commander and we all know it.

          (Mine's the one with the printout of the config.sys and autoexec.bat to get 632KB free base memory, with mouse support, in the pocket)

      3. Fruit and Nutcase Silver badge
        Thumb Up

        Re: VM/CMS

        Get yourself an IBM Model M keyboard. And feel the "best keyboard" again!

        I've not had first hand experience of one of these, but the same design, now with a USB interface

        https://www.pckeyboard.com/page/product/UB40PGA

        the above was from a reference by a commentard a few months ago.

        1. MonkeyCee

          Re: VM/CMS

          Cherry keyboards are also pretty much up to scratch as a Model M.

          I picked up a half bust one as part of a gayboy* full of ditched IT kit that I ethically recycled**, via eBay and assorted scrappies. About 60% of the keys worked.

          Sent them an email, they felt it was still covered by warranty and as long as I sent the bust one back for QC review I got a free replacement. With brown keys, since I'm never sure if I want it hard or soft ;)

          *a small skip :)

          ** the PMs plan was flytipping.

      4. Anonymous Coward
        Anonymous Coward

        Re: VM/CMS

        I was once working as a summer intern at IBM - editing a document in "PROFS" - did some formatting change and was waiting, waiting, waiting for the screen to update. Then the phone rings - it was an operator from the Warwick data center - asking me what I was doing, as it was taking up something like 95% cpu on the mainframe. Needless to say there was nothing much I could add - the process was terminated by the operator and I decided better to re-structure the document differently than attempt the same thing and trigger whatever the bug in the application again.

        1. VTAMguy

          Re: VM/CMS

          PROFS was XEDIT fancied up with a database to hold documents "centrally", but it was a known resource hog. It was initially seen (by IBM) as suitable for use by clerical level staff. Then it morphed into a do-all monster and imploded in on itself. Not a shining example of software technology from that era.

          1. yoganmahew

            Re: VM/CMS

            @VTAM (I'm sure I've just blocked a virtual route doing that)

            "Not a shining example of software technology from that era."

            You say that, but it would work on the end of a dodgy copper wire that SITA would string on trees in the arse end of nowhere. At least until the wire got nicked for the copper.

          2. Fruit and Nutcase Silver badge
            Joke

            Re: VM/CMS

            PROFS...

            Then it morphed into a do-all monster and imploded in on itself

            ...PROFS was replaced by NOSS?

            and finally, the move to Notes

            Then it morphed into a do-all monster and imploded in on itself

      5. fruitoftheloon

        @missing semicolon: Re: VM/CMS

        ms,

        I use Apple adjustable ergonomic keyboards with an ADB to USB adapter.

        They are the canines gonads..!

        Cheers,

        Jay.

      6. rnturn

        Re: VM/CMS

        One of my Model Ms--plus a gaggle of adapters--is the only way I can type accurately on a Mac nowadays.

      7. hmv

        Re: VM/CMS

        Not quite.

        Older IBM mainframe terminals used a beamspring keyswitch. The IBM model F keyboard was a reduced cost version of that (the original IBM PC keyboard and a number of others) and the IBM model M keyboard was a reduced cost version of the model F.

        Not that it's a bad keyboard - it's probably the best you can buy new today. Old beamsprings are rather rare, hideously expensive, and require "interesting" methods to adapt to modern computers (replace the controller with an Xwhatsit controller).

    2. VTAMguy

      Re: VM/CMS

      Agreed on the quality of IBM keyboards. Minor correction: 3270 technology used coax cables. And the CP command is "IPL" (Re-IPL is used as a verb).

      And yup, it wouldn't take long to get back into the swing of old friends like EDGAR again (Edit Data Graphically And Recursively!). One little-remembered VM CP command was DIAL which was used in place of LOGON to put the terminal under control of a guest OS (eg ACP, Airline Control Program).

      -Grayhaired VM/370 systems programmer

      (Systems programmers write code, in BAL, that runs in privileged mode. I'm not a friggin 'admin' and I don't have to login as root. Don't call me admin or I'll delete your A-disk.)

    3. Greybeard3

      Re: VM/CMS

      3270 used coax. 5250 (for AS/400) used twinax. I do not believe any 3270 used twinax. Just sayin'.

      1. jake Silver badge

        Re: VM/CMS

        Concur on the Model M. If you don't have one, get one. Especially if you're an old-school touch typist. Typing on one now, and for the last 30ish years. These things aren't toys, they are built to last. They double as an offensive weapon, and so are probably illegal in Blighty.

    4. dakra

      Re: VM/CMS

      3270 terminals for mainframes attached via 75 Ohm RG59 or 93 Ohm RG62 coax with BNC connectors.

      5250 terminals for system 3x and AS/400 attached via twinax

  5. John Brown (no body) Silver badge

    del *.*

    del *.* in the root directory in the days of MS-DOS is the worst I ever did. On a colleagues PC at that! Easily fixed though, after a few moments of heart stopping panic. His autoexec.bat and config.sys were pretty much factory default and del *.* only deleted the files in root, not directories and as per normal practices, there was barely anything in root in terms of files.

    1. naylorjs

      Re: del *.*

      A colleague did an rm -rf / on a production system, he hit ctrl-C pretty quickly afterwards. The phone calls started a few seconds later. It was fun to watch from the outside!

    2. Stuart Castle Silver badge

      Re: del *.*

      Friend of mine did something similar, except he was on his bosses Windows 3.1 machine, and did del *.* in the root. Normally, this wouldn't be an issue, except his boss had managed to installed pretty much everything to the root of drive C.

    3. Anonymous Coward
      Anonymous Coward

      Re: del *.*

      Deltree *.* could be even more devastating. Regardless of what subdirectory you are currently in, in DOS6.22 Deltree assumed you were pointing at the root.

      I recall an old DOS for Dummies book that recommended you remove the deltree command!

    4. Roger Kynaston

      Re: del *.*

      In my final year at uni I had an Atari ST on which I wrote my dissertation. I then lent it to a friend to write hers. She had trouble saving it to the floppy disk so I went over to help. It said the disk was full so I grabbed another disk at random and saved dissert.doc on there. Just as the floppy kicked into gear I realised that it had my dissertation on it and it was called dissert.doc. Cue lots of panic and visions of having to rewrite the whole thing in three days. Then I realised that Atari created a copy called file.bak and was able to recover mine from there. It was a short but sharp brown underpants stuff and I don't think that my friend really twigged how serious it was.

      1. onefang

        Re: del *.*

        "It was a short but sharp brown underpants stuff and I don't think that my friend really twigged how serious it was."

        She didn't notice the bad smell coming from your suddenly colour changed undies?

    5. 2Nick3

      Re: del *.*

      Back in '95 I was working at a help desk where a number of colleagues from the Windows support team were upgrading their WfW 3.11 machines to Win95 (1,2), and a number of them were having issues copying the '95 install files to their machines over the network (3). The drive wasn't full, but the copy job was failing for everyone at just about the same spot in the file list.

      I looked at the screen, and noted two things: They were at C:\> and the end of the dir/w showed 512 files and directories (max for the root of a FAT12 volume). On pointing this out the user of that machine typed in "del *.*" and was about to hit Enter when I yelled "Stop!" This got a bunch of angry glares (4), to which I said "Think about what that will do."

      At which point I was (rather impolitely) told to go back over to my side of the call center.

      1. Not me - I was happily running OS/2. Warp 3.0 on one machine (probably beta at that time), 2.11 on a PS/2 Mod70

      2. No idea why they weren't doing a fresh install instead of an upgrade/migration

      3. CDROM drives were scarce, so one person copied the CD to their system and everyone else was copying that directory to their local hard drive

      4. There was a lot of animosity between the Windows and OS/2 support teams, and the fact I had identified the problem was already a poke in the eye.

  6. Anonymous Coward
    Anonymous Coward

    Yep been there done that

    At least the deflect blame bit, and got away with it.

    I was working at a small company in mainland Europe and needed to plug in a new monitor or something. I took the power plug to my extension block and pushed it in, it wouldn't go, so I pushed harder. The plastic flexed and within the extension two wires touched. It was a modern building and so the trips for the floor went, no bangs or pops. The lights went out, the computers went off, and shortly after some angry people came walking down the corridor. They were four hours into a hardware debugging session apparently, not to mention other peoples work (maybe 20 or so) who were similarly inconvenienced.

    I (for a few minutes at least) genuinely believed that I had a dodgy extension block and informed everybody that it was me, but that it wasn't my fault. A short while later the penny dropped, but I kept my mouth closed. I then proudly helped my boss put said extension block into the bin, and normality was restored. Or at least normal for that company, whatever that was...

    I still cringe when I think of it, and I'm sure that said hardware developer would still find and kill me if he could trace me, so anonymous.

    1. Anonymous Coward
      Anonymous Coward

      Re: Yep been there done that

      Yay for shutting down an entire cluster of VM's in one go.

      Have one (laxative-free lager) on us BOFH's :)

      *snerk*

    2. Killfalcon Silver badge

      Re: Yep been there done that

      What actually happened there, AC? Were you using the wrong sort of plug?

    3. DropBear

      Re: Yep been there done that

      If uninsulated wires or bare terminals were close enough to each other that forceful handling* was able to cause a direct short, you DID have a dodgy extension block.

      *If it involved liberating chunks of plaster and wall anchors, I might come back to that statement...

      1. Richard Pennington 1

        Re: Yep been there done that

        That one reminds me of a story dating back to 1971. When the UK introduced decimal currency, it didn't take long for a small child to discover that a 1p piece fits neatly into a UK 3-pin electrical mains plug, touching all three conductors.

        1. Killfalcon Silver badge

          Re: Yep been there done that

          Is that (and similar hazards) why three-pin plugs have insulation at the base of the two powered pins?

          1. DropBear

            Re: Yep been there done that

            To the best of my understanding the insulated base is there to prevent touching energized pins while the plug is neither "in" nor "out" - ie. by the time the pins connect to the socket only the insulated part is supposed to be exposed.

  7. MatsSvensson

    OMG WE GOT HACKED!!!

    *head in hands*

    *peeks through fingers*

Page:

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like