we’re on the hunt for spooky tales for a Halloween special of On Call
To be titled 'Ghost in the Machine'?
Welcome once more to Who, Me?, our weekly column in which readers confess to their worst IT cock-ups. This time, we meet “Freddie”, who must have a good supply of luck on his side, as he managed to not only get away with his error, but win praise for fixing it. At the time, Freddie was a busy man. He worked for a managed …
Which is why you don't work direct at the server but over RDP unless something really critical has happened, because then "shutdown" is something you can only really do with a command-line (or if some installed program offers to do it for you, but then you test that on a non-server first, and generate yourself a mental install script, right?)
Ehm, no. you can do a shutdown from RDP too - even if it will warn you - if you have the permissions.
Anyway, there was a time when MS server applications had great remote administration applications, which let you working from your workstation without any need to access the server directly, or via RDP or whatever.
Then came the Linux crowd where there were no such applications, and everything must be done directly connected to a server via SSH to manipulate some text files (while maybe wiping your machine with rm -rf *, or shutting it down if you happen to work in the wrong terminal).
Unluckily MS saw how cheaper it was (and easier to offshore), compared to developing full remote administration applications, and copied it with PowerShell. Now, even management applications are wrappers over PowerShell scripts, so everything is far slower, bloated, and error handling a joke.
Well, who wants remote clicky-mousy for server administration instead of a text file that can be copied, edited, copied back just in case, etc. might actually be "wiping your machine with rm -rf *".
From the point-of-view of the universe, he or she might also do less harm in a Windows environment.
CLI works for linux because every application is decoupled... With Windows its a peace of dogs meat, as all they've done is take the same monolithic products and stripped away the GUI - For analogies,
Linux is like a sleek F1 car, each part is self-contained and held together with screws. It can be hard to get the components working together at first, but they're easily swappable.
Windows is like a big American RV, humongous and put together with rivets... it works and is very comfortable, but don't expect to swap anything without rebuilding the whole thing.
Troll, cause its Monday morning and I'm stuck on a train going out of my mind --->
"Linux is like a sleek F1 car, each part is self-contained and held together with screws. "
Nowadays, that's unfortunately less true than ever. To illustrate this, I suggest you try the Gentoo installation process. When you start with a stage 3 tarball and use OpenRC rather than systemd, the number of dependencies required to get a *lightweight* functioning desktop with a suite of useful applications which use your hardware properly is scary.
>> "Linux is like a sleek F1 car, each part is self-contained and held together with screws. "
>> Nowadays, that's unfortunately less true than ever. To illustrate this, I suggest you try the Gentoo installation process. When you start with a stage 3 tarball and use OpenRC rather than systemd, the number of dependencies required to get a *lightweight* functioning desktop with a suite of useful applications which use your hardware properly is scary.
> what are you on about?
> xorg-server mate slim firefox thunderbird conky dconf-editor libreoffice galculator corefonts dejavu roboto vlc audacity spotify ghex gimp conky
> there. you have a gentoo desktop.
Um. You put conky in twice. I know it's good, but it is not that good. Good effort, though, so please accept the virtual pint --->
And for most normals, that would be part of a truly scary command line, which would be accompanied by pity when you proudly point out you composed it from memory.
"...try the Gentoo installation process. ... the number of dependencies required to get a *lightweight* functioning desktop with a suite of useful applications which use your hardware properly is scary."
At least Portage deals with all those dependencies for you automagically, even if you do need to type a few package.use incantations to keep it all clean & tidy and free of unnecessary cruft and avoid filling the OCD-pleasing 4GB partition. (I'm afraid I had to cave in and resort to allocating 8GB on my desktop systems some years ago.)
Getting a fully-tricked-out Windows system requires downloading Mammon knows what from Baal knows where (they'll give you hints if you bribe them with shares in your soul), then trying to install it all in the right order without accidentally letting any of those packages install the bundled spyware, take over a bunch of file associations that you would rather stayed with some other package or replace your preferred media player with SuperVideoMediaDoober Plus Max (Cloud Edition). Then you find that one of the things you need only works properly if you install .Net Framework 1.1 _after_ you've installed .Net 4 but before you install .Net 2, unless you install .Net 3.5 first, in which case you have to install the Visual C++ 10,11,12,13,14,15,16,17 and 18 runtimes in an order determined by consulting a medium, after slaughtering a goat at New Moon.
(Oh wait, I'm getting confused. The bit with the goat is from the NetBSD install procedure.)
"Getting a fully-tricked-out Windows system requires downloading Mammon knows what from Baal knows where"
Ninite. One installer, no spyware, done.
To stick with the point about upgrading a CLI based OS to a full GUI, in Windows you just do: install-windowsfeature server-gui-mgmt-infra,server-gui-shell
.
*No dooont... stooooppp... my sides are hurting...*
Psst! Don't tell anyone, but there's a Linux port of PowerShell! There's even a Debian repository:
https://docs.microsoft.com/en-us/powershell/scripting/setup/installing-powershell-core-on-linux?view=powershell-6
"Linux is like a sleek F1 car"
You have to replace half the components after every few hours of use, and just chuck the old ones away?
A rally car might have been a better choice (they're designed to have components swapped in and out), but even then it's still not a good analogy.
Just dreamin':
[Quote]
Linux is like a sleek F1 car, each part is self-contained and held together with screws. It can be hard to get the components working together at first, but they're easily swappable.
[/Quote]
Then there was systemd: https://youtu.be/hYOWIdPHXts
A sleek F1 car... where everyone brings one piece of the car along when they come to the racetrack. They all go out to the pit and put the car together piece by piece, arguing non-stop about what kind of car they are supposed to be building. Violent fights erupt between KDE / Gnome and VI / emacs, Linux/Debian user groups ... In the end the bodywork is not installed because nobody can agree on the color scheme to be used. There's not steering wheel ,shifter or pedals. Instead there's a panels with some cryptically labelled buttons. And a MAN page explaining in which order to press the buttons to steer the car, accelerate and brake...
The car takes off without paint or bodywork but is declared a huge success because.. well look at the 'open'-ness of it... you can see every part ... and it's very lightweight too... so it's fast. But it still is only half baked. If it breaks the people who built it simply tell you : we gave you the list of parts (the sourcecode) you go fix it. We've moved on to the next car. If you dare raise the issues of missing bodywork and paint : prepare to be scoffed at. As for the steering steering wheel and pedals : this is not a kids bike. Only little children need steering wheels and pedals.
coat..
Linux is like a sleek F1 car, each part is self-contained and held together with screws. It can be hard to get the components working together at first, but they're easily swappable.
Ha Ha Ha Ha Ha Ha
Linux is like a series 2 land-rover defender with bits and bobs screwed on from Later models, discoveries and Range rovers.
Of course in reality some bits aren't screwed on and need welding or you may have to drill some holes in the chassis rails or even remove the engine.
Some bits wont work together at all and may cause your defender to stop in the middle of a stream.
..."where I don't have access to restart the box"
That is quite rare nowadays though. Usually they'll be VMs which you just start up through the relevant console or for hardware boxes/vm hosts then you normally have a management interface that works with the box switched off (but obviously still powered). Failing that the intelligent power supply can be cycled and as long as you have it on power on on power restore that should work. I've never let a truly remote box get zombied by an errant shutdown.
"“I had just finished with five minutes to spare, but instead of hitting 'sign-out', I accidentally hit 'shut-down' on the primary server,” Freddie said."
Placing the logout button next to the button for "shutdown" is one of the worst pieces of design in the entirety of Windows, and that's saying something. Freddie must be one of thousands of people who have made and continue to make this mistake.
With Windows 10, the buttons have been parted (but only by half an inch), and are no longer in the same submenu. And they are better marked, but it is still one of the most obtuse and needless mistakes ever to come out of Microsoft. Putting a massive risk at customers' feet for no reason.
When using robocopy to move an empty (as in no users) EDB file to a new drive, make sure you specify the Database Name in the script.
We were running out of space so added some more storage, then moved some of the databases to that new drive.
We had 9 days worth of space left so it was a bit of a rush job once the storage was ready.
Anyway, without the variable set, I robocopied the half empty drive over the live one...
Script cancelled but damage done.
Thankfully Exchange sorted it all out, failing over to another copy with about 10-15 mins downtime. It took it's time to think about it, but it sorted itself out! Phew.
You learn by your mistakes!!!
Our previous contractor was asked to leave his employment as a result of an errant Robocopy statement.
He left the MIR switch in his script when copying over some errant files that hadn't copied due to permission issues, 2 days after the file server had been replaced. Result? We lost 2 days data. His biggest mistake though was doing all this before checking to make sure we could actually access the backups of the new file server. Small mercy being only 2 days data lost, and it wasn't the database server.
At least on later versions of Windows Server (from 2008 onwards I believe), you're required to provide a reason for the shutdown/restart that goes into the event log. This is usually about the same time that you realise you hit the wrong option when trying to select "Log off", and does at least give you chance to cancel the impending doom you've initiated.
Having heard all of the horror stories of people shutting down servers since the 2k days I have always put in a GPO on my sites to remove the "shut down" and "restart" options from the start menu of my servers. (plus other housekeeping bits like logging off people who leave their sessions connected after a decent period.)
This forces shutting machines down to be done with the "shutdown" command (start->run "shutdown /i") which forces very deliberate action be taken in shutting machines down.
@Peter2
"I have always put in a GPO on my sites to remove the "shut down" and "restart" options from the start menu of my servers"
I tried that with one particular customer. They were in the habit of rebooting the server whenever there was any problem, so I thought that removing the menu items would stop them. Instead, they resorted to yanking the power cable every time.
I did the same thing, accidentally shutting down the file and print server for the London HQ of AT&T (this was a very long time ago, so I feel safe enough naming the company) at 14:30ish on a Thursday afternoon. I was not sure I had really hit "shutdown" until I saw all the lights go off on the server. Starting up was a slow process - the disk arrays spun up their drives sequentially and the whole thing was offline for 5-10 minutes.
We had all the big wigs in the building, the hundreds of staff etc. but there was not a single call logged. SMB mounts on clients would only have noticed if they tried to r/w while the server was down and obviously also no-one tried to print anything (or just tried it again 5 minutes later and it worked).
Not a truly impressive story I will admit, but in those minutes before I could check that everything was back online I was convinced I was going to get the sack. :-)
To be honest, Captain, I think you gave me too much time...
So how long will it really take, Laddie? Eh? You told him exactly how much time it would take to fix? How are you going to keep your reputation as a miracle worker doing that?
Montgomery Scott.