BOFH: The Great Backup BACKDOWN "So I see that backup check went well?" the Boss asks, trying to drag me into a conversation that will have nothing to do with me, but sounds technical enough that I would be an idiot and say... "What backup check?" the PFY pre-empts me. NGAAAAAAARGH! "The backup check," says the Boss. "The Financial Director wanted to be …
Sadly, that is the story of our desktop support groups life.
We have a product that will backup specific data from a systems HDD (Word, Excel, and a few others...) BUT the system needs to be on the network for more that 5 minutes at a time.
We have a group of travelling types that are rarely in an office and it's even rarer for them to be on VPN any length of time.
Needless to say, their shit don't make it in to the storage vault.
Give us time. Assuming that you're on M$ we'll disable the write functionality to memory sticks with a GPO.
Then we'll wait a month or so for you to work out how to circumvent the GPO via the registry, spread the knowledge round your colleagues etc before blocking it again using an obscure option in the AV software.
Finally we will agree that some writing to USB is required, but not before agreeing to an insanely complicated exception procedure that expires at a random point in the future.
Before setting a rule that only encrypted USB sticks maybe used. We'll then specify an out of production model as the only one allowed.
Anon because I'm not making this up!
I remember disabling USB storage devices, I had to hide for weeks!
Weeks after several people brought in infected drives and they had to hide from the IT department (Its no wonder they outsourced us and changed our name).
Not Anon because I can't be bothered to click the Anon button
RE AC with the encrypted USB pens etc. I've seen this happen too. I strongly suggested against using pre-made encrypted USB pens in the event that a hole be found in their security, and instead suggested using an industry standard software suite for encrypting/decryping (the sort you can run off an unencrypted part of the drive to open it up) as if a hole is discovered, it can be patched.
Suffice to say when those devices were found to be worthless thanks to a glaring security hole (non-random random number generation or something that made decryption trivial - I forget the exact details) I said, in front of the 'wrong people' (IE the management who overruled me) "I told you so" and sloped off for a smoke.
You'd remember me if this were the same place; trenchcoat, scottish, grumpy, sarcastic...but I suspect this sort of shit happens *everywhere*
"I do so every friday to the USB3 disk drive that they did not supply using software that they did not supply."
I was tasked with recovering a very important USB drive where everything was deleted, and the off-site end user thoughtfully suggested an "unerase" utility, and how I could go about registering such software. Even though we worked for the same company, he clearly was not aware of the purchasing and capex policy of for software that did such frivolous things as data recovery.
Either way, I'm still not sure why he didn't store it on the company server, which is backed up nightly, of which he had a VPN connection into - I mean, he couldn't have been able to do too much work otherwise...
"Either way, I'm still not sure why he didn't store it on the company server, which is backed up nightly, of which he had a VPN connection into - I mean, he couldn't have been able to do too much work otherwise..."
Do. Not. Get. Me. Started.
There must be some universal unwritten rule employees learn AFTER their basic IT orientation that says employees should not use their assigned server folder.
Yes the servers go down sometimes and so does the network itself but is it that much of an onerous and Herculean task to make a copy? One for your PC and one on the server and then, you know, check the damn dates?
>There must be some universal unwritten rule employees learn AFTER their basic
>IT orientation that says employees should not use their assigned server folder.
1. $MEGACORP have never given me any IT orientation, basic or otherwise.
2. $NASTY_FACILITY_COMPANY have set my server folder quota to 5G 'because I am home based' and I have something like 500G of project data per year.
3. All project data folders on servers have been deleted in favour of a project management databse system that:
3A. Has no BLOB capability large enough to accept software projects (i could create a massive repository structure for hundreds of individual source files. If the PLC development environment saved them like that. It doesn't)
3B. I'm not authorised to access it anyway, even though I generate the actual working product. Itis full of sales memos, quotes, and copies of the brochures sent to customers. But no drawings, software, site reports, photos, signed acceptances.
From the other side of the coin, the company I work for recently (2012 I think) "upgraded" every employee's network folder from 1GB to 2 or 4 depending on your job role, but kept a requirement to hold 3 years of offline mail for legal reasons. Couldn't keep it on a server because our Exchange quota was still only 1G. I got lucky and was moved to a new unit after that was allowed to keep their rouge IT folks, so they kicked my quotas up to 40GB (personal share and my portion of a group share), and my maximum mailbox size to 15GB.
Before that, there was no point using the network folder because even compressed, I had ~18GB of data that wasn't squeezing into a 1GB folder.
I once lost some personal data despite having 3 different backup tools.
1) the Corporate desktop backup didn't backup my local Exchange folder because it in a different directory to the corporate standard due to an OS upgrade.
2) the TSM backup to the mainframe apparently stopped working because whomever was paying
the IBM license fee decided to stop and the admin cancelled all the backups without notifying me.
3) My Linux network backup failed because the building maintenance guys did a power/generator test and the Linux backup server did not boot
Moral: You can't have too many backups.
After all, fast, well-backed up samba shares have been around for years, and how is an old technology going to let us do our business in the modern world? Plus this is safe, it's in the cloud and backed up automatically, we'll never lose files!
Until we discovered, in a round about way, that whilst it's backed up, the best restore they can do is restore the ENTIRE workspace. Essentially, if you delete a file by accident (unless you explicitly turn on version control) you're fucked.
This is also, almost the case for Dynamics on cloud too...
"Oh you have some data missing, we can restore the entire database to a week last thursday, when you KNOW the data was there...."
"Uh Wut? You're shitting me, are there any alternative solutions"
"Well thats the easiest sir"
"Easiest for you, or for me thats trying to keep a f£!##@! business running"
"Well we could restore the database to a test environment & then manually export the data & reimport it into your current instance of Dynamics"
Man...the thought process at MS tech support is horrific sometimes...i still fundamentally like Dynamics though...
You can boost your beloved Sharepoint, but some want a physical medium for backup.
What kind of snooping-free promises do these services provide? None afaics.
No secrets outside work (a few, not secret to co-workers), but sure don't want to put anything in the cloud, where it is certain to be stolen if there is any interest.
We have a nice shared filesystem with a 5-tier permission system that is working relatively well here. automatically mirrored and backed up, nice. Only there's 5 TB of space available for ~700 of us. And No. Fucking. Quotas. So of course it's chronically full to the brim, and we have a locally-managed NAS box in a cupboard for our backups. Which had to be set up by the network guys so that it is accessible by the lusers without having to "configure" anything.
But it's locally managed, so the netops promptly forgot about it (2 years ago) and just switched the static subnet it was part of to dynamic... just before the holiday, as it were. When contacted, it becam (slowly) evident that they had forgotten everything about the config or the admin password that they set. We're headed to a factory reset as I type. As I am the cautious type I mirrored the part I have access to just before The Events, but a few of my colleagues did not see the need for it and are now well and truely screwed (that is, until I tell them I can pull the -single- drive out and restore from that, but I'm going to let them marinate a bit before I do that)
Pierre, if you're lucky the device *might* have a one touch backup button - throw a disk on of the right capacity, hit the button, leave it overnight.
Then plug it into a machine, scrape the data off. I think the Netgear and Syno devices both support this.
Question is, is the data worth the couple of hundred quid a 6tb disk will cost you?
Worth a quick google to see if the device does support it, and whether the backup files it spits out will be 'plain text' as it were - that is, just flat files.
Sadly, I suspect you may have already investigated that....also, tell your IT peeps to set up a local wiki server and start fucking documenting stuff!
(genuinely)HTH
Steven R
> Some businesses are really tight with the wrong budgets obviously.
Yup, I'd say that. Well to be honest there's perhaps 10 times that in bulk when you account for the RAID array and the offline backups, and it's all expensive 1st-tier drives, but it's still only 5TB accessible to the lusers. Endemic underfunding of research and all that...
5TB for 700 people? wtf?
Some businesses are really tight with the wrong budgets obviously.
No, it's probably sizing storage to meet needs. How many business letters fit in 5TB? How many records in a typical blob-free database? Remember that child benefit data loss a few years back - the entire database that's the core business of 3,000 people fitted on a couple of CD-ROMs.
That's par for the course these days - simple business records take next to no space by modern standards. It's media, video especially, that's driving storage growth now and the typical business has no need for a few thousand movies on their network.
> the device *might* have a one touch backup button
Yeah, we were kinda hoping to avoid that actually, especially as I have no idea whether the data in there has any value at all (Most of it doesn't, that I know for sure).
The good news is that I just went and plugged my laptop into the ethernet port of the NAS box, took a bit of fiddling to ifup with an IP in the right range but thanks to wireshark I can now talk to it in samba. The shared part is now saved, I just need to gather login/password info from the half-dozen of other users to check if there is anything of value on there before we can wipe everything and start fresh!
As most of them undoubtedly use the same password for banking I expect a bit of friction, but hey, if it's either that or lose their precious excel templates...
Or, "but if it's mirrored we don't need to worry about backing up to tape" (When it was pointed out that the weekly backup would take a month to complete)
Said about a multi TB storage array holding thousands of files that are quite important. Oh how we laughed!
Anon, but they know who they are!
Ahh, the classic I have a mirror therefore I have a backup idea.
Until it is pointed out firmly that all a mirror means is you will happily duplicate the missing file.
Or worse as I found many years ago, you duplicate the corrupted Master File Table, and lose everything on both disks.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
