IBM warns itself of possible outages in lab shift screw-up

"I wonder if this is a somewhat drastic method of finding out what services they need to keep... turn 'em off for two weeks, and if no-one complains, they obviously weren't required."

That would make sense. I was in charge of a World Wide DC move for a 8+ B USD company in 2010.

At one point, during our coordination of a set of rack moving to the other site with the movers, my colleague cut his hand badly, resulting in a lot of discussions outside of the DC (security, HR etc ...).

Turned out no-one was supervising the movers, and they unplugged a couple of racks (still running !) by mistake. We basically shut down half a dozen of Live applications for which no shutdown was requested.

Since no-one ever complained, I was always wondering how much of the whole, in my opinion largely oversized DC was used. Before leaving the company, I really wanted to know, so spent my last month setting up Cacti on every DC server.

Turned up: 95% of the systems would do nothing, on a period of one month, nothing at all. Only a dozen of them would run some workload. All was due to poor process (new app = new system) and never any decommissioning or capacity mgmt.

Re: And yet IBM are selling their expertise in Cloud

Yep, isn't the cloud supposed to almost completely mitigate downtime in this kind of situation? Perhaps the IBM Cloud is too expensive for IBMers.

Re: And yet IBM are selling their expertise in Cloud

By 'cloud' in this case, they mean Tivoli and related products. IBM rebranded it as 'cloud'... because there is automation involved or something. It is the VMware definition of "private cloud", what most people would consider to be the opposite of cloud (on prem management software).

Re: 2 weeks downtime?

"What they doing, turning them off, putting them in boxes, sticking them in the post with a 2nd class stamp and then putting them back in one at a time?"

Probably. Well, except for the "2nd class stamp" part. But if you don't have money for new servers, that's how you do a move. If you're lucky you can stagger it to reduce the workload and limit downtime for any one system, but not if you have to get it all on one truck (for budget reasons, again.) I've done moves like that in academia. Moving a full rack of servers single-handedly took me just about a week from shutting everything down to powering back up the last machine, including the two days it took to get the servers picked up and driven across town. By the end I felt like one of the guys trapped inside the data center in "When Sysadmins Ruled The World".

Re: 2 weeks downtime?

What they doing, turning them off, putting them in boxes, sticking them in the post with a 2nd class stamp and then putting them back in one at a time?

Yes, that's exactly how IBM does it. I know because I've DONE an IBM datacenter/department shutdown.

also leads to

lack of internal expertise (in this case in their own products) leading to fossilisation of current configuration and nobody left to specify replacement/uplifts where needed

far more limits on use cases able to be tested (likely internally charged/services more limited)

very little feedback into the cloud development teams (making future offerings even worse)

May as well kill the entire idea and just let customers rent someone else's cloud to test IBM's

End of cynical rant...

As for "IBM Cloud", that's the joke that keeps on giving, much like IBM Watson Automation is billed as "AI", when it is about as far from AI as anyone can get...

Not if you define "AI" as "Artificial Idiocy".

Re: extreme short term viewpoint

Or asset strip and load up the company with a huge amount of debt (often incurred by acquiring the company in the first place).

I guess there is good and bad private equity...