Microsoft Office 365 and Azure Active Directory go TITSUP* It seems Microsoft's Office 365 is having an unscheduled nap as users across the world report difficulties logging into the administration portal. Office 365 is having issues since 04:51 AM ESThttps://t.co/3GGuh8EZs8 RT if you're also affected #office365down pic.twitter.com/ixHgD7YHv1 — Outage Report (@ReportOutage) April 6, …
Because it tends to be cheaper and easier than running your own resilient email system, if you're only managing a handful of users. Sure, it goes off from time to time, but that passes and in a few days it's forgotten about.
It's like when you drink tea that's too hot and it burns a bit on the way down, but a few seconds later you're fine and keep drinking the tea once it's cooled a little. People are funny like that.
Substitute coffee / hot chocolate / hot beer / hot absinthe or whatever you like to drink hot in lieu of the tea...
We run our email in-house, over multiple servers, in multiple sites.
Don't use it as your ONLY resource, I think should be obvious.
Such cloud options are great. As one option per cloud. But you should have in-house stuff too, or you're entirely reliant on a) your Internet connection and b) Microsoft. There's no reason you can't run a secondary email server, AD server, etc. in-house. If you do it right, I can't see a reason that external users would notice anything had even happened if the entire cloud went down.
But it's "lazy IT". Let's just pay a monthly subscription, then that's us sorted because "it's Microsoft". They don't give any thought to business continuity. Sure, it can work at some levels of business. But if you are a corporation of any size and you're affected by this,.. you obviously put too many eggs in the same basket.
@Lee D
"But it's "lazy IT". Let's just pay a monthly subscription, then that's us sorted because "it's Microsoft". "
That 2nd sentence is an MBA laden CxO talking, not rank and file "proper" IT. IT much of the time doesn't make its own budgets or decisions on stuff like this. A couple of dinners with some MS sales guys at the CxO levels, and this sort of thing gets pushed through over IT's objections. (been there, got the t-shirt). I would say it's probably more Lazy management that's trying to trim both the personnel and equipment budgets while simultaneously getting nice meals...
Many other people had already learned about this, long before the latest occurrence. (Based on what I've seen from you here, you're one of the people who saw this coming).
So perhaps the question that really needs answering might be "why does persistent IT incompetence on this scale (and with no improvement in sight) seem not to matter to the people who pay the IT budgets?"
NB "IT budgets" also needs to include anything where computers and software play an important part of product functionality, e.g. automation equipment, 'self driving cars', a whole pile of computer-dependent stuff.
Suggestions welcome.
Question:
"why does persistent IT incompetence on this scale (and with no improvement in sight) seem not to matter to the people who pay the IT budgets?""
Answer:
Most high level management these days is basically a bunch of gamblers who are taking the gamble that the reductions in resilience (or in MBA speak.. "savings") that generate large bonuses for them will not come back to bite them in the arse before they retire or move on to the next schmuck of a company that believes their interview bull.
Simples
"...that generate large bonuses for them will not come back to bite them in the arse before they retire or move on to the next schmuck of a company that believes their interview bull."
I don't think it's that. I mean most bad IT decisions don't actually save any money, not even in the short run. In fact many even have short run negative consequences.
My hypothesis is that there are many IT departments which believe _anything_ a salesperson will tell them. That's why companies _still_ run antivirus software, even though it's benefits have long been disproven by both theory and practice. That's why companies still invest in "office productivity" software like Microsoft Office or Open Office, even though those mostly cause your employees to waste time on things they don't know how to do, like making a printed document look good.
The problem is that the people making the purchasing decisions don't understand technology at all, so they don't question what sales people or random websites are telling them.
Most such purchasing decisions are not made by the IT department, but even the IT dept often don't have much of a clue either. The requirement for staff increased much more quickly than the availability of skilled staff, so companies have to take whatever they can get - including people who don't have much of a clue.
"why does persistent IT incompetence on this scale (and with no improvement in sight) seem not to matter to the people who pay the IT budgets?"
It is, perhaps, because people who know how computers work, generally don't work in computer administration any more, they move on to higher paying jobs like programmers.
There are few exceptions like that game company where the employee handbook leaked recently. They had a system where you had desks with wheels you could move around by yourself. As soon as you plugged it in on your new location the floor plans would automatically get updated.
Since most IT-departments are horribly bad at what they are doing, most people have never experienced a good IT-department, which means that they don't demand it to be actually usefull. Good IT is to rarely seen as an enabler of success and an effective motivator of your staff.
@Christian Berger
It is, perhaps, because people who know how computers work, generally don't work in computer administration any more, they move on to higher paying jobs like programmers.
I have come across programmers who appear not to know how computers work.
Because the people that are paying the IT budgets like shiny things and savings. They are fooled by so called digital directors who are just hipsters in disguise with no clue about real IT work. These so called directors want to be "Infrastructure free" because it will be a "massive cost saving", clearly showing they haven't a clue about how the cloud or local IT actually works.
Problem again is, the people that pay the budgets listen to this fool.
'So perhaps the question that really needs answering might be "why does persistent IT incompetence on this scale (and with no improvement in sight) seem not to matter to the people who pay the IT budgets?"'
Because they collected their handsome bonus for "cost saving measures" and left the company before the consequences became obvious. They're the same people who were happy to be sold a bunch of outsourcing crap; the same ones who paid small fortunes for endless streams of suits wrapped around MBAs from three- and four-letter professional-grade-bullshit (PGB) outfits, to be told, during an elaborate Death By Powerpoint, how to synergistically leverage the business process enhancement matrix for maximal mission-driven shareholder return which, strangely enough, meant cutting costs by sacking the very few remaining greybeards who knew how things actually worked.
In short, "the people who pay the IT budgets" have long since learned to trouser the bonus and skip out before the scale of their monumental ignorance and disastrous incompetence become apparent. Many of them have CVs to leave you leave you breathless with admiration, when you know that they have had 20-year careers fucking up every single thing they have touched.
But do not imagine that they are entirely worthless meat: to the saleslizards who work for the aforementioned PGB consultancies, such executives are as a herd of plump gazelle to the gaze of drooling hyenas ... fat, dumb and oh so tasty.
Unless you can actually check each item on a CV for veracity, it is wise to assume it to be at best wildly inaccurate and a heavily manicured version of the truth, and at worst complete fiction. By way of example, a former colleague of mine describes the time when we worked together (as low-end dev-ops on a Remedy ARS system) as the time when he was the team leader of that entire section.
Which, to be brutally honest, was complete and utter balderdash, wild exaggeration and outright fabrication of what he was actually doing.
Employing a person on the basis of information that they provide which cannot be independently checked is utter folly.
All brilliant Ideas in theory (not relying on the cloud, or any single source).
I suspect my situation is like many others.
Can you imagine what the answer would be (or in fact, was) when you ask for all the relevant kit, software and licences to have an on-prem exchange box (for example) *AND* the cloud offering, when the previous iteration of people in your job sold them on "the cloud" to save money?
I'll give you a hint. Two letters, started with "n", ended with "o".
The upshot was we've had approx 1,5 hours of slightly flaky emails, and skype refused new log-ons for a bit (but worked for those logged in). Now all fixed (10ish-12ish today).
In our business neither thing being down was a significant cost or risk, so correct decision made (as those who said "no" would see it, they're probably right).
Oh, I'm quite sure there are tens of thousands of places that do just that.
Cost-benefit analysis is fine, so long as someone did it.
It's when people whine that "Oh, my 4G credit card reader is down, I can't take cards, my business is in ruins" or "WE CAN'T GET OUR EMAILS!", when they haven't bothered to take such a loss into account that bothers me.
To be honest, everything in the world from GMail to Azure, AWS to IBM will be down for things on that order of magnitude, no matter what they promise. They have to be. But it's what YOU do about it for your circumstances that matters.
Emails will delay an hour or so and then come in later if it comes up. Skype shouldn't be a business-critical tool. Your remote workers not being able to get in on VPN for a bit is no worse than someone tripping over a plug. But when you whine that your work-at-home telesales can't dial into your VoIP VPN for a fraction of a second and it's costing you money, I have to just think "Okay, so what was your backup?"
@Lee D Absolutely.
In my Yoof I worked for, cough, *a* yorkshire based ISP.
With every single outage would come the angry phone calls and even speculative invoices from people who "can't operate my business" and are "losing thousands an hour here".....
All from their 9.99 a month (single) home ADSL account that happened to be down.
*shrugs*
Same deal innit.
@rmason
o365 isn't actually cloud. It's hosted. *IF* if was actually cloud, it would be more resilient and less susceptible to the issues that keep downing it. :)
Sidenote - In a meeting with MS o365 and Azure Sales and Tech, the o365 drone said "cloud", and I corrected him, with the Azure guys nodding with me. :) They weren't happy about the o365 guys throwing the word "cloud" about when they are a hosted service...
@Amos1
Hosted is someone else's computer - and o365 is totally hosted, with little in the way of resiliency.
Strictly speaking, Cloud is quite dynamic/automated and much more resilient and distributed. However... Marketing weenies will call everything on the internet "cloud" to keep a marketing hypewagon of buzzwords rolling. So at this point, the term cloud is virtually meaningless...
Cloud is a buzzword, simple as that. No real meaning, except what that shiny-suited salesman wants it to mean.
A cloud-hosted solution is a data centre full of servers on which the service is running. The only difference is that the cloud-hosting operator can probably do it a bit cheaper than you can, through economies of scale. The *other* difference is that you are not only relying on the cloud hosting machines to stay up, but also all the networking kit between you and them, plus other associated gubbins like DNS and the like.
When it all works, it is cheaper. When it doesn't, what ho, you got what you paid for.
*ALL* Microsoft hosted services were inaccessible for our business during this outage. Skype for Business, OneDrive, SharePoint Online, Outlook etc. Much more disruptive than just the publicised "Can't access the admin portal for Office 365" statement. Communication from Microsoft during this major EMEA wide outage has been very poor.
> as those who said "no" would see it, they're probably right
Yes. If you're big enough to afford a proper IT staff, and the necessary software licenses, and redundant hardware, then maybe you can do it better in-house. But for smaller outfits the cloud often makes more sense.
Sure there's going to be occasional downtime, no matter how many 9's your cloud provider claims, and of course you still need to keep backups somewhere other than in that same cloud, but unless you can afford multiple competent IT people (more than one, since they tend to take vacations sometimes) and redundant hardware (elsewise you have a single point of failure in your one crucial server, and how long will it take to re-build the server when it eventually fails) you can probably get better uptime in the long run if you use cloud solutions.
It's like mains power: a few sites really do need a large onsite generator and etc so they can ride out multi-hour power outages, but that's very expensive. So most people just have enough UPS so they can run for a few minutes and then safely shut things down if the power doesn't come back quickly. Long power outages are rare enough that for most businesses it doesn't make economic sense to maintain your own fully-capable power-generation equipment. So it is becoming with IT.
What is this "vacation" of which you speak? You keep using that word, I do not think it means what you think it means.
For a small business Office 365 is more cost effective than running exchange. If you get more than about 25-30 users it becomes less so over time. Problem is convincing people to spend the dosh to do internal servers in a CAS array for a small business is not going to happen most of the time.
The cloud is not always suitable for small businesses either. One has to consider data privacy laws such as the US HIPPA that have strict requirements about data access, etc. The cloud is also nothing more than moving an internal server farm to an external server farm with supposed cost savings (or more accurately more obvious costs) and possibly some operational efficiencies.
One major difference with external vs internal is how many are affected when the service goes down. Internal only affects one company while external affects numerous.
@pdh
Yes. If you're big enough to afford a proper IT staff, and the necessary software licenses, and redundant hardware, then maybe you can do it better in-house.>
I worked for a client who are not a small outfit by any means. They went over to o365, and I think was the right decision, even though at the time I did not think so. And that opinion is purely from having subsequently seen the inertia that appears to be endemic in their IT department. So, if Microsoft can deliver a better service, good for them - the in-house IT only shot themselves in the foot, IMHO
On the main tenant we manage we never experienced any issues. Maybe because we run a hybrid setup and adfs authentication, we’ll see when (or if) they post an RCA.
Another tenant gave an error that said something along the lines of “could not authenticate at the time”, but a third one (and the first one I checked when calls came in 2 to 3 hours ago) said “user unknown on this system”. I very nearly shat myself.
It seems the crisis has passed, and fortunately before beer o’clock. Another notch on the “bad microsoft” stick, but I still wouldn’t go back to running it all in house... the absolutely critical, losing-money-if-its-down stuff : yes. Email ? IM ? Sharepoint ? Hell no. I wouldn’t be able to get the uptime of most cloud providers without spending huge amounts of dosh.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
