back to article Microsoft confirms: We fixed Azure by turning it off and on again. PS: Office 362 is still borked

Microsoft is recovering somewhat from a bad case of the Mondays that left some of its subscribers unable to use multi-factor authentication to log into their cloud services. The Redmond giant said that around 2130 UTC today it had managed to get its Azure Cloud back up and running as per normal. Meanwhile, Office 364 is still …

  1. Anonymous Coward
    Anonymous Coward

    Customers advising Azure Tech Support:

    "Have you tried rebooting it?"

  2. Paul 129

    U2F

    Picked up a cheap fido u2f key.... These are great and work well in linux, easy config change. Browser support? You have to turn it on via config in firefox, enabled in chrome. Works with gmail nicely if you set your browser to remove cookie on close.

    Windows,... well no. Apparently there is V 1.2 of the standard which has been called U2F 2.0 microsoft is a partner. The standard is still in the process of being ratified (As far as I can tell) and hardware is scant and difficult to tell what version. i.e. a mess.

    What is the link? Microsoft somehow really messes up two factor authentication. Accident or embrace, extend, extinguish?

    My guess..... both.

  3. Dwarf

    Compound problem

    Its a shame that this happened on the same day that the public transport system suffered a significant failure requiring a far higher number of people to work from home. I expect that the two are linked in some way.

    You have to ask though, why didn't MS capacity management notice that they were getting close to a hard limit in one of their platforms, I thought this was one of the big pluses of Cloud - <sarc>you don't have to worry about all of that, we do it for you and do it better than you can</sarc>

    I wonder if MS should look at AWS Elasticache, that can be scaled out elastically and it uses redis too, so migration to it should be simple.

    1. Yet Another Anonymous coward Silver badge

      Re: Compound problem

      >you don't have to worry about all of that, we do it for you and do it better than you can

      True if you are the customer of a highly skilled cloud operator. If however you ARE the cloud operator then you are the on-premises idiot running round rebooting servers

    2. macjules

      Re: Compound problem

      Build with confidence with 99.99% VM uptime

      Translation: We think we know what an SLA is but we just never thought it might apply to us.

    3. John Robson Silver badge

      Re: Compound problem

      "Its a shame that this happened on the same day that the public transport system suffered a significant failure requiring a far higher number of people to work from home. I expect that the two are linked in some way."

      Well yes - without access to their cat pictures by email the drivers couldn't get in...

  4. Chris Gray 1
    Black Helicopters

    Hmmmm

    "Mitigation: Engineers deployed a hotfix which eliminated the connection between Azure Identity Multi-Factor Authentication Service and a backend service."

    What kind of back-end service needs to connect to the Multi-Factor Authentication Service? See icon.

    1. Nolveys
      Gimp

      Re: Hmmmm

      What kind of back-end service needs to connect to the Multi-Factor Authentication Service?

      Don't all Microsoft services count as "back-end"?

  5. steviebuk Silver badge

    But this should never happen

    The cloud is amazing and never goes down. How will we sell the bullshit notion of "Infrastructure Free" when this happens!

    1. Domquark
      Mushroom

      Re: But this should never happen

      Of course! The cloud is always on.

      Snake-Oil Salesmen sticks fingers in ears and chants "La-la-la" very loudly......

  6. bombastic bob Silver badge
    Devil

    This deserves a theme song...

    To the tune of the "Car 54 where are you" theme...

    *ahem*

    There's a network bottleneck

    Routers blinking all those lights

    Network traffic jams are forming

    Backup data not in sight

    Users screaming like a child

    Workload forms a monstrous pile...

    Office 364 where are you???

    1. Diogenes

      Re: This deserves a theme song...

      Boy does that(!) bring back memories - best forgotten

  7. ThatOne Silver badge
    Coat

    > Redmond said that it is still working on the problem

    Wouldn't it be better if they worked on the solution instead?

  8. Anonymous Coward
    Big Brother

    GCHQ here

    Would it help if we released info on a back-door to bypass MFA?

  9. MacroRodent

    Office 300 + rand () % 100

    Seems I am not the only one with trouble remembering which number starting with 3 is in the cloudy office name.

    1. Dan 55 Silver badge
      Headmaster

      Re: Office 300 + rand () % 100

      rand()? In this day and age? arc4random_uniform() at the very least.

    2. Baldrickk

      Re: Office 300 + rand () % 100

      uh oh, I spy some bias here.

    3. informed

      Re: Office 300 + rand () % 100

      I suspect the joke is that Office 365 assumes 365 days of operation and so far, we've lost a couple of days.

      1. Dan 55 Silver badge

        Re: Office 300 + rand () % 100

        No, this is.

  10. T. F. M. Reader

    "Passwords also cannot be reset by users."

    I imagine there is a fair number of people who's reaction to this was "Oooh, time to hack!"

    1. Sir Runcible Spoon

      Re: "Passwords also cannot be reset by users."

      aka engineers.

  11. Piro Silver badge

    Warm and fuzzy

    Always a good day when there's a problem with Azure or Office 365.

    1. johnnyblaze

      Re: Warm and fuzzy

      There does seem to be a large number of them too - every other week almost.

  12. bigfoot780

    Will it be Office 360 because of all the cycling?

  13. Anonymous Coward
    Anonymous Coward

    Multi factor is working this morning

    Not that I lost any sleep over being unable to read corporate email last night.

  14. Pipme

    Numberwang!

    Seriously... How difficult is it to type a 5?

    1. diodesign (Written by Reg staff) Silver badge

      "How difficult..."

      About as difficult as it is for Microsoft to run Office for 365 days of the year.

      C.

      1. Pascal Monett Silver badge

        I propose that El Reg officially names Office 3xx for the number of days it can still remain online. So, on January 1st, it is Office 365. If it falls over for a day somewhere during February, it becomes Office 364.

        Feel free to dole out decimals for hour outages, but where it falls over doesn't matter (ie if it is only the European users that cannot access, it still counts in the global number).

        1. Mr Sceptical
          Thumb Up

          Seconded!

          This should enter the El Reg standard units of uptime (or something).

          ^ More upvotes above please ^

          1. MyffyW Silver badge

            Office 340

            Process plants are commonly designed to run for 8000 hours/year, allowing time for routine maintenance and upgrades throughout that year.

            I'd be quite happy to pitch this to my users if MS would provide a 10% reduction in fees. And serious recompense if they fell below that.

            As it stands Microsoft seem to be achieving about 0.93 of a year, or about 1 unit of factory uptime.

            As to the name of the El Reg unit, I nominate it "the Tony" after Factory Records founder Tony Wilson.

  15. Giovani Tapini

    Turning it off and on again

    Is how people OUTSIDE Microsoft have been managing their technology for a decade or two now...Nice to know they are starting to keep up with the rest of their customers.

    1. macjules

      Re: Turning it off and on again

      Because with Windows Vista, "turning if off" usually mean that it did not turn back on again afterwards.

  16. alain williams Silver badge

    Please remind me ...

    why cloud was supposed to be better than on-premises ?

    How much local sysadmin time is really saved, factor in lost productivity and what is the nett cost difference ?

    1. Pascal Monett Silver badge

      Re: Please remind me ...

      It's called progress. Now an outage can inconvenience millions of users at a time across entire continents, instead of just the users of a single company.

  17. WonkoTheSane
    FAIL

    Still stuck for me

    MFA popup refuses to accept my Authy-generated code. "Send me a text" doesn't send a text.

    Guess it's going to be another quiet day...

    1. bombastic bob Silver badge
      Coat

      Re: Still stuck for me

      maybe the hard-coded SMTP password [to send an e-mail to the 'email to text' gateway] got changed and nobody fixed it in the server code...

      either that, or nobody fed the pigeons

    2. WonkoTheSane
      Facepalm

      Re: Still stuck for me

      It just came back up. Had to re-add Office365 into Authy though.

      Downside is, I have to do workings now...

      1. Anonymous Coward
        Anonymous Coward

        Re: Still stuck for me

        Service is still shaky.

        App Passwords having difficulty, Azure Admin Portal mysteriously can't communicate with the service service, user mfa setting's not available. Use at your peril

  18. thexfile

    Around and around microsoft goes, chasing their own tail.

  19. vanrobstone

    Totally waiting for office 303 to come out

    I like to be behind a few versions

    1. Jay Lenovo

      Re: Totally waiting for office 303 to come out

      When it comes to control, there is a certain advantage to the "onsite" cloud.

  20. informed

    >In a way, Microsoft is saying its cloud couldn't handle the weight of multi-factor login requests.

    Ohh god, I'm so sorry - that was me. I enabled MFA for two test accounts on Sunday night!!

    1. This post has been deleted by its author

    2. Fred Flintstone Gold badge

      >In a way, Microsoft is saying its cloud couldn't handle the weight of multi-factor login requests.

      Ohh god, I'm so sorry - that was me. I enabled MFA for two test accounts on Sunday night!!

      .. aaaand we have our comment of the week. Wonderful sarcasm.

      :)

  21. Anonymous Coward
    Anonymous Coward

    Office 0

    It's times like this I'm glad I mostly still use Office 2010.

    1. Anonymous Coward
      Anonymous Coward

      Re: Office 0

      The company I'm working for right now deployed O3xx office suite a while back. Fortunately I managed to interrupt the process so it didn't delete all the 2010 components, so I just repointed all my files handles back to the 2010 components and I'm a happy camper :D

  22. 4711

    Office 404...until it has been rebooted.

  23. This post has been deleted by its author

  24. Stevie

    Bah!

    In NY weather was not severe on Monday.

    Must be the other East Coast.

  25. CaptainBlue

    One way to secure your email: stop anyone (including you) from accessing it.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like