back to article Virgin Media's Brit biz broadband goes TITSUP: Total Inability To Support Upset People

We knew Virgin Media was going down the toilet – but not quite this way. For several hours on Tuesday, its business broadband has been offline in the UK, leaving folks unable to reach the internet. "Our customers may be experiencing a total loss of service on broadband and data services," the cable ISP admitted earlier this …

  1. Fizzle

    Virginal

    Ah well, you can't get into it easily, it's painful when you do, it's over too quickly and it doesn't last very long.

    Time to move on, in my opinion.

  2. Lee D Silver badge

    Hasn't affected two leased lines.

    When you say "business" you mean ADSL / basic business fibre, presumably?

    If your business is THAT reliant on the Internet being up, about time you bought yourself a backup line, a 4G stick or an actual, real connection, methinks.

    1. richardcox13

      > If your business is THAT reliant on the Internet

      Exactly.

      Likely they didn't realised how much they were until it failed, but new #1 priority because any single point of failure …

    2. diodesign (Written by Reg staff) Silver badge

      Leased lines

      Given it allegedly involves a corrupted database, used to authenticate subscribers, I'm not surprised it affects non-leased lines.

      C.

    3. Anonymous Coward
      Anonymous Coward

      This fault was on the VM Cable network and affected Business customers specifically.

      The VM leased lines are run just like everyone else's.

      The Cable network is run differently.

  3. Nexus1974

    Mine is fine, broadband been working perfect all night

    Not one dropout, didnt notice anything until i read the article here on The Register.

    Does seem to be only business broadband affected however.

    1. This post has been deleted by its author

    2. Anonymous Coward
      Anonymous Coward

      Re: Mine is fine, broadband been working perfect all night

      Ditto for me. Not a single glitch noticed when working from home on a residential connection and I am in one of the red hotspots.

  4. Dwarf

    Resilience

    "Our business is internet based and thus the outage has cost us thousands. This is the worst service I have ever encountered. Anyone thinking of moving to Virgin Media, stay away."

    Business grade resilience isn’t provided by hosting at home, if it’s important, then put it in a data centre and build it with resilience, it’s not expensive, particularly when you factor in the cost of outages on the business.

    Don’t whine when your home grade Internet link goes down - just remember how much you are paying for it and possibly get a 4G MiFi adapter or second link for backup

    </rant>

    1. brym

      Re: Resilience

      It might not be an uncontended line. But VMB are still marketing it as a business grade service. And while it's open for one and all to buy into, meaning not everyone using it may be a genuine business or startup (have a genuine need for greater upload speeds), those of us who sincerely rely upon it are rightly justified in complaining.

      It's also worth noting that VMB have known about the underlying issues for the better part of 2 years (since inception), yet continued to sell the Voom Fibre services. What attracts people (certanly myself) to this line of services is the price point. Sure, you get what you pay for. But everyone has to start somewhere, and uncontended fibre simply isn't an immediate option for everyone (out of budget, not fully cabled in your post code yet, etc).

      That said, I agree with you that it would be short-sighted to operate in such conditions without adequate fallbacks. I have two such connections to fall back on, so I haven't been as affected as customers who've claimed it's cost them thousands.

  5. hplasm
    Facepalm

    Wah Wah worst ever.!!

    Go on - try Talk Talk or BT- 100% reliable...

    or you could pay and get Business connections...

  6. Sirius Lee

    Are you really sure of your facts? I am no lover of Virgin Media but I live in London (one of the alleged hot spots) am a VM subscriber and work from home spending most of my day on a remote machine. If there were a problem affecting connectivity, surely I would have experienced it but I did not. Even if I'd fallen asleep at the times it is claimed the the outage occurred, as my connection to the remote machine is on all the time, I would have noticed a problem because it would have disconnected while asleep. But it didn't.

    1. Dan 55 Silver badge

      I'm guessing you don't have a business line.

  7. Anonymous Coward
    Anonymous Coward

    "This is starting to get ridiculous," one vulture fan fumed. "There's no information at all. I have now lost almost an entire day of working."

    I wonder how much productivity has been lost because businesses have made a decision to put all of their systems in the cloud, and so can't reach them when their ISP lets them down. Possibly they could have continued to do some work if they'd had systems hosted locally.

    Just a thought, YMMV, etc.

    1. Anonymous Coward
      Anonymous Coward

      re: no information

      "This is starting to get ridiculous," one vulture fan fumed. "There's no information at all. I have now lost almost an entire day of working."

      So there isn't a Starbucks, Costa, Supermarket with a Cafe, library or even a Pub that you could have adjourned to and carried on working then? (not forgetting 4G and no, I'm not on the moon!)

      Anyone with half a brain has a 'Plan B' for when their normal internet goes TITSUP. After all going TITSUP happens every few months does it not?

      We must all be getting soft/dumbed down or something...

  8. GruntyMcPugh Silver badge

    How can the entire clustered, replicated database go down? Oh wait,... SPOF?

    1. TechDrone
      FAIL

      Replication replicates errors too

      Your database may well be clustered / replicated / distributed across multiple racks, datacenters and continents. But if you get a data error then you'll get that same error replicated everywhere too. End result: Your highly available replicated database isn't.

      I wonder if they're now reviewing their restore, DR and incident management processes?

  9. Anonymous Coward
    Anonymous Coward

    I worked for an ISP where we lost the Free Radius database hours before our Xmas party. Solution was just to open access to the world, on the basis that not many people would realise it was effectively 'free' until we got in the next morning to try to cure both it and our hangovers.

  10. andy gibson

    support line closed

    I noticed that the support line was closed due to an "emergency evacuation".

    Perhaps that means "Christ the phones are ringing off the hook, lets all go out to the pub".

  11. mrs doyle

    Any business should have a backup solution if it is mission critical to stay online. All networks have outages at some time. end of.

  12. astarisit

    Workaround / temporary fix

    If you have a Virgin Media Business connection with a Static WAN IP, delivered over COAX - then to be able to give you that static address - Virgin create a GRE tunnel between the Hitron router and their head end.

    Nb. this is not the same as the sticky IP addresses VMB tend to give out.

    The problem appears to be that following a firmware update - the Virgin head end is not provisioning it's half of the GRE tunnel correctly. The Hitron is getting it's half - but the head doesn't think its expecting one (confirmed by the customer service saying "there's no static IP on this customer account" which is incorrect and aligns with the corrupt database issue outlined in the article) - so doesn't set it up.

    If you run a traceroute from the web interface on the Hitron - you'll see that the source IP is the static / assigned by GRE address and not the general dynamic DOCISIS one.

    No traffic can be routed = no usable internet connection

    A factory reset does NOT include wiping the details of the GRE tunnel - so you have to put it into modem mode to get it to forget about the tunnel completely. Then factory reset to make it a router again.

    You lose your static IP and any / all custom configuration you might have made - which is dull.... but at least you're back online again..

  13. kms

    This is starting to get ridiculous

    This outage is nothing new, just a lot longer than usual.

    My stats show this is my 16th interruption in service so far this year and the 73rd in the last twelve months. It's usually down for around 5 minutes or so but still a real pain when it strikes in the middle of a vi session over an ssh connection.

    And they call it a business service!

  14. Anonymous South African Coward Bronze badge

    One of the reasons why I tend to shun cloudy storage solutions - especially if you don't have redundant links.

    Or the company is stingy with the budget and tries to get by on an el-cheapo home link.

  15. Tubz Silver badge

    Check your contracts and SLA's, if in breach get compensation and hit the VM muppets were it only registers, in the wallet !

  16. The Dogs Meevonks Silver badge

    affecting residential too

    Outage is still ongoing as of this evening, and I've been told it will continue to be erratic until the 7th. Logs show it started on my residential line on the 28th and has been up and own more than a prozzies g string. Current outage has been on going for 11hrs so far.

    So I cancelled my service based on the fact that they have been unable to provide the service I pay for over the last three months... 100mb service that has averaged less than 20mb since November.. Actually since before, but kept records since the 28th.

    Signed up with new ISP thats been getting better reviews, new service active by the 12th and guaranteed to be 3 times faster than the miserable VM average or contract is void... And it.s cheaper too.

    Only issue is that VM can be utter arsehats about transferring numbers back... But I'm gonna fight them all the way on this.. 12 months of shitty service has made me very angry and anything to hit them in the wallet is fine with me.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like