"only impacted website metadata stored in our MySQL databases"
... which of course is the only valuable part of Github.
The actual repository data is completely mirrored in every single git checkout that you have.
Programmers, your snow day is well and truly over: GitHub's website has finally cleared its 24-hour outage, and reckons everything is operating normally again. Last year, CEO Chris Wanstrath said the company was shooting for zero downtime, or at least five-nines of uptime. Well, that comes out at about five minutes of non- …
This post has been deleted by its author
Any competent DBA will tell you that you should be running multiple replication slaves of any mission-critical MySQL database server, and they should be using different back-end storage systems than the master. These replication slaves can provide read-only copies for load-balancing, and can also be promoted to become a read/write master in the event that the master suffers a hardware failure.
Someone at GitHub/Microsoft evidently didn't get beyond chapter 1 of the "How to be a production MySQL DBA" notes.
OT, but that reminded me of the "pep talk" we recently acqui-hired remnants of a once-promising startup were given on arrival at the (larger head-count than the town I grew up in) megacorp. The speaker was the (insert plausible title translating to "high muckety-muck") of global marketing, and he reassured us that the emphasis will still be on commitment to quality and reliability, promising we would deliver "nine fives". A fellow newly-borged coworker and I looked at each other, agreeing that this lot could probably achieve it.