Hey guys,
Corbin here, just filling out a brief status update. We’re back up and running, as far as I can tell. I haven’t found anything broken in a few hours, so I’m moving us from “everything is broken and also on fire” to “any problems you may see are actually features in disguise” mode.
This is the official post-mortem. The short version: Some hardware failed. This is a completely natural thing. Due to some awkwardness and a lack of proper preparation on my part, we weren’t able to immediately get back up on our feet, but I just want to stress that this was an act of God and not something we could have guarded against any better. Please don’t email flames to Dreamhost; they’ve been really great throughout the process and gave us a pretty good turnaround for a weekend failure.
The technical details aren’t that interesting. We didn’t have snapshot backups, so it took a bit of leg work and facepalming to get everything back in place. (We now have snapshot backups. I learned the lesson.) Fortunately, the database was unharmed, so all of your comments are still there. (Yes, even the stupid ones.)
Thank you for flying Manly Air, and we look forward to many more comics with you all!
~ C.
