Update: Wiki and issue tracker outage

Daniel Beck August 13, 2015

I recently wrote about the two day outage of our wiki and issue tracker:

While this was a rather lengthy outage, it could have been much worse. We lost none of the data, after all.

OSUOSL have since published their post mortem. I was really wrong about not losing any data:

A further complication was that our backups were pointed at mysql2, which was out-of-date with mysql1, due to the initial synchronization failures. Fortunately, we had the binary logs from the 17th through the 30th. This means that though most data could be restored, some data from between the 15th and the 17th was lost.

For our issue tracker, that means that issues JENKINS-29432 to JENKINS-29468 were lost, as well as comments posted from about July 15 12:20 PM to July 17 2 AM (UTC). We know this thanks to the jenkinsci-issues mailing list where the lost issues and comments can be looked up for reposting.

We unfortunately don’t have such a record from our wiki.

infrastructure

About the author

Daniel Beck

Daniel is a Jenkins core maintainer and member of the Jenkins security team. He was the inaugural Jenkins security officer from 2015 to 2021. He sometimes contributes to developer documentation and project infrastructure in his spare time.