Forum Update and Outage

We started yesterday with what we thought would be a simple Debian upgrade on the forum server, but ran into troubles with the old customized setup. So we recreated the server virtual machine and restored the whole forum from our backups. The bad news is that the forum was down for half a day. There are a few good developments that came out of this:

  • Our backup and restore process has now been proven to work!
  • There is now a more automated restore process.
  • All packages now come from Debian, including Docker.
  • It is now running on a new clean install, wiping away any cruft.
  • It now runs on a standard base image without customization. The old VM was set up a long time ago, back when running Docker in Xen VMs required some kernel config customization.

Notes on the restore process

For anyone interested in the technical details about the restore process, here are some of our notes.

  • To restore, we used containers/app.yml and a .tar.gz from Discourse’s backup system where the dump.sql was manually edited to remove entries it refused to restore.
  • Bootstrapping the Discourse container takes a LONG time. They’ve built it very differently from most containers, when it first launches it builds and compiles a lot of stuff and then caches that. Their system works but it does make starting it the first time very slow. I’ll give an update when I can start restoring the backup.
  • The restore function of Discourse threw an exception. After manually deleting each broken database entry for each new database issue, the data was finally imported. We lost 1 post (which was missing a created_at) and 4 used tags.

Thanks to @TheLastProject for all her work on getting it going again!

9 Likes

It felt like a couple of hours :+1:

1 Like

Thanks a lot to everybody for their work, and also specially for this detailed report.
Kind regards!

1 Like

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.