Deployment: July 20, 2008

If all went well, this would be the last deployment for a long time. We were pretty sure that we’d fixed all of our issues, and this release was simply to “turn on” all of the functionality for all users—now that it was working, we’d let them all use it.

01:30–01:35 (5 minutes): We all logged onto the conference bridge, and confirmed that we were ready to go.

01:35–01:44 (9 minutes): We deployed the application.

01:44–02:19 (35 minutes): We Sanity Tested the application.

02:12–02:31 (19 minutes): The users did their Landing Tests. This was overlapped a bit with the Sanity Testing, because the person doing Sanity had to reboot her computer halfway through the process. However, based on the nature of the changes—i.e. that there were no changes, just a configuration change—we saw very little danger of Sanity failing, so we felt safe in letting the Landing Test start before Sanity was completely finished.

Everything worked fine.

Overall deployment: 01:30–02:31 (1 hour and 1 minute).

Deployment: July 11, 2008

This was sort of a continuation of the previous deployment. If all went well, we’d be able to kiss our recent troubles goodbye, and get back on to a regular schedule, without so many deployments all the time.

01:30–01:33 (3 minutes): We all logged onto the conference bridge, and confirmed that we were ready to go.

01:33–01:40 (7 minutes): We made some configuration changes to some of the servers. (There was no “deployment” this time, per se, just configuration changes. But it still caused an outage, and so had to be controlled.)

01:40–01:55 (15 minutes): We performed our Sanity Testing. Everything was fine. Normally Landing Tests would follow, but in this case, since there were no changes to the application itself, it wasn’t deemed necessary.

01:55–01:58 (3 minutes): We verified with the user base that everything was back up and running.

Overall deployment: 01:30–01:58 (28 minutes).

Deployment: July 7, 2008

This was partially a bug-fix deployment, but it was also a configuration deployment. We had a number of configuration changes to make to our web servers, in preparation for some future changes to our network/physical architecture.

22:00–22:01 (1 minute): We all logged onto the conference bridge, and confirmed that we were ready to go.

22:01–22:28 (27 minutes): We made the appropriate configuration changes to our web servers.

22:08–22:38 (10 minutes): We down-graded the logging level on our application servers. Because of the recent troubleshooting we’d had to do, we’d increased the logging greatly, but we now felt it was time to decrease it, and stop chewing up so much disk space with our logs.

22:38–22:50 (12 minutes): We deployed the application.

22:50–23:11 (21 minutes): We performed our Sanity Testing. Everything was fine.

23:11–23:18 (7 minutes): The users did their Landing Test. Everything was fine. (It’s not usual that Sanity Test takes longer than Landing Test, but I guess the users had their testing down to a science, after so many deployments lately—many of which I missed, since I was on vacation.)

23:18–23:22 (4 minutes): We switched back on everything that had to be switched on, and verified with the user base was completely back up and running.

23:22–23:55 (33 minutes): Just as we had all signed off on the deployment, and were about to hang up, one of the users claimed that there was an issue. We spent this time re-testing the functionality, but there was no issue. (Error between keyboard and chair.)

Overall deployment: 22:00–23:55 (1 hour and 55 minutes).