Sunday, May 11, 2008

Deployment: May 11, 2008

This was our biggest deployment. Maybe not the most important—release 1 would have qualified as the most important—but with this release, we were taking our application and migrating it over to a completely new technology. That meant a new platform, new servers, new software… new everything. It also meant lots of testing.

00:00–00:11 (11 minutes): We all logged onto the conference bridge, and confirmed that we were ready to go.

00:11–00:55 (44 minutes): We began the backup of our database, before executing the DB scripts.

00:11–00:33 (22 minutes): Concurrently with the database backup, we began configuring our web servers.

00:33–01:00 (27 minutes): We began some configuration of our new application servers.

00:55–01:30 (35 minutes): We executed the DB scripts.

01:30–01:50 (20 minutes): We deployed the application to the application servers.

01:50–02:20 (30 minutes): We did our Sanity Testing.

02:20–06:40 (4 hours and 20 minutes): The users performed Landing Testing. There were issues, but they were all deemed minor enough that we could leave the deployment in place.

06:40–07:10 (30 minutes): We turned on the first “phase” of users. For this rollout, since it was so big, we actually decided to roll the users out in phases; it’s a web-based application, and all users go through the old application, which decides what “phase” the user is supposed to be on; if the user is in a “current” phase, s/he gets redirected to the new application’s URL. So we had to verify not only that the right users were on the right phase, but also that users on “non active” phases were still getting the old version of the application (i.e. that they were not getting redirected).

All in all, the deployment went about as I’d expected it to. I knew that there would be a lot of testing to do, and I’d expected us to find some issues—you can’t rewrite your entire code base in a new technology and not expect to find issues, even late in the game—and I was relieved that none of the issues were deemed serious enough to roll back the deployment.

At this point, we had to monitor the application, and see how it handled the load we were throwing at it.

No comments: