When you’re sound asleep, we’ll be performing a major upgrade to make our purple gladlyCast server harder, better, faster, stronger:

We are performing the upgrade on Aug 11 (Tuesday), at 3 am in the morning as that’s the time where the least people use our services. The update will take maximum 1 hour. If you send an SMS during our downtime, it will be queued safely on another server and delivered to Twitter once our upgrade is completed.

No messages is expected to be lost during this transition. We treat your SMSes with care and provide refreshments to make them happy. We would update our gladlyCast Twitter regularly during which time.

The August 11 plan for the upgrade is to:

1. 3:00 am — Pull the plug and upgrade.
2. 3:20 am — Move everything over and wake the server.
3. 3:25 am — Wake our Twitter service and redeliver all queued messages sent between 3:00 am and 3:25 am.
4. 3:30 am — Wake our web server with sign ups re-enabled.
5. Spend the next 30 minutes ensuring all components are functioning.

(During this transition, this blog will go down too. This is something we haven’t thought about and I’m feeling rather dumb right now.)

Update 3:30 am: Our web server is up. (:
Update 3:33 am: Sign ups are not re-enabled at the moment.
Update 3:43 am: Components are functional except for phone association. Still working on that one.
Update 4:10 am: Sign up is re-enabled. Upgrade completed.

Tags: downtime, message queue, scalability, twitter, upgrade, vertical scaling

This entry was posted on Sunday, August 9th, 2009 at 9:16 pm and is filed under gladlyCast, gladlyCode. You can follow any responses to this entry through the RSS 2.0 feed.