We have had some technical oppses and have since resolved it. Tweets have been delayed, however. We have increased the flood caps to send out the tweets faster.

This time, the issue has been mostly a mistake on our part. We have failed to take sufficient measures to keep the gladlyCast code base up-to-date and we succumbed issues concerning software regression.

In time, we should be pushing out all the tweets by midnight. We are observing the gladlyCast system closely for a couple of days.

Happy tweeting!

[All tweets have been sent out.]

The Singapore SMS device was malfunctioned this morning. We did not get notified until @chrissyypoo alerted us. Sorry for the delayed messages, we have successfully delivered all delayed messages from Singapore.

We are currently investigating what went wrong and update you guys on what actually occurred.

The Singapore SMS feature is up and will be monitored closely.

Update: A glitch occurred on the software that checks the SMS device and it stops checking for new messages until we restarted it.

Our Twitter service is resting for 5 min as we patch an error. SMSes will be queued and delayed for 5 min pending this emergency update.

Update: It’s back up.

(What actually happened is that Twitter threw us an unexpected error and gladlyCast didn’t know how to react so it went into sleep mode till we addressed this issue.)

When you’re sound asleep, we’ll be performing a major upgrade to make our purple gladlyCast server harder, better, faster, stronger:

We are performing the upgrade on Aug 11 (Tuesday), at 3 am in the morning as that’s the time where the least people use our services. The update will take maximum 1 hour. If you send an SMS during our downtime, it will be queued safely on another server and delivered to Twitter once our upgrade is completed.

No messages is expected to be lost during this transition. We treat your SMSes with care and provide refreshments to make them happy. We would update our gladlyCast Twitter regularly during which time.

The August 11 plan for the upgrade is to:

1. 3:00 am — Pull the plug and upgrade.
2. 3:20 am — Move everything over and wake the server.
3. 3:25 am — Wake our Twitter service and redeliver all queued messages sent between 3:00 am and 3:25 am.
4. 3:30 am — Wake our web server with sign ups re-enabled.
5. Spend the next 30 minutes ensuring all components are functioning.

(During this transition, this blog will go down too. This is something we haven’t thought about and I’m feeling rather dumb right now.)

Update 3:30 am: Our web server is up. (:
Update 3:33 am: Sign ups are not re-enabled at the moment.
Update 3:43 am: Components are functional except for phone association. Still working on that one.
Update 4:10 am: Sign up is re-enabled. Upgrade completed.