Last Thursday, an error in the wordpress.com software caused some user settings to be overwritten, which resulted in loss of settings for some customers. The site was taken down for checks, and an hour later, 99% of users were back online.
The cause of the error? A coding error of a single character. Certainly checks and balances are needed, but according to Matt Mullenweg, founder of WordPress.com, they are already using reviews and testing.
It was less than a month ago that Toni Schneider, CEO of Automattic, wrote in glowing terms about the use of “continuous deployment” at wordpress.com. Is this event going to lead to the death of “continuous deployment” at WordPress? I suspect not.
In fact, Paul Graham described in a paper how he used Lisp for Viaweb in just this fashion. Viaweb was bought by Yahoo! and became the Yahoo Store. Viaweb would fully implement features before it had even become mainstream.
Let this WordPress.com downtime be a lesson as to what a single character can do, and also a lesson in how none of us are immune from such mistakes.
UNIX Administratosphere 