UPDATE: Everything went smoothly last night! Please do bookmark http://www.etsystatus.com and follow the @EtsyStatus account on Twitter — these will be the best ways to stay in touch with what’s going on with Sys Ops behind the scenes.
We used to do a lot of site maintenance wherein we’d take Etsy down for periods of time. We’ve been enjoying quite a respite, but tonight (early Saturday morning, really) we need to do some preventative work on one of our databases. It requires taking the site down, so we’re loath to do this. At 2:00 a.m. ET (view in your time zone here) we’re going to take the site offline while we do this work. While it’s likely to take 20 minutes, it could take as long as an hour.
Short story on why there’ll be downtime:
We’ve got some settings we need to change on one of the critical databases, and while it’s a relatively straightforward process, we’d like to do it with the site offline to be safer. We’re doing it at a time when most of you should be in bed.
Longer and nerdier story about it:
The database we’re talking about is a central database that contains a large portion of the data for Etsy, but not all. At peak, this monster of a server does about 6000 to 7000 database queries every second, and these days it is pretty bulletproof. However, there are a couple of improvements that we need to make that will allow it to use its disk storage a lot more efficiently. There’s also a point release change in the Postgres version we have that solves a number of bugs and other issues, as well as some performance fixes.
We’ve made these changes before elsewhere in the infrastructure, so we have high confidence in the changes. For the super geeky details: We’re changing the filesystem to not update access times on every disk IOPs, and we’re changing the I/O scheduler to one that performs better under random access conditions.
How we’re going to communicate to you about it:
We’re also going to take this opportunity to use some more communication tools at our disposal. In the spirit of being as communicative as possible, we’re going to have an Etsy Blog post (you’re reading it right now), a Forums announcement, and a new Etsy Status blog that we have at this URL: http://www.etsystatus.com. Some of you who are familiar with the fix.etsy.com blog know that we’ve used it to communicate those things in the past, and we’ll post updates there as well tonight. The Etsy Status blog will also tweet for all updates to the @EtsyStatus account on Twitter, so you’ll be able to get updates there as well.
We’ll let folks know via those channels after the site is back up as well.
You can also get an RSS feed of all TechUpdates here.