The engineering team has been so incredibly busy working on the site that I haven’t written an update in a while! By way of re-introducing myself to the members of the Etsy community who have joined since I last wrote, I run the engineering team here at Etsy and have been at Etsy since September 2008. As we head into the holiday season, I wanted to give you a few updates on what we have been doing and our special preparations for the holidays.
We’re continuing to invest in making the site stable and fast, and 2010 so far has been the best year in Etsy’s history for uptime and speed. We’ve also invested an incredible amount of time, energy, and money into our testing infrastructure this year. All of this adds up to a site that is faster and more stable than it has ever been as we enter the holiday season. Those who have been with us at Etsy for a while will recall that in 2009, we took the site offline for planned maintenance for hours as often as every two weeks to make major infrastructural changes to provide long-term, consistent stability. That work paid off and those types of major interruptions are ancient history. Every week, we’re making invisible changes to make the site better and faster for you.
I understand that this year, it may seem like Etsy is making changes all the way up to the peak holiday season, and we are. Many of the features you are seeing roll out have been underway for months and we would have liked to have gotten them out a little sooner. When we thought about timing, we felt that some of the features would be particularly useful for the holiday season (like coupon codes, which depended on our new checkout, and renew sold and copy), so we committed to making the final push to get those into Etsy members’ hands before the season kicks into high gear. Some of the other features we are releasing help shoppers discover unique gifts on Etsy, and we want everyone to have a successful holiday season, whether you’re selling, buying, or both. We understand that it’s important to you that the core site remains stable, available, and fast during what can be a hectic time. That is of primary importance to us, too, so as we head into Thanksgiving week, that will be top of mind for the engineering team.
You’ve probably also noticed that over the last several months we have been rolling out features to randomly selected groups of Etsy members before rolling out to everyone. The goal is to get quantitative feedback from members of the Etsy community and make sure the feature works well under real system load before rolling the feature out to everyone. Testing new approaches with small groups of people is a common practice online, and if you’ve ever used Google, Yahoo, or Facebook, you may have been part of what is known as a “bucket test” — where a percentage of users are put into a “bucket” of users who are testing a new feature — or “A/B test” — where two approaches are tested with two different groups of users. Most sites don’t tell you when you’re being bucket-tested, but we do because we want you to know when you might see something new. We read the Forums, watch Twitter, and talk to Etsy members, of course, but watching the overall data trends as we roll out new features gives us a clear picture of how features are being used by all members and if the larger community is successful in whatever they are trying to do (list items, search for gifts, etc.)
Behind the scenes, we have developed a number of systems in the past several months that provide constant data about how folks are using features, so as we roll out features, we watch how changes affect those numbers, many in real-time. One example of this is the wider listing page. As Sean wrote, we rolled out our new designs for the listing pages in late September and tested if the new designs resulted in more “Add to Cart” actions (they did) and also if they led people to explore shops more (they did). We’ve released other features that have had a notable impact on sales. For example, visitors to the site who use the “Items You Might Like” module on the home page buy at a significantly higher rate than visitors who do not use the module. We’ve been watching our metrics very closely as we’ve updated the checkout process to make sure the impact of any changes has been positive. If you ever have a chance to visit Etsy’s offices, you’ll see a number of large screens that display key systems and business metrics live. We watch these and many other metrics like hawks, and these numbers give us an immediate indication whether a particular change has been effective or not.
Photo by Zhi-Da Zhong
As we head into peak holiday shopping season, I also wanted to let you know how we staff our systems and network operations team. We operate this way year-round, but we have an especially heightened awareness this time of year, as I know many of you do as well. The Etsy site operations team is a 24/7 team and we now have systems operations team members around the globe, from Portland, Oregon to London. Our team members sleep with their phones by the bed and they (and their very patient families) stand at the ready when problems arise. Around 2,000 automated checks will alert the operations staff about any issues, and anyone on Etsy’s staff can page operations on call and get help in minutes at any time of day or night.
We wish you the best this holiday season, and we’re standing by to make it an awesome one!