Chad Dickerson joined the Etsy team on September 2, 2008 as our first CTO (Chief Technology Officer). Below Chad shares his personal account of his first two weeks at Etsy as well as some important information about the site, search functionality, and looking to the future.
Welcome to Etsy … Now Get to Work!
I just completed my second week at Etsy, and I wanted to give you an update from the technology trenches. As Etsy’s first CTO, I realize that it might not be entirely clear what a CTO does, so explaining my perspective on this new role at Etsy might be helpful. The CTO role is not simply about technology, but the relationships among people, process and technology. Success for a CTO goes far beyond choosing the best technologies. In many ways, the people and process parts of the equation are more important than the technology used. The people part is obvious, but process might seem opaque. Process is all about how you build services, test them, roll them out, monitor them and communicate internally and to the community, whether it’s a feature rollout or a bug. Above all, it’s about communicating well, which is why I am writing this now, sitting on the floor of my new apartment in Brooklyn among stacks of boxes that just arrived on Sunday from the West Coast.
The experience of coming in as a new CTO is like opening a door and walking into a room full of devices with random wires and blinking lights. Very quickly, you have to determine which ones are doing good things and functioning well, which ones are benign and which ones are ticking time bombs. For the ticking time bombs, you have to decide which ones to try to defuse first, and once you make that decision, you’re looking at a tangle of often unlabeled wires. Sometimes a seemingly positive or benign device is wired into one of the ticking time bombs, so you have to trace all of the wires in the room and understand the relationships across all of them. While those bombs are ticking, you have to get to know your team and their capabilities in real-time. Who on the team knows how to defuse this particular type of bomb? (Oh, and what is your name again?) This scenario is true of Etsy, and it’s been true of my early days in other CTO roles. It takes a little time to get a sense of how things work in a new place regardless of your past experience, but you also have to be prepared to make some quick decisions.
Learning About Etsy and Learning From the Best
I also want to give you a quick glimpse into what I have been doing in my first ten days on the job. I started on September 2, and we spent my first week in an Engineering meetup at the Etsy offices learning about how Etsy works at a very fundamental technology level. In those sessions, we discussed who currently does what, how those things are done, and the nuts-and-bolts technology environment at Etsy (again — people, process, technology). We closed the meetup with a visit from my former Yahoo! colleague Rasmus Lerdorf, a top-notch technologist who is best known for inventing PHP, the programming language that powers a large chunk of the web. Rasmus took us through his Simple is Hard presentation, which provides some very practical tips on how to tune PHP-driven web sites like ours for performance. Bringing in people like Rasmus to visit with the team illustrates another key part of the philosophy that I bring to Etsy: it is impossible to know everything, so when you need to learn about something, learn from the best. I feel very fortunate to have worked with some of the best in my 10 years in the Bay Area, and I plan to seek their counsel as much as possible in order to make Etsy a better place for you.
Aside from the technology discussions in the Engineering meetup, one of the themes I discussed was "intellectual honesty" and how important it is to a well-functioning team. To me, intellectual honesty is about admitting when something is not working so you can work together to figure out how to fix it. It’s also about measuring your successes and failures so you can properly reward for excellent work and learn from your failures so you can strive to never make the same mistake twice.
Intellectual Honesty: Search
As an example, search on Etsy is not working well in overall performance and speed. While I’m going to focus on search since it is a pressing issue that everyone can understand, it is truly emblematic of a lot of the challenges on the technology side. Building Etsy up to this point and keeping it running was a huge undertaking that took intense commitment, and I appreciate the passion that went into it. It is no small feat to build something like Etsy. That being said, the Etsy technology operation is in a state of transition that I think is very positive, but like all transitions of this type, there are growing pains. I am examining and evaluating the entire operation (again — people, process, technology) to make sure that Etsy will be able to handle current growth and the growth ahead. This is all happening against the backdrop of my very recent arrival and significant changes in the technology team just before I arrived and real day-to-day issues that need to be dealt with while I’m learning in real-time.
But back to search specifically … The performance of search is indirectly related to other issues, like the perceived delays in the posting of listings and many other site functions related to Etsy’s master database of listings and its replication out to our "slave" databases. In relative layman’s terms, the "slave" databases are copies of the master database that are synchronized periodically to the master database to distribute the computational load for services like search across more hardware. To be explicit about the search issue, some searches are taking as long as 60 seconds to return results and others are timing out altogether. I’ve seen the issues myself by using the site and I’ve heard from all of you in the forums, direct messages on Twitter, in blogs, etc. For some time now, band-aids have been applied to the problem for short term performance gains. I strongly believe that the underlying technology and architectural approach for search on Etsy are reaching their limits. We need to look at different approaches while we actively deal with the current issues.
In the short term, you’re probably thinking: "what is the schedule for fixing search now?" The honest answer is that after ten days on the job, I don’t have a firm timeline yet on the longer-term fix. But we are taking action in the short term. The existing search has been built and hacked on since the beginning of Etsy’s existence, so it is an ingrained system that can’t be ripped out and replaced overnight without potentially significant disruption. To mitigate the current performance issues, I pushed through an order for a batch of new hardware that we will be putting in place that should increase performance soon. We are looking at some other tweaks (too many to list here) to help in the meantime.
For the longer term, I have mobilized the Engineering team inside Etsy, and I have connected the team here to engineers working on search at some very large and successful sites that you probably know, use and love. I’m finding that a lot of those people love Etsy in the same way that led me to make the jump.
There are issues to address now and there will most certainly be bumps on the technology side as Etsy grows, but I want all of you Etsians to know that it is an honor and a privilege to be serving the Etsy community as your CTO. If you can give me some time and understanding here in my early days at Etsy, I hope to make you proud.
For more information on the business and the people behind Etsy, be sure to read The Long View: Rob & Maria from Etsy founder Rob Kalin and CEO Maria Thomas. Here’s our first post about Etsy hiring Chad Dickerson.
We’re closing commenting on this Storque post so we can follow your comments on this forum thread.