Dan Siroker helps companies discover tiny truths, but his story begins with a lie. It was November 2007 and Barack Obama, then a Democratic candidate for president, was at Google’s headquarters in Mountain View, California, to speak. Siroker—who today is CEO of the web-testing firm Optimizely, but then was a product manager on Google’s browser team—tried to cut the enormous line by sneaking in a back entrance. “I walked up to the security guard and said, ‘I have to get to a meeting in there,’” Siroker recalls. There was no meeting, but his bluff got him in.
At the talk, Obama fielded a facetious question from then-CEO Eric Schmidt: “What is the most efficient way to sort a million 32-bit integers?” Schmidt was having a bit of fun, but before he could move on to a real question, Obama stopped him. “Well, I think the bubble sort would be the wrong way to go,” he said—correctly. Schmidt put his hand to his forehead in disbelief, and the room erupted in raucous applause. Siroker was instantly smitten. “He had me at ‘bubble sort,’” he says. Two weeks later he had taken a leave of absence from Google, moved to Chicago, and joined up with Obama’s campaign as a digital adviser.
At first he wasn’t sure how he could help. But he recalled something else Obama had said to the Googlers: “I am a big believer in reason and facts and evidence and science and feedback—everything that allows you to do what you do. That’s what we should be doing in our government.” And so Siroker decided he would introduce Obama’s campaign to a crucial technique—almost a governing ethos—that Google relies on in developing and refining its products. He showed them how to A/B test.
Over the past decade, the power of A/B testing has become an open secret of high-stakes web development. It’s now the standard (but seldom advertised) means through which Silicon Valley improves its online products. Using A/B, new ideas can be essentially focus-group tested in real time: Without being told, a fraction of users are diverted to a slightly different version of a given web page and their behavior compared against the mass of users on the standard site. If the new version proves superior—gaining more clicks, longer visits, more purchases—it will displace the original; if the new version is inferior, it’s quietly phased out without most users ever seeing it. A/B allows seemingly subjective questions of design—color, layout, image selection, text—to become incontrovertible matters of data-driven social science.
by Brian Christian, Wired | Read more:
Photo: Spencer Higgins; Illustration: Si Scott
