One scary tidbit: rolling big changes to SERP design without testing, and then waiting a fiscal quarter to notice AOL users had revolted. Here's that section:
Grant zeroed in on the multimedia character of AOL query results: If you searched for "Radiohead," for instance, you would get not just text articles on the band but also images, video, and links to their songs. Grant saw this differentiation as a weakness -- it slowed load time, and the rich results meant that users were less likely to refine their searches, thus delivering below-average page views. He ordered a change to the page, making it look and operate exactly like Google's. Yet it turned out there were ways in which some users actually preferred the old format. It was certainly different, offering people a reason to go to AOL rather than Google. And the below-average page views could be seen as a sign that users were finding what they wanted the first time through. Also, according to former executives, the old search page actually produced more revenue per search than Google's.
In any case, changing the page backfired, badly. Search revenue fell to $156 million, from $232 million the previous quarter. As a result, AOL missed its revenue targets. "Management was blindsided by how disruptive the change to search was," says Pali Research analyst Richard Greenfield. "It's troubling that they didn't know what the impact of the search change would be. This raises serious concerns about their ability to run the business and turn it around."
Perhaps a basic split test could have save a few hundred million dollars?
It is sad to see such a great internet pioneer so reduced...
Article link: Dead Man Walking