Testing Models of Consumer Search Using Data on Web Browsing and Purchasing Behavior

(with Ali Hortaçsu and Matthijs R. Wildenbeest, American Economic Review, 102(6): 2955–80, 2012.)

Using a large dataset on web browsing and purchasing behavior we test to what extent consumers are searching in accordance to various search models. We find that the benchmark model of sequential search with a known price distribution can be rejected based on recall patterns found in the data as well as the absence of dependence of search decisions on prices. Our findings suggest fixed sample size search provides a more accurate description of search behavior. We then utilize the fixed sample size search model to estimate demand elasticities of online bookstores in an environment where store preferences are heterogeneous.

[Published version] [Working paper version]

Search Engine Optimization: What Drives Organic Traffic to Retail Sites?

(with Michael R. Baye and Matthijs R. Wildenbeest, Journal of Economics & Management Strategy, forthcoming)

The lion's share of retail traffic through search engines originates from organic (natural) rather than sponsored (paid) links. We use a dataset constructed from over 12,000 search terms and 2 million users to identify drivers of the organic clicks that the top 759 retailers received from search engines in August 2012. Our results are potentially important for search engine optimization (SEO). We find that a retailer's investments in factors such as the quality and brand awareness of its site increases organic clicks through both a direct and an indirect effect. The direct effect stems purely from consumer behavior: The greater the brand equity of an online retailer, the greater the number of consumers who click its link rather than a competitor in the list of organic results. The indirect effect stems from our finding that search engines tend to place better-branded sites in better positions, which results in additional clicks since consumers tend to click links in more favorable positions. We also find that consumers who are older, wealthier, conduct searches from work, use fewer words or include a brand name product in their search are more likely to click a retailer's organic link following a product search. Finally, the brand equity of a retail site appears to be especially important in attracting organic traffic from individuals with higher incomes. The beneficial direct and indirect effects of an online retailer's brand equity on organic clicks, coupled with the spillover effects on traffic through other online and traditional channels, leads us to conclude that investments in the quality and brand awareness of a site should be included as part of an SEO strategy.

Searching for Physical and Digital Media: The Evolution of Platforms for Finding Books

(with Michael R. Baye and and Matthijs R. Wildenbeest, in NBER's Economics of Digitization, ed. by S. Greenstein, A. Goldfarb, and C. Tucker. University of Chicago Press, forthcoming.)

This chapter provides a data-driven overview of the different online platforms that consumers use to search for books and booksellers, and documents how the use of these platforms is shifting over time. Our data suggest that, as a result of digitization, consumers are increasingly conducting searches for books at retailer sites and closed systems (e.g., the Kindle and Nook) rather than at general search engines (e.g., Google or Bing). We also highlight a number of challenges that will make it difficult for researchers to accurately measure internet-based search behavior in the years to come. Finally, we highlight a number of open agenda items related to the pricing of books and other digital media, as well as consumer search behavior.

Working Papers

Optimizing Click-through in Online Rankings for Partially Anonymous Consumers

(with Sergei Koulayev, invited for third round review, Marketing Science)

The vast amount of information available online has revolutionized the way firms present consumers with product options. Presenting the best alternatives first reduces search costs associated with a consumer finding the right product. We use novel data on consumer click-stream behavior from a major web-based hotel comparison platform to estimate a random coefficient discrete choice model and propose an optimal ranking tailored to anonymous consumers that differ in their partially revealed price sensitivity. We are able to customize rankings by relating price sensitivity to request parameters, such as the length of stay, number of guests, and day of the week of the stay. In contrast to a myopic popularity-based ranking, our model accounts for the rapidly changing prices that characterize the hotel industry and consumers' search refinement strategies, such as sorting and filtering of product options. We propose a method of determining the hotel ordering that maximizes consumers' click-through rates (CTR) based on the information available to the platform at that time, its assessment of consumers' preferences, and the expected consumer type based on request parameters from the current visit. We find that CTRs almost double when consumers are provided with customized rankings that reflect the price/quality trade-off inferred from the consumer's request parameters. We show that the optimal ranking results in an average consumer welfare 173 percent greater than in the default ranking.

Search with Learning

(with Ali Hortaçsu and Matthijs R. Wildenbeest, invited for third round review, Journal of Business & Economic Statistics)

This paper provides a method to estimate search costs in a differentiated product environment in which consumers are uncertain about the utility distribution. Consumers learn about the utility distribution by Bayesian updating their Dirichlet process prior beliefs. The model provides expressions for bounds on the search costs that can rationalize observed search and purchasing behavior. Using individual-specific data on web browsing and purchasing behavior for MP3 players sold online we show how to use these bounds to estimate search costs as well as the parameters of the utility distribution. Our estimates indicate that search costs are sizable. We show that wrongfully assuming consumers are not learning while searching can lead to severely biased search cost and elasticity estimates.

E-book Pricing and Vertical Restraints

(with Matthijs R. Wildenbeest)

This paper empirically analyzes how the use of vertical price restraints has impacted retail prices in the market for e-books. In 2010 five of the six largest publishers simultaneously adopted the agency model of book sales, allowing them to directly set retail prices. This led the Department of Justice to file suit against the publishers in 2012, the settlement of which prevents the publishers from interfering with retailers' ability to set e-book prices. Using a unique dataset of daily e-book prices for a large sample of books across major online retailers, we exploit cross-publisher variation in the timing of the return to the traditional wholesale model to estimate its effect on retail prices. We find that e-book prices for titles that were previously sold using the agency model decreased by 18 percent at Amazon and 8 percent at Barnes & Noble. Our results are robust to different specifications, placebo tests, and synthetic control groups. Our findings illustrate a case where upstream firms prefer to set higher retail prices than retailers and help to clarify conflicting theoretical predictions on agency versus wholesale models.

Do MSRPs Decrease Prices?

(with In Kyung Kim and Dmitry Lubensky)

The nature of manufacturer's suggested retail prices (MSRPs) and whether their effect is pro- or anticompetitive is not well understood. We exploit a policy experiment in which a ban on MSRPs was imposed and then lifted a year later. Prices increased by 2.3 percent as a result of the ban and decreased by 2.6 percent when the ban was lifted. We find no indication that MSRPs lowered prices by acting as binding price ceilings and outline an alternative mechanism in which recommendations affect prices indirectly by providing information to searching consumers. We demonstrate that recommendations can increase search and reduce prices.

What's in a Name? Measuring Prominence, and Its Impact on Organic Traffic from Search Engines

(with Michael R. Baye and and Matthijs R. Wildenbeest)

Organic product search results on Google and Bing do not systematically include information about seller characteristics (e.g., feedback ratings and prices). Consequently, it is often assumed that a retailer's organic traffic is driven by the prominence of its position in the list of search results. We propose a novel measure of the prominence of a retailer's name, and show that it is also an important predictor of the organic traffic retailers enjoy from product searches through Google and Bing. We also show that failure to account for the prominence of retailers' names---as well as the endogeneity of retailers' positions in the list of search results---significantly inflates the estimated impact of screen position on organic clicks.

Consumer Search on the Internet

This paper analyzes search frictions in online markets using novel data on the web browsing and purchasing behavior of a large panel of consumers. This dataset is unique in that consumer search behavior prior to a transaction is observed. Although recent models have shown that large price dispersion persists in various markets, and this dispersion is directly related to the magnitude of consumer search costs, little attention has been given to quantifying these costs. I use data on consumers shopping for books online to link prices and consumer search patterns at different bookstores to estimate consumer search costs in the context of an equilibrium search model. The search patterns indicate that consumers visit relatively few firms and exhibit a strong search preference for prominent retailers. I control for search intensities at different retailers during consumers' search process and find that search cost estimates are lower than when assuming consumers sample equally among alternatives. Accounting for unequal consumer search reduces search cost estimates in half from $1.8 to $0.9 per search. I examined the search cost heterogeneity by using a rich set of consumer characteristics and relating them to search patterns and search costs estimates. I use a flexible random effects model in which the number and order of firms visited by the consumer is her optimal ordered choices, allowing search cost cutoffs to depend on regressors. The estimates indicate that consumer search costs are related to their observable characteristics, such as income, where individuals with income greater than $100,000 incur relatively higher search costs.

Invited Publications

The Evolution of Product Search

(with Michael R. Baye and and Matthijs R. Wildenbeest, Journal of Law, Economics & Policy, 9(2): 201-21, 2013)

This paper examines the evolution of product search. We provide an overview of product search in the pre-internet era, and discuss how online search evolved from directory based search in the early 1990s to "vertical" search engines by the late 1990s. We also document the prominence of price comparison sites in the mid-2000s, and the challenges these platforms faced through 2010. We then use comScore qSearch data to closely examine trends in product search between 2010 and 2012. We find that, today, the vast majority of shoppers conduct product searches at retailer sites and other marketplaces, whereas traditional price comparison sites have become less important.