Many years ago, I rented The Italian Job, which has a prominent getaway scene involving Mini Coopers—the first time I had ever heard of the cars.  Nearly a decade later, I viewed an advertisement for Minis when watching a video on YouTube, which triggered a memory of Mini Coopers being interesting.  After viewing the video I started researching the car.   After about a week’s worth of research, I decided that I wanted to purchase one.  I did a quick search for Mini Coopers in the Bay Area and was directed to a local dealer’s website by a sponsored link that appeared on the side of the search results.  I stopped by the dealer and later purchased my Mini.

In buying my car, I traversed what marketers call the sales funnel.  At many points along the way I was exposed to various forms of advertising.  These exposures, or touches in marketing terminology, may or may not have contributed to my purchasing decision.  For instance, the marketing group at BMW could claim that the product placement in the movie informed me of the the existence of the brand, and that the video ads on YouTube rekindled my interest.  The dealer’s search engine marketer could claim that the sponsored search link delivered me straight to their door, rather than some other dealership.

Determining what advertising actions affected my purchase is a question of attribution. There is a multi-billion dollar online advertising industry that is relentlessly devoted to figuring it out, because reasoning about how to employ this information allows us to engineer the most effective experiences for customers.^[1]

Modeling Attribution

Historically, advertisers lamented an attribution problem due to data scarcity.  Perhaps the most oft-heard expression of this is a quip attributed to John Wanamaker:

Half the money I spend on advertising is wasted; the trouble is I don’t know which half.

As the science of advertising progressed, different channels started to emerge.  Some more amenable to experimental analysis than others.

Fortunately, the abundance of rich data on the Internet has partially alleviated the scarcity problem near the end of the funnel—in many cases, we know what advertisements users were exposed to immediately before a potential purchase and what actions the users took.  In fact, the distinction campaigns at various points in the funnel is so stark, we classify them into performance (near the end of the funnel) and brand (near the begining of the funnel) campaigns in online advertising.  The differences in the types of campaigns are also reflected in how the respective advertisers choose to pay for ad placement, which we discuss subsequently.

In performance advertising scenarios, largely due to the existence of the aforementioned data, we can accurately predict how likely someone is to make a purchase soon after they view an online advertisement.  In my view, this is a Herculean task undertaken by some of the best experts in machine learning that I know.  Some claim we have reached a sort of phase transition in marketing from Mad Men to Math Men.  It is not clear to me that we are quite there yet, many of the institutions from the prior era still exist today.  Perhaps they have evolved or are yet to be disrupted. The history of ad agencies themselves is interesting and there is a great infographic about the origins of various agencies, courtesy of VitaminTalent:

As we move further up the funnel, attribution modeling becomes more difficult.  In particular, when users have exposure to multiple advertisements, which ones did the trick?  Figuring this out correctly is a very difficult problem and, in practice, often requires a well designed experimentation system.  As Randall Lewis and David Reiley exposed, it is even easy for experts to significantly error in their analysis.  For small advertisers without trained econometricians, this problem can be even more acute.  The core of the problem is that when estimating the lift (gain in revenue) of a given advertisement using data from a control and test group,  great care must be taken so that the populations are comparable.  Consider this simple example.

A restaurant owner wants to test the effect of handing out coupons to potential customers.  The owner offers $2 off a $10 meal (the only meal the restaurant offers).  The restaurant decides to hand out coupons.  The restaurant happens to be the only restaurant situated along a right branch of a fork in a walking path.  The restaurant owner stands a little ways down the right path and hands out coupons and has another employee, that stands in the middle of the left path, simply observe the people walking by.  At the end of the day, they tally their results.  Of the 100 people walking down the right path (all got coupons), 50 purchased a meal.  Of the 100 people walking down the left path, 5 purchased meals (without coupons).  The day before 200 people also passed through the fork and 50 people purchased meals.

Is it worth it for the restaurant owner to hand out coupons?  Well, using the left branch as a control, we see that handing out coupons (right branch) gives a (50 – 5) / 5 = 900% lift in customer purchases.  That amounts to an expected $450 in extra revenue at a cost of $90.  A net gain of $360, for a total of $460 net revenue after accounting for the coupon costs.  Well worth it!  Or is it?

If we look at the previous day’s totals, we see 50 meals were purchased for a total of $500 in revenue.  If both days were typical days, what could have gone wrong in the restaurant owners analysis?  It is likely that the two populations of people (the ones going left and the ones going right) were fundamentally different.  The ones going right may have taken that direction because the restaurant is on that branch.  In effect, the owner was giving coupons to people that were already going to purchase meals at the restaurant.  Thus, a net loss in revenue when compared to not using coupons.  A better experiment would have been to randomly give coupons to those traveling on the left and right branches, tracking the population of travelers that did not receive a coupon and comparing it to those who did.  Of course, this is a clear example of how the owner got the analysis wrong and lost money in the process.  Even so, these errors are easy to make and happen all of the time when attributing the affects of advertising.  Unfortunately, designing a good experiment can be difficult and expensive, so many firms knowingly render flawed analysis.

This article is not just about modeling attribution, however.  This article investigates how we would use a model of attribution to improve the effectiveness of online advertising.  What we have found is that successfully employing multiple-attribution models is every bit as difficult as coming up with the models themselves.

Employing Attribution Models

The previous section highlighted how important it is to correctly estimate the effects of advertising.  This section focuses on the particulars of online advertising, specifically how we can incorporate multiple attribution into current pricing schemes.  The figure below, produced by InfographicLabs, gives a nice illustration of the flow of information in online advertising, for those that are unfamilar.

 

Online Advertising by Infographiclabs

In recent years the online advertising industry has witnessed a shift from the more traditional pay-per-impression model to the pay-per-click (PPC) and more recently to the pay-per-conversion model.  Historically, website publishers sold banner advertisements through direct sales on a cost-per-mille (CPM) basis.  Remnant inventory was then sold to ad networks at, typically, a substantially lower price.  The ad networks then bundled and resold the inventory to affiliated advertisers, allowing advertisers to get greater audience reach as the network’s inventory broadened.

In parallel, sponsored-search auctions arrived as a viable method for purchasing advertising space, conditioned on user intent.  Initially, slots were sold on a CPM basis, but soon search engines allowed advertisers to pay only after a click occurred.  These PPC revenue models facilitated an explosion of long-tail advertisers in addition to large advertising firms.  No longer did individual advertisers have to reason about the rate at which an ad would be clicked, that risk was transferred to the search engines that were in a much better position to model and mitigate the risk though the large number of auctions served.

Not to be outdone, display ad networks evolved into ad exchanges, such as the Right Media Exchange (RMX), which offered multiple models for payment.  Advertisers can choose from a variety of payment options: pay each impression, pay on a click, pay on a conversion (sale or other advertiser-defined event).  The amount an advertiser pays is determined by the exchange rules and whether or not a payment event occurred.   In practice, what ads are rendered on a page and how much the advertisers are charged for the opportunity to display their ads are determined via an auction that is facilitated by the exchange.  For contingent payments, the exchange mechanism accounts for the estimated rate at which payment occurs for an ad when determining which advertiser’s ad is shown on the page.  When a payment event occurs, the exchange determines the auction associated with the payment (attribution) and the resulting transaction costs between the advertiser and publisher.  For impression events, the attribution is simple: the associated auction (and publisher) is the auction where the ad was served to the user.  For a click event, exchanges attribute the click to the last auction that served the ad to the particular user.  This is called last-event attribution and seems, intuitively, to be a reasonable model: the likelihood that I click on an ad is not influenced by previous views of the ad.  For conversion events, the situation is a bit more complicated.

Whether or not I make a purchase may be influenced by a sequence of views.  How many times I have viewed the ad influences where I am in the funnel—how more or less likely I am to purchase.  This presents a problem for the exchange, how should payment be distributed when multiple events influence a purchase (conversion).  Incorrect attribution by the exchange can cause inefficiencies in the marketplace.  We can illustrate this with a simple example:

[Consider] one pay-per-conversion advertiser that has a value of $1 per conversion.  Assume a user sees the ad of this advertiser four times on average.  The probability of converting after viewing the ad for the first time is 0.02, and after the second viewing this probability increases to 0.1.  The third and the fourth viewing of the ad will not lead to any conversions.  Also, assume that this ad always competes with a pay-per-impression ad with a bid of 4 cents per impression.

First, consider a system that simply computes the average conversion rate of the ad and allocates based on that. This method would estimate the conversion rate of the ad at (0.02 + 0.1 + 0 + 0)/4 = 0.03. Therefore, the ad’s effective bid per impression is 3 cents and the ad will always lose to the competitor. This is inefficient, since showing the ad twice gives an average expected value of 6 cents per impression, which is more than the competitor.

If we employ frequency capping and restrict the ad to be shown at most twice to each user, the above problem would be resolved, but another problem arises. In this case, the average conversion rate will be (0.02+0.1)/2 = 0.06, and the ad will win both impressions. This is indeed the efficient outcome, but let us look at this outcome from the perspective of the publishers (website owners). If the two impressions are on different publishers, the first publisher only gets 2 cents per impression in expectation, less than what the competitor pays. This is an unfair outcome, and means that this publisher would have an incentive not to accept this ad, thereby creating inefficiency.

Finally, note that even if the conversion rate is estimated accurately for each impression, still the usual mechanism of allocating based on expected value per impression is inefficient, since it will estimate the expected value per impression at 2 cents for the first impression. This will lose to the 4 cent competitor, and never gives the ad a chance to secure the second, more valuable, impression.

By now it should be clear that the standard pay-per-conversion auction mechanism is not efficient.  What can we do?  First, we can just accept this inefficiency, however this leaves open an opportunity for a clever arbitrageur to enter the market and profit.  Second, the advertiser could just pay per impression and the exchanges problem would be solved.  However, this assumes that a lone advertiser has the requisite information to correctly model multiple attribution, which is unlikely for the reasons discussed in the previous section.  A third option involves the exchange, or some intermediary, purchasing the inventory per impression and selling it per conversion to the advertiser.

In an external paper (Patrick Jordan, Mohammad Mahdian, Sergei Vassilvitskii, and Erik Vee, 2011, 31-43), coauthors and I describe a novel approach to this problem.  We develop a fairly general model to capture how a sequence of impressions can lead to a conversion, and solve the optimal ad allocation problem in this model. We show that this allocation can be supplemented with a payment scheme to obtain a mechanism that is incentive compatible for the advertiser and fair for the publishers.

The crux of the idea is that the intermediary must account both for the immediate value of the impression, as well as the expected future value given the prevailing market conditions.  Determining the correct amount to bid involves a recursive formulation that can be solved with dynamic programming.

1. I recently learned there is a marketing term for this: sales process engineering. ^

TAC/AA is part of the general 2011 Trading Agent Competition (TAC).  TAC 2011 was held in July, 2011 in Barcelona, Spain, in conjunction with IJCAI-11 and the workshop on Trading Agent Design and Analysis (TADA-11).

The TAC/AA competition consists of two basic rounds: qualifying and finals.  During the qualifying round agents will participate in a round-robin style tournament.  Agents pass the qualifying round by meeting a minimal standard for agent competence.  The TAC/AA tournament finals consists of multiple stages, with the particular elimination structure to be determined based on the number of entries.

Eleven teams from around the world entered the competition (see table below).  Nine agents displayed the minimum standard of competence during a round-robin qualifying stage held in June, thus advanced to the final tournament at the main TAC event in July.  The dates for the TAC/AA competition are as follows:

• Qualifying Rounds: 22-23 June  [ day 1 and day 2 ]
• Final Rounds:   17 July [ semifinals ]   19 July [ finals ]

The 2011 TAC/AA Qualifying Rounds completed on June 23rd.  The results from each day: day 1 and day 2.  Teams that made it through the qualifying rounds are shown in the table below.

The tournament finals are divided into two phases: the semifinal stage and the final stage.  During the semifinal stage, one team will be eliminated, leaving eight agents to participate in the final stage.  In the final stage agents are ranked by their average profits.  The tournament winner is the agent with the largest average profits.  This year’s tournament consists of many strong competitors from past tournaments.  In particular, TacTex remains undefeated over history of the competition.

The top three most profitable teams from the 2011 tournament are, in order, are TacTex from the University of Texas at Austin,  Schlemazl from Brown University, and tau from Tel Aviv University.  The full list of scores is given in the table below.

Congratulations to all the teams that competed.

Further Reading

In its short history, numerous papers have been written about TAC/AA:

• Designing the Ad Auctions Game for the Trading Agent Competition
• Strategy and mechanism lessons from the first ad auctions trading agent competition
• TacTex09: A Champion Bidding Agent for Ad Auctions
• A Particle Filter for Bid Estimation in Ad Auctions with Periodic Ranking Observations
• A First Approach to Autonomous Bidding in Ad Auctions
• A new sequential classification to assist Ad auction agent in making decisions
• A Bidding Agent for Advertisement Auctions: An Overview of the CrocodileAgent 2010
• A Knapsack-Based Approach to Bidding in Ad Auctions
• Rank and Impression Estimation in TAC AA

In addition, TAC/AA has garnered some interest in industry:

• The Search Agency

Thanks to Andrea Zabel for reading drafts of this post.

From an advertiser’s perspective, determining an optimal bid can be complicated.  For instance, an advertiser may calculate the likelihood you will take certain actions once you view the ad, such as purchasing a product, as well as model the bidding behavior of its competitors and the rules of the auction.  As advertisers have become more nuanced in their valuation of advertising opportunities, different mechanisms have emerged to facilitate sophisticated transactions between advertisers and publishers.

Ad exchanges such as Yahoo’s Right Media Exchange and Google’s DoubleClick Ad Exchange handle billions of advertising opportunities a day.  The exchange rules themselves are complex.  For instance, one advertiser may specify that it will pay the publisher only when a user purchases an item on its site after viewing an ad on the publisher’s site, whereas another may specify that it will pay only when a user clicks on its ad.  The auction mechanism provided by the exchange must compare these offers for the publisher and finally determine which advertiser wins the auction and how much they should pay if contingent events (clicks, purchases, etc) occur.

For each economic opportunity, requesting, determining, and ranking bids spans a duration of milliseconds.  Therefore, not only must the advertisers’ bidding algorithms and the auction mechanism be sophisticated, it must be fast.  This mix of economics and computer science has produced a growing field called computational or algorithmic economics.

However, research in bidding agent strategies is often limited to stylized models.  Few companies would open their proprietary bidding models to external scientific investigation.   Since 2000, the Trading Agent Competition (TAC) series of tournaments has spurred researchers to develop improved automated bidding techniques for an array of challenging market domains.  TAC provides an important venue for research teams to openly compete in realistic markets.  Additionally, the repeatable nature of the simulations allows researchers to draw statistically valid conclusions.

The original TAC game presented a travel-shopping scenario; subsequent games have addressed problems in supply chain management, market design, and ad auctions. By continually introducing new games, the TAC series engages the community in an expanded set of strategic issues bearing on trading agent design and analysis.  A key feature of these games is that, like most realistic market environments, they are sufficiently complicated to defy analytic solution.  Such games often exhibit severely imperfect and incomplete information revealed over time throughout dynamic activity.

Participants in TAC develop autonomous agents that bid in various markets.  In doing so, the designers face many issues found in electronic markets today.  For instance, in the Ad Auctions game (TAC/AA), agents representing Internet advertisers bid for search-engine ad placement over a range of interrelated keyword combinations.   Advertiser strategies combining online data analysis and bidding tactics compete to maximize profit over the simulated campaign horizon.

Lessons learned in early competitions helped to distill processes by which designers construct bidding agents for a given scenario.  In particular, participants in TAC have produced general agent architectures applicable to a variety of market domains.  These architectures modularize two core problems agents face: prediction and optimization.  For instance, the winning agent in the 2009 and 2010 TAC/AA tournaments developed sophisticated models for predicting the bids of opponents in each keyword auction.  These predictions allowed the agent to select the most profitable positions.  As it happens, this is one of the key problems faced by search-engine marketing companies as they optimize advertising campaigns for their clients.

Participating research teams come from many of the top international institutions.  Participating agent designers, as well as mechanism designers, have produced a considerable amount of academic research as a result of the competitions.  In addition, many of the competitions’ participants have applied the lessons learned in TAC to industry: Yahoo, Google, Microsoft, various advertising agencies, etc.

The Twelfth Annual Trading Agent Competition (TAC-11) will be held in July, 2011 in Barcelona, Spain, in conjunction with IJCAI-11 and the workshop on Trading Agent Design and Analysis (TADA-11).  TAC 2011 is comprised of five scenarios:

• Power TAC,
• Lemonade Stand Game,
• Ad Auctions,
• Market Design,
• Supply Chain Management.

To register for TAC-11, please fill out the registration form.  For general information about TAC, see the TAC website or the Trading Agent Organization.

TAC/Travel

TAC/Travel is an eight-player travel-shopping scenario where each player acts as a travel agent, with the goal of assembling travel packages. Each agent acts on behalf of eight clients, who express their preferences for various aspects of the trip. The objective of the travel agent is to maximize the total satisfaction of its clients (the sum of the client utilities) minus expenditures. An annual competition for the game was run every year from 2000–2006. Over 70 entries have competed over the entire history of the yearly competitions, with many teams submitting entries over the course of multiple competitions. There are over 30 scholarly publications, including one book, that analyze the entries and the game itself.

TAC/SCM

TAC/SCM is a six-player supply chain management scenario where players act as manufacturers, who must compete with each other for both supplies and customers, and manage inventories and production facilities. The objective of each manufacturer is to maximize its accumulated profits over the course of a simulated year. An annual competition for the game has run every year since 2003. Participants have created over 150 entries and over 50 corresponding scholarly publications.

TAC/Market Design

TAC/Market Design is a variable-player double auction market scenario where agents act as market specialists. Players in the game create respective marketplaces that facilitate transactions between potential buyers and sellers currently associated with the marketplace. Players compete over three objectives: profit share from fees, market share, and transaction efficiency. The corresponding competition has run every year since 2007. Participants have created over 30 entries and at least 10 corresponding publications.

TAC/AA

TAC/AA is an eight-player sponsored search advertising scenario where agents act as advertisers. Players manage their respective ad campaign by selecting bids and ads to be displayed for each day. The objective of each advertiser is to maximize its accumulated sales profits net advertising expense over the course of a simulated 2-month campaign. Participants created 15 entries for the July 2009 inaugural competition.

TAC/Power TAC

Sustainable energy systems of the future will need more than efficient, clean, low-cost, renewable energy sources; they will also need efficient price signals that motivate sustainable energy consumption as well as a better real-time alignment of energy demand and supply. In Power TAC, agents act as retail brokers in a local power distribution region, purchasing power from a wholesale market as well as from local sources, such as homes and businesses with solar panels, and selling power to local customers and into the wholesale market. Retail brokers must solve a supply-chain problem in which the product is infinitely perishable, and supply and demand must be exactly balanced at all times.

Lemonade Stand Game

It is summer on Lemonade Island, and you need to make some cash. You set up a lemonade stand on the beach (which goes all around the island), as do two other entrepreneurs. There are twelve places to set up, evenly spaced around the island. Your price is fixed, and all customers go to the nearest lemonade stand. Every night, everyone moves under cover of darkness (simultaneously) and in the morning, their locations are fixed. There is no cost to move. After 100 days of summer, the game is over. The utility of the repeated game is the sum of the utilities of single-shot games.

Thanks to Kevin Lochner,  Dan Reeves, David Pardoe, and Andrea Zabel for reading drafts of this post.

Congratulations to all the teams that competed.

See last year’s results here.

Abstract

Sponsored search auctions provide a substantial source of revenue for online publishers, amounting to billions of dollars annually. Given the impact of these online ad auctions, a growing body of researchers has started to investigate both the mechanism design problem faced by publishers as well as the strategic problems faced by advertisers. Developed at the University of Michigan, the TAC Ad Auctions (TAC/AA) game is one such avenue of research into online advertising auctions. The TAC/AA game provides a challenging simulated sponsored search advertising scenario, and was developed to explore features of these auctions beyond the reach of current analytical methods. In this presentation, we present an overview of the game along with some analysis of the 2009 TAC/AA tournament. In particular, we discuss the strategic choices that advertisers made when bidding for slots across multiple interdependent auctions.

The 2009 TAC/AA competition had two basic rounds: qualifying and finals. During the qualifying round agents participates in a round-robin style tournament. Agents pass the qualifying round by meeting a minimal standard for agent competence. The TAC/AA tournament finals was held during the Trading Agent Design and Analysis (TADA) workshop as well as the main IJCAI conference in July 2009.

The tournament consisted of two stages: semifinals and finals. During each stage of the finals approximately half of the agents were eliminated, while the remaining agents continued on to the next stage. In total, fifteen teams participated in the competition with eight moving on to the final stage of the finals round. The semifinal stage consisted of a total of 88 games, while the final stage consisted of 80. Teams in the semifinal stage played in either 44 or 48 games, while all agents in the finals played in each of the 80 games.

Congratulations to all the teams that competed. The results of the competition can be found at the TAC/AA site and general information about the 2009 competition on the main TAC site.

We regressed this model on the semifinals test set, then evaluated by calculating the standard error of each finalist’s score. This method has an average standard error of $581.71. Without the adjustments of this model, the average standard error is $1,129.34. Thus we have a decrease in standard error of 48%. The table below gives the coefficients and means selected as the control variates’ model for the 2009 server.

Guha Balakrishnan has written an excellent parser that you may download software to perform the control variates calculation. The software (run via a shell script) is available via the following links

• Agent Adjusted Score Parser [zip]
• Agent Adjusted Score Parser [tgz]

The adjusted score parser is run as follows:

agent-adjusted-score-parser game.slz.gz

This will produce a YAML formatted output. For instance, running the parser on game 323 of the 2009 finals returns:

---
"TacTex": 73244.03720799176
"Schlemazl": 78888.52399318124
"MetroClick": 65975.26726281317
"munsey": 64521.04637546821
"UMTac09": 63280.05978207753
"AstonTAC": 76017.1659715028
"QuakTAC": 78955.42417305509
"epflagent": 62374.29730182024

The TAC Ad Auctions (TAC/AA) game presents a realistic sponsored search environment for a simulated advertising scenario. Advertisers represent retailers in a simplified home entertainment market, bidding to place ads before users searching on product keywords. Game entrants design and implement bidding strategies for the advertisers, while behaviors of the search engine and users are simulated by the server. The TAC/AA tournament pits advertiser agent strategies against each other, evaluating each in terms of sales profit net of advertising costs.

Our hope is that by tackling this challenging problem competitively, researchers and practitioners participating in TAC/AA will produce new ideas about bidding strategy for advertising (and in general), as well as insights about sponsored-search mechanisms and ways to improve the model.

Visit the TAC/AA website for participation information as well as code for developing an agent.