While I can’t talk about the products under development, it is fun when I can talk about things that everyone can use today to do something better than what you can do elsewhere.

With that in mind, I want to talk about Live Search Cashback. I have been interested in the concepts behind Cashback for a long time, since I was an early advocate for Cost Per Acquisition (CPA) as a future disruptive trend in ecommerce.

I wrote about this in connection with SNAP, which I helped Mayfield invest in back when I was an entrepreneur in residence there.

In CPA, the merchants pay based on completed actions by customers (e.g. purchases), rather than just paying to display ads or for clicks by users who may not actually purchase anything.

CPA has potential to ad value for all parts of the ecommerce ecosystem:

• It is great for merchants because they only pay when they get real value, so they don’t have to worry about losing money on advertising.
• It’s great for search engines (once there is enough scale) because they know just how much money they can make from advertisers if they can send the right traffic there, without worrying about limited advertising budgets, and they also get data about transactions so they can improve their service.
• It’s great for searchers, because search engines can rank ads (or even organic search results) based on the transactional information from other users. You would rather click on an ad or a search result that has led to happy purchases by other shoppers than click on one that attracted user interest but no sale. This is a good way to filter our spam of many kinds.

Microsoft’s Live Search Cashback has all these potential benefits, but with an added twist: Live Search actually gives some of the advertising revenue back to the users who make the purchases. Here is how it works:

• Search for a product on Live Search. There are now two different ways to buy using cashback:
  • You see a “gleam” on some of the ads that alerts you about the cashback opportunity. The ads list the cashback percentage.
  • You see a link to a Live Search product detail and “compare prices” page. On that page, you see the typical list of prices for your product across a wide variety of merchants. However while typical engines sort by total price including shipping and taxes, Live Search Cashback also shows you the total prices after the cashback discount.
• When you click over to one of those merchants from either the search result ad or the cashback price comparison page, Microsoft asks for your email so it can remember you when it needs to give you cashback if you make a purchase.
• Then you shop on the merchant page (any of the merchants you clicked on).
• Any product you buy from a selected cashback merchant qualifies you for the stated cashback percentage. You get an email from the Microsoft cashback program telling you how much cash you will get back. You have to create a Windows Live account once in order to set up the program, but that’s easy and also gets you access to all the other Microsoft online services (including search personalization, hotmail, msn messenger, Healthvault, etc).
• Then after a waiting period (60 days at present in most cases, which is long enough to prove you aren’t going to return the product and cancel your purchase), the cash shows up in your paypal account or other account you have specified.

That sounds great in principle, but like everyone, I wondered whether this actually works. In particular, I wondered: Could I use Live Search Cashback to get the absolute lowest price for products I am shopping for?

Here are some reasons why, in theory, the cashback system could fail to deliver the best deals:

• It is possible that the lowest-price merchants haven’t signed up for the cashback program yet
• Online sellers offering cashback might raise their prices knowing that buyers would be getting cashback. In that case, sellers would benefit from the program, but buyers would not and so it woudl be better to shop elsewhere for lower prices.

I have heard a lot of excitement about cashback inside Microsoft, and rumors of people getting some good deals, but so far I had not seen any proof that Live Search Cashback could get the best deal if you really tried all possibilities. So that’s what I set out to do.
First, a little about my shopping style. For high value items, I am really a price-conscious shopper. I first do research to determine the product that I want to buy. I use a lot of tools for this, but that will be the subject of another post. For now, it’s all about price. So once I have chosen the product I really want, I shop using all the tools and techniques available to get the lowest possible price. It’s perhaps a bit silly in my case since I do value my time more than the actual dollar savings I achieve, but I also value the feeling that I used smarts and tools to get the “best bargain”.
So with that in mind, here is my experience buying some pricey consumer electronics products on Cyber Monday this week. You can follow along at home and if you do it now you should get the same results.
I wanted to buy two products:

• Panasonic TH 50PZ80U – 50″ plasma TV
• Panasonic DMP-BD35K Blue-ray DVD player

The links above point to the Live Search product and results pages for these products, so you can try this at home.
For the plasma TV, it’s an expensive item (list price is $2,000 plus shipping, as I write this), so I really wanted to shop around and get a great price. I searched for this product on Amazon, Shopping.com, PriceGrabber, NextTag, CNET, Google, Yahoo, Ebay, and Live Search. And I clicked on many of the ads that came up on these sites offering special deals, coupons, other comparison shopping engines, etc. Here is what I found using each of these (I list total price including shipping and taxes):

• Shopping.com: They have a nice feature that lists the lowest price from a trusted merchant. In this case, it is Adorama Camera, for $1,269.00. (There was a lower price quoted from another less trusted merchant, but the price on the site when you click was actually higher than the price listed on Shopping.com. Annoying bait and switch tactic, perhaps?).
• Pricegrabber and NextTag: Both listed Butterfly Photo at $1263, but it is not a highly trusted seller, and then Adorama Camera for $1269.
• Amazon: In the Amazon Marketplace, the lowest price merchant is again Adorama Camera, but through Amazon the price is $1296.00. It looks like you pay $27 to buy through Amazon instead of going direct.
• CNET: There was one merchant, 6TH Ave, that listed at $1244 but they had a low trust rating, so I discounted them. The lowest price trusted merchant on CNET, B&H Photo, offered it at $1279.
• Google: The lowest price merchant was Universal LCD, for $1265. (Note Google listed B&H Photo with a 1,235.50 total price, but that was for a demo item with a scratch. The ones without scratch at B&H Photo are $1279, as listed on CNET.)
• Yahoo: The lowest price merchant was Universal LCD, for $1265, same as on Google.
• Ebay: The lowest price seller offering buy-it-now was for $1469. (It’s possible I could have got it for less waiting a few days for the auction, but so far I’ve hardly ever seen a hot item go on auction for less than the cheapest prices elsewhere, and I’m really only comparing things I can order right now that are in-stock.)

Having tried all the other services, I then tried Live Search Cashback. I tried both the “cashback ad” path and the “cashback price comparison” path (as described earlier).

• Ebay via Live Search Cashback: The best ad was the Ebay ad, offering cashback of 25%. All you do is click on that Ebay ad, which takes you to Ebay. Then anything you buy on ebay using Paypal and buy-it-now gives you that 25% (be sure you go to eBay after clicking on the ad, not directly, in order to get the cashback). I found that same trusted seller offering it for $1469, but once I walked through the buying path it showed me that I would get $200 cashback (the cashback maximum is $200). So that nets out to $1269, which matches the lowest price I could find from any other service.
• Best merchant from the Live Search comparison page: Adorama. They offered 3% cashback of their base price of $1269 (the same price I found using shopping.com) for a total net price of $1230.93. This is the one I wound up buying.

So Live Search Cashback found me the best deal on the internet for my 50″ Panasonic plasma TV! It was interesting that despite the 25% cashback deal with Ebay, it was still better to use another cashback merchant. I wondered if that was true for all products, or if sometimes it might be a better value to buy on Ebay with those big cashback deals.
This turned out to be true for the Panasonic Blue-ray DVD player. I bought it on eBay for $259 and got $52 cashback, for a net price of $207.
By comparison: shopping.com, nexttag, pricegrabber, yahoo all had B&H Photo at $239.95 (with occasionally a lower-trusted merchant coming in at $224). Amazon Marketplace had it at $229.72 from Electronics Express, thus beating all the other deals except Live Search Cashback on this product.
So the bottom line:
I shopped for two high-quality, high value consumer electronics products that I wanted to buy. I tried all online services I could think of, and for each of these products I got the best deal on the internet using Live Search Cashback!

This is exciting for me personally and I am happy to share the exciting news with my friends and family, especially in the current economy where everyone is watching expenses.
On a broader note, I must admit I was skeptical about whether cashback would really work and whether it would draw in new users to Live Search. Now that I have done my own research and found that it really can get the best deals, I believe a lot more users will check with Live Search to see if there are cashback deals whenever they are shopping for something pricey at least. Whether or not this impacts overall search market share in the near-term is still an open question, but at least it will serve to expose more users to some of the search innovation that is brewing inside Microsoft. I will write about some of these innovations in the coming weeks.
A note if you are following in my footsteps here: The Live Search price comparison page was tricky (that is, buggy!). Unlike many of the more mature shopping engines, the Live Search comparison shopping page did not show whether the products were in stock and did not including shipping costs. And some of the prices listed on this page were not the same as you see when clicking over to the merchant. Data feed consistency is a problem for many of these comparison shopping engines, so you have to go and check each result to make sure it is as offered — if it looks too good to be true, it probably is. I expect that these kinds of issues will be improved soon.

Amazon has spent 12 years and $2 billion perfecting many of the pieces behind its online store. By most accounts, those operations are now among the biggest and most reliable in the world. “All the kinds of things you need to build great Web-scale applications are already in the guts of Amazon,” says Bezos. “The only difference is, we’re now exposing the guts, making [them] available to others.”

This article was the first to announce that Powerset is one of the major early customers for Amazon’s new Electric Compute Cloud (EC2) Web service. Here are the relevant paragraphs, which mention Powerset and some of our key angel investors:

Consider Powerset, the secretive search startup backed by A-list angel investors, including PayPal Inc. (EBAY ) co-founder Peter Thiel and veteran tech analyst Esther Dyson. Co-founder and CEO Barney Pell harbors ambitions of out-Googling Google with technology that he says would let people use more natural language than terse keywords to do their searches. By analyzing the underlying meaning of search queries and documents on the Web, Powerset aims to produce much more relevant results than the current search king’s.
Problem is, Powerset’s technology eats computing power like a child munches Halloween candy. The little 22-person company would have to spend more than $1 million on computer hardware, two-thirds of that just to handle occasional spikes in visitor traffic, plus a bunch of people to staff a massive data center and write software to run it. That’s when Pell heard about Elastic Compute Cloud. He was sold. Based on tests so far, using the Amazon site for part of the company’s computing power could cut its first-year capital costs alone by more than half.
… Highly anticipated search upstart Powerset Inc. plans to use the Amazon computing service, even though it’s still in test mode, to supplement its own computers when it launches its service sometime next year.

I love the Halloween candy analogy — it is both appropriate and timely (I’ve just eaten a bag of candy corn while composing this article).
Jeff Bezos will be talking about Web Services on Wednesday morning at the Web 2.0 Conference in San Francisco, and I expect he will be mentioning Powerset!
My Powerset Co-Founder and COO Steve Newcomb debuts his own blog column by giving his initial perspective on this partnership with Amazon and what it means for startups. Here is Steve’s article on Amazon and Powerset. Steve has a more detailed article about EC2 that will come out later in the week, after Jeff’s talk.

From Bill Gates’ book “Business at the Speed of Thought”, it seeemed that Microsoft truly embraced the concept, but now Google is likely to have taken it to a new level.
I wish that Google would take its internal collective intelligence tool suite (wikis, bug tracking, resistance to powerpoint, and now prediction markets) and make them open source or otherwise available so that other organizations could adopt them. Of course that requires cultural inclinations that are not common outside Google.
In any case, when people talk about Google’s greatest competitive assets, I think their “CI culture” should rank high up there, although I don’t recall seeing it discussed by any of the analysts.

It might seem puzzling why you would invest in a general
search company (perhaps as opposed to
vertical search companies that might have more specific focus and could be
acquired and merged into offerings by the incumbents), in the presence of so many giants. The href="http://www.usatoday.com/tech/news/techinnovations/2005-07-19-snapgoogle_x.htm">article about Snap in USA Today suggests that Snap only wins if they can unseat Google. However, I think Snap
represents an good investment, without any need that they become the new #1 or #2 general search engine. Below are my thoughts about the general search landscape and some major themes that Snap addresses. (Disclaimer: I
am currently an Entrepreur in Residence at Mayfield and have a personal
interest in Snap; the thoughts below are my own and should not be taken to
represent the thoughts of Mayfield or Snap).

Online advertising is today a massive market that continues to grow
rapidly. The U.S. paid search component of this market was $4B in 2004 and
is predicted to grow to $6B by ’06, with a worldwide paid search market
growing to $23B by 2010. The paid search market comprises both search
portals (e.g. Google Adwords), where users go specifically to search, and
contextual advertising, where users are exposed to ads in the context of
viewing publisher’s websites (e.g. Google Adsense ads on NYT.com). The
contextual advertising market is growing even faster than search portals (a
desktop client advertising market is also growing quickly, but represents a
much smaller portion of the market).

While the business (in both these categories) is dominated by GOOG and YHOO,
there is room in the market for new entrants. Users, Advertisers, and
Publishers are willing to try multiple solutions. Users regularly use
multiple search engines – both broad and focused. For searches with
commercial intent (40-60% of all searches), search engine marketing tricks
are making organic result rankings increasingly less effective. In addition,
users still have to do a lot of searching among today’s unstructured results
to make purchasing decisions, which is leading to an emphasis on focused
search with new UIs and added structure for different verticals (e.g. jobs,
shopping). Advertisers are frustrated with lack of transparency and control
and the difficulty of budgeting and managing pay-per-click campaigns.
Publishers want the greatest revenue per page and transparency into their
advertiser and user behavior, and they routinely experiment with different
advertising networks.

While current leaders are innovative and aware of these issues, there are a
variety of “innovator’s dilemmas” they face in responding to new entrants
and resulting market shifts. New payment models, such as a shift to
cost-per-action (CPA), along with increased transparency, will be hard to
address initially without losing revenue. This pattern was observed in
previous shifts from cost-per-impression to cost-per-click, where rates hit
a new floor and gradually built up as the market matured. New user
interfaces and emphasis on market segments with commercial intent pose
similar challenges for incumbents. While a new player can offer a different
interface and attract users that value it, an incumbent must conduct careful
experiments off the home page and only adopt changes that suit their
existing broad user base.

This combination of (1) major industry shift to CPA, (2) opportunity for UI
innovation, (3) focus on searches with commercial intent, (4) ease of
switching, (5) PR and word-of-mouth momentum for standout products, and (6)
innovators dilemmas for incumbents to address these issues has opened up a
significant white space of opportunity for new search entrants to find a
path to rapid growth. Even a small share of this profitable market, in the
presence of fierce competition among incumbents, promises high exit
valuations.

Snap plays to all of these themes with a team who has done it
before, a promising initial product, encouraging user adoption and
testimonials, and an attractive valuation. While it is certainly high risk
to enter a place with such strong competitors, the rewards are equally high.

The discussion above is not even comprehensive of the innovations visible
already today in Snap (with more to come, of course!). I encourage everyone
to try out Snap with at least the following:

• search for “digital camera”. Note you get both query completion, and then
  are taken to a nice vertical search results page.

• search for “tools”. Check out the result for Craftsman Tools, that lets you
  select from pull-down list of the type of tools you want to browse, then
  takes you directly to that section of the site.

• search for “ebay”. Note the results page includes most popular links within
  ebay; a screen shot of the homepage; a company snapshot (including stock
  chart); news headlines; popular searches related to ebay; ebay-specific
  offers like downloads and discounts; and even a summary of important
  auctions on ebay right now. This looks nothing like what you find on
  existing search engines and points to how much room there is for
  innovation in search user experience.

• search for “hybrid cars”. Note the logos in the listings, which helps you
  figure out which ones are real and which are spam. Note that none of them
  are spam…

• search the news (click on the link at the top right corner of the snap
  homepage). Try out the refine-as-fast-as-you-type search (similar to X1 for
  desktop search). You can search for articles with a word, and then search
  within those articles restricting by date/time, headline, and source.

Lastly, go download Snap Ultrasearch (promoted on the homepage), and try it
out. You won’t be sorry! I’ll write more tomorrow about what I like about
this new Ultrasearch product. But needless to say, it definitely shows Snap is thinking outside the search box.

Tag: verticalleap

moderator: Niki Scevak, Jupiter Research

panelists:

• Phil Carpenter, SideStep
• Beatrice Tarka, Mobissimo
• Scott Jampol, Yahoo Travel / FareChase
• Vajid Jafri, Cfares

Here are my raw notes. Further processing to come later.

niki: Recently left to start own venture. Took a deep look into vertical
search. Travel and retail form the backbone.
Travel has unique trend in that 30 years ago the listings were trapped
inside mainframes. The internet webified those systems and brought them
online, but kept intact the agency model of listings being centralized, and
people transacting with the listing agency.

Post 9/11, airlines going bankrupt spawned discount airlines with closer
connection to their customer.

Jafri: Travel is 40% of all ecommerce. Building a search company as believe the
next generation of travel companies

Niki: How much is consumer demand in these 3 factors:
1. fragmenting of listings
2. more people buying travel online
3. market spending more money on search engine marketing

Phil: Travel is incredibly fragmented. Messy markets are good when looking
at search as you can aggregate it all for consumers.
Overwhelming rush of people and demand is good as well. This is the right
sector, fusion of search and travel is a good mix. The peanut butter and
chocolate combinations that taste great together.

Beatrice: Growing demand for onlie information search. You can just go
online and search for tickets. In the past booking through travel agent were
expensive systems. Many of those suppliers chose to establish their own
channel. In US had 3 independent suppliers, in Europe over 50.
Mobissimo caters to those independent suppliers, especially international.
In the future, growing globalization of economics means we need search to
locate inventory and prices.

Scott: Travel search represents the way some people shop for travel. A large
and growing segment want to compare prices and availability, but want more
than price comparison, want value comparison. Continue to see it as a blend
going forward.

Jafri: $52B in transactions, going to $76B next year. The other two
industries that come close are illegal…

The challenge for travel search engines is how to collect all this
information. You wouldn’t know you could buy a $2000 ticket wholesale for
$500. Search is also of great value to suppliers, gives them market info they
have historically never received because legacy systems don’t capture it.
They can use the info to dynamically change the price and increase market
share. So we see this as a great opportunity for market to evolve from front
ends to legacy systems, to a whole new infrastructure.

Niki: A lot of people claiming the lowest price, with guarantees. Is that consumer
confusion a good or back thing for consumer search?

Jafri: Never go to sleep at night thinking you’ve got the best deal. How
comprehensive, how many sources, how deep can you drill, how relevant
content can you bring back.
30% of the market is just in the wholesale market, which consumers can’t get
access to. Is there a search engine that can deliver that content? Otherwise
it’s not providing full value. If the supplier says I’m willing to lower my
price, but only if you give me real-time market info — I’ll lower my price
by $10 to get a sale of $1000. Those are the new business methods that
search engines will bring to the market.

Niki: Issues with crawling expedia’s listings, etc, and where that will
play out?

Scott: Our policy is not to crawl anyone without explicit permission. We
spend a lot of time building relationships. You see online travel agencies
moving slowly into travel search as it’s up to us to provide and demonstrate
value for partners. Companies that make it possible for travelers to find
the best trip for them, not just the lowest price, will add the most values.
We’re in early discussions with a lot of partners. There are concerns on
both sides. Within FareChase we want to main unbiased and accurate
results. We don’t charge for free clicks. We believe travelers need to know
that rankings aren’t based on inventory, but what’s most relevant to search.

Jafri: United did a study and found a consumer would switch carriers for a
$5 price difference. Success of low cost carriers is all driven by
price. The medium carriers need to lower their price. For the consumer it’s
important to get best value for their dollar. In our research we look at
price-driven consumers if it’s apples to apples. People are looking for
experience and relevant, richer content and go to sites with more
comprehensive experience.

Beatrice: Bring some suppliers that were reluctant to work with meta search
engines. American Airlines went so far as to sue a meta search
engine. Mobissimo now has America Airlines in our search engine. The market
evolves, and suppliers come to recognize the value to them. The suppliers,
and travel sites are in a review period. In the past, Expedia, Travelocity,
and Orbitz touted themselves as search engines. Now we’re saying we are the
search engines, you’re just representing some of the inventory.

Phil: We see some holdouts, but many more players are turning to travel and
meta search and embracing it. Intercontinental hotels pulled out of expedia
and are working with travel search. Orbitz and Hotels.com are working with
sidestep, but not other players in travel search. Partners are selective,
looking for (1) partners that are partner centric, and (2) big enough
audience to be interesting. I’m confidence the holdouts will come to join
us.

Niki: Who are your direct competitors, the general search engines or
online travel agencies?

Phil: The easy money is shifting transactions away from traditional online
agencies. Hoteliers don’t view travelocity fondly, it’s an expensive way for
them to sell. When we approach them with a different model, driving
transactions direclty totheir website, bulding direct relationships with
their consumers and building their own brand, not just the online agencies’
brand, that’s attractive to them. Hoteliers are having better luck driving
transactions through google and yahoo, but should play with travel search
engines as well.

Beatrice: Marketing money is still mostly offline. So the first shift is to
move that money online. Travel suppliers can directly track ROI from online
spend.

Niki: What kind of cost difference is there for cost offline, through
search engines, travelocity, etc? How much cheaper, how compelling?

Beatrice: Certain travel keywords on general search engines run from
$2-5. The meta search engines are moving to CPA, so no risk, or when there’s
a referral. CPC levels range from .50 to 1.60. Definitely below the prices
on the general search engines. So there’s a big opportunity there, and for a
more targeted opportunity to get a user who is shopping, not just to
research information.

Scott: For Farechase, we built product with search monetization. We surround
free results with paid/branded results on a more traditional advertising
basis. That’s how Yahoo is positioning different for vertical search. It
does a lot for consumer confidence and trust, hard to come by in the travel
industry. And extremely good value proposition for advertisers. We’ve seen
that with suppliers.

Jafri: Airlines pay $30 per customer acquisition through traditional
channels (sabre, travel agents online or brick and mortar). From search
engines, they’re paying $5 per customer acquisition. So they can cut the
cost by a big amount, enough to restore profitability to the airlines.

Niki: Advertising pricing in search world is typically per click. But in
travel it’s per acquisition / transaction. Is that a sign of an immature
market or a permanent phenomenon?

Beatrice: Depends on suppliers needs. Some are driven by ROI, who will opt
for CPA. Others will be interested by CPC as they also have advertising on
their sites so they can monetize each visit. Others are driven by
brand. It’s difficult to convey a brand message about one hotel on a google
text ad, but an image can do this better. So the media will likely shift.

Phil: We think it’s important to be flexible aboutg models. Who are we, as
emerging companies, to tell them how they must do business. They have
different groups, with different methods for success. So we have to listen
to them and price accordingly.

q: Mike Gibson. You’re doing a lot of crawling. Need for real-time data
sounds important. As airlines might be thinking how to price their seat
inventory, web crawling seems to run contrary to real-time data.

Beatrice: Web crawling is one of the methods to get real-time
availability. We don’t have the luxury of feeds. Airline has limited
inventory per class of bookings, which may disappear each minute. If you
want to find a ticket in the summertime to Europe that can be incredibly
frustrating. Sometimes the partners do cache part of the inventory and you
don’t get the most up to date image of the latest inventory. Don’t see a way
ultimately other than XML feeds crawled in real-time.

Niki: Are those conversations about XML feeds starting to happen?

Jafri: We’re in discussions about getting XML feeds. But we’re dealing with
30 year old store and forward technology. They aren’t where they can provide
real-time XML feeds. Some companies are taking that forward data feed,
converting to XML feeds.

q: Ofer from Raw Sugar. How to discover information within destinations,
rather than just pricing to get there?

Scott: Travel planning is a multi stage multi decision maker process. We
look at it as a funnel: Inspiration, research and comparison, booking, on
trip, and post trip experience. There are products and services to provide
at all those stages. Yahoo Travel is being integrated into the farechase
experience. Like a mapping tool to show you attractions around a particular
hotel. It’s very important for travel search engines to get beyond just
providing info about the product and prices, but helping people to plan and
book travel.

Niki: WIth local search panel, IYP 20% of the quries are related to
travel. So travel is a big factor around local search as well.

q: What incentive do suppliers have to work with one or few of you,
vs. getting to as many channels as possible? E.g. What got American to sign up
with Mobissimo?

Beatrice: We are the most comprehensive travel aggregator. Half of our
queries are coming from international search. We aren’t spending money
buying keywords, so we don’t compete with hotels. We educate users on
benefit of using a search engine. The product has an international
component. And query management system. Travel suppliers are based on very
old structure that doesn’t permit tracking, they have to pay for every query
to those systems. Sending a query that will convert is the issue. As you
aggregate more searches you also need a system to manage the queries.

Jafri: Value to suppliers is critical, they have to buy into it. First
distribution was through online travel sites. They made it clear that that
model is broken, they’re spending too much money on this. They have 3
problems: They spend $10B on distributing their products, they have $30B in
unsold inventory, and these travel sites are owning the customers.
Any travel search engine that addresses these issues will get the support of
the suppliers.

Phil: You not only provide qualified queries, but also volume. If any of you
manage search engine marketing campaigns yourselves, you think about range
of search engines on which you advertise. Are you going to advertise on
Canoodle or to more and deeper with an engine that can provide volume? It’s
the combination of quality and quantity of leads.

q: Growing distribution through affiliate programs?

Jafri: That’s important. There is no real leader in travel search yet. To
get traction in marketplace we must team up with sites that can generate a
lot of traffic.

Prediction markets provide more reliable forecasts of election results than polls. They can also be used to get improved forecasts on sales levels, project completion dates, or to get more detailed data on consumer preferences than focus groups provide.
We’ll discuss how they are being applied in business contexts to incorporate divergent views into business plans.We’ll introduce the idea and its history in the first session, then proceed to current examples, tools, and discuss the future of this new approach.

For a useful overview of prediction markets, see the See Time magazine article: “The end of management”.
I have been interested in this idea ever Robin Hanson (the founder of the field of prediction markets) and I were summer students in the AI Lab at NASA Ames Research Center, both working under Peter Cheeseman. This was right around the time when Robin wrote his first papers on the topic: “Idea Futures: An idea whose time has come?”, and “Could gambling save science?”. In order to match the work its NASA funding, Robin created a demonstration of how a set of Mars Rovers could place bets on the presence of scientific interest at different locations on Mars. The result would be the emergence of consensus beliefs that could be more accurate than the knowledge of any individual.
I also remember the time Robin hosted an Murder Mystery evening with a prediction market as an added twist. Like a normal such party, actors would read out each scene. But in Robin’s version, the audience members would then place bets on the identity of the murderer by buying options. Based on the market dynamics, a ticket marked “This ticket is worth $1 if Professor Plum was the murderer”) might start out having equal value as the other suspect tickets (so that you could be the whole set of them for $1), but then the price would fluctuate as unvents unfolded until the prices ultimately went to $0 for all but the actual murderer. It was really fun, an improvement on the original version.
My raw notes from the session are below.

Robin Hanson (gmu) and David Pennock (yahoo): background on prediction markets

- problems
manipulation
sabotage
…
combinatorial
Robin’s slides should be available online soon.

Bernardo Huberman (HP): predicting the future

o aggregating info
collective intelligences good at coop problem solving
how to agg ind skills to generate useful info?
o how do orgs predict?
ask experts and consultants
have meetings
designate someone as forecaster
take a vote (not very good)
o alternative: markets
markets aggregate and reveal info (hayek, lucas, etc)
to predict outcomes, use markets where the asset is info (rather than a physical good)
e.g. iowa electronic markets
o experiment: use markets in hp to predict printer sales etc
results weren’t very good.
- markets within orgs: problematic because:
low participation
illiquidty
info traps
hard to motivate
easily manipulated
o a new mechanism:
identifies participants that have good predictive talents, and extracts their risk attitudes
induces them to be truthful
avoiding pitfalls of small groups
aggregates info in a nonlinear fashion
(two published papers on this topic)
what is it based on?
- people are not all the same
think of info in people’s heads as the assets and use portfolio theory
use a market mechanism to determine an individual’s risk attitudes and
performance
- then, ask people to forecast and performan a nonlinear aggregation of
their results taking into account their risk characteristics
info gathering process is simple, decentralized in time, and inexpensive to
implement
stages:
1. market for contingent securities
provides behavioral info, such as risk attitudes – synchronous
2. participants generate predictions no outcomes, which are then aggregated.
asynchronous
stage 1: lottery-like securities
marbles from an urn.
can buy and sell colors. things quickly converge to a price.
can extract risk attitudes: ratio of cash vs marbles people accumulated over
the game. risk averse people keep the cash, risk seeking over-invest in
certain colors.
stage 2: forecasting
participants given 100 tickets to be allocated among 10 securities
this determines probabilities
true state pays according to the number of tickets allocated to it
aggregating predictions: change exponent based on ind risk attitudes
result: just 9 players aggregated give a complete distribution
the info aggregation mechansism: group did better than the best individual.
IA also did better than the market.
possible contamination with external market info.
can you find an interesting and hard problem, where people are willing to
get an answer and devote resources to test it?
o predicting revenues and profits inside an organization
HP services: predict revenue month to month on first week of the month
did pilot for 5 months.
15 managers worldwide, part of hp services finance office.
have a tool that HP sends them where they can see the predicted revenue.
every first week they made a prediction, we got a number and compared with
official prediction.
actual value was straight on the mode of the prediction.
people placing forecast anonymously.
group prediction wasn’t always below official projection, sometimes was above.
managers (esp good ones) said they’d be happy to have their names published.
now doing an internal one, to predict on volate commodities HP uses to make
computers.
o other examples:
- shell oil executive suggested running with top customers to predict demand
from refineries. a big problem is to decide production.
- prediction of success of products
must be very precise about measurable outcome of these projects.
1. we don’t know how far into future a mechanism like this can predict.
10 years from now might not be interesting.
2. we don’t know how it scales as the # of participants increase.
www.hpl.hp.com/research/idl
q: what markets failed and why?
lack of motivation
for our scenario, we don’t pay that much. once you put a $ on the table, the
behavior is very different.
q: face to face interaction among participants?
only in the initial game
q: training procedure?
yes we have a protocol, and a test, until they pass.
q: rohit: agoric computation (markets to allocate computing resources). ecology of
computation book: what are predictions for prediction markets?
a: we are on threshold of utility computing (eg. Sun selling cpu for
$1/hour). transition will have to take place. like shit from internal
generators to electric utilities.
the difference is we’re dealing with information that is valuable to
companies, so people get nervous.
but eventually believe it will have to happen.
q: learning over time, ability of highly informed traders to compensate for
uninformed traders. how robust to uninformed participants?
a: we are using methods to id people who have expertise (looking at emails exchanges,
etc).
the learning piece we measure: how is risk attitude changing over time.
haven’t looked at other dynamical variables.
q: jp: used a 100 person forecasting center 15 years ago for allocation of
resources in systems integration. You shouldn’t be able to see what someone
else is doing, as that influences you. This avoiding corruption and bias is
a different anonymity issue.
a: in our system you can’t see what others are doing.
q: is HP actively using these predictions to make business decisions?
a: we are using it internally and with some customers. but there are issues.
in process or trying to spin out something.
question on how much you’d make if you did this.
might use this just as a product differentiator.

Emile-Servan Schrieber, CEO and president of Newsfutures:
The business value of prediction markets

o quotes
1. thomas malone: There is an increase in human freedom in business that may, in the long
run, be as important a change for business as change to democracy was for
governments.
2. key value we see is prediction markets have the potential to extract the
best essence from group knowledge as an alterantive to majority decision.
(ie beyond democracy)
business application: forecasting and decision support through critical
knowledge aggregation
(unclear which apps will have the most value)
o markets as brains
intelligence emerges from interaction of many relatively dumb parts.
learning by reinforcement and prunjing improves performance
real money market: you don’t control money flow. not as much correlation
between who is knowledgeable and who is rich.
play money market: everyone starts the same, accumulate money by being successful
o About Newsfutures company
global 1000 clients
over 40,000 markets operated since 2000
online properties:
world news exchange
innovation futures (tech review)
tech buzz game (yahoo)
patent applications on “multi-outcome” trading engine
market for each possible bin, each is alternative to every other
o Newsfutures Business model
tech licensing
public markets
advertising revenue
creates branded prediction trader panels (market research)
corporate solutions: use these branded trader panels
(not necessary to install systems inside the company)
prediction trader crowds
internal / external
public / restricted
internal restricted: frontline employees
external restricted: consumer panel, invited experts, supply chain
external public: innovation futures, tech buzzz, HSX, pharma traders
o HP experiment
o eLilly: beating official forecast
a dozen sales people, forecst monthly drug sales
usually get 10-15% participation
market accuracy advtange over official forecast: sometimes >75%
o beyond HP
18 companies (maybe 25 by end of this year) have implemented a prediction market (ramping fast)
(wisdom of crowds book got people very interested)
horizontal, across varied industries
o coprorate applications:
sales forecasting
demand forecasting
need to know 1.5 years in advance whether to create a bigger production
capacity. bestbuy is gauging early adopters. but expect retailers will
have to drop the price, so then what happens to the demand.
price forecasting
company mfg paper needs to forecast, as not traded in the market.
similar problem for chemicals like benzene, no organized market
companies are publishing the prices of deals in real-time. now we’re using
their customers to create futures markets within buyer community.
project prioritization
product design, feature selection
campaign evaluation
candy company wants to figure out potential impact on sales of different
types of ads on tv. use marketing people to evaluate the campaigns.
regulatory monitoring
eli lilly wondered in 2003 what would happen in congress re: drug reimportation and benefit in medicare.
(Siemens recently sold its cellphone units to taiwanese company because their sales prediction problem was so bad…).
o some crowds:
sales people
marketing people
consumers (eg benzene) / readers (eg tech review)
supply-chain partners
external experts
general population (self-selection is very useful)
doesn’t matter if trading population is small %
o beyond accuracy:
not just: accuracy, liquidity, money, incentives, sw, trading
mostly about:
business value
focusing attn on issues
accountability
haveing a voice / recognition
user-friendly design
dialogue
o challenges to internal markets
threat to traditional management (see Time magazine article listed above)
some questions are better left unanswered (unasked!)
e.g. when will this plant get built?
engaging busy professionals
getting the answer means publicizing the answer
must have, or nice to have?
conclusion: not one size fits all…
o state of the art
rounding the bend: many companies engaged, awaiting results
some early wins
some early losses (people just using sw didn’t get very far)
a lot of learning
look for case studies to come out in 6 to 12 months
o example: sales forecasting (siemens)
agg opinions of 50 hq sales personnel in brazil to predict phone sales in
brazil. (25 trading)
very nice results so far
o public prediction trader panels
thinks that’s where most of the business value will be
“will a new drug benefit be introduced under Meidcare?”
head lobbyist within eli lilly was telling bosses it would never happen but market was saying at least 40% chance it would happen, even at the worst of times
“how many new molecular entities will be approved by the FDA in 2003?”
people figured it out very accurately
“how much will access to physicians increase or decrease 2002 vs 2003)? spot on: 7.4%
innovation futures (launched fall 04, 10K traders)
sponsored market (by cisco)
“how many voip subscribers will the US have at the end of 2005?”
Google IPO market: both prediction sites were closer to actual result than google official estimates. real and play money did the same.
yahoo tech buzz
o take home:
if you own a comunity whose “brain” you’d like to leverage via a prediction market, Newsfutures can partner with you with operational experience, technology, and business models
q: how do you charge?
$25K up to 120K/year
q: assessing political risk in emerging markets?
(short Turkish Lira based on outcome of next parliament election)
no.
q: how to overcome cultural resistance for betting on the future?
a: we called a VP forecasting of a large car company. Told her people are investing in future outcomes, works well for elections.
she said: “It’s like gambling”. we said: “Yes, HP is using it to improve sales forceasts”. She got it immediately. Businesses don’t mind gambling as long as it produces results.

Walter Yuan, social science experimental lab at Caltech: jMarkets

o functional featues
multiple markets
continuous
anonymous
open-book market
order-driven markets with price priority and time precedence
role-assigned traders
constarint-regularted trading (short-sale and credit limit)
o non functional features:
fast (<30 traders, 20-4 markets)
trader friendly gui
easy to configure a marketplace sessino
easy to deploy
open source
trader friendly GUI demonstration
fast transaction and cooling
o advantages of experimental market software
flexible config, simple, abstract, strong domain expertise, open source
o disadvantages
security, scalability beyond 100+ participants, lack of business domain specific customization, clearing and settlement
o applications of jmarkets
1. president's day weekend ski equipment sales
rebalance inventory, stock up new equipment, adjust cash flow
- markets vs ebay:
two-sided
multiple units held, offered or put up for sale
market making opportunities
very hard to make a market in ebay, as you'd have to buy the products
and then try to sell them
efficiency widely established
price discovery
2. inter-outlet markets in sales personnel during busy holiday shopping
periods
sales personnel act as sellers of their service time; stores act as buyers
for competing limited sales service time
different approach: stores act as buyers and sellers of their salesperson time
similar problem: airline crew allocation
3. trading just-issued treasury bills among a group of insurance companies
ad-hoc, centralized market of forward contracts with designated participants
market is short-lived
opportunities for market makers
price discovery provides more info on later auction
dealers can be both buyers and sellers - market making; primary dealers
natural sellers; secondary dealers natural buyers, but not nec.
compare to present non-transparent, phone-based system
q: what institutional rules in these markets? keeping track of earnings?
right now only credit limit and short sell
have a module can view as a clustering engine.
q: disadvantages (security, settlement clearing, customization) seem
relatively trivial to solve in open source community. but scaling >100+
participants – some fundmental constraint that doesn’t make it scale?
a: just haven’t tested
q: really about markets, not predictions. (yes). did you try predicting
salespeople demand? (no)

The Internet is shaking up the once-staid and lucrative business of classified advertising. Newspaper publishers that once enjoyed a virtual monopoly on the classified market now face increasing competition from Web sites like eBay.

This comes just days after Ebay agreed to buy Shopping.com. I think the eBay/Shopping.com deal seems to be more strategic, because eBay was already in the business of helping users comparison shop for products. But I do agree that the decline of the newspaper classified advertising business is driving major activity by all the papers. Following on these lines, I expect there will be a string of acquisitions of vertical search engines, particularly those related to classifieds, within the next year. The next one I would predict: Oodle, a classified search engine that aggregates across many sources of classified ads and provides a nice faceted search interface.

Analysts in several articles emphasize the acquisition as a way to attract more merchants and a community platform. For example, Motley Fool writes:

The deal certainly expands the business of eBay. It gets new customers, a sophisticated community platform, and a large merchant base. What’s more, eBay now has two more strong brands — Shopping.com and Epinions.com.

Similarly, from an article in Forbes:

POTENTIAL MATCH. But the purchase has the potential to plug a longstanding hole for eBay. Its own marketplace appeals mostly to small merchants, who sell a wide variety of second-hand and overstock goods at auction and for fixed prices, and to customers seeking unique items and deep discounts. eBay has been trying for years to appeal to merchants that want to sell new and in-season merchandise as well — with middling success.
The result of this acquisition could be a new venue for merchants that want to sell products to eBay’s more than 60 million active buyers. “I can see things that [the purchase] brings to eBay — mainly targeted buyers,” says Derek L. Brown, an analyst with Pacific Growth Equities, which has an underperform rating on eBay. “If eBay can match that targeted demand to its listings, that could drive sales.”

This all makes sense. However, if it were just a matter of acquiring merchants, then Ebay could have bought one of the private companies (e.g. nexttag, bizrate/shopzilla) who also aggregate many merchants, and could have done this at at a much lower price. I think additional value comes from Shopping.com’s faceted user interface and deep product normalization capability. These can greatly improve search on eBay’s site and multiply eBay’s revenue through improved browsing and discovery.
This transaction also provides further validation for the potential valuations for vertical search Ecommerce plays. It will be fun to discuss this at the upcoming Vertical Leap Vertical Search Workshop on June 28th.

• The early history of Yahoo!
• The alternating and intertwined importance of algorithmic search and editorial directories in the past and the future of
  Yahoo!

• Yahoo!’s history and current developments in media and entertainment,
  including the Yahoo’s subscription music service

• Communications and Mobile services
• The feature wars with Google and other competitors, including the
  increasing focus on personalization and community

For me, the most interesting part of the whole discussion concerned the
acknowledgement that search is king today, but that tags and personalization
are ushering in a new importance for editorial browsing:

David: People start with words, but in the results the directory is
there. People have become very comfortable with typing in words.

Jerry: Philosophically, as people want to navigate they may choose to search,
choose to browse. As we see more heightened awareness around tags, it brings
us closer to a filtered or browse scenario. I think we’re just starting to
see the back and forth scenarios. Yahoo has that capability. We can do it
in a very integrated way. Eventually those two finding metaphors will
coexist.

The rest of this blog entry contains my notes.

Note: While I’m pretty good at capturing sessions in real-time, so this almost looks like a transcript, I don’t claim that I was accurate in the notes — I did not capture everything, and abbreviated or interpreted as I absorbed the content.

• All Things Digital Conference
• David Filo and Jerry Yang, Co-Founders of Yahoo!
• Interviewed by Kara Swisher and Walt Mossberg (Wall Street Journal)
• May 24, 2005