March 23, 2009
Wolfram Alpha: A New Kind of Question-Answering System
There has been much excitement recently over the upcoming launch of Wolfram Alpha. This is a new question-answering system developed by Stephen Wolfram, inventor of Mathematica, and it is scheduled for a beta launch in May. Wolfram has been providing demos to industry insiders. I haven’t had a demo yet, but I have learned what I could from reading articles by Nova Spivak (“Wolfram Alpha computes answers to factual questions. This is going to be big”) and Doug Lenat (“I was positively impressed with Wolfram Alpha”). And this weekend I spoke with William Tunstall-Pedoe, CEO of True Knowledge, who also got a demo. Many of my examples and conclusions come from conversation with William (thanks!). Since life is short and so is the attention of web readers, I’ll give the rest of my thoughts in bullet form.
What it is: A new kind of question-answering system.
- Math: “2+2″ and then a few simple math questions: “integrate xsin^4xdx”, “what is the square root of 18″ etc.
- Business: “gdp france” showed amount and graph of how it changed over time. “gdp france/germany” showed graph with both amounts and the ratio
- “internet users in Europe”: Showed total, and a chart of usage by country in Europe, at the current time, specifically highlighting the biggest and smallest
- “ISS”: generates a graphic rendition of the international space station orbiting earth and updating in real-time
- “tides in san Francisco”: showed a graph of tides over time, where the times were listed in the local time regime current in the late 19th century for those data points. “tide NYC 11/12/1922” gave a single answer.
- “weather”: showed graph of average temperature in Cambridge, MA (where Stephen was when doing the demo). Based on reverse IP lookup.
- Computational fluid dynamics: typing in the name of a specific aerofoil produced a picture of that aerofoil along with its differential equations.
- stock prices: “MSFT CSCO” showed comparison chart
- chemicals: Substances at temperature or pressure, got physical properties calculated. “H2SO4” showed a diagram and chemical properties. “5 molar h2s04″ did something cool, I don’t know what.
- genome sequences: “AGTAG” shows sequences from the human genome that match that pattern
- data about people: “How old is Barack Obama” gives his age now. “When was Alan Turing born” gives the answer. “How old is Alan Turing” (a trick question) gives an error message with no human-readable explanation (True Knowledge, by contrast, tells you exactly why this is a trick question).
Coverage of data: It answers questions over the following types of structured data:
- static tables and databases (e.g. a database of internet usage by country by year)
- dynamic data feeds (e.g. historical stock market data, position of space shuttle, weather)
- numerical inference (e.g. math questions)
- numerical computations and simulations (e.g. tides, astronomy, chemistry)
December 3, 2008
How Live Search Cashback got me the best online deals!
Now that I am part of Microsoft Live Search following the acquisition of Powerset, I am making a point of trying out all the latest offerings from Live Search (including those under development) and many other Microsoft products in general.
While I can’t talk about the products under development, it is fun when I can talk about things that everyone can use today to do something better than what you can do elsewhere.
With that in mind, I want to talk about Live Search Cashback. I have been interested in the concepts behind Cashback for a long time, since I was an early advocate for Cost Per Acquisition (CPA) as a future disruptive trend in ecommerce.
I wrote about this in connection with SNAP, which I helped Mayfield invest in back when I was an entrepreneur in residence there.
In CPA, the merchants pay based on completed actions by customers (e.g. purchases), rather than just paying to display ads or for clicks by users who may not actually purchase anything.
CPA has potential to ad value for all parts of the ecommerce ecosystem:
- It is great for merchants because they only pay when they get real value, so they don’t have to worry about losing money on advertising.
- It’s great for search engines (once there is enough scale) because they know just how much money they can make from advertisers if they can send the right traffic there, without worrying about limited advertising budgets, and they also get data about transactions so they can improve their service.
- It’s great for searchers, because search engines can rank ads (or even organic search results) based on the transactional information from other users. You would rather click on an ad or a search result that has led to happy purchases by other shoppers than click on one that attracted user interest but no sale. This is a good way to filter our spam of many kinds.
Microsoft’s Live Search Cashback has all these potential benefits, but with an added twist: Live Search actually gives some of the advertising revenue back to the users who make the purchases. Here is how it works:
- Search for a product on Live Search. There are now two different ways to buy using cashback:
- You see a “gleam” on some of the ads that alerts you about the cashback opportunity. The ads list the cashback percentage.
- You see a link to a Live Search product detail and “compare prices” page. On that page, you see the typical list of prices for your product across a wide variety of merchants. However while typical engines sort by total price including shipping and taxes, Live Search Cashback also shows you the total prices after the cashback discount.
- When you click over to one of those merchants from either the search result ad or the cashback price comparison page, Microsoft asks for your email so it can remember you when it needs to give you cashback if you make a purchase.
- Then you shop on the merchant page (any of the merchants you clicked on).
- Any product you buy from a selected cashback merchant qualifies you for the stated cashback percentage. You get an email from the Microsoft cashback program telling you how much cash you will get back. You have to create a Windows Live account once in order to set up the program, but that’s easy and also gets you access to all the other Microsoft online services (including search personalization, hotmail, msn messenger, Healthvault, etc).
- Then after a waiting period (60 days at present in most cases, which is long enough to prove you aren’t going to return the product and cancel your purchase), the cash shows up in your paypal account or other account you have specified.
That sounds great in principle, but like everyone, I wondered whether this actually works. In particular, I wondered: Could I use Live Search Cashback to get the absolute lowest price for products I am shopping for?
Here are some reasons why, in theory, the cashback system could fail to deliver the best deals:
- It is possible that the lowest-price merchants haven’t signed up for the cashback program yet
- Online sellers offering cashback might raise their prices knowing that buyers would be getting cashback. In that case, sellers would benefit from the program, but buyers would not and so it woudl be better to shop elsewhere for lower prices.
I have heard a lot of excitement about cashback inside Microsoft, and rumors of people getting some good deals, but so far I had not seen any proof that Live Search Cashback could get the best deal if you really tried all possibilities. So that’s what I set out to do.
First, a little about my shopping style. For high value items, I am really a price-conscious shopper. I first do research to determine the product that I want to buy. I use a lot of tools for this, but that will be the subject of another post. For now, it’s all about price. So once I have chosen the product I really want, I shop using all the tools and techniques available to get the lowest possible price. It’s perhaps a bit silly in my case since I do value my time more than the actual dollar savings I achieve, but I also value the feeling that I used smarts and tools to get the “best bargain”.
So with that in mind, here is my experience buying some pricey consumer electronics products on Cyber Monday this week. You can follow along at home and if you do it now you should get the same results.
I wanted to buy two products:
The links above point to the Live Search product and results pages for these products, so you can try this at home.
For the plasma TV, it’s an expensive item (list price is $2,000 plus shipping, as I write this), so I really wanted to shop around and get a great price. I searched for this product on Amazon, Shopping.com, PriceGrabber, NextTag, CNET, Google, Yahoo, Ebay, and Live Search. And I clicked on many of the ads that came up on these sites offering special deals, coupons, other comparison shopping engines, etc. Here is what I found using each of these (I list total price including shipping and taxes):
- Shopping.com: They have a nice feature that lists the lowest price from a trusted merchant. In this case, it is Adorama Camera, for $1,269.00. (There was a lower price quoted from another less trusted merchant, but the price on the site when you click was actually higher than the price listed on Shopping.com. Annoying bait and switch tactic, perhaps?).
- Pricegrabber and NextTag: Both listed Butterfly Photo at $1263, but it is not a highly trusted seller, and then Adorama Camera for $1269.
- Amazon: In the Amazon Marketplace, the lowest price merchant is again Adorama Camera, but through Amazon the price is $1296.00. It looks like you pay $27 to buy through Amazon instead of going direct.
- CNET: There was one merchant, 6TH Ave, that listed at $1244 but they had a low trust rating, so I discounted them. The lowest price trusted merchant on CNET, B&H Photo, offered it at $1279.
- Google: The lowest price merchant was Universal LCD, for $1265. (Note Google listed B&H Photo with a 1,235.50 total price, but that was for a demo item with a scratch. The ones without scratch at B&H Photo are $1279, as listed on CNET.)
- Yahoo: The lowest price merchant was Universal LCD, for $1265, same as on Google.
- Ebay: The lowest price seller offering buy-it-now was for $1469. (It’s possible I could have got it for less waiting a few days for the auction, but so far I’ve hardly ever seen a hot item go on auction for less than the cheapest prices elsewhere, and I’m really only comparing things I can order right now that are in-stock.)
Having tried all the other services, I then tried Live Search Cashback. I tried both the “cashback ad” path and the “cashback price comparison” path (as described earlier).
- Ebay via Live Search Cashback: The best ad was the Ebay ad, offering cashback of 25%. All you do is click on that Ebay ad, which takes you to Ebay. Then anything you buy on ebay using Paypal and buy-it-now gives you that 25% (be sure you go to eBay after clicking on the ad, not directly, in order to get the cashback). I found that same trusted seller offering it for $1469, but once I walked through the buying path it showed me that I would get $200 cashback (the cashback maximum is $200). So that nets out to $1269, which matches the lowest price I could find from any other service.
- Best merchant from the Live Search comparison page: Adorama. They offered 3% cashback of their base price of $1269 (the same price I found using shopping.com) for a total net price of $1230.93. This is the one I wound up buying.
So Live Search Cashback found me the best deal on the internet for my 50″ Panasonic plasma TV! It was interesting that despite the 25% cashback deal with Ebay, it was still better to use another cashback merchant. I wondered if that was true for all products, or if sometimes it might be a better value to buy on Ebay with those big cashback deals.
This turned out to be true for the Panasonic Blue-ray DVD player. I bought it on eBay for $259 and got $52 cashback, for a net price of $207.
By comparison: shopping.com, nexttag, pricegrabber, yahoo all had B&H Photo at $239.95 (with occasionally a lower-trusted merchant coming in at $224). Amazon Marketplace had it at $229.72 from Electronics Express, thus beating all the other deals except Live Search Cashback on this product.
So the bottom line:
I shopped for two high-quality, high value consumer electronics products that I wanted to buy. I tried all online services I could think of, and for each of these products I got the best deal on the internet using Live Search Cashback!
This is exciting for me personally and I am happy to share the exciting news with my friends and family, especially in the current economy where everyone is watching expenses.
On a broader note, I must admit I was skeptical about whether cashback would really work and whether it would draw in new users to Live Search. Now that I have done my own research and found that it really can get the best deals, I believe a lot more users will check with Live Search to see if there are cashback deals whenever they are shopping for something pricey at least. Whether or not this impacts overall search market share in the near-term is still an open question, but at least it will serve to expose more users to some of the search innovation that is brewing inside Microsoft. I will write about some of these innovations in the coming weeks.
A note if you are following in my footsteps here: The Live Search price comparison page was tricky (that is, buggy!). Unlike many of the more mature shopping engines, the Live Search comparison shopping page did not show whether the products were in stock and did not including shipping costs. And some of the prices listed on this page were not the same as you see when clicking over to the merchant. Data feed consistency is a problem for many of these comparison shopping engines, so you have to go and check each result to make sure it is as offered — if it looks too good to be true, it probably is. I expect that these kinds of issues will be improved soon.
September 11, 2008
Marissa Mayer on the Future of Search
The Official Google Blog is running a series in which search experts pontificate on the future of search. In the first installment, Google’s VP of Products, Marissa Mayer, writes about the future of search.
I really liked the article. Here are a few thoughts I had while reading the article.
This past Saturday, I kept track of the things that came up in conversation that I wanted to search for right then but couldn’t:
Are “fab,” “goy” and “eely” words? (There was a Scrabble game going on.) What time does J.C. Penney open on Saturday? Which school has a team called the Banana Slugs? What is the team mascot for San Jose State? How much power does that hydroelectric dam generate? What do you call a group of turkeys? What time does Tropic Thunder show? What’s the name of that great Irish flute player, first name James? What’s the name of the largest city in Russia after Moscow and St. Petersburg? Which is older, a redwood or a cypress? What’s the oldest living thing and how old is it? Who sings “Queen of Hearts”? What kind of bird is that flying over there? Is the “LF” in San Francisco on Union Square or Union Street? What are the dance steps to the Charleston? What day of the week was The Lawrence Welk Show on? What are the lyrics to “In the Mood”? How does Coumadin differ from aspirin in its blood thinning effects? What was the story behind the naming of the number “googol”?
Looking at this list, two things are very clear: (1) I could do a lot more searches and (2) search still has a lot of opportunity for innovation, change, and progress. There are lots of ways that search will need to evolve in order to easily meet user needs. Let’s look at some of my unanswered questions from Saturday and consider how search might change over the next 10 years.
Thinking about the questions one might have asked, but didn’t, is a nice way to recognize some gaps in search. Mayer concludes that she could answer all of her queries with search today, using the right keyword query, but that there must have been easier ways of getting there. It would be really interesting to see how much work it would take ordinary searchers to get these same answers using keyword search, or even how many tries it took Marissa to get the intended results.
The points Marissa makes as follow-ons (including the value of natural language, voice, context, disambiguation, multimedia, and mobility) are all big and important problems.
I also liked her summary of the ideal search engine:
Your best friend with instant access to all the world’s facts and a photographic memory of everything you’ve seen and know. That search engine could tailor answers to you based on your preferences, your existing knowledge and the best available information; it could ask for clarification and present the answers in whatever setting or media worked best.
One interesting aspect of this definition is that it envisions that search engines will still exist as a category in the ideal future. I think there will always be value in having an automated, intelligent conversational partner, and I am a strong proponent of such a future vision. But I also think that increased search intelligence will find its way into the flow of our daily lives and tasks. While there is value in answering factual questions in just the right way, there might be at least as much value in helping us with the task-oriented context in which the questions arise (why do we want movie times, why are we asking about pain killer ingredients) and in helping us to read the content once it is finally returned.
Anyway, I look forward to the rest of the series on the future of search. It’s a good time for this dialogue as I start out in my new role as search strategist and evangelist at Microsoft.