Comments on: Powerset and Natural Language Search

By: stufflix

stufflix — Tue, 03 Apr 2007 14:18:54 +0000

I remember as a student at Cambridge Univ ORL the video retrieval project that was searching for passages in video content that provided specific visual content, the procurement of which was obtained by seaching audio sentences information mostly I think. I think for English the task would be far easier than for other inflexion based languages though so I am concerned that the expansion of project would be far more complex than the” add another same type of google box approach to search country specific pages”.
Well…I think a novel UI approach would also help greatly to assist in the construction of the sentences to be used as search artifacts and I think might be quite key to the project.

By: stufflix

stufflix — Tue, 03 Apr 2007 13:37:26 +0000

By: Barney Pell

Barney Pell — Fri, 10 Nov 2006 15:45:30 +0000

Patrick: You are right that Google is now using specific passage retrieval algorithms (ala Bill Woods) that take word proximity, variation, and order into account. It certainly makes the search better. And that said, you are right that there is much more to do even for Google.
Natural language search, for me, means the ability to use natural language to do anything you would want to do with a search engine. This includes informational, navigational, and transactional queries. It also includes statements of intent and context.
Moving beyond just document search, including ideas you suggest, is highly relevant.

By: Patrick Herron

Patrick Herron — Fri, 10 Nov 2006 00:40:47 +0000

Without futzing around we might safely (though possibly wrongly) assume the keyword-ese search engine not mentioned by name here is Google. Google might be the right commercial standard, so for the purposes of conversation I’ll take the risk of deluding myself and assume you really mean Google.
Anyway, this notion of keywordese where Google strips stop words (function words really) is somewhat strawlike, for a lack of a better word (or a lack of a word altogether). A simple test of Google suffices. If you compare search results for “a who” and “the who” you will notice the results are different. Same too when I tried “a mule” and “the mule.” If the functional word didn’t matter, then they would be stripped and mule would be the query inside the black box. But that’s not the case, so Google does not really fit the description. But what other search engines truly matter today, assuming a mass-market audience?”
Google’s search appears to expand the query by what I might call both functional ambiguity and boolean ambiguity as well as proximity-based weightings. Further, the terms in the title tags are more important than terms in other places. in this sense, Google already appears to consider functional words, collocation/proximity, and even document context as well. So by your definition of natural language search it seems that Google is already there.
But it isn’t, obviously.
The value of Google’s search is based almost entirely on the search/searcher context. If the searcher has a weakly formed preconception of a satisfactory search result (namely because the person is not a domain expert) and the search string is dumb (1-2 words) chances are the search results will be satisfactory–given PageRank over a large enough network. But if you begin pumping up user domain expertise then the language of search query will necessarily become richer while the user’s standards for satisfaction will rise. Finally, if the user is a domain expert s/he will likely already know the right subnetworks to stick to. Google frankly begins to suffer in such an environment as I witnessed first-hand in a study I designed for a large statistical software company. People who rely on their support search either like Google users and flail or they search with less keywordese-like language (longer query strings, more functional words) and experience greater success. Several search engines including Google were tested.
I remain befuddled by the phrase “natural language search.” If I had to imagine what is meant by natural language I would imagine certain things to be present that are typically absent: (1) user context modeling; (2) semantic network expansion (plug in WordNet or something like it and expand your input).
But efforts at disambiguation will diminsih naturalness for reasons too extensive to discuss here. Even POS tagging strips out POS ambiguities which are rich in the english language. So grappling with ambiguity using an estimate of likely semantic preferences for specific users might get you to naturalness. But then you might also need recognizing, making, and refining mistakes. Or an ability to recognize the differences in meaning of a term over time.
What I wonder even more about is, why stick with search? Search seems to be a marriage of keywordese-in to document-out. information extraction at least breaks the document barrier but it still ends up tied to the document. if you’ge going to have enough computing power under the hood to have natural language search queries (what I imagine that means) and rewrite/expand queries you might as well go for intelligently processing results into a coherent natural language whole. How about inductive reasoning (e.g., Muggleton’s work) or multidocument summarization approaches? Something synthetic, something that generates new information from the din of the old, something far more valuable and compact. Of course that’s easier for me to say than to do however.
I’m eager to see this search engine. Good luck.

By: lohit

lohit — Fri, 06 Oct 2006 13:35:02 +0000

Good luck with the project.
It would be really interesting to see to what extent NLP are used for search. Having a larger dataset like the web makes its much more challenging. Will there be any beta version fore registered users to try the service? Would be be interested in seeing the results.

By: direwolff

direwolff — Fri, 06 Oct 2006 01:20:44 +0000

Also note that Keywordese is just fine when you know exactly what you’re looking for. If I type “Christophe’s Restaurant Sausalito, CA” into Google, the top result gets me what I want, their contact info. Search improvements are required when the task is a discovery one, where I’m not exactly sure what I’m looking for so I need something that will understand what I mean. In this regard, Don Dodge’s recommendation above, to focus on specialized vertical markets is great advice worth considering.
Good luck to you on this Barney, it’s a heck of a play.

By: azeem

azeem — Thu, 05 Oct 2006 13:33:33 +0000

Barney
Totally with you on this. Really clear exposition of today’s search.
Some points:
1. Average keyword length is drifing up. When I ran marketing at Albert, which did some query disambiguation while heading (but not reaching) a decent NLP, average internet searches were 1.4 words trending towards 2. (This is data looking at 1996 to 1999). That is, I guess, because users are getting more sophisticated, data sets are broader so demanding (and rewarding) longer searches.
According to Yahoo!, average query length has now reached 3.3 words. (See http://blogs.zdnet.com/micro-markets/index.php?p=27 )
So contrary to what Danny O’Sullivan has said, user behaviour can and does change.
2. I think we have fallen into the category of learned helplessness about search. We basically accept what we’re given and take what we can get. The lack of innovation in search–and let’s face it, there hasn’t been any since PageRank–has taught us not to rebel.
aa

By: Don Dodge

Don Dodge — Thu, 05 Oct 2006 08:50:48 +0000

Barney, Great post, excellent analysis of the search problem. We may have met years ago when I was director of engineering at AltaVista. We had a series of meetings with the SRI guys about NLP.
My advice is to focus on enterprise search or specialized vertical search markets. Your unique technology will be more appreciated and valued by those customers. If you are really stuck on doing consumer search than I would suggest you try to specialize in News Search, People Search, Medical Search, or some other vertical where NLP power can be an advantage.
Good luck!
Don Dodge
The Next Big Thing
http://dondodge.typepad.com/the_next_big_thing/2006/10/powerset_natura.html

By: Steve Bryant

Steve Bryant — Thu, 05 Oct 2006 08:21:45 +0000

Hey Barney, I totally dig what you’re saying. I have much respect for you for giving the search engine game a go. But please don’t mischaracterize what I said, which is that competing with Google involves more than search technology. That’s all. If you look at the history of my posts you’ll see I’m not an unabashed Google lover by any stretch of the imagination.
I wish you all the success in the world.
-Steve
p.s. I would take issue with “users don’t like typing and will not enter more than 2-3 words.” Most non-tech savvy people I know type in godawful long sentences, which then break the engine. Either way though, your point stands.

By: anthropocentric

anthropocentric — Thu, 05 Oct 2006 01:34:40 +0000

What you are saying makes sense. It is clear that the search user interface will be improved in the coming years.
Encouraging users to make use of the stopwords and providing better results based on that usage is really interesting:
books by Bradbury about carousels
sales data for Apple iPod
recipe for onion soup
I wonder what proportion of searches will actually benefit from this enhancement? In many cases, the inclusion of the stopword and the added meaning that it brings is just not helpful (example: “recipe for onion soup” vs. “recipe onion soup”).
Is this a great new feature to incrementally improve the best search engines or is this truly a paridigm shift and a new type of search engine?