By: jshrager

jshrager — Fri, 13 Oct 2006 00:29:15 +0000

I think that some important points have been missed by all parties in this discussion. First, keywords (more general keyword logics) are the way we have now of doing “trick semantics”. In order to understand this, one has to understand that search is a two-sided coin; there is the query, and there is the database. Let’s take it for the moment that the database is The Web. The oft-overlooked point is that NEITHER a query NOR the database are expressed in pure semantics. Put in more plain terms: You don’t write what you think, nor does the person who write the web page you’re looking for; rather, you both use a surface code (NL) that bears a complex relationship to what what one “really means”. Keywords are a way that we have learned to see through this problem, and expert searches know how to do it. Suppose, for example, that you’ve interested in when marsupials first diverged from mammals. I have several options: I could write: “When did marsupials diverge from mammals?” If these were just interpreted as keywords, then if there happens to be a paper that contained the same words, I might win. But let’s assume that that’s not the case. Instead, there might be a paper that discusses (perhaps over a paragraph): “The evolution of kangaroos, and when they split off from the bears, their nearest mammalian counterparts.” (I’m making this up, I doubt that kangaroos and bears are closely related!) Here’s where the two headed coin comes in: You need to not only understand my query in some semantics terms, but you also need to understand the databse (that is, the sentences in the web pages) in similar semantic terms. Now, expertise in keyword search usage is, I claim, NOT merely a matter of knowledge of natural language, but a specalized skill that involves NL, but also involves understanding how to flip that double-headed coin — that is, how to take a query, in whatever terms, and create a new query that is likely to be able to find the appropriate things on the other side of the coin. This is not, I claim, the same as, nor even closely related to natural langauge processing, and indeed, I hypothesize that skill in NL is not highly correlated with it. (Everyone in my lab can speak several languages, but everyone in my lab comes to me because I can find things they can’t on google!) My hypothesis is that the step from NL queries to semantics is actually NOT the critical step in this process. Rather it is the step from Query-semantics (now approximated by keywords) to the other side of the coin: DB-semantics (now approximated by index terms), and the figuring out how to do THIS is going to be the critical step. I have a theory of how to do this… which I’ll tell you if you hire me!
Cheers,
‘Jeff

By: JoeDuck

JoeDuck — Thu, 12 Oct 2006 15:04:30 +0000

I’m bullish about your prospects Bernie but don’t think that 20% poll means much. More people than that simply *want* Google to falter. But best of luck.
Also: We need MUCH better search paradigms, so hurry up dude!

Comments on: The Powerset Blogstorm: 1 week later

By: jshrager

By: JoeDuck