July 3, 2008

Microsoft to acquire Powerset

On Monday, Microsoft and Powerset announced that Powerset is being acquired by Microsoft.

In terms of timing, the companies announced that the deal was signed. There is still the customary period before the deal is officially closed (at which point, I expect we're going to have a great party).

I'm including, below, the text of the announcements from the blogs of Powerset andMicrosoft.
I think these sum up pretty well the logic behind the acquisition on both sides.

It took a lot of work by many people to make this happen. Most significant, of course, was the entire team at Powerset, who executed so well to build and launch a wonderful product that showed the world what is now possible.

Immediately following the announcement, we had a day of calls with members of the press, which resulted in a lot of coverage. I'll try to post a collection of links next week.

One press meeting that I really enjoyed was a podcast with me, Ramez Naam (Group Program Manager for Microsoft Live Search), and Mike Arrington for TechCrunch. That link provides an article, transcript, and the full audio of the interview.

There is a lot more to say about Powerset, Microsoft, the acquisition, and what it means for the future of search, linguistic technology, semantic web, etc. I am excited to be staying on with Microsoft in a strategy and evangelist role and I am looking forward to the chance to talk and write a lot more about this, and from a whole new perspective, soon.

Here is the text of Powerset's blog announcement:

We’re excited to announce officially that Microsoft has signed an agreement to acquire Powerset.

Powerset has always been a small company with big dreams, with the ultimate goal of changing the way humans interact with computers through language. We set out to improve search by indexing Web pages based on the meaning expressed in them rather than just the literal words. Powerset licensed breakthrough technology from PARC, hired world-renowned computational linguists and search engineers, and recently released a search and discovery experience for Wikipedia articles. Our technology helps to improve search results and also makes new features possible, such as Factz, which aggregates information from many articles to summarize a topic.

With any startup, the challenge is to take the seeds of an idea and grow it into a viable company. At Powerset, we transformed our idea into a world-class semantic search platform, demonstrating the future of search with our Wikipedia search experience. But building a large-scale semantic search engine is expensive, requiring an engineering effort and computing resources beyond what most start-ups could ever imagine. Because our goals around improving search align so well, Powerset has decided to team up with Microsoft. We believe that this is the fastest way to bring our technology to market at a large scale.

Microsoft shares our goal to improve search through deeper analysis of queries and documents, and understands that our technology and expertise will play a key role in the evolution of search. With an existing search infrastructure, incredible capital resources, unlimited data, a leading search team, and clear mission to revolutionize the search landscape, Microsoft can rapidly accelerate our progress in building semantic search technology and bringing it to full Web scale. When we launched our first product, we heard: this is great, but when and how will we get Powerset to go beyond Wikpiedia? Microsoft accelerates our ability to move Powerset to the entire Web faster than anyone could have imagined.

Powerset will continue to operate much as we currently do, working in the same building, with the same organizational structure, and with the same uniquely talented and growing team (apply on our jobs page). We’ll continue to tackle the hardest problems in parsing, semantics, ranking, indexing, scalable computing, user experience and all of our other specialties. But now we’ll do it with the support of Microsoft and the vast resources of the entire Live Search team.

Over the past couple of years Powerset has made amazing progress. Starting with just a big idea, we licensed the best linguistic technology, recruited a top-notch team, built out our datacenter, engineered a world-class semantic search platform, tackled deep natural language issues, improved relevance, innovated an interface and launched a great product. So few start-ups ever tackle such deep, scientific problems successfully and create the kind of value we’ve delivered in such short order.

For now, Powerset.com will continue to host our Wikipedia Search & Discovery and we’ll be continuing to experiment with our product, based on user feedback. But, expect many announcements from us in the coming months about how we’re integrating our technology and features into Live Search.

And here's the text of Microsoft's blog announcement:

Powerset joins Live Search

We're excited to announce that we've reached an agreement to acquire Powerset, a San Francisco-based search and natural language company.

Powerset will join our core Search Relevance team, remaining intact in San Francisco. Powerset brings with it natural language technology that nicely complements other natural language processing technologies we have in Microsoft Research.

More importantly, Powerset brings to Live Search a set of talented engineers and computational linguists in downtown San Francisco. This is a great team with a wide range of experience from other search engines and research organizations like PARC (formerly Xerox PARC).

We're buying Powerset first and foremost because we're impressed with the people there. Powerset CTO and cofounder Barney Pell is a visionary and incredible evangelist. When he introduced our senior engineers to some of the most senior people at Powerset — Search engineers and computational linguists like Tim Converse, Chad Walters, Scott Prevost, Lorenzo Thione, and Ron Kaplan — we came away impressed by their smarts, their experience, their passion for search, and a shared vision.

That shared vision is to take Search to the next level by adding understanding of the intent and meaning behind the words in searches and webpages.

We know today that roughly a third of searches don't get answered on the first search and first click. Usually searchers find the information they want eventually, but that often requires multiple searches or clicks on multiple search results. Two specific problems are the most common reasons for this:

* Differences in phrasing or context between a user's search and the way the same information is expressed on webpages. Search engines don't understand today that "shrub" and "tree" are similar concepts. We don't understand that "cancer" sometimes refers to a disease and sometimes refers to a horoscope and when a query or a webpage refers to which.
* Lack of clarity in the descriptions for each webpage in the search results. Sometimes a result looks relevant from its short description on the results page but turns out to be not so relevant when you visit the actual page. As a result, searchers frequently click results and then rapidly click back when they realize they aren't what they're looking for.

These problems exist because search engines today primarily match words in a search to words on a webpage. We can solve these problems by working to understand the intent behind each search and the concepts and meaning embedded in a webpage. Doing so, we can innovate in the quality of the search results, in the flexibility with which searchers can phrase their queries, and in the search user experience. We will use knowledge extracted from webpages to improve the result descriptions and provide new tools to help customers search better.

Working with our existing Search team and other Microsoft teams that focus on natural language, Powerset will help us address all of those problems and opportunities.

We're looking to add even more talented engineers to the San Francisco team to accelerate our shared progress. If you're interested in joining the team, drop us a line.

We'll have more to say about the things we're doing in understanding searches and webpages through natural language technology in the coming months. In the meantime, please join me in welcoming Powerset to Microsoft!

Satya Nadella, Senior Vice President, Search, Portal, and Advertising

Posted by barney at 3:50 PM | Comments (0) | TrackBack

May 11, 2008

Powerset launched today!

Sunday 5/11/2008: Powerset has launched our first open product to the world!

Our initial product offers users a whole new way to experience Wikipedia and Freebase content, based on our unique natural language understanding technology.

Powerset Homepage

A write-up about Powersets Wikipedia product is available on the Powerset blog.

I will write more over the next few days about the product and it's role in the ecosystem of search, content, linguistics, and semantic technology, but for now I'm just incredibly excited. I'll just note a couple highlights from the evening.

We were planning to launch at 9pm PST. But in an unusual twist for a software company, one of our eager engineers actually flipped the switch to make everything live 15 minutes ahead of schedule. Since everything was working, we just decided to go with it!

Within the next couple of hours, the first press articles came out. Pretty much across the board, the journalists and bloggers captured the essence of our initial product. They got what was special about it, and also recognized it for the initial step that this represents (finally freeing us of the Google Killer hype that is impossible for a small startup to live up to).

Within 1 hour of launch, we received a note from a VC asking about possible investment in the company.

And 2 hours after we were live, we had our first denial of service attack. An automated script sent a never-ending sequence of bizarre queries at our system. Fortunately, our own engineers had been preparing for this kind of thing already and we managed to stay up and weather the storm.

The whole company was gathered in the office. We spent time alternating between: making speeches and toasts, reading press articles, looking at the traffic and load, and watching the initial queries float by. The last part was the most exciting: real users and real queries!

Since we launched on Sunday night on Mother's day (thanks, Mom!), we had it pretty easy with relatively light traffic. I think Monday is going to be an exciting day.

Posted by barney at 11:59 AM | Comments (0) | TrackBack

February 26, 2008

In 5 years we will search more with voice than typing

David Vogelpohl wrote an article, Will Microsoft Resurrect Natural Language Search, citing a recent AP article about Bill Gates and voice-based search. Here are some quotes from the AP article:

People will increasingly interact with computers using speech or touch screens rather than keyboards, Microsoft Corp. Chairman Bill Gates said.

“It’s one of the big bets we’re making,” he said during the final stop of a farewell tour before he withdraws from the company’s daily operations in July.

In five years, Microsoft expects more Internet searches to be done through speech than through typing on a keyboard, Gates told about 1,200 students and faculty members Thursday at Carnegie Mellon University.

David conjectures, as do I, that when people speak their searches they are more likely to use natural language than to use keywordese, and that this could change the game in search.

I personally can envision Microsoft trying to integrate speech based data entry as closely as possible with our normal style of speaking. Perhaps the phrase “Where can I buy a hd tv?” would be more natural for searchers when you take away the limitations of the keyboard.

Wide spread speech based data entry will almost certainly impact the way Microsoft and subsequently all other search engines deal with search queries.

It's interesting to see Bill Gates predicting this to happen within 5 years. In the blink of an eye, an entire industry is going to change dramatically.

While on the topic of predictions about voice and language, here's one of my predictions that I have been meaning to write up:

Within 8 years from now (2016), every category of consumer electronics will have some linguistic interface as a standard feature.

By "linguistic interface", I mean voice interactions or text-based interaction that is linguage-based. Not that these devices won't still have nonlinguistic interfaces too (e.g. there will still be buttons, most likely). And by "every category", I mean you will not find a category of consumer electronics that does not have some product in that category with that feature.

For example, users will expect to be able to talk to cameras, tvs, stereos, ipods, phones, watches, microwave ovens, refrigerators, cars, etc. There will still be some cameras that aren't language-enabled, but every category will have some products that are.

As my friends Cliff Nass and Scott Brave write in their book, Voice Activated, when people interact with devices using voice, it also invokes the rest of their social apparatus. You can't hear a voice without ascribing some kind of personality, gender, race, social status, etc to the source of the voice. So in addition to expecting linguistic capability, we're also going to start expecting personality within the next decade.

I'll stop here before I get carried away to the singularity...

Posted by barney at 8:30 PM | Comments (0) | TrackBack

February 25, 2008

LA Times on Founders Brunch and the PowerStache

My friend Jessica Guynn just wrote an article that appeared online in the LA times today entitled: Brainstorming over bagels: Silicon Valley entrepreneurs seek camaraderie and capital at brunch.
The article will appear in the LA Times print edition tomorrow morning.

The articles covers the Founders Brunch, a networking event for founders of companies that I attend regularly.

Many of my friends are quoted in the article, and there are photos of Auren Hoffman and Keith Rabois (our host this time). Peter Thiel expressed the networking aspect of this kind of event well:

Founders Brunch is important for the same reason Silicon Valley is important: There are all of these subtle network effects," said Peter Thiel, a 40-year-old former PayPal executive now bankrolling some of the hottest Internet companies. "Otherwise why wouldn't you start a tech company in Fresno where everything is cheaper? The advantage to being in Silicon Valley and the San Francisco area is that so many other people are doing the same thing."

Jessica noted that I had a new beard, and I explained my recent decision on growing it:

Barney Pell, the 39-year-old co-founder of Powerset, a natural-language search engine trying to challenge Google, sported a new beard he vowed not to shave until his San Francisco start-up launched its new product.

To be more accurate, I vowed not to shave off my beard until the launch, but I didn't vow that I wouldn't shave at all. I made that mistake during graduate school. I thought I was ready to submit my PhD thesis in about 3 months, and vowed not to shave or cut my hair until it was done. This was partly a way to motivate myself to finish, and partly a way to let my friends stop asking about my progress as they would clearly know when was done. As it turned out, my thesis advisor thought I had more work to do, and I wound up taking a full year before finishing. So by the time I was actually ready to submit my thesis, I had really long hair and a very full beard indeed. I'm not going to do risk that again...

Anyway, you might think I'm a maverick, but it turns out that most of Powerset is in on the gig. Almost all our employees are growing moustaches and/or beards in preparation for our upcoming launch. Even women who can't grow nearly as nice moustaches as the men have painted them on from time to time. And our folks even registered a domain name and created a website, PowerStache.com, featuring photos taken over time as people grow their beards and moustaches.

It's pretty silly and really wasn't initially a coordinated effort, but it's fun and reflects the excitement inside the company as we are nearing the time when the early version of our product will be available to the general public.

Posted by barney at 7:52 PM | Comments (0) | TrackBack

Powerset in Forbes article on the Language of Search

Forbes.com has a special issue on language, including interesting articles and interviews by some of my favorite writers on Language.

I'm happy that natural language and semantic search was included in the special issue. Andy Greenberg from Forbes.com published his piece on language and search engines devoting a good portion of the article to Powerset and Hakia, featuring interviews with me and with Hakia's founder Riza Berkan. The article, entitled "Language Web-lish" starts off with Andy using Powerset's metaphor comparing people's current use of search engines to communicating like cavemen:

A question in English, like "What year was Hillary Clinton born?" becomes what he calls a primitive "keywordese": "Hillary Clinton born year."

"We have this great gift of human intelligence based around language," says Pell, "and now we have to translate it into a grunting pidgin language to interact with machines."

Andy described an example I showed him from Powerset:

When a user enters the question, "In what year was Hillary Clinton born?," Powerset's algorithm doesn't simply scour the Web for this collection of words in close proximity. Instead, it looks at pages with an eye for their meaning. Reading the sentence "Born to Dorothy and Hugh Rodham in 1947, Hillary Clinton is a New York senator," Powerset will disassemble the sentence's grammar and extract the fact of Hillary Clinton's birth date. That fact is then connected with the user's question, even if the word order of the result and the query didn't originally match.

Andy also went through an example from Hakia:

Taking the question "What drug is best for treating a urinary tract infection?" Riza Berkan points to the word "drug." Hakia's algorithm, he says, understands that the word contains a massive subset of concepts including synonyms and specific names of medicines. When it spots a term that falls into that subset, like "Amoxicillin," Hakia can substitute the medicine's name for the word "drug" in the result.

"You don't want the word 'drug,' you want the name of the drug," says Berkan. "That's a hidden failure in search engines, and people don't even know what they're missing."

Other natural language and semantic search companies mentioned included Cognition Search and Lexxe.

As is typical, my friend Peter Norvig at Google gets the last word in the article:

Google's Peter Norvig, the search giant's director of research, knows just how complex semantic algorithms can be: His Berkeley Ph.D. thesis tried to develop one in 1978. Every sentence of text, he says, took weeks to analyze. "The result was kind of like a dancing bear," he says. "It was amazing that it could dance at all, but we didn't expect it to star in the Moscow Ballet."

But that doesn't mean Google's engineers are idly watching semantic search from a distance, says Norvig. The company's thousands of engineers are looking at how to incorporate semantic analysis into a search algorithm. But semantic analysis is just one of many directions that Google's teams are exploring... "Basically, we just do whatever works," says Norvig. "Instead of trying to understand everything, we're trying to understand something about billions of pages a week."

But does that pragmatic approach leave Google vulnerable to an innovative start-up willing to risk its fate on building meaning-based search from scratch?

"It's unlikely," says Norvig. "But even car companies have to worry about anti-gravity machines."

I think that analogy is quite a stretch. It's more like big car companies having to worry about smaller companies focused on electric cars. They don't have to worry about this immediately but, at some point, this is going to be the future of their industry.


Posted by barney at 12:16 AM | Comments (0) | TrackBack

November 19, 2007

Natural Language and the Semantic Web: ISWC Keynote talk

I gave an invited keynote talk last week at The 6th International Semantic Web Conference and the 2nd Asian Semantic Web Conference, 2007. The abstract for the talk is below. The image below links to the original video and presentation slides.

The live presentation (and video) contains technical demos that aren't in the slides. Some of the demos are already available inside Powerlabs (e.g. Powermouse, which lets you browse and query our semantic database of facts extracted from Wikipedia), while some of these are still internal (e.g. an open search box, and output of our natural language system on full sentences). I also gave some detailed walk-through showing how Powerset takes advantage of external semantic resources like Wordnet and Freebase.

For me, the most fun part of the talk was toward the end, where I got to speculate on how ecosystem effects can make natural language search and the semantic web become deeper and more powerful more quickly than people might expect. For example, advertisers, publishers, and vertical search sites will be able to contribute ontologies that enable them to get more users, better internal search, and more revenue, while having as a side effect that the broad search engines get more knowledgeable about different domains. The questions afterward were also challenging and interesting.





POWERSET - Natural Language and the Semantic Web

Continue reading "Natural Language and the Semantic Web: ISWC Keynote talk"

Posted by barney at 8:29 PM | Comments (0) | TrackBack

November 1, 2007

Management changes at Powerset

In this posting, I want to talk about some significant management changes at Powerset. The main changes are:

Let me give you some historical context. When I first had the idea for Powerset and was looking to build my initial management team, I knew that as a first-time CEO my strengths were around the technology and vision and not necessarily around management of a large organization. I sought out a strong operating partner to share many of the C-level responsibilities while letting me do the things I was great at. And that's how I found Steve Newcomb, who became my cofounder and our COO. This partnership worked well during the formation and rapid growth of the company. While I focused on the strategy around Powerset’s vision and technology and connecting Powerset with the outside world, Steve lead the company internally and brought strengths in execution on several other fronts. We can all be proud of what we accomplished during Powerset’s early days but with the company’s very rapid growth and the team’s great progress, we knew it was time to re-evaluate.

After extensive thought and reflection, the Board and management team decided that the time was right for us to bring in a new CEO to take the company to the next level and for me to transition into the role of CTO. The Board evaluated what this change would mean for Steve, and concluded that bringing in a world-class CEO who is a strong operational manager would make the COO role redundant. By helping us get to this point, Steve did the job he signed up for and he has now left the company. As many of you know, Steve has been a strong force for the company and a key part of what made Powerset an early success. He has also been a champion and protector of our corporate culture. These influences are now part of our DNA and we will continue to invest in and protect the inspirational culture that Steve helped to build. Steve will remain a friend of the company and a major shareholder and maintains the best wishes for the success of Powerset and the team. He has personal passions in some new directions which he will no doubt be writing about on his blog.

I consider this kind of deliberate reflection in order to make the best choices for the company a strong testament to Powerset’s management team. The result is truly what we all think is the best path for the company going forward. Bringing in a new world-class CEO will help the company grow and take advantage of the great opportunity ahead of us. I am proud of what we accomplished to get the company this far, and I really look forward to working and learning from a great CEO during our next stage of growth. And, as a major shareholder in the company, I see this transition as something that will result in great long term value for the company.

While I enjoyed being CEO during the initial growth phase of the company, pulling together the early team and investors, and defining the vision and core strategy for the company, I believe the CTO role at this point plays to my best strengths and my passion. It also makes it easier for me to contribute ideas and technical solutions without people taking them as directives from the CEO. This was not obvious when I decided to be CEO during the earlier growth of the company and in this sense it makes it easier and more appropriate for me to be part of the creative team. As Founder and CTO, I will also continue as the technology visionary and evangelist for Powerset to the outside world. Ron Kaplan, who has been our CTO and Chief Science Officer, will transition fully to the CSO title. This also gives Ron more time to guide the core science at the heart of Powerset's differentiation.

We have recently kicked off a search to find the right CEO. We have already talked with some excellent candidates and are confident that we will bring in someone of up to Powerset’s level of quality. If you are or know someone who could be a great CEO for a company with Powerset's vision and visibility, I would love to talk with you.

So that's the background for the current changes. With that, I want to give some perspective on the development of startup companies, which may be useful for other early management teams facing similar stages of growth. The talents, roles, and personalities that work best for running a company are often different at different stages of the company's growth. Each stage brings with it a challenging transition. Powerset is unusual only in the reflection and cooperation that the management team has demonstrated in making the right changes to propel the company through the next stages of growth.

In thinking about these changes, it is an interesting point for reflection about where Powerset is now and where we are going. It has been a little over two years since we incorporated the company and just one year since we raised Series A funding. What was largely a potential back then has become much more of a reality now. One year ago we had only a prototype, didn't have a license or source code to our core technology, had a small team in general and no search team at all and people were asking why natural language might matter for search, wasn't this impossible and hadn't it already failed. Today, all of that has changed in ways that are beyond what anyone might have expected.

The changes we are making now position us for a next phase that promises to be really exciting. We will bring our technology out in real products that users will enjoy and that will trigger changes across the entire ecosystem of search. I think the next year is going to be an amazing time for Powerset and I am as passionate as ever about Powerset, our technology, our team and our future.

Posted by barney at 10:51 PM | Comments (0) | TrackBack

September 12, 2007

Tim Converse on Proximity is a Hack

Powerset's Tim Converse wrote a great article entitled: Proximity is a Hack.

In the article, Tim says that the two biggest improvements in web search were the use of links (including anchor text) and term proximity. The article explores the benefits of term proximity and argues that works to the extent that it approximates linguistic relationships in the text.
He concludes that natural language processing of the documents should have the ability to more accurately capture linguistic relationships even if the query itself is in keywordese (as opposed to a natural language query with internal linguistic structure).

To recap: proximity is both a wonderfully powerful relevance feature, and a total hack. It helps enormously, but it’s not what you really want, it’s just sorta somewhat correlated with what you really want. What you need for what you really want is the underlying structure of all that web content: the real syntactic structure of the sentences, how the sentences connect to each other, how the facts relate, and (maybe) how the discourse flows and the topics connect. We’ve squeezed all the juice we can out of webpages considered as word-vectors; now it’s time to parse this stuff and get at the real structure.

Can that be done? A couple of years ago I would have said no, but I hadn’t seen the PARC natural language technology then, and didn’t know that an effort this concerted and well-funded was on the way. Now, do I think that Powerset will do it? I still don’t know, frankly - there’s so much more to do to make it real and debugged and scaled the way it needs to be. But it’s clear to me that the next big thing in web search is either this or something a whole lot like this, and I think we have the best shot of anyone. And that’s why I’m at Powerset.

The article is definitely good reading for people interested in search and the potential benefits of NLP.

Posted by barney at 9:03 PM | Comments (0) | TrackBack

August 9, 2007

Powerlabs internal launch

Today was an exciting milestone for Powerset. We released the first version of the Powerlabs platform for our employees to try out. The Powerlabs platform is a framework for innovation in which a community of users can generate and refine ideas as they interact with products and concepts. It combines elements of social networking, crowd-sourcing, and social search (among other buzzwords that, in our case, really make a difference).

It turns out that the product team have been using Powerlabs to improve Powerlabs itself, so there were already a large set of ideas and evaluations by the time the rest of the employees got to try the system out. And we are already finding the system to be addictive: within a couple hours of internal release (the time it took Product Manager Mark Johnson and me to play a few matches of Dance Dance Revolution), already over 50 ideas had been generated and evaluated!

With this much interest from our own small number of employees, it is amazing to think about the kind of ideas, creativity, and feedback we are going to get from the 16,000 people already signed up for Powerlabs launch in September! (You can sign up at the Powerlabs Website).

Powerlabs is so cool, in fact, that we have already started talking about potentially offering this as a service to other companies who want community innovation around their products (both internal employees and outside users). So the race is now on to see which takes off faster: a radical new way to search using natural language, or a radical new way to create products!

Posted by barney at 8:49 PM | Comments (0) | TrackBack

Technology Review on Building a Better Search Engine

Technology Review recently had an article featuring Powerset: Building a Better Search Engine, by Michael Reisman. In addition to Powerset, the article also mentions Hakia and Cognition Search, and closes with a discussion of a semantic search project inside IBM.

I am an avid reader of Technology Review and am really excited to have an article about us in this great publication.

The full article is worth reading.

Here are just a few excerpts about natural language search and Powerset's technology:

The company claims that the engine finds the best answer by considering the meaning and context of the question and related Web pages. "Powerset extracts deep concepts and relationships from the texts, and the users query and match them efficiently to deliver a better search," Powerset CEO Barney Pell says.

Powerset chief technology officer Ron Kaplan has led PARC's XLE team since the 1970s and is the author of much of the technology behind XLE that has been licensed to the company. Kaplan says that he and Pell began to collaborate on the idea about two years ago. Current methods of searching used by more traditional engines focus on isolated keywords and broad but shallow content coverage. This leaves a lot of room for improvement, Kaplan says.

"They are really not getting at relationships," he notes. "The best that they do to approximate relationships are words that are close to other words." He adds that a much deeper level of analysis is required.

The article came in time to announce the upcoming launch of Powerlabs, our early user community:

The company plans to release demo versions of the search engine on its Powerlabs website, where consumers can test-drive the product beginning in September. User feedback will be taken into consideration as Powerset makes the final product, which is slated for release next year.

"The key challenge is to get the system to the point where people can understand how to use it and get real value out of these systems even though they are not perfect," Pell says. "We are finally at the point where we are going to cross that threshold."

Posted by barney at 8:21 PM | Comments (0) | TrackBack

Information Week on The Ultimate Search Engine

An article in Information Week this week, The Ultimate Search Engine, by Nick Hoover, gives an overview of many approaches and providers of next-generation search.

The article begins with a statistic about search frustration that I have heard several times but have not been able to find the data:

People search for 11 minutes on average before finding what they're looking for, and half abandon searches without getting that far, according to Microsoft. By Gartner's estimate, half of potential Web sales are lost because visitors simply can't find what they want.

The article covers the following ideas being explored for next-generation search:

The discussion on natural language and semantic search features Powerset and Hakia.
Here are a few relevant excerpts:

Continue reading "Information Week on The Ultimate Search Engine"

Posted by barney at 7:26 PM | Comments (0) | TrackBack

May 28, 2007

Powerset's Nitay Joffe in New York Times today

In Fierce Competition, Google Finds Novel Ways to Feed Hiring Machine - New York Times

The New York Times today features an article about the fierce competition for talent in the high-tech industry. Powerset engineer Nitay Joffe (see photo) is quoted in the article as having turned down Google to join Powerset:

All three companies say their toughest recruiting challenges come from start-ups, who snap up people like Nitay Joffe.

Mr. Joffe, who had summer internships at Google for the last two years, expected to go to work there. But before Mr. Joffe, a recent computer engineering graduate of the University of California, San Diego, accepted a job, a friend suggested he check out a San Francisco start-up, Powerset, which is trying to build a rival search engine.

“Powerset had everything that Google had in terms of what I was looking for — smart people, interesting projects, great amenities,” Mr. Joffe said. Powerset also had one thing Google could not offer: the potential to strike it rich with the Internet equivalent of a lottery ticket.

“When you get a stock option at 5 cents and it goes to $50 ...,” Mr. Joffe said, before his voice trailed off. With Google’s shares hovering around $480, it no longer offers the same potential. “Google isn’t going to $4,000,” said Mr. Joffe, who began working at Powerset recently.

The article omitted one of Nitay's key reasons for join Powerset, in addition to the potential startup stock value. He was really excited about making a huge difference to the company, as opposed to being one of 10,000 people at a big company like Google. And this has certainly come true -- he is working on the very core of our system and is now central to the company!


Posted by barney at 1:35 PM | Comments (0) | TrackBack

April 30, 2007

Barney speaking at U. Washington

I am giving a talk tomorrow (Tuesday, 5/1/07) at the University of Washington, hosted by Professor Oren Etzioni.

UPDATE: The video for this talk is now available.

Here are details of the talk. The abstract is below.

Powerset and Natural Language Search

Search has become a critical part of our daily lives and the primary gateway to information and services of all kinds. Much of the success has been driven by the growth of the internet and behind-the-scenes improvements in search technology. However, the search user interface, based on keywords and advanced search operators, has changed very little since the earliest information retrieval systems.

In this talk, we discuss the concept of natural language search. Central to this is a new user experience, in which users express queries in natural language and the system responses respect the linguistic information in the query.

To realize this vision at broad scope and scale will require advances in a variety of technology areas, including natural language processing, information extraction, knowledge representation, and large-scale search indexing and retrieval systems.

In addition, it will require innovations in user interface. Issues include changing user behavior, educating users about the affordances and constraints of the technology, supporting users in formulating effective queries, and managing expectations.

With a new search architecture centered on natural language, linguistic and lexical knowledge translate directly into improved capabilities and experiences for end users. This creates a challenge to encode enough knowledge to cross the threshold for a broad application. It also creates an opportunity to establish a framework for introducing new knowledge through automated and community-based approaches.

While the challenges are formidable, the opportunities are great. Natural language search has the potential to transform and improve all aspects of the search experience: precision, recall, power, usability, presentation and interaction.

We will conclude the talk with a discussion of Powerset, a startup company that is tackling these challenges in an attempt to bring natural language search to the world.

Posted by barney at 12:01 AM | TrackBack

April 7, 2007

When is Natural Language useful?

In talking about Powerset and natural language search, I am frequently asked "When is Natural Language search useful?". The idea here is that maybe there are some specific situations where you really want natural language search. My general response is that this is like asking "When is Natural Language useful?" to talk to other people? The very question assumes that there are some particular situations where you want to use natural language, and others where you would prefer to just grunt out a few words.

I am aware of a number of situations where it is really clear that you want the power and usability of natural language for search, including:

Continue reading "When is Natural Language useful?"

Posted by barney at 8:07 PM | TrackBack

Wall Street Analyst Downgrades Google, Mentions Powerset

As reported by Eric Savitz on Barron's Blog:

On April 3rd, Trip Chowdry, of Global Equities Research, downgraded Google (GOOG) to Neutral from Buy, and set a price target of $425, nearly 10% below the current price.

Among several reasons (including Youtube legal and revenue issues and Moritz leaving the board), he says the company is losing the technology lead in search to the start-up PowerSet, and that Google could lose talent to the newcomer.

Trip does note that Powerset has not yet launched. And he is known for being contrarian. I would certainly not bet against Google's stock continuing to rise for a long time. But I do agree with Trip that the potential for a disruption in search technology (the shift to natural language search being one example) shakes up the game and makes Google's long term dominance at least a little less certain.

My Powerset cofounder Steve Newcomb blogs about this as well.

Posted by barney at 7:47 PM | TrackBack

February 17, 2007

More on pigeons and search

I have written previously that using "keywordese", the language of keywords used to query search engines today, is like using a "grunting pidgin language" instead of a human natural language to communicate our intent to the search engine. This also explained the joke on our Powerset Series A Funding Party t-shirts with the grunting pigeon.

In an unexpected confirmation of the connection between pigeons and keyword search technology, a reader pointed out to me an article on Google's website about Google's PigeonRank technology.
This technology employs huge arrays of pigeons that are trained to peck at pages that are more likely to be high quality results. Although each pigeon is fallible, the wisdom of the crowds means that the results of the whole coop can have high quality.

So far, the PigeonRank technology appears to be applied only to rankings, not to query languages. But it is possible that such work is happening in secret.

Posted by barney at 1:47 PM | TrackBack

February 14, 2007

Powerset Series A Funding Party

Powerset had party last Saturday night to celebrate our Series A funding! The party took place at Frisson, a trendy bar/restaurant in San Francisco's financial district. We wound up with perhaps 500 people attending, completely filling up the space (and all available parking for blocks around) for most of the evening.

Party supported by our sponsors and investors

While the event was lavish and brought back memories of the heady dotcom boom days, when some companies spent a significant portion of their funding on their launch parties, we actually were fortunate to have the cost subsidized by our investors and partners. I want to thank them here.

Continue reading "Powerset Series A Funding Party"

Posted by barney at 11:57 PM | TrackBack

February 13, 2007

Barney interviewed by Bambi Francisco on Marketwatch about PARC/Powerset deal

Bambi Francisco, from Marketwatch, interviewed me last Friday about the PARC/Powerset strategic deal announcement. The video is now available.

We discussed the following questions:

The interview was brief and to the point, and I think we shared some useful perspective on the significance of this deal.

I still need to post an entry to an earlier video interview with Bambi last November. It's hard keeping a blog up to date when so much is happening with your startup company.

Posted by barney at 9:07 PM | TrackBack

February 4, 2007

Barney on Cooking With Geeks (Podcast)

I was invited a few weeks ago to attend an experimental dinner event called "Cooking with Geeks". The idea was to combine a salon with a cooking lesson and podcast it to the world. The Cooking With Geeks podcast is now available (the previous link is to part 1, after which you can watch part2).

Here's the writeup from the CWG website:

We got a group of geeks together, had a great chef come in and teach us how to cook, handed out some wine, and kicked off an interesting conversation. Come on along for a San Francisco geek dinner.

Geeks involved are: Barney Pell, CEO, Powerset; Mary Hodder, CEO, Dabble; Henri Poole, founder/director, Civic Actions; Kathleen Lyman, CEO, LaunchMedia; Steve Gillmor. Our co-hosts were Robert Scoble [of Scobleizer fame] and Fred Davis, co-founder of Wired Magazine. Videographers are Eddie Codel and Glenn Gullmes. Chef is Rozana Ogneva of www.AreYouBeingServedCatering.com. Our host is Jeannine Barnard.

I knew some of these folks before, in particular Mary Hodder is a friend of mine. I didn't know just how expert Mary was about food. The food itself was delicious and we all gave our compliments to the chef. In addition to food, we covered several interesting (to us) topics. Here are a few that I remember after a month (I haven't watched the videos yet).

I enjoyed the event and look forward to doing this again.

For folks who don't live in the Bay Area, I think this dinner gives a pretty typical glimpse into how folks here in the tech industry combine work and play in social contexts. It captures pretty well the way that everyone is passionate about ideas, technology, and startup companies.

Posted by barney at 10:01 PM | TrackBack

January 10, 2007

CNBC Squawk Box interview: It just (barely) happened

Note: The video is live on CNBC site for 24 hours only.
Search for the Next Google
This search engine provides data through users asking questions rather than typing in key words, with Barney Pell, Powerset CEO and CNBC's Becky Quick

I just got home from my CNBC Squawk Box interview. In addition to the challenges I expected from having to get up at 3am, there were some challenges I did not expect.
When I got to the front door of KRON-TV (Channel 4 in SF) and called the newsdesk reception, nobody was expecting me there! I explained that I was here for CNBC and they said they would look into it and send someone down. But a few minutes later nobody had arrived. I called CNBC in New York, who said that people were expecting me and someone would be right down.

Continue reading "CNBC Squawk Box interview: It just (barely) happened"

Posted by barney at 5:30 AM | Comments (2) | TrackBack

January 9, 2007

Barney on CNBC Squawk Box Wednesday 1/10/07 7:40AM EST

I am going to be interviewed live tomorrow on CNBC Squawk Box, talking about Powerset.

Today CNBC came in to take video screenshots of our old demo, so some of this content may be in the background during my interview.
The video shoot of our demo today was fun. The videographer, Stan, jumped right in and bossed everyone around. The funniest part was when he insulted our assistant Ryan's clothes, saying he used to shop at Baby Gap too. Scott Maddux jumped to Ryan's defense, saying that it was better than the Geranimals that Stan was wearing (stay tuned for photos so you can judge for yourself).

The interview will be live at 7:40AM EST, or 4:40AM PST.

While the attention is exciting, I have no idea how coherent I will be so early in the morning...

If you want to watch it, you have to tune in to CNBC on your tv at that time (so if you're on the west coast like me, you can share my sleepiness at that early time in the morning). We are hoping to get an archival version that we can serve from the Powerset website.

Posted by barney at 11:11 AM | TrackBack

January 1, 2007

Happy New Year: Powerset in the New York Times

Powerset was featured in the New York Times today in an article by Miguel Helft entitled: "In Silicon Valley, the Race Is On to Trump Google". The article talks about the wave of startups going after aspects of the search market. I originally thought the article was going to be titled: "What are they thinking?", as the emphasis is on the mindset of the entrepreneurs and the investors going after a large market in the presence of such formidable players. The article begins and ends with a discussion of Powerset. In between, the search startups discussed in the article include Powerset, Hakia, Snap, A9, ChaCha, and Wikia.

Continue reading "Happy New Year: Powerset in the New York Times"

Posted by barney at 4:34 PM | TrackBack

November 19, 2006

Google and Powerset in the Sunday Times

Powerset was mentioned today in The Sunday Times in an article entitled: Quest for last word in search, by Paul Durman.

The article interviews my friend Peter Norvig, now director of research at Google, and discusses natural language search. Here are some relevant excerpts:

Continue reading "Google and Powerset in the Sunday Times"

Posted by barney at 10:39 PM | Comments (0) | TrackBack

November 4, 2006

Amazon Web Services and Powerset in Business Week article

Jeff Bezos Cover of Business Week

Rob Hof at Business Week just came out with an article called Jeff Bezos's Risky Bet. The article talks about Amazon's Web Services initiative, in which Amazon is enabling other companies to take advantage of the massive technology infrastructure Amazon has developed to power its own operations:

Amazon has spent 12 years and $2 billion perfecting many of the pieces behind its online store. By most accounts, those operations are now among the biggest and most reliable in the world. "All the kinds of things you need to build great Web-scale applications are already in the guts of Amazon," says Bezos. "The only difference is, we're now exposing the guts, making [them] available to others."

This article was the first to announce that Powerset is one of the major early customers for Amazon's new Electric Compute Cloud (EC2) Web service. Here are the relevant paragraphs, which mention Powerset and some of our key angel investors:

Continue reading "Amazon Web Services and Powerset in Business Week article"

Posted by barney at 2:57 PM | Comments (0) | TrackBack