In the beginning, there was music. Childhood and young adulthood floated by to a soundtrack of lyrics and rhythms and searing guitar riffs that consumed you, became you, constituted your identity, galvanized your intent, spoke your soul.

But time passes, classrooms fade to cubicles, and a vast landscape of new music turns foreign and unexplored. For Jeff Hersh, 31, the stereo came to double as Proust's madeleine, its purpose to invoke memories rather than create them.

"Finding music was easier when I was younger," says Hersh, a vice president at Smith Barney in New York. "In college I lived in a fraternity house with 70 guys all around me at all times, listening to various kinds of music. But as you get older, you work more, you get isolated."

Then in November, a friend told Hersh about Pandora.com, an inventive "Internet radio" website that generates music streams — "stations" — based on one's favorite artists or songs. He started his own private thread of music that was a combination of Neil Young and Pearl Jam, Hersh says, and in an hour he heard more new music he liked than he had in the last decade, much of it from obscure bands that shared musical traits with Young and Pearl Jam.

IBM plans to give away key search technologies for corporate data retrieval that use concepts and facts instead of simpler "keyword" searches relied upon by consumer Web companies such as Google Inc., the world's largest computer company said on Monday.

While simple but powerful keyword searches have revolutionized how Internet users locate and retrieve information, IBM is looking to transform how office workers sift through the piles of data stored inside organizations.

"I don't see any of the major players moving into this area," Arthur Ciccolo, head of search technology at IBM Research, said of how major consumer Internet search companies such as Google, Yahoo Inc. and Microsoft have focused on the public Internet instead of private record data retrieval.

IBM plans to openly offer other software developers its Unstructured Information Management Architecture (UIMA), a technology that can analyze text within documents and other media to understand latent meanings, relationships and facts.

The Netscape IPO wasn't really about dot-commerce. At its heart was a new cultural force based on mass collaboration. Blogs, Wikipedia, open source, peer-to-peer - behold the power of the people.

Ten years ago, Netscape's explosive IPO ignited huge piles of money. The brilliant flash revealed what had been invisible only a moment before: the World Wide Web. As Eric Schmidt (then at Sun, now at Google) noted, the day before the IPO, nothing about the Web; the day after, everything.

Computing pioneer Vannevar Bush outlined the Web's core idea - hyperlinked pages - in 1945, but the first person to try to build out the concept was a freethinker named Ted Nelson who envisioned his own scheme in 1965. However, he had little success connecting digital bits on a useful scale, and his efforts were known only to an isolated group of disciples. Few of the hackers writing code for the emerging Web in the 1990s knew about Nelson or his hyperlinked dream machine.

At the suggestion of a computer-savvy friend, I got in touch with Nelson in 1984, a decade before Netscape. We met in a dark dockside bar in Sausalito, California. He was renting a houseboat nearby and had the air of someone with time on his hands. Folded notes erupted from his pockets, and long strips of paper slipped from overstuffed notebooks. Wearing a ballpoint pen on a string around his neck, he told me - way too earnestly for a bar at 4 o'clock in the afternoon - about his scheme for organizing all the knowledge of humanity. Salvation lay in cutting up 3 x 5 cards, of which he had plenty.

Although Nelson was polite, charming, and smooth, I was too slow for his fast talk. But I got an aha! from his marvelous notion of hypertext. He was certain that every document in the world should be a footnote to some other document, and computers could make the links between them visible and permanent. But that was just the beginning! Scribbling on index cards, he sketched out complicated notions of transferring authorship back to creators and tracking payments as readers hopped along networks of documents, what he called the docuverse. He spoke of "transclusion" and "intertwingularity" as he described the grand utopian benefits of his embedded structure. It was going to save the world from stupidity.

I believed him. Despite his quirks, it was clear to me that a hyperlinked world was inevitable - someday. But looking back now, after 10 years of living online, what surprises me about the genesis of the Web is how much was missing from Vannevar Bush's vision, Nelson's docuverse, and my own expectations. We all missed the big story. The revolution launched by Netscape's IPO was only marginally about hypertext and human knowledge. At its heart was a new kind of participation that has since developed into an emerging culture based on sharing. And the ways of participating unleashed by hyperlinks are creating a new type of thinking - part human and part machine - found nowhere else on the planet or in history.

Not only did we fail to imagine what the Web would become, we still don't see it today! We are blind to the miracle it has blossomed into. And as a result of ignoring what the Web really is, we are likely to miss what it will grow into over the next 10 years. Any hope of discerning the state of the Web in 2015 requires that we own up to how wrong we were 10 years ago.

Ask most companies how they bring value to the market and they’ll point to their products. Cycorp is a bit different. The 10-year-old company cares about the services it sells—but mainly because they bankroll its true quest: creating a “knowledge base” called Cyc that can endow computers with something approaching common sense. This quest has been so time-consuming that most venture capitalists would long ago have written off their investments—or demanded the CEO’s head on a platter. That Doug Lenat and his 54 employees have avoided this fate is a lesson in managing long-term, visionary R&D projects.

Aristotle was one of the world's greatest thinkers ever. Digital Aristotle, on the other hand, knows as much about chemistry as a reasonably bright high school student, and nothing else. Yet a Seattle boutique investment firm, Vulcan Inc., is spending millions of dollars over the next few years trying to turn Digital Aristotle into, well, a digital Aristotle.

And why not? That's just a fraction of a percent of Vulcan founder Paul G. Allen's net worth. Even a noble failure would surely be at least as worthwhile as the Portland Trail Blazers professional basketball team, another of Allen's many interests. Allen, whose fortune derives from his standing as Bill Gates's original partner in Microsoft Corp., created Vulcan in 1986 to manage his investments.

Digital Aristotle began in 2003 as a contest, dubbed Project Halo. Three sets of high-powered researchers competed to create software that could do well on a high school advanced-placement exam in chemistry. They all succeeded. The winning program, written by a collaborative team from SRI International, in Menlo Park, Calif.; the University of Texas at Austin; and Boeing Phantom Works, in Seal Beach, Calif., scored a 3.00 on the exam out of a possible 5.00. That's better than the human student median grade of 2.82.

Suddenly, the computer world is interesting again. The last three months of 2004 brought more innovation, faster, than users have seen in years. The recent flow of products and services differs from those of previous hotly competitive eras in two ways. The most attractive offerings are free, and they are concentrated in the newly sexy field of "search."

Google, current heavyweight among systems for searching the Internet, has not let up from its pattern of introducing features and products every few weeks. Apart from its celebrated plan to index the contents of several university libraries, Google has recently released "beta" (trial) versions of Google Scholar, which returns abstracts of academic papers and shows how often they are cited by other scholars, and Google Suggest, a weirdly intriguing feature that tries to guess the object of your search after you have typed only a letter or two. Give it "po" and it will show shortcuts to poetry, Pokémon, post office, and other popular searches. (If you stop after "p" it will suggest "Paris Hilton.") In practice, this is more useful than it sounds.

Microsoft, heavyweight of the rest of computerdom, has scrambled to catch up with search innovations from Google and others. On Dec. 10, a company official made a shocking disclosure. For years Microsoft had emphasized the importance of "WinFS," a fundamentally new file system that would make it much easier for users to search and manage information on their own computers. Last summer, the company said that WinFS would not be ready in time for inclusion with its next version of Windows, called Longhorn. The latest news was that WinFS would not be ready even for the release after that, which pushed its likely delivery at least five years into the future. This seemed to put Microsoft entirely out of the running in desktop search. But within three days, it had released a beta version of its new desktop search utility, which it had previously said would not be available for months.

Meanwhile, a flurry of mergers, announcements and deals from smaller players produced a dazzling variety of new search possibilities. Early this month Yahoo said it would use the excellent indexing program X1 as the basis for its own desktop search system, which it would distribute free to its users. The search company Autonomy, which has specialized in indexing corporate data, also got into the new competition, as did Ask Jeeves, EarthLink, and smaller companies like dTSearch, Copernic, Accoona and many others.

I have most of these systems running all at once on my computer, and if they don't melt it down or blow it up I will report later on how each works. But today's subject is the virtually unpublicized search strategy of another industry heavyweight: I.B.M.

The flow of a document, including the topics covered and the ways those topics relate to each other, is clear to people. It would be useful if computer systems that process documents could also learn how to consider topic information.

Teaching a computer to discern a document's topics and create a summary that puts the topics in the correct order is a bit like teaching it how to put together the pieces of a jigsaw puzzle. Current methods focus on finding the right match for a given piece.

MIT and Cornell University researchers have developed a system that does the equivalent of putting pieces that show parts of a mountain and pieces that show parts of the sky into separate groups, and putting the sky pieces above the mountain pieces.

After training on subject-specific sets of documents and document summaries, the researchers' automatic classification algorithm, or content model, can extract the topic structure of a group of related topics. It selects and orders topics to generate to summary.

(DANNY HILLIS:) I have always envied Alexander the Great, because he had Aristotle as a personal tutor. In those days, Aristotle knew pretty much everything there was to know. Even better, Aristotle understood the mind of Alexander. He understood which topics interested Alexander, what Alexander knew and did not know, and what kinds of explanations Alexander preferred. Aristotle had been a student of Plato, and he was himself a great teacher. We know from his writings that he was full of examples, explanations, arguments, and stories. Through Aristotle, Alexander had the knowledge of the world at his command.

Of course no one today knows all that is known, in the sense that Aristotle did. Now there is far too much knowledge for that to be possible. The scientific revolution, and the technological revolution that followed it, led to a self-reinforcing explosion of knowledge. The explosion continues. Today not even the most highly trained scientist, the most scholarly historian, or the most competent engineer can hope to have more than a general overview of what is known. Only specialists understand most of the new discoveries in science, and even the specialists have trouble keeping up.

This problem isn't new. In 1945, Vannevar Bush wrote an essay for Atlantic Monthly about out the problem of too much knowledge. He wrote,

Engineers at Purdue University are developing a system that will enable people to search huge industry databases by sketching a part from memory, penciling in modifications to an existing part or selecting a part that has a similar shape.
first step in the search process.

Until recently, researchers and their assistants spent countless hours poring over seemingly endless volumes of journals and scientific literature for information pertinent to their studies in fields such as cancer, AIDS, pediatrics and cardiology.

But thanks to new software developed by bioinformatics researchers at UT Southwestern Medical Center at Dallas, scientists can now easily identify obscure commonalities in research data and directly relate them to their studies, saving money and speeding the process of discovery.

The computer application is unique because it "emulates the scientific thought process" in researching data, said Dr. Harold "Skip" Garner, professor of biochemistry and internal medicine, who with former graduate student Dr. Jonathan Wren developed the system.

The great strength of computers is that they can reliably manipulate vast amounts of data very quickly. Their great weakness is that they don’t have a clue as to what any of that data actually means.

Bow (or libbow) is a library of C code useful for writing statistical text analysis, language modeling and information retrieval programs. The current distribution includes the library, as well as front-ends for document classification (rainbow), document retrieval (arrow) and document clustering (crossbow).

Paul Allen, who co-founded Microsoft Corp. with Bill Gates, claimed preliminary success in a hitherto secret project to enable computers to answer questions they've never seen before, and to state their reasoning.

The project seeks to develop a so-called Digital Aristotle, named after the Greek philosopher who, in a far simpler day, is said to have known the answer to any question about science.

Link to dissertation

Penn State researchers have developed new software that can help decision-making teams in combat situations or homeland security handle information overload by inferring teams' information needs and delivering relevant data from computer-generated reports.
The agent software called CAST (Collaborative Agents for Simulating Teamwork) highlights relevant data. This helps improve a team's decision-making process as well as enhances members' collaboration.