DARPA Developing Epic search Engine To Police The Internet

[RT] The Pentagon’s research arm that fosters futuristic technology for the military will soon begin working to surpass current abilities of commercial web search engines. Yet, once it masters the “deep Web,” the agency doesn’t say much about what comes next.

The Defense Advanced Research (DARPA) said its “Memex” project will be able to search the far corners of internet content that is unattainable by modern, mainstream search engines, offering DARPA “technological superiority in the area of content indexing and Web search on the Internet.”

DARPA said earlier this month in its solicitation announcement for Memex proposals that the system will initially be used to counter human trafficking, which often thrives in web forums, chat rooms, job postings, hidden services and other websites.

To root out trafficking operations within the invisible corners of the web, commonly referred to as the “deep web,” Memex (a melding of “memory” and “index”) “will address the inherent shortcomings of centralized search by developing technology for domain-specific indexing of Web content and domain-specific search capabilities.”

With Memex, DARPA hopes to achieve the ability for decentralized, automated, topic-precise searches that can leverage image recognition and natural language technology.

DARPA has asked researchers to develop advanced web-crawler software to reach sites and resources that have sophisticated crawler defenses. Memex operators would then be able to access the indexed domain-relevant content with much greater precision and ease than is currently possible.

Read full story