Distributed search engine

Distributed search engine

A distributed search engine is a search engine where there is no central server. Unlike traditional centralized search engines, work such as crawling, data mining, indexing, and query processing is distributed among several peers in decentralized manner where there is no single point of control.

Contents

History

InfraSearch

In April 2000 three programmers built a prototype P2P web search engine based on Gnutella called InfraSearch.[1] It was meant to run inside the participating websites' databases creating a P2P network that could be accessed through the InfraSearch website.[2][3][4]

Opencola

On May 31, 2000 Steelbridge Inc. announced development of OpenCOLA a collaborative distributive open source search engine.[5] It runs on the user's computer and crawls the web pages and links the user puts in their opencola folder and shares resulting index over its P2P network.[6]

YaCy

On December 15, 2003 Michael Christen announced development of a P2P-based search engine, eventually named YaCy, on the heise online forums.[7][8]

FAROO

In February 2001 Wolf Garbe published an idea of a peer-to-peer search engine,[9] started the Faroo prototype in 2004,[10] and released it in 2005.[11][12]

Wowd

Some time in 2006 Borislav Agapiev started thinking about a distributed search engine.[13] Then on October 20, 2009 he publicly launched Wowd.[14]

Majestic-12

In August 2005, Majestic-12 hit version 0.1.5, with 1 billion URLs searched [15]

As of August 2011, Majestic-12 is at version 1.7.5, with 547 billion URLs searched [16]

Seeks

In March 2010 Emmanuel Benazera released the first implementation of the Seeks collaborative distributed search engine.[17][18]

References

  1. ^ Justin Hibbard. "Can peer-to-peer grow up?". Red Herring. http://www.redherring.com/Home/9528. 
  2. ^ Simon Foust. "Move Over Yahoo, Here Comes InfraSearch". 'Dmusic. Archived from the original on 2000-10-13. http://web.archive.org/web/20001013141235/www.dmusic.com/news/news.php?id=2614. 
  3. ^ Sean M. Dugan. "Peer-to-peer networking is poised to revolutionize the Internet once again". InfoWorld. Archived from the original on 2000-10-18. http://web.archive.org/web/20001018022633/http://www.infoworld.com/articles/op/xml/00/07/17/000717opprophet.xml. 
  4. ^ John Borland. "Napster-like technology takes Web search to new level". Cnet. http://news.cnet.com/2100-1023-241223.html. 
  5. ^ David Akin. "Software launched with a little pop". Financial Post. Archived from the original on 2000-06-21. http://web.archive.org/web/20000621132128/http://www.nationalpost.com/financialpost.asp?f=000531/303636.html/17/000717opprophet.xml. 
  6. ^ Paul Heltzel. "OpenCola-Have Some Code and a Smile". Technology Review. http://www.techreview.com/web/12360/?a=f. 
  7. ^ "YaCy: News". Archived from the original on 2005-11-24. http://web.archive.org/web/20051124084140/www.yacy.net/yacy/News.html. 
  8. ^ Michael Christen. "Ich entwickle eine P2P-basierende Suchmaschine. Wer macht mit?". heise online. http://www.heise.de/newsticker/foren/S-Ich-entwickle-eine-P2P-basierende-Suchmaschine-Wer-macht-mit/forum-50682/msg-4744034/read/. 
  9. ^ Wolf Garbe. "BINGOOO - Die Transformation des World Wide Web zur virtuellen Datenbank" (in German). Wirtschaftinformatik (magazine). http://www.pubzone.org/dblp/journals/wi/Garbe01. "... Wir setzen dem das Konzept einer verteilten Peer-to-Peer-Suchmaschine entgegen [We counter with the concept of a distributed peer-to-peer search engine] ..." 
  10. ^ Bernard Lunn. "Technical Q&A With FAROO Founder". ReadWriteWeb. http://www.readwriteweb.com/start/2009/12/technical-qa-with-faroo-founder.php. "... When I started to work on the first prototype in 2004 ..." 
  11. ^ "FAROO: History". Archived from the original on 2008-03-22. http://web.archive.org/web/20080322000927/http://www.faroo.com/english/download/history.html. 
  12. ^ "Revisited: Deriving crawler start points from visited pages by monitoring HTTP traffic". Faroo. http://blog.faroo.com/2010/01/03/revisited-deriving-crawler-start-points-from-visited-pages-by-monitoring-http-traffic/. 
  13. ^ Taylor Buley; Quentin Hardy. "Wowd: Searching The Darkness". Forbes. http://www.forbes.com/forbes/2010/0301/technology-internet-wowd-networks-skype-searching-darkness.html. "... He left in 2006; he saw the popularity of Skype and wanted to do the same with search. ..." 
  14. ^ Borislav Agapiev. "Wowd Public Launch". http://distributedsearch.blogspot.com/2009/10/wowd-public-launch.html. 
  15. ^ "Majestic-12 Archived news". http://www.majestic12.co.uk/archived_news.php. 
  16. ^ "Majestic-12 : DSearch : Stats". http://www.majestic12.co.uk/projects/dsearch/stats.php. 
  17. ^ Emmanuel Benazera. "Seeking help with Seeks". P2PNet. http://www.p2pnet.net/story/42507. 
  18. ^ Lee Schlesinger. "Seeks delivers new search engine paradigm". Sourceforge. http://sourceforge.net/blog/seeks-delivers-new-search-engine-paradigm/. 
v · FAROO · YaCy · Sciencenet · Seeks · Wowd · InfraSearch · Opencola · Alvis
Distributed web crawlers
Grub · Boitho Distributed Crawler · Majestic-12: DSearch
italics = defunct

Wikimedia Foundation. 2010.

Игры ⚽ Нужно решить контрольную?

Look at other dictionaries:

  • Search engine image protection — (SEIP), also known as search engine reputation management, combines the expertise of Search Engine Marketing (SEM) and PR, and is designed to protect a company or brand against incorrect negative publicity via the Internet. By using SEIP… …   Wikipedia

  • Search engine technology — Modern web search engines are complex software systems using the technology that has evolved over the years. The largest search engines such as Google and Yahoo! utilize tens or hundreds of thousands of computers to process billions of web pages… …   Wikipedia

  • Index (search engine) — Search engine indexing collects, parses, and stores data to facilitate fast and accurate information retrieval. Index design incorporates interdisciplinary concepts from linguistics, cognitive psychology, mathematics, informatics, physics, and… …   Wikipedia

  • Web search engine — Search engine redirects here. For other uses, see Search engine (disambiguation). The three most widely used web search engines and their approximate share as of late 2010.[1] A web search engine is designed to search for information on the Wo …   Wikipedia

  • Collaborative search engine — Collaborative Search Engines (CSEs) are an emerging trend for Web search and Enterprise search within company intranets. CSEs let users concert their efforts in information retrieval (IR) activities, share information resources collaboratively… …   Wikipedia

  • Sphinx (search engine) — Infobox Software name = Sphinx caption = author = developer = Andrew Aksyonoff released = 2001 latest release version = 0.9.8 latest release date = release date|2008|07|15 programming language = C++ operating system = Linux, Windows, Solaris,… …   Wikipedia

  • Database search engine — There are several categories of search engine software: Web search or full text search (example: Lucene), database or structured data search (example: Dieselpoint), and mixed or enterprise search (example: Google Search Appliance). The largest… …   Wikipedia

  • Grub (search engine) — Grub is an open source distributed search crawler platform. On July 27, 2007 Jimmy Wales announced that Wikia, Inc., the for profit company developing the open source search engine Wikia Search, had acquired Grub from LookSmart. [… …   Wikipedia

  • Distributed web crawling — is a distributed computing technique whereby Internet search engines employ many computers to index the Internet via web crawling. Such systems may allow for users to voluntarily offer their own computing and bandwidth resources towards crawling… …   Wikipedia

  • Distributed hash table — A distributed hash table (DHT) is a class of a decentralized distributed system that provides a lookup service similar to a hash table; (key, value) pairs are stored in a DHT, and any participating node can efficiently retrieve the value… …   Wikipedia

Share the article and excerpts

Direct link
… Do a right-click on the link above
and select “Copy Link”