Yahoo! Slurp is a web crawler

Web crawler

A Web crawler is a computer program that browses the World Wide Web in a methodical, automated manner or in an orderly fashion. Other terms for Web crawlers are ants, automatic indexers, bots, Web spiders, Web robots, or—especially in the FOAF community—Web scutters.This process is called Web...

 from Yahoo!

Yahoo!

Yahoo! Inc. is an American multinational internet corporation headquartered in Sunnyvale, California, United States. The company is perhaps best known for its web portal, search engine , Yahoo! Directory, Yahoo! Mail, Yahoo! News, Yahoo! Groups, Yahoo! Answers, advertising, online mapping ,...

 that obtains content for the Yahoo! Search

Yahoo! Search

Yahoo! Search is a web search engine, owned by Yahoo! Inc. and was , the 2nd largest search engine on the web by query volume, at 6.42%, after its competitor Google at 85.35% and before Baidu at 3.67%, according to Net Applications....

 engine. Slurp is based on search technology Yahoo! acquired when it purchased Inktomi

Inktomi

Inktomi Corporation was a California company that provided software for Internet service providers. It was founded in 1996 by UC Berkeley professor Eric Brewer and graduate student Paul Gauthier. The company was initially founded based on the real-world success of the search engine they developed...

.

Slurp identifies itself by using the following User agent

User agent

In computing, a user agent is a client application implementing a network protocol used in communications within a client–server distributed computing system...

 strings:

* Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)
* Mozilla/5.0 (compatible; Yahoo! Slurp China; http://misc.yahoo.com.cn/help.html)

External links

• Yahoo! Search Resources for Webmasters
• Yahoo! Slurp's official FAQ
• How can I reduce the number of requests you make on my web site?
• Yahoo! Site Explorer is a webmaster
  Webmaster
  A webmaster , also called a web architect, web developer, site author, or website administrator is a person responsible for maintaining one or many websites...
  
  tool that lets you access the information Yahoo! has about a site's online presence. You can see which sites and subpages are indexed by Yahoo! Search, track sites that link into web pages, and view the most popular pages from any site. More...
• Yahoo! Search Blog
• Yahoo! Weather Report Blog Posts