[htdig] Re: * Web Crawler Help!** (PR#982)

Subject: [htdig] Re: * Web Crawler Help!** (PR#982)
From: Geoff Hutchison (ghutchis@wso.williams.edu)
Date: Sat Dec 23 2000 - 09:56:18 PST

On Fri, 22 Dec 2000 mikemillin123@yahoo.com wrote:

> Ok... So the way I see it is that I am going to use a
> index robot and tell it to index the 6 websites and
> give me a data file. Then I'll tweak the search

Install ht://Dig and set the start_url to point at all six websites.
(Despite the name, it can take more than one URL). Index. Use htsearch and
you'll have an index of all six sites--if you want to restrict the search,
use the "restrict" and/or "exclude" fields in the search form.

-Geoff Hutchison
Williams Students Online

------------------------------------ To unsubscribe from the htdig mailing list, send a message to htdig-unsubscribe@htdig.org You will receive a message to confirm this. List archives: <http://www.htdig.org/mail/menu.html> FAQ: <http://www.htdig.org/FAQ.html>

This archive was generated by hypermail 2b28 : Sat Dec 23 2000 - 10:07:30 PST