Subject: Re: [htdig3-dev] Failing to get any pages indexed . .
From: Sphboc@aol.com
Date: Fri Mar 10 2000 - 09:56:50 PST
True cause-and-effect, after a little more testing, appears to be similar to:
A. FULL TEXT of each limit-urls-to entry is used for matching; INCLUDING any
"http://www: prefix(es).
B. Any urls in which the high-order node(s) is/are not identical to the full
text fail to match.
Cure appears to be to specify only "smarterkids.com" in a
separate entry.
By the way, is the matching case-sensitive? URL's themselves are not, but
how is matching accomplished?
In a message dated 3/10/00 6:37:36 AM US Mountain Standard Time,
ghutchis@wso.williams.edu writes:
<< At 3:10 AM -0500 3/10/00, Sphboc@aol.com wrote:
>New server: smarterkids.com, 80
>0:0:0:http://smarterkids.com/: redirect
>htdig: Run complete
>htdig: 1 server seen:
>htdig: smarterkids.com:80 1 document
In this example, there's a redirect to www.smarterkids.com. But I'll
guess that this is not in limit_urls_to, so it's rejected.
-Geoff Hutchison
Williams Students Online
http://wso.williams.edu/
>>
------------------------------------
To unsubscribe from the htdig3-dev mailing list, send a message to
htdig3-dev-unsubscribe@htdig.org
You will receive a message to confirm this.
This archive was generated by hypermail 2b28 : Fri Mar 10 2000 - 10:02:47 PST