Re: [htdig] "Deleted, invalid" messages solved


Subject: Re: [htdig] "Deleted, invalid" messages solved
From: Geoff Hutchison (ghutchis@wso.williams.edu)
Date: Wed Jul 26 2000 - 10:02:00 PDT


On Wed, 26 Jul 2000, Gilles Detillieux wrote:

> According to D.J.Adams@soton.ac.uk:
> > Now I have to investigate why certain pages are flagged as
> > "Deleted, noexcerpt"!
>
> Main causes:
> - disallowed in robots.txt
> - indexing turned off by meta robots or noindex tags
> - no indexable text in documents
> - server_max_docs exceeded

Also when merging:
- duplicates between the two databases (oldest is removed)

--
-Geoff Hutchison
Williams Students Online
http://wso.williams.edu/

------------------------------------ To unsubscribe from the htdig mailing list, send a message to htdig-unsubscribe@htdig.org You will receive a message to confirm this.



This archive was generated by hypermail 2b28 : Wed Jul 26 2000 - 00:00:23 PDT