Re: [htdig] Meta data

Subject: Re: [htdig] Meta data
From: Geoff Hutchison (
Date: Sat Jul 22 2000 - 16:06:54 PDT

At 3:22 PM +0100 7/7/00, Abdul Jabbar wrote:
>Is there anyway to program Htdig to only search for Metadata and not index
>whole websites?. Some of the websites that I index Htdig absoultely
>cripples them is there anyway to slow down the relentless pressures? i.e.
>limit the number of hops?, please help. Thanks in advance.

Gilles said just about everything, but I'll also point out that just
indexing metadata won't make any difference in how hard the server is
hit. After all, it has to download the document anyway before it can
get the metadata. There's no way using HTTP to just get the META
information since this can take up an arbitrary amount of the

-Geoff Hutchison
Williams Students Online

------------------------------------ To unsubscribe from the htdig mailing list, send a message to You will receive a message to confirm this.

This archive was generated by hypermail 2b28 : Sat Jul 22 2000 - 06:10:55 PDT