Re: [htdig] Strategy for a dynamic site

Subject: Re: [htdig] Strategy for a dynamic site
From: Geoff Hutchison (
Date: Sat Jan 20 2001 - 09:01:12 PST

At 2:45 PM -0800 1/18/01, Richard Seymour wrote:
>I was thinking of generating a few bogus pages that provided links to
>all the content that can't be accessed using standard links, and
>generating these bogus pages on the fly scheduled just before the htdig
>indexing runs. Is this a good strategy?

This is pretty reasonable. If you don't want to index the "bogus
pages," make sure they have META robots tags in them, e.g.
<META name="robots" content="follow,noindex">

-Geoff Hutchison
Williams Students Online

------------------------------------ To unsubscribe from the htdig mailing list, send a message to You will receive a message to confirm this. List archives: <> FAQ: <>

This archive was generated by hypermail 2b28 : Sat Jan 20 2001 - 09:29:41 PST