Subject: Re: [htdig] Strategy for a dynamic site
From: Geoff Hutchison (ghutchis@wso.williams.edu)
Date: Sat Jan 20 2001 - 09:01:12 PST
At 2:45 PM -0800 1/18/01, Richard Seymour wrote:
>I was thinking of generating a few bogus pages that provided links to
>all the content that can't be accessed using standard links, and
>generating these bogus pages on the fly scheduled just before the htdig
>indexing runs. Is this a good strategy?
This is pretty reasonable. If you don't want to index the "bogus
pages," make sure they have META robots tags in them, e.g.
<META name="robots" content="follow,noindex">
-- -Geoff Hutchison Williams Students Online http://wso.williams.edu/------------------------------------ To unsubscribe from the htdig mailing list, send a message to htdig-unsubscribe@htdig.org You will receive a message to confirm this. List archives: <http://www.htdig.org/mail/menu.html> FAQ: <http://www.htdig.org/FAQ.html>
This archive was generated by hypermail 2b28 : Sat Jan 20 2001 - 09:29:41 PST