Re: [htdig] Strategy for a dynamic site


Subject: Re: [htdig] Strategy for a dynamic site
From: Geoff Hutchison (ghutchis@wso.williams.edu)
Date: Sat Jan 20 2001 - 09:01:12 PST


At 2:45 PM -0800 1/18/01, Richard Seymour wrote:
>I was thinking of generating a few bogus pages that provided links to
>all the content that can't be accessed using standard links, and
>generating these bogus pages on the fly scheduled just before the htdig
>indexing runs. Is this a good strategy?

This is pretty reasonable. If you don't want to index the "bogus
pages," make sure they have META robots tags in them, e.g.
<META name="robots" content="follow,noindex">

--
-Geoff Hutchison
Williams Students Online
http://wso.williams.edu/

------------------------------------ To unsubscribe from the htdig mailing list, send a message to htdig-unsubscribe@htdig.org You will receive a message to confirm this. List archives: <http://www.htdig.org/mail/menu.html> FAQ: <http://www.htdig.org/FAQ.html>



This archive was generated by hypermail 2b28 : Sat Jan 20 2001 - 09:29:41 PST