[htdig] Duplicate pages


Subject: [htdig] Duplicate pages
From: anr@att.net
Date: Tue Sep 19 2000 - 09:55:31 PDT


The site I am indexing is a bit peculiar. The following
is an example of the setup, where each page is exactly
the same.

www.domain.com/subdirectory/
www.domain.com/subdirectory/index.html
www.domain.com/Subdirectory/
www.domain.com/Subdirectory/index.html

I assumed that in the case where there is no index.html
that it was just loading the index.html. Here's the
problem. htdig recognizes this as 4 different pages,
and indexes all of them. I can see where it would think
it is 2 different because of the s and S. Is there any
way to prevent the duplicates?

Thanks!
Adam

------------------------------------
To unsubscribe from the htdig mailing list, send a message to
htdig-unsubscribe@htdig.org
You will receive a message to confirm this.
List archives: <http://www.htdig.org/mail/menu.html>
FAQ: <http://www.htdig.org/FAQ.html>



This archive was generated by hypermail 2b28 : Tue Sep 19 2000 - 09:58:44 PDT