Re: AW: [htdig] Going for the big dig


Subject: Re: AW: [htdig] Going for the big dig
From: nets@searchtools.com
Date: Wed Dec 20 2000 - 14:30:59 PST


At 9:35 AM +0100 12/20/00, Reich, Stefan wrote:
>Not quite sure if this helps, but maybe ;-)
>
>If I'm right, Lotus.com is running on Notes Domino something servers.
>
>We've experienced lots of problems with this notes servers, because of their
>meshed link structure.

I did some testing on this a while back and recommend that robots
should ignore all URLs that contain but do not end in "OpenDocument"
and "OpenView&CollapseView" and ignore all URLs that contain
"ExpandView" or "OpenView&Start.

Hope it helps,

Avi

-- 
_________________________________________________
Complete Guide to Search Engines for Web Sites, Intranets, 
   and Portals: <http://www.searchtools.com>

------------------------------------ To unsubscribe from the htdig mailing list, send a message to htdig-unsubscribe@htdig.org You will receive a message to confirm this. List archives: <http://www.htdig.org/mail/menu.html> FAQ: <http://www.htdig.org/FAQ.html>



This archive was generated by hypermail 2b28 : Wed Dec 20 2000 - 14:51:26 PST