Re: AW: [htdig] Going for the big dig

Subject: Re: AW: [htdig] Going for the big dig
Date: Wed Dec 20 2000 - 14:30:59 PST

At 9:35 AM +0100 12/20/00, Reich, Stefan wrote:
>Not quite sure if this helps, but maybe ;-)
>If I'm right, is running on Notes Domino something servers.
>We've experienced lots of problems with this notes servers, because of their
>meshed link structure.

I did some testing on this a while back and recommend that robots
should ignore all URLs that contain but do not end in "OpenDocument"
and "OpenView&CollapseView" and ignore all URLs that contain
"ExpandView" or "OpenView&Start.

Hope it helps,


Complete Guide to Search Engines for Web Sites, Intranets, 
   and Portals: <>

------------------------------------ To unsubscribe from the htdig mailing list, send a message to You will receive a message to confirm this. List archives: <> FAQ: <>

This archive was generated by hypermail 2b28 : Wed Dec 20 2000 - 14:51:26 PST