[htdig] Re: [htdig3-dev] Failing to get any pages indexed . .


Subject: [htdig] Re: [htdig3-dev] Failing to get any pages indexed . .
From: J. op den Brouw (MSQL_User@st.hhs.nl)
Date: Fri Mar 10 2000 - 05:27:08 PST


Are the pages in correct HTML syntax? I once had a the problem
that someone over here wrote bad HTML; netscape presented it
"correctly" whereas htdig failed to index the (bottom of the)
page where all the links were (the last ... lines where seen
as comment because of a bad comment start).

Also use htdig -vvv to see more what htdig is doing.....

Sphboc@aol.com wrote:
>
> I've been getting, from several different sites, messages of this type;
> manual inspection of the site reveals that there are in fact pages present.
> (and, there's no evidence of any robots.txt existing) . .
>
> Does anyone have any insights?

--jesse
----------------------------------------------------------------------
J. op den brouw Johanna Westerdijkplein 75
Haagse Hogeschool 2521 EN DEN HAAG
Faculty of Engineering The Netherlands
Electrical Engineering +31-70-4458936
----------------------------------------------------------------------
           Linux - because reboots are for hardware changes

------------------------------------
To unsubscribe from the htdig mailing list, send a message to
htdig-unsubscribe@htdig.org
You will receive a message to confirm this.



This archive was generated by hypermail 2b28 : Fri Mar 10 2000 - 05:32:43 PST