Re: [htdig] Redigging - Don't redig urls in DB


Nicolas.Poizot@alcatel.fr
Thu, 12 Aug 1999 10:00:36 +0200


>
> Tim Perdue wrote:
> > It would be really nice, on an update dig, if htdig would not re-hit
> > pages that are already in its index. This creates a total nightmare
on
> > large sites when you are trying to do an update dig.
>
> Yes, we've heard this request. There's code in the 3.2 tree that does
> this, so it will be in the next release.
>
Super it will be very nice :-)
I have another asking on this case.
I use the hack of Tim. I make a new database for the new site and after
i'm merging with the current database. It's ok for the maximum of
documents. But in such cases, some documents are marked Invalid and so
removed of the database. But the question is:
What is the reason for a document that it was marked "invalid"? I have
parsed the code html of this document and i don't see why...

So htdig is very nice product. I like particulary the maximum possibility
of configuration. It's very flexible :-) and very simple to configure.

Nicolas Poizot

> --
> -Geoff Hutchison
> Williams Students Online
> http://wso.williams.edu/
>
> ------------------------------------
> To unsubscribe from the htdig mailing list, send a message to
> htdig@htdig.org containing the single word unsubscribe in
> the SUBJECT of the message.
>

------------------------------------
To unsubscribe from the htdig mailing list, send a message to
htdig@htdig.org containing the single word unsubscribe in
the SUBJECT of the message.



This archive was generated by hypermail 2.0b3 on Thu Aug 12 1999 - 01:01:40 PDT