Re: [htdig] pause


Subject: Re: [htdig] pause
From: Franck Horlaville (fhlist@online.co.ma)
Date: Thu May 25 2000 - 05:43:38 PDT


>>I'd like to pause the htdig crawl and restart it where it left off, is
>>there any way to do this ?
>
>Sure. Pass the -l flag when you call htdig and it will write out the
>"todo" list of URLs before it quits. Then when you restart with this
>flag, it will read this list in and resume.

thanks - I guess I had missed it in the docs ... Which btw don't mention
what signal is to be sent to interrupt : a simple kill or a kill -s INT ?

Next question : how do I know when the crawl has finished to do merge etc
?

My idea :

repeat forever
do
start crawl at 1.15 am (using a work db)
pause crawl at 6.15 am
until done
merge databases
end repeat

thanks a bunch

Franck Horlaville,

Technical Director - Athena Online s.a.
Web site creation and hosting

--
<http://www.athena.online.co.ma/>

------------------------------------ To unsubscribe from the htdig mailing list, send a message to htdig-unsubscribe@htdig.org You will receive a message to confirm this.



This archive was generated by hypermail 2b28 : Thu May 25 2000 - 03:30:18 PDT