RE: [htdig] Exclude_urls


Subject: RE: [htdig] Exclude_urls
From: bs@hi.is
Date: Fri Mar 24 2000 - 03:48:41 PST


These 6000 are student pages which I dont want showing up in the results.
If I exclude all ~ I am excluding some 2000 pages for staff and others that
I want to show up.
So any help is appreciated.

11:33 24.03.00 +0000 skrifaši Budd, S ...
>could you just
>exclude_urls: ~
>
>> -----Original Message-----
>> From: bs@hi.is [SMTP:bs@hi.is]
>> Sent: Friday, March 24, 2000 10:54 AM
>> To: htdig@htdig.org
>> Subject: [htdig] Exclude_urls
>>
>> Hi.
>> I'm experiencing some mayor difficulties trying to exclude some 6000 urls
>> to userpages for my local domain.
>> The urls vary so I generate a list before every dig which I then include
>> into the conf I am using. First off I tried putting each url on a
>> different
>> line so the conf file looked like this :
>>
>> exclude_urls: cgi-bin
>> exclude_urls: ~foo
>> exclude_urls: ~bar
>> ..
>> exclude_urls: ~jon
>>
>> Since the dig using this conf only used the last exclude_urls: I put
>> everything on one line making it look like this:
>>
>> exclude_urls: cgi-bin ~foo ~bar ... ~jon
>>
>> A dig that normally took 4 hours is now still going after 14 hours. A
>> request to /cgi-bin/htsearch is now hogging the system for a long time and
>> not returning anything to the user. I tried removing all the files undir
>> db/ but the /cgi-bin/htsearch never reaches that far as to report an error
>> about the db not being there.
>> Since everything pointed to problems with the conf I switched it out for
>> an
>> old one that did not have all these excludes and everything is back to
>> normal .. at least the /cgi-bin/htsearch is not locking up and hogging the
>> system.
>> So I guess my questions are : Can I exclude these 6000 user urls ? If so,
>> how ?
>>
>> Thanks.
>> --------------------------------
>> B. Stefansson
>> Computing Services
>> University of Iceland
>> bs@hi.is
>>
>> ------------------------------------
>> To unsubscribe from the htdig mailing list, send a message to
>> htdig-unsubscribe@htdig.org
>> You will receive a message to confirm this.
>

------------------------------------
To unsubscribe from the htdig mailing list, send a message to
htdig-unsubscribe@htdig.org
You will receive a message to confirm this.



This archive was generated by hypermail 2b28 : Fri Mar 24 2000 - 02:41:22 PST