[htdig] problem limiting urls

Subject: [htdig] problem limiting urls
From: Chad Cunningham (ccunning@math.ohio-state.edu)
Date: Sat Jan 15 2000 - 11:57:20 PST

I've been trying to figure a way around this with no luck. I want to use
htdig to index a message board. The problem is that I only want the read
page containing the actual messages indexed, and not the main listing
page. If I set limit_urls_to to read.php, it does this fine, but only
for the first page. The rest of the pages are all the same url index.php
with different query parameters. But, if I tell it to also limit urls to
index.php, it indexes all the pages but also indexes index.php itself,
when I just want the read.php pages listed on index.php to be searched.
I hope that makes sense... Basically, how can I get htdig to follow a
link without indexing the page the link is to, instead linking only
valid pages linked to on that page?

The only thing I have come up with is making a single page that links to
all the messages, but there are about 130,000 messages and that would be
quite a page, so I'd prefer to find another way to do it if possible.


Chad Cunningham ccunning@math.ohio-state.edu

"I'll tell you what kind of guy I was. If you ordered a boxcar full of sons-of-bitches and opened the door and only found me inside, you could consider the order filled."

-Robert Mitchum

------------------------------------ To unsubscribe from the htdig mailing list, send a message to htdig-unsubscribe@htdig.org You will receive a message to confirm this.

This archive was generated by hypermail 2b28 : Sat Jan 15 2000 - 11:57:02 PST