Re: [htdig] problem limiting urls

Subject: Re: [htdig] problem limiting urls
From: Gilles Detillieux (
Date: Mon Jan 17 2000 - 11:04:11 PST

According to Chad Cunningham:
> I've been trying to figure a way around this with no luck. I want to use
> htdig to index a message board. The problem is that I only want the read
> page containing the actual messages indexed, and not the main listing
> page. If I set limit_urls_to to read.php, it does this fine, but only
> for the first page. The rest of the pages are all the same url index.php
> with different query parameters. But, if I tell it to also limit urls to
> index.php, it indexes all the pages but also indexes index.php itself,
> when I just want the read.php pages listed on index.php to be searched.
> I hope that makes sense... Basically, how can I get htdig to follow a
> link without indexing the page the link is to, instead linking only
> valid pages linked to on that page?
> The only thing I have come up with is making a single page that links to
> all the messages, but there are about 130,000 messages and that would be
> quite a page, so I'd prefer to find another way to do it if possible.

The only solution I can think of would be to add meta tags to the index.php
documents, as described in

Gilles R. Detillieux              E-mail: <>
Spinal Cord Research Centre       WWW:
Dept. Physiology, U. of Manitoba  Phone:  (204)789-3766
Winnipeg, MB  R3E 3J7  (Canada)   Fax:    (204)789-3930

------------------------------------ To unsubscribe from the htdig mailing list, send a message to You will receive a message to confirm this.

This archive was generated by hypermail 2b28 : Mon Jan 17 2000 - 11:04:40 PST