Subject: Re: [htdig] won't make url encoded queries?
From: Torsten Neuer (email@example.com)
Date: Thu Nov 11 1999 - 23:42:09 PST
Chris Gough wrote:
> Geoff Hutchison wrote:
> > At 2:47 PM +1100 11/12/99, Chris Gough wrote:
> > >It seems my new htdig is not following links with "url encoded"
> > >addresses, such as
> > >http://18.104.22.168/WebSearch/program.php3?id=DP-AJ66NA
> > There's a patch on the website:
> > <http://www.htdig.org/files/contrib/other/htdig-3.1.3-urlparmbug.patch>
> To me it reads as though that patch is for a bug that turns
> "..?foo=123&bar=456" into "..?foo=456". My URL has no "&" in it. Have i
> misunderstood the scope of the bug? Could their be some other reason why
> htdig is not following the links?
You're right - the bug is about multiple parameters. But there
might also be some side effects which also affect single parameters.
To be sure, it is not the case, I'd suggest you apply the patch
Second, you should check "robots.txt", the "robots" META tag and
the settings of "bad_querystr" and "exclude_urls" in your Ht://Dig
Third, Ht://Dig won't follow the links if they are written using
if they are triggered by <FORM>s (I have a couple of pages where
I use this to exclude pages from being indexed).
-- InWise - Wirtschaftlich-Wissenschaftlicher Internet Service GmbH Waldhofstraße 14 Tel: +49-4101-403605 D-25474 Ellerbek Fax: +49-4101-403606 E-Mail: firstname.lastname@example.org Internet: http://www.inwise.de
------------------------------------ To unsubscribe from the htdig mailing list, send a message to email@example.com containing the single word unsubscribe in the SUBJECT of the message.
This archive was generated by hypermail 2b25 : Thu Nov 11 1999 - 23:52:58 PST