[htdig] Fw: [htdig] problem with php links

Subject: [htdig] Fw: [htdig] problem with php links
From: fx (fxbois@cybercable.fr)
Date: Tue Feb 01 2000 - 06:17:04 PST

> > I want that htdig indexes my news'page
> > unfortunately it only indexes url like "http://www.toto.com/news.php3"
> > and never url like "http://www.toto.com/news.php3?id=2"
> > is there any way to tell to htdig to through that kind of url
> >
> > second problem : htdig don't take links like that : <a
> > (no problem if I write <a href="/activites/index.php3" > )
> htdig shouldn't have any problems with these URLs unless you actually
> configure it to exclude them. Version 3.1.0b2 (about 14 month ago) had
> problems with URLs containing "?", but other versions should allow them.
> Directory URLs like <a href="/activites"> should get passed back
> to the HTTP server (unless you're using local_urls_only in 3.1.4,
> which you shouldn't be if you want to index dynamic content like
> PHP). The HTTP server will just redirect http://server/activites to
> http://server/activites/ (i.e. it will add a trailing slash) to get
> a proper directory URL. If your HTTP server is configured to allow
> index.php3 as a directory index, then the redirect should lead htdig
> right to the document you want.
> If you can't get it to work, and you've ruled out any server configuration
> problems, try "htdig -i -vvv" and look through the output to see why the
> URLs are being rejected.

it still doesn t work ... I include my conf and a bit of the output of
"htdig -i -vvv"
(I use 3.1.4 and have no problem with apache )

database_dir: /home/web/inerd/htdig/db
database_base: ${database_dir}/inerd
#allow_virtual_hosts: true
valid_extensions: .html .htm .shtml .php .php3 .asp .php
exclude_urls: /cgi-bin/ .cgi
bad_extensions: .wav .gz .z .sit .au .zip .tar .hqx .exe .com .gif\
   .jpg .jpeg .aiff .class .map .ram .tgz .bin .rpm .mpg .mov .avi
maintainer: inerd
max_head_length: 10000
max_doc_size: 200000
no_excerpt_show_top: false
search_algorithm: exact:1 synonyms:0.5 endings:0.1
search_results_wrapper: /home/web/inerd/www/htdig/wrapper_inerd.html
nothing_found_file: /home/web/inerd/www/htdig/nomatch_inerd.html

+A tag: pos = 2, position = =/news/index.php3?idnews=3 class=news>
href: (La troisième)

   Rejected: Extension is not valid!

*A tag: pos = 2, position = ="/services" class="navig1">
href: (services)

   Rejected: Extension is not valid!

do you have any suggestion ?
(I ve really tried a lot of things ... )


To unsubscribe from the htdig mailing list, send a message to
You will receive a message to confirm this.

This archive was generated by hypermail 2b28 : Tue Feb 01 2000 - 06:19:50 PST