Re: htdig: htdig not indexing server
Fri, 10 Jul 1998 10:26:15 +0930 (CST)

On 9 Jul, James Slater wrote:
> Hi,
> Just installed ht://dig & I'm trying to index an internal web server.
> However, on running htdig i get:
> #./htdig -vis
> New server: <web server>, 80
> 0:0:0:http://<web server>/: not found
> htdig: Run complete
> htdig: 1 server seen:
> htdig: <web server>:80 1 document
> Now, if i add a couple more v's, I can see that it's picking up the
> index.htm document, reads it, but doesn't do anything with it. the
> server itself can be ping'd quite happily, so is there a problem with
> using servers with index.htm as their root document? I can index another
> internal web server fine. I've changed the web server name to <web
> server> as we're behind a firewall ;)
> Cheers!
> James Slater.

This is going back a while, but I remember having a similar problem
which I _think_ I solved by using the full URL of the start point in
the config file. And this meant I had to use a specific entry for
limit_urls_to, thus:

# This attribute limits the scope of the indexing process. The default is to
# set it to the same as the start_url above. This way only pages that are on
# the sites specified in the start_url attribute will be indexed and it will
# reject any URLs that go outside of those sites.
# Keep in mind that the value for this attribute is just a list of string
# patterns. As long as URLs contain at least one of the patterns it will be
# seen as part of the scope of the index.
# ${start_url}

This quite cheerfully indexes my site.


David Robley

WEBMASTER | Phone +61 8 8374 0970 RESEARCH CENTRE FOR INJURY STUDIES | AusEinet | Flinders University, ADELAIDE, SOUTH AUSTRALIA Visit the PHP mirror at

---------------------------------------------------------------------- To unsubscribe from the htdig mailing list, send a message to containing the single word "unsubscribe" in the body of the message.

This archive was generated by hypermail 2.0b3 on Sat Jan 02 1999 - 16:26:52 PST