Re: htdig: htdig not indexing server


webmaster@www.nisu.flinders.edu.au
Fri, 10 Jul 1998 10:26:15 +0930 (CST)


On 9 Jul, James Slater wrote:
> Hi,
>
> Just installed ht://dig & I'm trying to index an internal web server.
> However, on running htdig i get:
>
> #./htdig -vis
>
> New server: <web server>, 80
> 0:0:0:http://<web server>/: not found
> htdig: Run complete
> htdig: 1 server seen:
> htdig: <web server>:80 1 document
>
> Now, if i add a couple more v's, I can see that it's picking up the
> index.htm document, reads it, but doesn't do anything with it. the
> server itself can be ping'd quite happily, so is there a problem with
> using servers with index.htm as their root document? I can index another
> internal web server fine. I've changed the web server name to <web
> server> as we're behind a firewall ;)
>
> Cheers!
>
> James Slater.

This is going back a while, but I remember having a similar problem
which I _think_ I solved by using the full URL of the start point in
the config file. And this meant I had to use a specific entry for
limit_urls_to, thus:

#
start_url: http://www.nisu.flinders.edu.au/navpage.html
#
# This attribute limits the scope of the indexing process. The default is to
# set it to the same as the start_url above. This way only pages that are on
# the sites specified in the start_url attribute will be indexed and it will
# reject any URLs that go outside of those sites.
#
# Keep in mind that the value for this attribute is just a list of string
# patterns. As long as URLs contain at least one of the patterns it will be
# seen as part of the scope of the index.
#
limit_urls_to: http://www.nisu.flinders.edu.au
# ${start_url}

This quite cheerfully indexes my site.

Cheers

-- 
David Robley

WEBMASTER | Phone +61 8 8374 0970 RESEARCH CENTRE FOR INJURY STUDIES | http://www.nisu.flinders.edu.au/ AusEinet | http://auseinet.flinders.edu.au/ Flinders University, ADELAIDE, SOUTH AUSTRALIA Visit the PHP mirror at http://au.php.net:81/

---------------------------------------------------------------------- To unsubscribe from the htdig mailing list, send a message to htdig-request@sdsu.edu containing the single word "unsubscribe" in the body of the message.



This archive was generated by hypermail 2.0b3 on Sat Jan 02 1999 - 16:26:52 PST