Re: [htdig] MODIFIED date


Subject: Re: [htdig] MODIFIED date
From: Torsten Neuer (tneuer@inwise.de)
Date: Tue Sep 26 2000 - 10:16:38 PDT


Roger Weiss wrote:
>
> OK, I'll bite. HOW?
>
> For background:
> Before running htdig, I run a perl script which creates an html file that
> contains links (in the form of hrefs) to all the index.html files in my
> domain.

It is much more convenient to have the start URLs listed in the
start_url
tag of the configuration file. You can include an external file for this
purpose.

> Htdig then starts at this file and follows the links to generate the db.
> Is that technique getting in my way at all?
> Or do I just need to change a parm in my apache server?

What kind of pages are you serving? Pure HTML or server-parsed stuff?

If the latter is the case, you have to emit the proper HTTP header
through
your scripts. Emitting a Last-Modified header from your script varies
with
the script engine you use. If instead your pages are pure HTML,
everything
should be fine without changing anything - except if you have
mod_usertrack
handling compiled in and activated, in which case the server will issue
cookies to tracks users which might cause the Last-Modified header not
to
be emitted by Apache.

What does the average HEAD request return? (Try telnet'ing the server on
port 80 and issue some HEAD request to see what HTTP headers are sent by
the server).

cheers,

  Torsten

-- 
InWise - Wirtschaftlich-Wissenschaftlicher Internet Service GmbH
Waldhofstraße 14                            Tel: +49-4101-403605
D-25474 Ellerbek                            Fax: +49-4101-403606
E-Mail: info@inwise.de            Internet: http://www.inwise.de

------------------------------------ To unsubscribe from the htdig mailing list, send a message to htdig-unsubscribe@htdig.org You will receive a message to confirm this. List archives: <http://www.htdig.org/mail/menu.html> FAQ: <http://www.htdig.org/FAQ.html>



This archive was generated by hypermail 2b28 : Tue Sep 26 2000 - 10:20:18 PDT