[htdig] httpd Internal Server Error


Subject: [htdig] httpd Internal Server Error
From: Greg Lepore (gregl@mdarchives.state.md.us)
Date: Wed Jul 12 2000 - 07:43:11 PDT


        OK here goes,

        I am getting an Internal Server Error from Apache when attempting to run a
search that returns a very large number of results (i.e. 25,000 pages and
up). Same search run at the command line executes perfectly. When the
number of indexed pages was less (ca. 80,000) the search worked, but now
that the number of pages is over 110,000 the Error occurs. The messages
file says "Premature end of script headers" and that is it.
        Running the search at the command line takes around 8 or 10 seconds; it
appears that Apache is giving the error after timing out waiting for a
response. I have set this timeout as high as 3 or 4 minutes. The machine
is not under a whole lot of strain via http.
        Question: Should sending the results from HTDIG to Apache be taking
several minutes longer than the search itself at the command line?

        Config:
        PII 400 with 128MB Ram on RedHat 6.1, HTDIG 3.1.3(I know it's a little
old), database size is 200-300MB

############HTDIG CONFIG FILE##################
        database_dir: /usr/www/htdig/db
start_url:http://www.mdarchives.state.md.us/megafile/msa/speccol/sc2900/sc29
08/000001/000001/html/am1--94.html
limit_urls_to:http://www.mdarchives.state.md.us/megafile/msa/speccol/sc2900/
sc2908/
exclude_urls: /cgi-bin/ .cgi .pdf .tif
max_head_length: 50000
max_doc_size: 200000000
search_algorithm: exact:1 synonyms:0.5
logging: true

allow_in_form: search_algorithm
common_url_parts: http://www.mdarchives.state.md.us/megafile/msa/speccol/
compression_level: 6
remove_bad_urls: false
allow_numbers: true
heading_factor_6: 1
create_url_list: false
backlink_factor: 0
timeout: 240
minimum_word_length: 3
local_urls:http://www.mdarchives.state.md.us/megafile/msa/speccol/sc2900/sc2
908/=/megafile/msa/speccol/sc2900/sc2908/
local_default_doc:index.html

Gregory Lepore
Maryland Electronic Capital Webmaster
410-260-6425
gregl@mdarchives.state.md.us

------------------------------------
To unsubscribe from the htdig mailing list, send a message to
htdig-unsubscribe@htdig.org
You will receive a message to confirm this.



This archive was generated by hypermail 2b28 : Wed Jul 12 2000 - 05:01:42 PDT