htdig: Ongoing tribulation with htwrapper script...

John Grohol PsyD (
Fri, 02 Oct 1998 17:29:30 -0400

Maybe I wasn't clear enough in my first e-mail, because
what I was looking for were specifics, not generalities.
I basically know what I *need* to do, it's just not working!

A lot of external sites, with our encouragement, use our
internal search engine on their Web site to help people
find information in our field (mental health). This
search engine (EWS) uses form variables which htsearch
does not use nor does it understand. Therefore I need a
wrapper script in order to convert the EWS form variables
into htsearch form variables. This should be a simple thing.

I need to write the wrapper in perl, because that's the
language I know and have installed (as do most Web sites).
I don't know PHP and don't have it running on our server.

I installed Andrew Bishop's patches to htsearch to allow
it to read in a configuration file from the command line.
This works nicely and *everything* works absolutely
smashingly from the command line. All searches return
expected results.

However, when I run the exact same script from my Web
browser, it returns the nomatch.html file and the
query string is empty. Running with the -vvvv option
shows all strings to be empty.

What I did:
I wrote a perl script to read in the old EWS search form,
and then write out the search terms to a temporary file
($lockfile). Then I throw that file at htsearch. Again,
this works fine from the command line. This is what I
use to call htsearch:

system("/home/lib/httpd/cgi-bin/htsearch -vvvv -c /etc/htdig2.conf < $lockfile");

I tried placing a line above this which sets the QUERY_STRING:

$ENV{'QUERY_STRING'} = "config=htdig2&words=depression&method=and&format=mhfind";

but I don't think that works that way.... or at least it
didn't seem to be working.

Full perl script is attached. $20 for anybody who
solves my mystery! :-)


---------------------- wrap.cgi --------------------------
#!/usr/bin/perl -w

$| = 1;

# Get the form variables

   read(STDIN, $buffer, $ENV{'CONTENT_LENGTH'});
   @pairs = split(/&/, $buffer);

   foreach $pair (@pairs) {
      ($name, $value) = split(/=/, $pair);
      $value =~ tr/+/ /;
      $value =~ s/%([a-fA-F0-9][a-fA-F0-9])/pack("C", hex($1))/eg;
      $value =~ s/<!--(.|\n)*-->//g;
      $value =~ s/<([^>]|\n)*>//g;
      $tags{$name} = $value;

# Create a random lockfile

        $lockfile = "";
        srand(time ^ $$);
        $lockfile = int(rand(60000));
        $lockfile = "/home/lib/httpd/cgi-bin/temp/$lockfile";

# Write the search query to the lockfile
# (For testing purposes, the query is static)
# Normally we would write out $tags{'search'}

        open(NUM,">$lockfile") || die $!;
        print NUM "depression\n";
        print NUM "mhfind\n";

# Call htsearch and throw the search query at it (from the lockfile)
# Actually, whether you comment the following line out or not doesn't
# seem to matter. Perhaps you can't set ENV variables in perl like this?

$ENV{'QUERY_STRING'} = "config=htdig2&words=depression&method=and&format=mhfind";

# Tried both, doesn't seem to matter?
# exec "/home/lib/httpd/cgi-bin/htsearch -c /etc/htdig2.conf < $lockfile";

system("/home/lib/httpd/cgi-bin/htsearch -c /etc/htdig2.conf < $lockfile");

# Remove the lockfile
# `rm $lockfile`;


To unsubscribe from the htdig mailing list, send a message to containing the single word "unsubscribe" in
the body of the message.

This archive was generated by hypermail 2.0b3 on Sat Jan 02 1999 - 16:28:28 PST