[htdig] Somethin' Stupid


Subject: [htdig] Somethin' Stupid
From: Fanac Webmaster (fanac@fanac.org)
Date: Fri Apr 14 2000 - 14:31:45 PDT


On the fanac.org site there is a cross reference listing of all the names that
I have been able to find that have been mentioned in the various documents that
the site holds. I do not want this listing to be indexed by ht://Dig because
all of these documents are reacable by other paths so I put <META NAME="robots"
CONTENT="noindex,nofollow"> in the header sectuion of its index. However I do
want to let the search engine to be able to find the entries in the index so I
created and html document (allnames.html) that looks like this:

<HTML><HEAD>
<META NAME="robots" CONTENT="index,nofollow">
<TITLE>FANAC Names Cross Reference Index</TITLE>
</HEAD><BODY>
<A HREF="names-ab.html">Ben Abas</A><BR>
<A HREF="names-ab.html">Tom Abba</A><BR>
.
.
.
<A HREF="names-we.html">Jammy Weasel</A><BR>
<A HREF="names-we.html">Jack Weaver</A><BR>
<A HREF="names-we.html">Sigourney Weaver</A><BR>
<A HREF="names-we.html">Tanaqui Weaver</A><BR>
<A HREF="names-we.html">Clifton Webb</A><BR>
.
.
.
<A HREF="names-z.html">Ben Zuhl</A><BR>
<A HREF="names-z.html">Gary Zukav</A><BR>
<A HREF="names-z.html">Gary Zukov</A><BR>
<A HREF="names-z.html">Edward Zwick</A><BR>
</BODY></HTML>

After updating my data base, using a procedure (DIGGER.TXT (attached) that
uploads to ../htdig/bin as digger ) that I cribbed from your rundig procedure,
a search for "Sigourney Weaver" produces the following results, completely
ignoring the "FANAC Names Cross Reference Index" file:

Documents 1 - 2 of 2 matches. More *'s indicate a better match.

Mt. Holz Science Fiction Society *****
        @@@@@ @ @ @@@@@ @ @ @@@@@@@ @ @ @@@@@ @@@@@ @@@ @ @ @ @ @ @ @ @
@ @ @ @ @ @ @ @ @ @@@@@ @@@@ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @
@ @ @ @ @ @ @ @ @ @ @@@@@ @ @ @ @ @@@@@ @@@@@ @@@ Mt. Holz
Science Fiction Society Club Notice - 12/31/99 -- Vol. 18, No.
27 Chair/Librarian: Mark Leeper, 732-817-5619, mleeper ...
        http://fanac.org/fanzines/MT_Void/MT_Void-1827.html 01/01/00, 19050
bytes

Stratus SF SIG News 1989 **
        Stratus SF SIG News 1989 1988 _ 1989 _ 1990 _ 1991 _ 1992-3 Stratus SF
SIG News #9---Monday, January 9, 1989
        ************************************************************************
******** NEWS Movies: Victoria Tennent has been cast as Offred in
Margaret Atwood's "The Handmaid's Tale." It ...
        http://fanac.org/fanzines/sfnews/sfnews89.html 08/22/99, 28906 bytes

I have attached the configuration file that I use (HTDIG.CON that uploads to
../htdig/conf as htdig.conf) but the important stuff is listed below:

database_dir: /home/fanac/www/htdig/db

start_url: http://fanac.org/allnames.html \
                        http://fanac.org/index.html

limit_urls_to: http://fanac.org/

exclude_urls: /cgi-bin/ .cgi /sffandom/ /frame-

Obviously I'm doing something incredibly stupid (nothing unusual in that) and
any help will be gratefully accepted.

Jack Weaver Fanac Webmaster
The Fanac Fan History Project http://fanac.org
Science Fiction Fandom WebRing http://fanac.org/sffandom


#!/bin/sh

#
# copied from rundig
#
# $Id: rundig,v 1.7 1999/01/31 04:27:02 ghutchis Exp $
#

DBDIR=/home/fanac/www/htdig/db
COMMONDIR=/home/fanac/www/htdig/common
BINDIR=/home/fanac/www/htdig/bin

TMPDIR=$DBDIR
export TMPDIR

date
$BINDIR/htdig -i -s
date
$BINDIR/htmerge -s
date

------------------------------------
To unsubscribe from the htdig mailing list, send a message to
htdig-unsubscribe@htdig.org
You will receive a message to confirm this.



This archive was generated by hypermail 2b28 : Fri Apr 14 2000 - 12:18:11 PDT