Re: [htdig] indexing mhonarc archives


Subject: Re: [htdig] indexing mhonarc archives
From: Markus Mohr (markus@nef.wh.uni-dortmund.de)
Date: Sun Jan 30 2000 - 07:53:11 PST


Hi!

To at least partially answer my own question (:):

I put the following lines in my mhonarc.rc file. It changes the
beginning of the message file and the date and thread indices. This is
mainly the standard plus the additional meta tag.

Giving the indices a ´noindex, follow´ and the messages a
´index,nofollow´ allows htdig to crawl through the indices, but stops it
from following the ´backlinks´ in the message-files.
Cut down my digging time roughly in half.

ciao
marksu

<MsgPgBegin>
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML//EN">
<HTML>
<HEAD>
<TITLE>$SUBJECTNA:72$</TITLE>
<LINK REV="made" HREF="mailto:$FROMADDR$">
<meta name="robots" content="index,nofollow">
</HEAD>
<BODY background="/EHG/images/background.jpg">
</MsgPgBegin>

<TIdxPgBegin>
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML//EN">
<HTML>
<HEAD>
<TITLE>$TIDXTITLE$</TITLE>
<meta name="robots" content="noindex,follow">
</HEAD>
<BODY background="/EHG/images/background.jpg">
<H1>$TIDXTITLE$</H1>
</TIdxPgBegin>

<IdxPgBegin>
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML//EN">
<HTML>
<HEAD>
<TITLE>$IDXTITLE$</TITLE>
<meta name="robots" content="noindex,follow">
</HEAD>
<BODY background="/EHG/images/background.jpg">
<H1>$IDXTITLE$</H1>
</IdxPgBegin>

------------------------------------
To unsubscribe from the htdig mailing list, send a message to
htdig-unsubscribe@htdig.org
You will receive a message to confirm this.



This archive was generated by hypermail 2b28 : Sun Jan 30 2000 - 07:56:20 PST