Re: htdig: Following Javascript links

Andrew Scherpbier (
Sun, 19 Jul 1998 08:02:12 -0700

Itamar S.-T. wrote:
> One of the sites I'd like to index with ht://dig uses alot of Javascript
> to link to the pages, mainly popup boxes with links.
> How can i get ht://dig to follow these links? i want the pages accessed
> by JS to be indexed too.
> TIA.

It may not work in your situation, but you can use Muffin as a proxy server
to htdig. If you set Muffin up to strip javascript, at least htdig won't see
it and get all confused. If, as you said, there are links that are generated
with javascript, I think you are out of luck.
Are these pages index by Altavista or excite at all?

A while ago I saw something about a GPL'd javascript parser. Anyone know if
that could be used in conjunction with htdig?

You can get Muffin from
Look in the docs to setup a HTTP proxy server in the htdig.conf file. The
attribute is "http_proxy"

Andrew Scherpbier <>
Contigo Software <>
To unsubscribe from the htdig mailing list, send a message to containing the single word "unsubscribe" in
the body of the message.

This archive was generated by hypermail 2.0b3 on Sat Jan 02 1999 - 16:26:53 PST