[htdig] SQL handling start_url

Subject: [htdig] SQL handling start_url
From: Curtis Ireland (cireland@solidum.com)
Date: Wed Dec 06 2000 - 13:16:42 PST

Hypothetical Situation:

I have an SQL database table of links I wish to present someone visiting
my site. However, I would like to make these links searchable from my
site. Normally, if these links were static, I would just list them in
the htdig.conf file.

Is there any way to have start_url get its list from an SQL back-end?
Has anyone already built a patch to handle this?

Here are a couple of solutions I can think of to bi-pass the problem,
but I'm sure I'm not alone in desiring this feature.

1) Build a PHP link built with links to all the sites we want to index.
Have htDig use this as its start_url
2) Before htDig starts its database build, dump all the links to a text
file and have the htdig.conf include this file

The one problem with these two solutions is how would the limit_urls_to
variable work? I want to make sure the links are properly indexed
without going past the linked site.

Just something for everyone to wrap your heads around.

Curtis Ireland	- cireland@solidum.com
Solidum Systems - http://www.solidum.com
(T) (613)724-6004 x284	- (F) (613)724-6008

------------------------------------ To unsubscribe from the htdig mailing list, send a message to htdig-unsubscribe@htdig.org You will receive a message to confirm this. List archives: <http://www.htdig.org/mail/menu.html> FAQ: <http://www.htdig.org/FAQ.html>

This archive was generated by hypermail 2b28 : Wed Dec 06 2000 - 13:28:10 PST