Re: [htdig3-dev] 3.2 goals (was On vacation this weekend)

loic@ceic.com
Tue, 3 Aug 1999 12:40:00 +0200 (MEST)

> parser which recognise <server: http://www.......>
> wait_time: 10
> max_hop_count: 999999
> ........... : .....
> </server>
> and insert it into config

 Just an idea, for the future. I don't know if everyone agree with
the fact that at some point htdig will use an SQL database for things
like documents database. If this is the case, I think that the
configuration file could be loaded in a SQL table.
 The main argument in favour of a database version of the configuration
file is scalability. I currently have a search engine (not htdig)
that contains 150 000 starting points for exploration (servers). It
is hard if not impossible to manage such a list in a flat file (people
submit new URLs to the search engine, two personns review from 600 to 1000
submissions each day).

 Cheers, 

-- 
		Loic Dachary

		ECILA
		100 av. du Gal Leclerc
		93500 Pantin - France
		Tel: 33 1 56 96 09 80, Fax: 33 1 56 96 09 61
		e-mail: Loic@Dachary.org URL: http://www.senga.org/

------------------------------------
To unsubscribe from the htdig3-dev mailing list, send a message to
htdig3-dev@htdig.org containing the single word "unsubscribe" in
the SUBJECT of the message.

This archive was generated by hypermail 2.0b3 on Tue Aug 03 1999 - 03:30:29 PDT