Subject: RE: [htdig3-dev] Re: SQL Backend
From: Tillman, James (JamesTillman@fdle.state.fl.us)
Date: Tue Dec 21 1999 - 12:35:45 PST
> I don't understand what you mean by incremental updates. By my
> definition, htdig does support them, in that it can recognize
> when most
> kinds of files haven't been updated, and not bother to index
> them anew.
> Subsequent digs are never as long as initial digs at sites I've seen.
Ah, I should have been more specific. I meant actually getting htdig to
incrementally update its database without having to initiate a "dig" on the
site. You simply say: "hey, this url at this site has been updated, so
don't dig the whole site, just index this part of the site". It would save
a hell of a lot of crawling, and would also save the developer the trouble
of having to crawl the site himself to get the relevant data and shove it in
the database. I prefer to let htdig do his job himself, but rather to make
more flexible about how he goes about doing it. Then again, there may be
things we can't achieve while remaining true to this philosophy...
> > I am working on a perl-htsearch interface and hope to have it
> > completed in early January. This will allow direct access to the
> > resultlist from htsearch, but won't persist the database
> Neat. I've been sitting on a plan to write some config files and
> templates and whatnot, to serve as a straightforward framework for
> writing htsearch wrappers in perl -- is this essentially the
> path you're
> taking, or are you actually parsing the db files?
I'm using a new object called Searcher that I introduced into the htsearch
source code (not the feature-frozen release branch, but the 'other' one).
I'm going to be using SWIG to wrap the Searcher C++ object and access the
results of a search directly from perlspace. So for perl, wrapping of the
command line htsearch would become obsolete, if I can pull it off...
> If you're looking for testers, I'm likely interested,
> especially if you
> can pull out arbitrary attributes per hit [like $(TITLE),
> $(DATE), etc].
For me, this capability would be a must.
To unsubscribe from the htdig3-dev mailing list, send a message to
You will receive a message to confirm this.
This archive was generated by hypermail 2b28 : Tue Dec 21 1999 - 12:50:05 PST