Subject: Re: [htdig] Including Pull-Down Menu Pages
From: Geoff Hutchison (email@example.com)
Date: Fri Oct 20 2000 - 14:05:03 PDT
On Fri, 20 Oct 2000, Douglas Kline wrote:
> All this boils down to the fact that, in the case of these menu bars, Web pages
> are reached by the search engine by a different means than that by which they
> are reached by the user. That is the ultimate source of the problems I have
> described. As these menu bars become more common, more pages will not be found
> by search engines which their users expect to find them.
You make fine points and they're all valid. The problem is that there are
hundreds of non-standard ways of indicating links on the web now and only
a very few can be followed by search engines (ht://Dig is not alone in
actual form/CGI pair.
that a specific form is for navigation and another isn't. For ht://Dig to
would need to contain at least a significant portion of a full
interpreter to run the code. But then you hit the wall of
the user interaction. It's an "undecidable" problem to use the
Now some have proposed that search engines should just "slurp" up all URLs
in a document as possible branch points. This would be possible if all
URLs were full and included, e.g. http://. But since most of us use
relative URLs, it would still need to recognize that "foo/" was likely to
be a relative URL.
Since you are running your own search engine with ht://Dig you also have
control over things like the start_url. In some cases, it may be more
efficient to generate a list of URLs outside of ht://Dig and put them in a
file before indexing or things along these lines. So beyond things like
tags, you can provide URLs through other means.
You also have access to the source and are free to modify it. (That's the
"free" of freedom in free software.) So if you feel your website needs
custom modifications to recognize your style of navigation, go to it.
We're willing to offer pointers into the code.
In short, it IS a problem, but it can only be solved by better web
standards from the HTML side of things (e.g. a stanrdard for representing
URLs for navigation purposes). But IMHO, you're *much* futher ahead of the
game with ht://Dig than with other packages for the reasons I just
-- -Geoff Hutchison Williams Students Online http://wso.williams.edu/
------------------------------------ To unsubscribe from the htdig mailing list, send a message to firstname.lastname@example.org You will receive a message to confirm this. List archives: <http://www.htdig.org/mail/menu.html> FAQ: <http://www.htdig.org/FAQ.html>
This archive was generated by hypermail 2b28 : Fri Oct 20 2000 - 14:10:46 PDT