Re: [htdig] Indexing never ends ...


Subject: Re: [htdig] Indexing never ends ...
From: Geoff Hutchison (ghutchis@wso.williams.edu)
Date: Tue Dec 05 2000 - 19:29:05 PST


On Wed, 6 Dec 2000, Zon Hisham Bin Zainal Abidin wrote:

> I ran the indexing at 11pm last nite and it's still not finish at 8am
> this morning. There are only 20 categories in the category table, 120
> subcategories in the subcategory table, 15 states in the state table and
> 152 towns in the town table.

Well, it's not clear if you can match these independently, but if you
could this would be "only"
20*120*15*152
5,472,000

Which in my mind would take some time. Even just 120*15*152 gives 273,600
pages. To index the latter in 9 hours would require indexing an average of
30,400 pages in an hour or better than 8 pages a second. (!)

> AND of course the footer will (dynamically) CHANGE) when different
> category/subcategory are chosen:
> category.phtml?catcode=BUS&subcatcode=BUS-AUT&statecode=STATE1 for
> STATE1
> category.phtml?catcode=BUS&subcatcode=BUS-AUT&statecode=STATE2 for
> STATE2 so on and so

OK. But I don't see how this would necessarily lead to an infinite loop.
If you see that the indexing is generating two URLs that lead to the same
page, e.g.:

category.phtml?catcode=BUS&subcatcode=BUS-AUT&statecode=STATE2
category.phtml?catcode=BUS&statecode=STATE2&subcatcode=BUS-AUT

To htdig, these are different, but these are probably the same to your
code. But from your description, you haven't given any sense that this is
happening, just that this seems to be taking longer than you expect.

--
-Geoff Hutchison
Williams Students Online
http://wso.williams.edu/

------------------------------------ To unsubscribe from the htdig mailing list, send a message to htdig-unsubscribe@htdig.org You will receive a message to confirm this. List archives: <http://www.htdig.org/mail/menu.html> FAQ: <http://www.htdig.org/FAQ.html>



This archive was generated by hypermail 2b28 : Tue Dec 05 2000 - 19:38:16 PST