htdig: valid_extensions


Joergen Haegg (jorgen.hagg@axis.com)
Sat, 27 Sep 1997 07:53:33 +0200


I made this little patch to select specifik extensions.
(Referring to my previous letter.)

It works, but I wonder if maybe htdig already has this capability
somehow already?
Has I reinvented the wheel? :-)

/Jörgen Hägg

--- Retriever.cc.org Fri Aug 15 23:32:20 1997
+++ Retriever.cc Sat Sep 27 07:47:34 1997
@@ -434,6 +434,7 @@
 Retriever::IsValidURL(char *u)
 {
     static Dictionary *invalids = 0;
+ static Dictionary *valids = 0;
 
     //
     // Invalid extensions will be kept in a dictionary for quick
@@ -452,6 +453,19 @@
             p = strtok(0, " \t");
         }
     }
+ // Valid extensions. Hack by jh@axis.com
+ if (!valids)
+ {
+ // A list of valid extensions, separated by spaces or tabs
+ String t = config["valid_extensions"];
+ char *p = strtok(t, " \t");
+ valids = new Dictionary;
+ while (p)
+ {
+ valids->Add(p, 0);
+ p = strtok(0, " \t");
+ }
+ }
 
     static String url;
     url = u;
@@ -477,6 +491,13 @@
     char *ext = strrchr(url, '.');
     if (ext && invalids->Exists(ext))
         return FALSE;
+ //
+ // See if the path extension is in the list of valid ones
+ //
+ char *dext = strrchr(url, '/');
+ if (ext && !valids->Exists(ext))
+ if (dext && dext < ext)
+ return FALSE;
 
     //
     // If any of the limits are met, we allow the URL
----------------------------------------------------------------------
To unsubscribe from the htdig mailing list, send a message to
htdig-request@sdsu.edu containing the single word "unsubscribe" in
the body of the message.



This archive was generated by hypermail 2.0b3 on Sat Jan 02 1999 - 16:25:05 PST