Re: htdig


Konstantin Tokar (tokar@apmsun.mpei.ac.ru)
Thu, 13 Aug 1998 18:19:34 +0400


Hi

I have made a few modifications, maybe, you to their sign. If it is
possible, maybe, you will make a format htdig.conf as at Apache (with
units for different servers and cases, with an opportunity to specify
section from the form html).

My server caseinsensitive, therefore I have added _ disallow. IgnoreCase
();
 
Server::Server(char *host, int port)
{
    if (debug > 0)
        cout << endl << "New server: " << host << ", " << port << endl;

    _host = host;
    _port = port;
    _bad_server = 0;
    _documents = 0;

/*KT*/
_disallow.IgnoreCase();
/*/KT*/

}

And for the analysis robots.txt
while (name && isspace(*name)) name++;

void Server::robotstxt(Document &doc)
{

.........
.........
        name = good_strtok(line, ":");
        if (!name)
            continue;
/*KT*/
        while (name && isspace(*name)) name++;
/*/KT*/

-- 
Konstantin Tokar                   
MOSCOW POWER ENGINEERING INSTITUTE, Applied Mathematics Dept.
Email tokar@apmsun.mpei.ac.ru
WWW:  http://www.mpei.ac.ru/tokar/



This archive was generated by hypermail 2.0b3 on Sat Jan 02 1999 - 16:27:17 PST