RE: Preventing Data Mining

["Sal Stangarone Jr." <sals@xxxxxxxxxxxxxxxxxxxx>]

Is this B2C?  How about securing the area, so only authorized users have
access?

-----Original Message-----
From: web400-bounces@xxxxxxxxxxxx [mailto:web400-bounces@xxxxxxxxxxxx] On
Behalf Of Wayne McAlpine
Sent: Monday, December 12, 2005 4:45 PM
To: web400@xxxxxxxxxxxx
Subject: [WEB400] Preventing Data Mining

We make available access to a large database using CGI programs.  More and
more frequently, I am seeing attempts to download the entire data base of
millions of records by hitting the server with a script to sequentially
access every record.

I'm trying to come up with a method of preventing this kind of wholesale
download.  Many web sites now use a challenge system consisting of an
authorization word in graphics format that the user must key in before being
granted access.  The problem is that this solution imposes a burden on our
legitimate users that shouldn't be necessary, so I'm looking for a solution
that doesn't involve any additional steps for our users.

The first few times this happened, I changed the Apache server configuration
to deny access to the offending IP address, but it's impossible to monitor
log files to keep up with the ever-changing IP addresses where these attacks
originate.

Anyone have any thoughts on how to approach this problem?

--
This is the Web Enabling the AS400 / iSeries (WEB400) mailing list To post a
message email: WEB400@xxxxxxxxxxxx To subscribe, unsubscribe, or change list
options,
visit: http://lists.midrange.com/mailman/listinfo/web400
or email: WEB400-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives at
http://archive.midrange.com/web400.