Calvin,

Yes, they are scraping the page and displaying it in a child frame. They are
then using javascript to prefill fields. So yes, I have considered:
1. obfusicating element names
2. using javascript to dynamically build elements

These both have drawbacks in maintenance as you pointed out. I was hoping
that maybe someone on the list could point me somewhere that I had not
thought of. As I said before, great minds think alike ;)

Mica

-----Original Message-----
From: straight_talking_java@yahoogroups.com
[mailto:straight_talking_java@yahoogroups.com]On Behalf Of Calvin Yu
Sent: Wednesday, July 15, 2009 8:45 AM
To: straight_talking_java@yahoogroups.com
Subject: Re: [ST-J] Help with Scrapers





It's possible that the software is not using the right referrer addresses,
so you can maybe detect that.
Do you know why they are using the scraper on your site? Is it to scrape
data from it, or is it to automate some interaction
repeatedly? I think any solution you come up with will have to hook
into what their
original intention is.

I was on
a project a couple of years ago where I did some scraper analytics and
researched some potential prevention techniques. In almost all cases,
it really becomes an arms race.
Almost any solution you come up with they will be able to defeat. And
since more techniques involve obfuscation almost any solution will have to
be weigh against how much more difficult it would be to work on that
functionality in the future.

Calvin

On Tue, Jul 14, 2009 at 6:05 PM, David Rosenstrauch
<darose@...>wrote:

>
>
> Scot Mcphee wrote:
> > on the login put a check box (with a CAPTCHA) they must fill out every
> > time they log in that says 'i agree with the terms and conditions' one
> > of which is no screen scrapers. use a variety of detection methods.
> > when detection occurs, disable the account for breach of terms and
> > conditions.
> >
> > also, another way - generate the entire interface programatically with
> > something like Dojo. make every Ajax request (which like, gets the
> > JSON code to generate the interface, so it's entirely necessary) send
> > the information as to whether a scraper is detected. if so, as well as
> > logging for the above breach of terms and conditions, send them a
> > tight javascript loop that eats 100% CPU for lets say, a minute.
> >
> > make all this code obfuscated, (take dojo into production mode and
> > it's pretty obfuscated already) and generate and rotate key method
> > names in the javascript.
>
> Maybe there's also some way to encrypt the traffic and/or parts of the
> page making it inaccessible to the ActiveX control.
>
> DR
>
>
>

[Non-text portions of this message have been removed]






[Non-text portions of this message have been removed]