Jump to content











Photo
- - - - -

bots


  • Please log in to reply
3 replies to this topic

#1 pscEx

pscEx

    Platinum Member

  • Team Reboot
  • 12688 posts
  • Location:Korschenbroich, Germany
  • Interests:What somebody else cannot do.
  •  
    European Union

Posted 27 November 2008 - 02:03 PM

Or what should be the reason that 200 guests at the same time are interested on the 'mouclass issue'?

My question: Is it possible to limit access of different users to the same post to maybe x (10?) reads at the same time?
And / or limit access from the same IP to maybe 4 reads.

Peter

bots.gif

#2 Nuno Brito

Nuno Brito

    Platinum Member

  • Team Reboot
  • 10452 posts
  • Location:boot.wim
  • Interests:I'm just a quiet simple person with a very quiet simple life living one day at a time..
  •  
    European Union

Posted 27 November 2008 - 03:34 PM

It's a terrible situation.

We took care of the spam bot registrations but now we need to handle these harvesting bots that slurp our resources.

I have placed a very complete robots.txt on the site root but I see that it's nowhere respected.. :(

---

Yesterday I was thinking about something similar to what you suggest.

My idea consisted on writing a simple page load counter.

Everytime a certain IP reached 50 (or more) page loads on a day, it would redirect onto a page asking to reset the page load counter along with some of our usual anti bot techniques like the add/subtract number operations prior to allow reset.

Bots would get stopped quite quickly as their efforts to spider the site would fail.

My only fear is that it would also keep away the google crawlers from doing their work and have a negative impact on the google results.

On the other hand, if you're addicted to boot-land this also ensures that your math skills seriously improve as time passes.. :)

#3 pscEx

pscEx

    Platinum Member

  • Team Reboot
  • 12688 posts
  • Location:Korschenbroich, Germany
  • Interests:What somebody else cannot do.
  •  
    European Union

Posted 27 November 2008 - 03:52 PM

My only fear is that it would also keep away the google crawlers from doing their work and have a negative impact on the google results.

The difference is that google.com seems to be a registered user, and nou an unregisterd 'Guest'

Peter

#4 Nuno Brito

Nuno Brito

    Platinum Member

  • Team Reboot
  • 10452 posts
  • Location:boot.wim
  • Interests:I'm just a quiet simple person with a very quiet simple life living one day at a time..
  •  
    European Union

Posted 27 November 2008 - 04:18 PM

Yes, guess we can try to apply this setting just for guests.

:)




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users