I’m getting really tired of “verify you’re not a bot” checks everywhere.

I understand these checks help prevent bots, but they seem to appear on so many websites now. Does anyone else find them frustrating?

  • Scrubbles@poptalk.scrubbles.techEnglish
    53·
    5 days ago

    I went from thousands of requests per minute to 10, tops. Bot prevention is a must have.

    If bot checks were removed then everyone would be complaining that it’s unusably slow.

    • ElectricVocalist@jlai.lu
      116·
      5 days ago

      It’s easy to block most bots. It’s impossible to block bots that target you specifically. I’m not blocking any bots on my websites, I like it better to optimize performance so that I can serve them all and have a lot of headroom

      • GenosseFlosse@feddit.org
        17·
        5 days ago

        No, it’s not. Had request from a bot network, always changing ip’s, user agent and request url. There is no pattern there I can block.

        • mesa@piefed.socialEnglish
          16·
          5 days ago

          I created a bot black hole. Specific recursive link that if the click it three times it blocks the IP. And the link is “click if ai” URL at the bottom. Works like a charm.

          • GenosseFlosse@feddit.org
            12·
            5 days ago

            No; the traffic i get seems like it had stored my sitemap from ~10 years ago because the requested urls there are long gone, and now tries to guess other url’s with a similar pattern. It does not seem to follow links from the frontpage.

            • America@sh.itjust.works
              91·
              5 days ago

              That’s fascinating. It almost sounds like they’re training a scraper. Which, really, is far and away the worst possible bot to have.

              • GenosseFlosse@feddit.org
                6·
                5 days ago

                I already looked into something similar, IP banning does not work. One day i got ~120k unique IPs, but the most active one only made ~100 requests. I would gain nothing if i run any script to lookup the IP with each requests (and using up more resources in the process), if it fails 99% of the time because the next hit is a different IP.

                • mesa@piefed.socialEnglish
                  4·
                  5 days ago

                  Ok. Sounds like your in a unique situation. Good luck. You seem fucked.

                  All the above worked for me.

                  • CorrectAlias@piefed.blahaj.zoneEnglish
                    1·
                    4 days ago

                    I also have the same problem as who you’re replying to on a few of my websites. The only way to block them is Anubis or Cloudflare WAF.

                    I think you might be in the unique situation, to be honest. It may only be a matter of time until your solution also stops working.