Fake bots that impersonate legitimate services are on the increase.
At VerifiedVisitors we're seeing an increase in fake bots using a legitimate user agent string to impersonate well known bot services. Currently across our range of sites, 12% of the bot visitors are actually impersonating legitimate service. The attackers are hoping that by allowing the agent onto your access list, you will give free rein to their bots.
Very few organisations audit their access lists, and once the agent is on the list, it tends to stay there, sometimes for years. The attack relies on the fact that the bot will only be verified by the user agent string alone, with no additional checks.
At VerifiedVisitors we've got an API to ensure that doesn't happen. Each of the user agents goes through a rigorous multi-factor authentication process to ensure these bot pretenders never get onto the access list in the first place. Our API is dynamically updated constantly, so we're always on top of the every changing bot landscape. Performing this bot verification and managing your access and block lists manually is a time consuming, and let's face it, pretty boring task. It's only too easy to allow based on agent and, as these entries tend to stay on your access list for a long time, allow fake bots onto your site.
It's a clever form of attack, why go to all the problem of disguising your origins if you can just pretend to be a genuine bot? Given some legitimate bots, particularly Bingbot, use up lots of system resources and bandwidth, it's easy to blend into this background noise, and program your fake bot, now helpfully allowed, to perform its nefarious activities unobstructed.
Once you integrate the VerifiedVisitors API service, we do all the background checking, automatically, on each and every bot, and only allow the genuine bots through you actually want.