To block unwanted crawler bots from your site you can employ the bellow code (example block a bingbot):
RewriteEngine On RewriteBase / SetEnvIfNoCase User-Agent "bingbot/2.0" bad_agent Deny from env=bad_agent
Add
SetEnvIfNoCase User-Agent "$BOTNAME" bad_agent
before the “Deny” line for every bot you want to block
Comments
0 comments
Article is closed for comments.