Tutorial Details
- Program: WordPress
- Difficulty: Intermediate
- Estimated Completion Time: 10 Minutes
- Files to Edit: .htaccess (Make a backup, just in case!)
Lately there have been a lot of WordPress sites compromised only due to the bots that roam the world wide web! There are a lot of plugins out there which can protect your WordPress baby by blocking these “roguish” bots!
In this article you will be learning an easy and useful method of adeptly configuring your .htaccess file to filter these bots which can infect your website and can eat up your server resources. So get your .htaccess file ready for editing!
Step 1 Preparing the Code
The code mainly consists of bot names. I have added the most famous bots in here that I can think of. If there is some bot missing, please mention it in the comments.
The code is pretty straightforward. Go ahead and copy the code below and paste it in your .htaccess file.
# Bot Blocker
<IfModule mod_setenvif.c>
SetEnvIfNoCase User-Agent ^$ keep_out
SetEnvIfNoCase User-Agent (pycurl|casper|cmsworldmap|diavol|dotbot) keep_out
SetEnvIfNoCase User-Agent (flicky|ia_archiver|jakarta|kmccrew) keep_out
SetEnvIfNoCase User-Agent (purebot|comodo|feedfinder|planetwork) keep_out
<Limit GET POST PUT>
Order Allow,Deny
Allow from all
Deny from env=keep_out
</Limit>
</IfModule>
Step 2 Testing the Code
To see whether the code is doing its job, I using recommend this website Bots VS Browsers. This website is a good place to simulate these types of attacks. Once on their website all you have to do is select any bot from the code, which you just added to your .htaccess file, and use that as the user agent. Enter the URL of your site and hit enter. If you see a “403 Error” this means that the code is doing its job. If not the code must’ve gotten messed up while being copied into your .htaccess file, so try again.
Step 3 Adding More Bots
Now you are familiar with the code and how to test it, we can add more bots to the code. You must have noticed the repetition in the code, and by using the same logic, you can add a dozen more bots to be blocked by setting the same parameters. Cool huh!
SetEnvIfNoCase User-Agent (i-IS-evilBOT) keep_out
As you can see in the code above, now I am blocking the “i-IS-evilBOT” (which I just made up). Other than that the name of the bot is not case sensitive and you can add it as per your liking. Go to the Bots VS Browsers page and this time enter the user agent which I just created, and voila, you’ll see that this user agent which was added to my .htaccess file is also blocked! You can add as many bots as you want to be blocked separated with a pipe character “|”
Conclusion
I said in the beginning that there are many plugins which can do the same thing and you can avoid this editing. But by manually editing the .htaccess file you can effectively block bad user-agents and bots with better efficiency and better site performance!
Enjoyed this Post?
Subscribe to our RSS Feed, Follow us on Twitter or simply recommend us to friends and colleagues!
By Bilal Shaheen
Join our newsletter!
Create a Tutorial, Get Paid!
Have something to teach the world? Want to earn money doing it? Tutorials, screencasts, and articles published here on Wptuts+ are largely contributed by readers just like you! We'll pay you great money for good content. Find Out More
Is there any method to applied those code into wordpress self-hosted in nGinx-based server?
This might get you started – http://forum.nginx.org/read.php?2,216599,216599
How do these Bots compromise WP sites? I don’t understand this yet. Why is this an issue now?
They can actually leave spam comments or will use up all the resources and can cause down-time. Casper bot can even delete all your wordpress file.
Hi.
After adding this code to my .htaccess file(I checked before saving, it was correct) I got a 500 Internal Server Error. So I had to remove the code.
And ideas? Could it be that this code overlapped with a plugin?
Try using the code provided in resources file. Otherwise I cannot think of any error. Or a better way to troubleshoot this is contacting support of your webhost provider. Hope this helps
Thanks for this – I can see it being very useful. Is there anywhere you can find out the detail of further bots that it would be wise to block?
Jeff Starr at perishable press has composed a nice long list. Have a look here http://perishablepress.com/press/2009/03/29/4g-ultimate-user-agent-blacklist/
Or you can google the bots. happy bot hunting! lol
Thank you for sharing this technique, I never used it before so I will give it a try a protect more all my websites.
Glad you liked it : )
If you want to make your site even more secure, check out the 5G firewall, which has the above code and more:
http://perishablepress.com/5g-blacklist-2012/
When used in conjunction with cloudflare, askimet, and better wp security, my site is pretty locked down.
This should be a big, bright red, link at the top of this article. Thanks so much for sharing.
Wow, that is awesome. I’m going to put this on a website after work.
Also, make sure you don’t overwrite your existing .htaccess file. Just put code like this somewhere in the middle or at the bottom. Otherwise you can cause problems.
Can I use this trick together with the plugin?
Hi Bilal ! Can you please suggest the best Plugin( Plugins ), which will replace the manual editing work.
Thank you.
I would not suggest a plugin because its not a good idea to edit your htaccess file with a plugin, because a plugin will force rewrite the file and its not recommended. See if you can get some help from a friend to set up your htaccess file for your blog.
I had been using a little plugin for this, buh thankx for a nice quick post.Hate plugins a lot.
Or simply switch to CloudFlare.
Their free plan takes care of bots and the like => less spam, save bandwidth
I have a relatively small website that I switched to CloudFlare 3 days ago (Feb 09, 2012) and here are some stats so far:
8,903 Page views
-> 6,609 from regular traffic
-> 28 from crawlers/bots
-> 2,266 from threats
1,137 Unique visitors
7 Unique crawlers
311 Unique threats
Requests Saved
20,533 requests saved by CloudFlare
32,645 total requests
Bandwidth Saved
285.4 MB bandwidth saved by CloudFlare
552.6 MB total bandwidth
Totally recommend them! (I am not in any way associated with CloudFlare, other than the fact that I use their free service)
Cloudfare is a great service but since it is reverse proxy so it will potentially mess with your analytics and adsense.
CloudFlare is awesome, but I don’t believe CloudFlare is a failsafe for protecting your blog. You still want to put this stuff in your .htaccess file.
Absolutely, I was probably just too excited
I believe that CloudFlare in conjunction with such htaccess directives is a very efficient way of protecting WordPress and reducing load on your server. My spam dropped to less than half since I started using it. So did the bandwidth usage. If I add Amazon S3 for the static files, it’s a complete package
@alex – don’t know about Adsense, but Google Analytics works as usual.
I’m not sure why, but this doesn’t seem to be working for me, I still see the page when testing.
However, it is the same when using the 5G link.
didnt work for me…
returned a 500 server error.
pasted the code from the source file after the .htacces code that was already there, so dont know what happened… does it need something extra from my host server in order to work?? greetings from Colombia