I have seen several blogs and articles suggesting that a simple CAPTCHA implementation can be created using a form field hidden by CSS and then excluding any form submissions that include the 'poison' field. I'm not going to single out any particular article, but they are easy to find.

This idea always felt like a bad idea to me and something I have never felt like trying, but I had never been able to convince users that it should be avoided. It appears that spammers have caught on, and I now have proof.

We use jCaptcha on our site and it does the job for us, although I am prepared to admit that our safety has much to do with the fact that our traffic is relatively low. We err on the side legibility so that we don't annoy our users, but this in turn makes the CAPTCHAs easier to break.

Before plugging the test into our forms, we were being attacked on many of our client sites, and were often receiving several thousand faked submissions a day. To test the behaviour of the CAPTCHA, our implementation includes the time and IP of all submissions as well as indicating whether the test field was:

• entered correctly
• entered incorrectly
• not entered (ie left blank)
• not sent - only possible via automation

The main aim was to measure the percentage of failed CAPTCHAs as a measure of the annoyance caused on users. It wasn't too bad. It also shows some interesting behaviour in (alleged) spam attempts. Large numbers of incorrect, missing or null values are scattered through the CAPTCHA logs, but also very common are paired entries like this:

2008-09-26 12:49:49 | aaa.bbb.ccc.ddd | Incorrect Value
2008-09-26 12:49:51 | aaa.bbb.ccc.ddd | NULL

That is: an IP address would send a form submission and would be rejected due to an incorrect value. Two seconds later, the same IP will send the form data again without the CAPTCHA data. Not empty, just not sent.

A few things that should also be pointed out: We send a clear 'message failed' message (403 - forbidden) when the CAPTCHA fails since we don't want logs filled with spammers that aren't sure whether their message is getting through or not.

Back to the original message, and the behaviour I have seen suggests to me that spammers are aware of the 'hidden field' and if they are able to send submissions as seen above and succeed 50% of the time, that is a huge percentage compared to the computation and success rate with text recognition.

I hope this is enough to convince others that hidden fields are not a substitution for a real CAPTCHA and spam prevention is a real problem requiring stronger solutions.