I recently received a report from our headquarters office that our corporate website was down. I tried accessing it from my home office and had no problem, but a few minutes later, I too saw the error. In Chrome, it manifested as a "No data received" message. Clicking the "More" button revealed the further description "Unable to load the webpage because the server sent no data" and the error code "ERR_EMPTY_RESPONSE".
I soon discovered the problem was sporadic. Sometimes I was able to view the site's homepage, sometimes not. Sometimes I was able to log in to the site's Wordpress admin page, sometimes not. Sometimes I was able to save changes to a post in Wordpress admin, sometimes not. There didn't seem to be any pattern to which pages produced the error or when.
The site uses WordPress and is hosted on a GoDaddy shared server. Several people have complained about similar problems with WordPress recently (such as here and here), but didn't offer any real solutions.
I contacted GoDaddy support. Their first move was to enable error logging for my hosting account. That led nowhere: although I replicated the problem several times while logging was enabled, and although we waited several minutes for log data to be recorded, nothing showed up in the logs. The support tech concluded that no errors were being logged because the requests were being blocked before reaching the layer where logging would occur.
His next move was to run some tests on GoDaddy's firewall and the servers behind it. These tests showed that everything was operating normally.
After some thought, he concluded that mod_security was to blame. He mentioned that WordPress has recently been the target of a lot of hacking, and that GoDaddy had therefore stepped up security measures, deploying mod_security. Mod_security was mistakenly identifying some, but not all, attempts to access the site, from both headquarters and my home office, as hostile. While we assumed that users in other locations weren't affected, we had no way of knowing.
The tech suggested some workarounds, including clearing cache and cookies, and using a private browsing sessions. Neither of these helped any of the affected users. The one suggestion he offered that did work was to wait.
He indicated that once mod_security identifies a request as hostile, it blocks the sender's IP address for 90 seconds. This explains the sporadic, now-it's-up, now-it's-down nature of the problem. He also indicated that, if several requests from the same IP are flagged as hostile, the duration of the blockage becomes longer. He wasn't able to tell me a formula for how long we needed to wait. I ended up waiting several hours before trying again. At that point, the problem had resolved itself, and it hasn't resurfaced, three days later.
We really have little control over the situation. Because it's a shared hosting account, we don't have access to mod_security settings, or to network and server logs that might help with troubleshooting. The tech indicated that upgrading to a dedicated or virtual dedicated server would give us greater control. So far, we've decided not to upgrade, since it would increase cost, cause some downtime, and take several hours of effort on our part to migrate the site to a new server.
We're keeping our fingers crossed that mod_security decides that we -- and our customers -- aren't evil after all.
