If you’re reading this post, it’s no thanks to Dreamhost. The past few days I have noticed my web traffic dropping off substantially. I pay $15/mo. plus an annual fee for their Private Server offering in order to host this and a few other blogs. I discovered a great many problems occurring when I checked the server, such as browsers getting a “500 Internal Server Error” on attempting to load documents on all of my WordPress sites.
Rifling through the error.log, I found these events clustering and generally taking the form:
[Mon Jan 19 23:15:08 2009] [error] [client 24.6.13.227] (12)Cannot allocate memory: couldn't create child process: 12: php5.cgi [Mon Jan 19 23:15:08 2009] [error] [client 24.6.13.227] (12)Cannot allocate memory: couldn't spawn child process: /dh/cgi-system/php5.cgi [Mon Jan 19 23:15:08 2009] [error] [client 24.6.13.227] (12)Cannot allocate memory: couldn't create child process: 12: php5.cgi [Mon Jan 19 23:15:08 2009] [error] [client 24.6.13.227] (12)Cannot allocate memory: couldn't spawn child process: /dh/cgi-system/php5.cgi [Mon Jan 19 23:15:21 2009] [error] [client 24.6.13.227] Premature end of script headers: php5.cgi [Mon Jan 19 23:15:21 2009] [error] [client 24.6.13.227] Premature end of script headers: php5.cgi [Mon Jan 19 23:15:48 2009] [error] [client 24.6.13.227] Premature end of script headers: php5.cgi [Mon Jan 19 23:15:48 2009] [error] [client 24.6.13.227] Premature end of script headers: php5.cgi
… thus leading to the 500 errors on the client.
I dutifully emailed support. I noticed this announcement from around the same time I started debugging, however it didn’t mention my specific Private Server. I got a response from support from Liz advising me that:
I usually see that error when there is a WordPress plugin not playing nicely with PHP. Since you should be able to get into the admin panel for the WP install (http://xxxxxxxxxx/wp-login.php, where “xxxxxxxxxx” is the domain name of each domain running WP), I would recommend going in and turning off your plugins, and then turning them back on one by one until you hit the one that breaks your site, and then seeing if it needs to be updated.
This seemed plausible since I had recently made extensive renovations to one of my sites. I spent hours ripping apart and tearing down changes I had made, baselining the offending blog. This yielded no improvement. I then downgraded PHP, turned off memory-intensive cacheing operations and similar gadgets, and executed a couple of reboots. I sent several replies to the Support email (which, stupidly, spawns new tickets rather than amending old ones) and ever since have not received a response.
Similarly, updates to the Dreamhost Status blog have ceased. It’s as though no one works there anymore.
The problem is always the same. My PS is good for a half hour or so after a reboot and then starts throwing errors like above, ultimately failing completely. Then I did some googling and I realized that I am not alone.
Specifically, the comments @ Dreamhost Status and Twitter are revealing that this is common and that there is presently a major outage. Disastrously, Dreamhost are not talking to anyone about it. No updates to their status blog, no email responses from Customer Support. Just a wild goose chase which wasted hours of my evening and morning debugging something that is essentially their issue.
This also seems to have revealed that Dreamhost has a standard practise of nuking processes, via script, at random when their virtual hosting architecture becomes stressed. That would certainly explain why my web site, which served only 28 people today, is maxing out and failing to allocate memory to new processes at an already more-than-adequate 150MB of memory service package with VPS.
I also discovered that Dreamhost vastly over-reports memory usage via its VPS control panel, which would encourage the unwise to make radical upgrades to their service accounts in order to try to address problems like mine, wasting their money:
… this does not reconcile with the Unix top command which reveals nothing extraordinary happening on my virtual private server:
If you see similar stats, don’t upgrade your VPS account — it’s a scam. Dreamhost is trying to bully you into upgrading your service.
Anyway, as of this posting I have rebooted my VPS 7 times in the past 24 hours, and had to do it again just in order to submit this entry. Not that any of you will be able to read it until I am finished moving to a new host, anyway..
So here are the five deadly sins I think we’ve caught Dreamhost in today:
- Misreporting a service-wide fault as a localized (and neutralized) problem
- Misleading customers via support instructing them to debug when clearly the situation is 1)
- Misleading customers via so-called “management” tools that report extremely inflated resource consumption statistics in order to upsell them
- Allowing such problems as 1) to carry on for 24+ hours without a useful update or resolution advice
- Sucking. hard.
So… bye guys. Enjoy the chargebacks!
UPDATE:Â see the exciting sequel to this story — Dreamhost Customer Service FAIL.
Anybody knows why I’d accumulate several sleeping php5.cgi process in the top command? I’m running Wordpres in DH shared host. The sleeping processes do not eat CPU but do eat memory, and I had resorted to a script to kill them periodically.
However, what’s the reason for them staying around? Shouldn’t php5.cgi go away once the wordpress page has been served?
Hello,
What i like with dreamhost PS is that they take care about everything.
Can you advice me any dedicated server easy to use for someone who knows nothing about servers ?
I want somebody to repair when it breaks, is there any company selling dedicated servers with a support as reliable as dreamhost one? (at least they reply fast!)
I want to move because my websites are not optimized and even set up at 4000 MB my websites show thousand of errors pages everyday, so better get 8GB dedicated for the same price as dreamhost 4GB PS !
Thanks for your help.
This is only Dreamhost problem so the solution is to change hosting …
We’ve been having the same problems, and we got the same damn form mail:
***
I’ve checked into why you’ve been receiving these internal server errors, and it seems your scripts have been getting automatically killed by our Process Watcher due to your sites going over Memory limits on the shared server:
I would highly recommend that you follow the steps in the following wiki
article in order to reduce your usage:
http://wiki.dreamhost.com/Finding_Causes_of_Heavy_Usage
Also, please be aware that just because this site is the one getting
killed, it isn’t necessarily the problematic one.
For instance, if Site A is using 90% of your allotted memory usage, and
then Site B attempts to use an additional 15%, Site B will be 5% over and will get it’s script killed.
I’d also recommend looking into any 3rd party plugins you may be running, especially if you happen to be running WordPress installs as they can be notoriously poor at memory management. (Secret Tip: Removing them from the plugins directory is more effective then simply disabling them)
Lastly, you may want to look into DreamhostPS, as you will be able to
raise your memory limits to whatever limit you’d like.
After you’ve taken the steps in the wiki above, if you find that you
require further assistance in reducing your usage please let me know.
***
I definitely want to switch hosts.
i’ve had exactly the same issues with dreamhost. At the end, it looks like that blame not entirely DH. My site had last PR rating because there were too much 500 erorrs, and i’ve switched to VPS.
Once on VPS, i’ve managed to hunt one culprit (JomComment for Joomla) which was SLOOOW and when there is many visitors VPS exausts all physical and virtual memory, and then 500-s starts to show.
So, upping memory does help if you have many visitors, but if there is something not working properly…it’l use all memory very fast.
I’m thinking of moving to hostgator, anways. Hope there is no problems like this.
I am having similar (or same?) problem with Dreamhost. I’ve upgraded to 1GB VPS, but it exausts all memory with tons of sleeping php5.cgi processes.
Top shows no CPU load but zero available memory (just like yours).
I am so switching to other hosting.
This sounds very similar to my problem at Dreamhost. In my case, I can’t attach any files to posts on a phpBB forum that I run – it eventually times out with a 500 error. I contacted support and after many exchanges was finally told that I was “pushing close to the limitations of memory usage” – and yes it was recommended that I “upgrade” to a Dreamhost VPS. I mentioned that I found this extremely odd since the forum I run has had maybe 1 or 2 visitors a day for many months now – very slow. I also have a blog with very little traffic. I had only the default plugins (WP-cache) which I have disabled but still I cannot add attachments to forum posts. From everything I have found the issue appears to have started a couple of months ago and seems to coincide with the issues that all of the other Dreamhost users were reporting. In the logs I am
seeing “Premature end of script headers: php5.cgi”. Has anyone had any luck resolving these errors?
Stumbled upon this while working on a similar problem…
However: “this does not reconcile with the Unix top command which reveals nothing extraordinary happening on my virtual private server”.
It seems to match up pretty well. You’re using 150MB of memory, and 80MB of swap memory, thus the 230MB of total used memory that is showing. I really think DH should give you more swap memory than that… but with my account at 400MB, I rarely end up using half of it.
I spent almost 9 days now thinking that my site’s been hacked and my visitors are going somewhere else. It started when on 8th february I suddenly noticed a 50% drop in my site’s visitors. When I checked the error logs It showed premature end of script error. I troubleshooted on my own since then and support replies are reporting that every thing is fine at their end and I must solve my site issues. I am not on a PS I am on a regular shared hosting package.
What I should do now? move to a better host? I cant afford to move I just paid them for another year.
heem.. this article make me nervous :)… It’s been a month use Dreamhost as my hosting. Just hopping miracle 😀
@martin – solution does not fix this as it is temporary/intermittent
Very interesting post and it confirms my suspicions that Dreamhost are bullshitting users. I went from regular DH hosting to Private Server and still unresolved. I activated and deactivated every plugin and still no joy.
They have a server cluster alright…a server cluster f***!
I’m pleased you wrote this post as DH were being unhelpful and the same support-by-rote story that it is my site. Not so! Time to move hosts. In the event it IS my site I pledge to correct this comment.
here you can find the solution that works for me: http://paininthetech.com/2009/01/07/dreamhost-fix-internal-server-error-without-tech-support
I am having this shit too many times, and i just must change my hosting company. This is not acceptable. They f*** us in the a***.
Funny enough. I’m Googling for the solution of this problem, and found this page, which leads me to a support thread on WordPress forums, and the thread started was me! That was 2 years ago though.
Let me know if you have a solution to this already. DreamHost offered a 66% off for their PS, but I didn’t go for it. The offer has expired though.
Seriously, let me know when you have a solution already. The internal server error is too annoying for me to bear. DreamHost isn’t helpful either, this is what they replied to my support ticket:
And a link to their Wiki page, wanting me to troubleshoot my own. If I can troubleshoot my own I wouldn’t have emailed them in the first place. Is every host the same or it’s just DreamHost?
And BTW as Gersham pointed out there is something interesting happening in the top command window above … memory is pegged to the maximum. If I’d pasted the entire output below you would have seen my virtual host spawning dozens of zombie instances of Dreamhost’s proprietary apache daemon… thus preventing other code within php5.cgi from executing.
This is actually quite remarkable … while the problem appears to have been ducktaped, Dreamhost still hasn’t responded to peoples’ support requests.