jsanderz

Hello everyone,
When I try to access my website I receive the error below:
Internal Server Error

The server encountered an internal error or misconfiguration and was unable to complete your request.
Please contact the server administrator, webmaster@informationaddicts.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.
More information about this error may be available in the server error log.
Additionally, a 500 Internal Server Error error was encountered while trying to use an ErrorDocument to handle the request.


I have use Wordpress and host it under Dreamhost.

My error log is showing :
[Wed Jan 21 00:22:32 2009] [error] [client 194.8.75.175] Premature end of script headers: php.cgi, referer: http://www.informationaddicts.com/ch...ur-composition
[Wed Jan 21 00:25:49 2009] [error] [client 65.55.208.50] Premature end of script headers: php.cgi
[Wed Jan 21 00:25:57 2009] etc....


Please can anyone help, I have contacted Dreamhost support and I am awaiting a reply, but I am doubtful that they can help.
Regards
Jeff.

Bruce Hearder

What sort of site have you got installed here?

Is it a blog, or a standard HTML site..

Almost certainly the problem revolves around your .htaccess file..

Ask your host to look at that, or failing thgat, give some more details here and I'm sure a few of use can help you out..

Take care

Bruce

jsanderz

Hello Bruce,
It is a blog, I wouldn't care it was alright first thing this morning.
I am waiting for a reply from Dreamhost support.
Regards
Jeff.

P.S. if you need anymore information just ask.

Bruce Hearder

Unless you have installed any plugins are anything else on the site, i'd say its almost certainly something happening on the server side of things.

They may have changed something that has caused it to stop working..

HostGator did this too me a few months ago, and was very frustrating..

Take care and best of luck getting things working

Bruce

---------------------------------------------------------------------------------------------------------
♦ Discover the website Secrets of the Pros, Visit WebsiteSecrets101.com Now!!, and get YOUR Website Moving!

jsanderz

Bruce,
I just received this message from Dreamhost:
It appears some scripts on your site are being killed by our process
watcher program. This keeps the CPU/Memory load balanced for all users
on the server. Unfortunately, while we can't change the settings on
that, there are ways to reduce the CPU and Memory footprint of your site.
One is to make sure you've disabled hotlinking of your images.
Sometimes the culprit is robot "crawlers", such as googlebot, which like
to get caught in loops on some dynamic sites, causing the load to jump on
the server.

You might find the following articles in our wiki useful in helping you
reduce the load your sites put on the server:

Finding Causes of Heavy Usage - DreamHost

User resource reporting - DreamHost

and

Bots spiders and crawlers - DreamHost

Failing any of the above, you may simply be outgrowing shared
hosting...if there's no further steps you can take to optimize your site
and/or reduce your resource usage, then you may wish to look into our
Dreamhost PS service at Web Hosting by DreamHost Web Hosting: Web Sites, Domain Registration, WordPress, Ruby on Rails, all on Debian Linux! as this would give you
dedicated resources isolated from other users.

Now I am a little lost here, I know very little on the web side of things. I am certainly not outgrowing the shared hosting side, I get very little traffic to this blog.
This is frustrating, all was work this morning.
Regards
Jeff.

jsanderz

Bruce,
I just received this message from Dreamhost:
It appears some scripts on your site are being killed by our process
watcher program. This keeps the CPU/Memory load balanced for all users
on the server. Unfortunately, while we can't change the settings on
that, there are ways to reduce the CPU and Memory footprint of your site.
One is to make sure you've disabled hotlinking of your images.
Sometimes the culprit is robot "crawlers", such as googlebot, which like
to get caught in loops on some dynamic sites, causing the load to jump on
the server.

You might find the following articles in our wiki useful in helping you
reduce the load your sites put on the server:

Finding Causes of Heavy Usage - DreamHost

User resource reporting - DreamHost

and

Bots spiders and crawlers - DreamHost

Failing any of the above, you may simply be outgrowing shared
hosting...if there's no further steps you can take to optimize your site
and/or reduce your resource usage, then you may wish to look into our
Dreamhost PS service at Web Hosting by DreamHost Web Hosting: Web Sites, Domain Registration, WordPress, Ruby on Rails, all on Debian Linux! as this would give you
dedicated resources isolated from other users.

Now I am a little lost here, I know very little on the web side of things. I am certainly not outgrowing the shared hosting side, I get very little traffic to this blog.
This is frustrating, all was working this morning.
I have a backup of all the files, shall I try restoring them via FTP?
Regards
Jeff.

Bruce Hearder

Jeff..

Maybe, its some plugin or somehting like that that is causing the problem. Assuming your hosting company is telling you the truth!

You can always check your web stats and see how much bandwidth you've used over the past month and judge for yourself..

Anyway, that doesn't fix your problem..

I suggest you un-activate all plugins that you have running, one by one, and see if thatfixes the problem..

If not, then I suggest you back up everything off your server, backup your mysql database, delete everything on the host and start again..

Sometimes this fixes.. I reckon you've got about a 20-50% chance that this is going to work for you, but atleast you won't have lost everything..

Failing that, changes hosts.. Thats all I can suggest..

Sorry I can't be of much more help..

Take care and best of luck with your site

Bruce

---------------------------------------------------------------------------------------------------------
♦ Discover the website Secrets of the Pros, Visit WebsiteSecrets101.com Now!!, and get YOUR Website Moving!

gcornelisse

jsanderz,

I have to agree with you. I doubt you're using too many resources. How long as this site been active? Did this just start happening one day, or were you updating the site's code? Or, is this a new site you're setting up?

My take: The 500 error is from the web server trying to display another (different) ErrorDocument. In other words, something caused an error, the web server started looking for the specific ErrorDocument for that error and didn't find it so it defaults back to a 500 error. I could be wrong, but it certainly wouldn't be the first time I've seen something exactly like this. If you reread the error you might see what I'm talking about.

500 errors are usually from a server misconfiguration. That doesn't mean your web site's code can cause it, but its much less likely. Bruce could be right about the .htaccess file. They can cause 500 errors too. The .htaccess file would be considered a server configuration file.

Gary

Gary Cornelisse
http://www.SummitMediaConcepts.com
Internet Application Engineering & Web Development

Monetize Wordpress MU: Pay-to-Blog Paypal Subscription plug-in

Bruce Hearder

About 18 months ago, i got an email from my then hosting company (a very reputable compnay, one that is mentioned many times on this forum), informing me that my 3 websites where using up too many resources and they where going to shut down my account.

The 3 websites got less than 100 visitors a week, combined, and b/w was less real low..

What they wanted me to do was upgrade to a dedicated server at $200 a month..

Talk about a rip off!!

Of course I am no longer a client of theirs and have mobed everything I had to another host that has been absolutely excellent in the past 18 months..

I guess the moral of the story is.. Don't always believe everything you hear from your hosting company, some of them have other agendas running..

Take care

Bruce

---------------------------------------------------------------------------------------------------------
♦ Discover the website Secrets of the Pros, Visit WebsiteSecrets101.com Now!!, and get YOUR Website Moving!

jsanderz

Thanks guys for your help,
I have taken a look at the bandwith, and it is as follows: Total bandwith Unlimited, Used so far 2.065GB,
Estimated usage for cycle 2.207GB. To me that means that my usage is
low, and it certainly doesn't look to me that I am outgrowing shared hosting (what ever that means).
I have tried to delete the files on informationaddict account but it won't let me delete the log files, "access denied message". I am currently trying to restore the old files.
Regards
Jeff.

Matt Fulger

Jeff,

First of all, let me say that I'm certain you do not need to get a dedicated server to accomplish whatever it is you are trying to do with your blog. You were right that your host probably can't (or won't) help you as you found out from their (probably canned) reply.

But just because you get an error that seems to be an internal server error or misconfiguration doesn't mean that's what it actually is either. There could be 1000s of different reasons why you are getting an error. Your host probably knows this and also knows it's unlikely they will be able to quickly & easily fix the problem even IF it really was an internal server error. Which is doubtful or it wouldn't be just your site and they would be bombarded with support requests.

With that said.....

I noticed the following:

Premature end of script headers: php.cgi

This tells me that your host is probably running php suexec. Which basically means php running as cgi.

With that information, I would say the most likely cause is the permissions on some files and/or directories. If you have them set to 777, you will keep getting errors. Try using your FTP client to change permissions (CHMOD) from 777 to 755.

That alone could fix your problem.

Hope this helps.

Matt Fulger

Well......

Looks like I didn't answer in time. LOL

I see now there is a different error, but now it could just be that you aren't finished uploading the backup of your site.

Let us know when you get it uploaded and if it works of not.

jsanderz

Will do Matt,
I have a gut feeling that it is not going to work.
Regards
Jeff.

Matt Fulger

I see you are still having problems....

Assuming that the files are actually on the server.... Now it looks like a .htaccess problem. I just had the same problem a couple days ago.

Are you manually installing wordpress or using their one click installer?

jsanderz

Just thought I would update you guys:
Firstly I have manage to restore the files and the site is up and running, however as per usual I find it slow to load, do you?
Secondly I received another email from Dreamhost support, take a look.

Hello Jeff,

The issue would be intermittent, as the process watcher only kills off a
process when it exceeds the CPU/memory limit which is allowed on shared
hosting, resulting in the 500 error you were seeing. This is a
completely separate number from bandwidth or storage (which is obviously
not an issue, as you've pointed out). Using the information in the Heavy
Usage wiki article (by copying and pasting the commands therein) which I
sent you in my last reply, I found the following:

burke:~/logs/informationaddicts.com/http$ tail -10000 access.log.0| awk
'{print $1}' | sort | uniq -c |sort -n

(...)

35 67.195.37.113
46 208.115.111.246
46 66.150.96.121
47 82.195.187.122
65 194.8.75.175
79 209.85.238.3
108 64.129.18.10
199 66.249.67.130
211 198.176.19.40
273 194.8.75.103
365 72.30.79.91

Some of the heaviest hitters on your site are googlebot, and a yahoo
crawler (198.176.19.40 and 72.30.79.91). As mentioned in the Bots and
Spiders article I sent you, these can cause serious problems and drive up
the resource usage on your site, particularly on dynamic sites, such as
Wordpress blogs. How you choose to deal with them is up to you, as
different folks have different needs...and for some, blocking search
robots would be the death of their site...for others, they don't care so
much. You can make this decision, and proceed with the instructions
found in:
Bots spiders and crawlers - DreamHost

Please read over that article carefully, and then feel free to let me
know if you have any questions.

What do you think, is it googlebot and yahoo causing me these problems?
Regards
Jeff.

Matt Fulger

Hey Jeff,

Yes, your site does seem to load a bit slow for me. But I honestly doubt googlebot has anything to do with it (unless googlebot just hangs out at your site spidering it all day). I hate to say this but the IP your host said is googlebot is NOT. A quick reverse dns lookup shows:

198.176.19.40 resolves to
"euro-gnat.newellco.com"

However, the other one is Yahoo:

72.30.79.91 resolves to
"llf531277.crawl.yahoo.net"

And googlebot is one of your heavy hitters:

66.249.67.130 resolves to
"crawl-66-249-67-130.googlebot.com"

I think your host means well and just pasted the wrong IP. But if you DO decide to use a robots.txt make sure you are blocking the right IPs. Not sure you would want to block googlebot though.

My guess is you have some Wordpress plugin sucking cpu power if this problem persists. Is that all you have installed on your site so far?

Or could there be some other script causing the cpu usage to spike?

Poorly written scripts can wreak havoc on a server, sucking all cpu and making everyone's sites slow down. That's why they have a process watcher in place to kill off whatever process slows the cpu.

So at this point I'm thinking your problem could be in a plugin or some other script you may have installed on that domain.

Let me know if you have anything else installed or if you think it might be a plugin.

Also, maybe you could ask your host if they have a log of what process they had to kill and if their logs might help you track down the problem. Or ask them which logs you may have in your account that would reveal the info you need to prevent this from happening again.

Good luck!

Peace & Prosperity,
Matt Fulger

jsanderz

Hi all,
Well Bruce and Matt it looks like you were right, it seems the problem was caused by a plugin.

I received this response from Dreamhost in reply to my email:

"Well, your best bet in this case would be to remove all of your plugins
and start reactivating them one at a time and check back to see if you
notice problems...And as far as the bots, there are options beyond just
blocking them:
Bots spiders and crawlers - DreamHost
Googlebot - DreamHost

Both of these tell you how to tell the bots "Hey there...slow it down",
without blocking them and losing your search ranking. Let me know if you
have any other questions!"


I will give credit to the guy, he did try, but he seems convinced that it is googlebot that is causing the high CPU.
Anyway, after many hours of trying to restore my website, I finally got it back up and running, only to find out a few hours later that it was showing 505 errors again.
I restored it again and quickly disabled the plugins, and what do you know the site is much faster, and has been ok for the last few days.
I have around 10 plugins, which have been ok for the last year, so not sure why one or more of them has started misbehaving. I am activating them one at a time to find out which one is the culprit.
Will keep you updated.
Guys thank you for your help and advice, I truly appreciate it.
Regards
Jeff.

Matt Fulger

Your welcome for the help. I know what a pain it can be sometimes. I used to run dedicated servers for awhile, so I kinda know a little about both sides of the coin. It's not easy for a server admin to track down specific issues, they can only tell you what they find in their logs.

I still don't believe googlebot is your problem. It's a damn wordpress plugin (which maybe googlebot gets caught in a loop because of?). I've had plugins completely cripple my blog before. I've also had them work one day and stop the next.

I'm not sure why this sometimes happens. But I do know that in some cases a MySQL upgrade can wreak havoc on your scripts (wordpress or not). It seems the developers deem it necessary to change the sql syntax with each update.

And in many cases with a cPanel Host, this is setup by default for automatic updates.

Anyway, good luck and let us know how it goes.

Darren Mothersele

Hey Jeff, I feel your pain on this one! I had some sites hosted with Dreamhost a while ago, and still keep some non-business related sites there. I moved anything important off to a more robust setup about 18 months ago.

I found that scripts that behaved perfectly on other machines just refused to run on Dreamhost. They have a process watcher that gives you very little access to resources. Scripts can run for a very short amount of time before they get terminated.

I hope you get this sorted. I remember reading somewhere that one Dreamhost user managed to get them to swap his site onto a less busy server and his blog worked fine since then. Could be worth asking nicely if there's anywhere they could move you to?

Drupal Developer

Matt Fulger

Hmmm.....

With this new information, I would suggest moving your sites to a better host that isn't quite as limiting. I've personally never used Dreamhost but it's very common for hosting companies to oversell their server space and then act like it's the customers fault when something doesn't work right (not saying dreamhost does this). But based on the comments above, it does show that they are very limiting.

There are tons of good hosts out there that have no problems running multiple scripts without slowing down or killing your site.

Again, good luck getting everything sorted.

jsanderz

Hello everyone,
Just thought I would update you all. Everything was working fine for 4 days with 2 plugins enabled, so I enabled another 2, and what do you know, my site will not load.
This is really becoming a drag, I have decided to leave it, but not sure what to do now.
I have no website and I am hoping to set up more niche sites. What shall I do, change my host or try something else?
I would really appreciate your advice.
Regards
Jeff.

milan

Jeff, what is the exact plugin that is causing the trouble?

It can be just a miss configured plugin, doesn't have to be a bad one.

A serious host wouldn't have a configuration where too many things could cause an "Internal Server Error"! It's up to you to decide whether you change your host, if you pay above average you can find a better service.

I have many shared hosting accounts at different places. Some of these hostings gave me "Internal Server Error" but in all cases it was because I didn't set the folder rights properly, or my script had errors.

There is a way to get more information when an "Internal Server Error" happens. Log in to the Cpanel right after it happens and read the error log.

jsanderz

Milan,
I am not sure as I enabled 2 plughins at once, I am not convinced that it is a plugin now. Could Dreamhost support be right, is it one of the bots?
I am trying to get my site back online, and I will enable one plugin at a time. Then I will see what happens.
Its a god damn nightmare, as I cannot really get on with anything else on the site, I cannot even logon to Admin account in Wordpress.
Regards
Jeff.

Bruce Hearder

Jeff..

Check you server stats, via cPanel.

If you look at AWSTATS there is a bit that shows you the bandwidth used by various incoming IP addresses.. Check this out, if IP address is using all your bandwidth, then simply ban that IP address via cPanel..

A couple of years ago, I had excite.jp pull 2GB of bandwidth on one of my sites within just a month.. Excite.JP - who cares about them, so their IP got banned, and bandwidth usage plummeted..

Worth checking into..
Hope this helps

Bruce

---------------------------------------------------------------------------------------------------------
♦ Discover the website Secrets of the Pros, Visit WebsiteSecrets101.com Now!!, and get YOUR Website Moving!

jsanderz

Bruce,
I have checked my server stats via the cpanel, and I cannot find the AWSTATS. There are many logs via (access and error logs) FTP but they are not very user friendly.
Help.
Regards
Jeff.

alexxfender

Hello..

The problem of getting 500 Internal Server Error with Dreamhost is making me mad! Check discussion.dreamhost.com [under general troubleshooting topic] and I hope you will get the picture.

I start using Dreamhost a few weeks ago, and get this error so many times already, very frustrating. I have two sites running with Gallery2 and php-fusion cms.. and they both have this error many times. I have transfer one site from other hosting company to dreamhost and get this error right from the start.. i never have this error in the previous hosting before(4yrs!)

stevenh512

If this is a Wordpress blog I have a few suggestions that will lower your CPU/memory usage a bit:

1) If you haven't already, upgrade to the latest version of Wordpress (at the time of this writing, 2.7.1 but 2.8 is right around the corner).

2) Disable any plugins you don't need. A few that are usually safe to keep (they don't cause much server load): all in one seo pack, google analyticator, akismet, google xml sitemaps, wp-recaptcha

3) Install the wp super cache plugin and configure it to serve cached pages whenever possible. Cached pages are actual .html files (your php code doesn't get executed at all if you configure it properly) and should cause almost zero server load.

4) Install the gd press tools plugin and optimize your database. You'll probably be surprised to see how much "overhead" there is in your database and this can cause extra server load (along with taking more time for wordpress to find things in the database). There are also some other "advanced" and "performance" settings in this plugin that you might want to tune, perhaps with the help of your server administrator unless you really know what you're doing, but personally I just use this plugin to keep my database optimized.

This signature intentionally left blank.

jsanderz

Hi stevenh512,
I will give this a try and see if it helps.
Many thanks.
Jeff.

Matt Fulger

Hey Jeff,

You mean to tell me 6 months later and you're still having problems with
WordPress on DreamHost? If so, you REALLY need to simply find a better
host. Steven has some good points, but you should have a good host that
isn't so limiting on your account(s).

Good Luck!

jsanderz

Hi Matt,
To be honest, I have only one plug in enabled on this particular site and it has been stable for the last few months. I have not been doing much online lately due to work commitments, but before I do anything else I need to upgrade Wordpress to the latest version, just as stevenh512 suggested.
Thanks for your concern, (so much to do, so little time).
Jeff.

Matt Fulger

Hi Jeff,

Yeah, upgrading is definitely the first step.
I totally understand what you mean with work commitments.

Seems like we need longer days to squeeze everything into our
busy schedules. lol

Matt

stevenh512

The nice thing is, once you get around to upgrading Wordpress to the latest version, you should never have to download the latest version and manually upgrade that blog again. Wordpress has one-click upgrade in the dashboard now (not 100% sure what version introduced this feature, I think it was 2.7 or 2.7.1).

This signature intentionally left blank.

danmorton

I had the identical thing happen to me. Same error messages and all.

I really had no experience looking at my log files, but found that only a few IPs had accessed the Wordpress blog I suspected was the problem. So I figured that bandwidth (etc.) probably was not the problem. And, my technical knowledge is really at a minimum here, too.

I really was at a loss until I read this post. I couldn't even log in to my blog, so I started deleting plugins through FTP. Most of the ones deleted I wasn't even using.

Then - voila - all was working perfectly.

Thanks for the ideas - this discussion sparked the cure.

Lesson learned? Watch those plugins!

Dan