7 {{ upvoteCount | shortNum }}
7 {{ upvoteCount | shortNum }}

Problem with my apache services on Linux

by marlon
5 replies
Hey,

I have a dedicated server with olm.net that is acting up.

We have spam filters off. Mail scanning off. Tomcat off.
BUT something is running a process or processes in Apache
enough that it's shutting down our server.

OLM support seems clueless about it. All they tell us is to restart
Apache, which doesn't fix the issue.

Anyone know how to troubleshoot this and find what's bogging down
apache?

Marlon
#apache #linux #problem #services
  • Profile picture of the author Voon
    Hm.. I'm sure apache's error log is a good place to start with?
    Signature

    .

    {{ DiscussionBoard.errors[1334058].message }}
  • Profile picture of the author RichPirate
    Have you checked the apache configs to see whats being loaded if a restart doesn't help then it's probably a service that loads during start up... also it could be that there is some process thats running apache, like a cron job or just someone trying a DOS attack on you. In which case you would want to check the access logs.
    Signature

    WSO: Earn $1,000 per Month With This Model. Guaranteed.

    {{ DiscussionBoard.errors[1334470].message }}
    • Profile picture of the author marlon
      Hi,

      So we have a whole bunch of domains in the apache error logs. Most are
      mine but some belong to other people. Those messages look like this:

      ===================================
      exclog: site20: domain prcash.com
      exclog: site67: domain pushbuttonchannel.com
      exclog: site75: domain affiliate11.com
      ===================================

      Like that affiliate11 isn't my domain I don't think. I saw some a Joe Vitale
      domain and a Mike Filsaime domain and others in that list.

      Then we have these error messages repeating quite a bit:

      I don't get this one:

      ================================================== =
      WARNING: MaxClients of 512 exceeds ServerLimit value of 256 servers,
      lowering MaxClients to 256. To increase, please see the ServerLimit
      directive.
      ================================================== =

      Then this one repeated a whole bunch today

      ================================================== ==
      Oct 30 02:10:15 2009] [warn] child process 10749 still did not exit, sending a SIGTERM
      ================================================== =

      Here are the other things in there:

      [Mon Oct 26 04:20:33 2009] [notice] SIGHUP received. Attempting to restart
      WARNING: MaxClients of 512 exceeds ServerLimit value of 256 servers,
      lowering MaxClients to 256. To increase, please see the ServerLimit
      directive.


      exclog: site75: domain affiliate11.com
      [Sun Oct 25 04:20:37 2009] [notice] Digest: generating secret for digest authentication ...
      [Sun Oct 25 04:20:37 2009] [notice] Digest: done
      [Sun Oct 25 04:20:37 2009] [notice] LDAP: Built with OpenLDAP LDAP SDK
      [Sun Oct 25 04:20:37 2009] [notice] LDAP: SSL support unavailable
      [Sun Oct 25 04:20:37 2009] [notice] httpdmon: httpdmon_init
      [Sun Oct 25 04:20:37 2009] [notice] bandwidth monitoring enabled (mapping file: /etc/virtualhosting/mappings/apache.domainmap)
      [Sun Oct 25 04:20:37 2009] [notice] Apache/2.0.54 (Fedora) configured -- resuming normal operations
      [client 61.160.216.63] script '/var/www/html/prx2.php' not found or unable to stat
      [Sun Oct 25 04:59:35 2009] [error] [client 113.14.106.161] File does not exist: /var/www/html/images/125X125.gif, referer: Cash Solos
      [Sun Oct 25 10:14:31 2009] [error] [client 113.14.106.161] File does not exist: /var/www/html/images/125-125.gif, referer: Text Ad Magic - Use Some Advertising Magic
      [Sun Oct 25 20:22:00 2009] [error] [client 64.246.165.210] File does not exist: /var/www/html/robots.txt
      [Mon Oct 26 00:13:49 2009] [error] [client 88.80.216.124] client sent HTTP/1.1 request without hostname (see RFC2616 section 14.23): /w00tw00t.at.ISC.SANS.DFind
      [Mon Oct 26 01:43:02 2009] [error] [client 66.249.68.113] File does not exist: /var/www/html/robots.txt
      exclog: signal received 15
      [Mon Oct 26 04:08:22 2009] [notice] SIGHUP received. Attempting to restart
      WARNING: MaxClients of 512 exceeds ServerLimit value of 256 servers,
      lowering MaxClients to 256. To increase, please see the ServerLimit
      directive.
      exclog: top-level log:/var/log/httpd/access_log
      [Mon Oct 26 04:09:02 2009] [notice] Digest: generating secret for digest authentication ...
      [Mon Oct 26 04:09:02 2009] [notice] Digest: done
      [Mon Oct 26 04:09:02 2009] [notice] LDAP: Built with OpenLDAP LDAP SDK
      [Mon Oct 26 04:09:02 2009] [notice] LDAP: SSL support unavailable
      [Mon Oct 26 04:09:02 2009] [notice] httpdmon: httpdmon_init
      [Mon Oct 26 04:09:02 2009] [notice] bandwidth monitoring enabled (mapping file: /etc/virtualhosting/mappings/apache.domainmap)
      Signature

      [CENTER]Free in depth weekly newsletters on my blog |

      {{ DiscussionBoard.errors[1334489].message }}
      • Profile picture of the author Gary Smith
        Originally Posted by marlon View Post

        Hi,

        I don't get this one:

        ================================================== =
        WARNING: MaxClients of 512 exceeds ServerLimit value of 256 servers,
        lowering MaxClients to 256. To increase, please see the ServerLimit
        directive.
        ================================================== =

        Then this one repeated a whole bunch today

        ================================================== ==
        Oct 30 02:10:15 2009] [warn] child process 10749 still did not exit, sending a SIGTERM
        ================================================== =
        ServerLimit is a number that is compiled into apache. The default is 256 so the only way you can get rid of the error message is to lower MaxClients in the apache config or recompile apache with a higher value for ServerLimit in the file prefork.c.

        If you are hitting the MaxClients value of 256 though, it could be that a specific script is never exiting. You could tell fairly quickly by using the lsof command. If your apache process runs as nobody then lsof | grep nobody will show the open files and you may even be able to tell with a ps -ax (if you don't understand any of that and you want me to have a look PM me. I've been an ISP since 1995 and manage a whole bunch of dedicated servers).

        Another problem that crops up quite regularly is the size of logfiles. I've seen plenty of servers where the logfiles aren't being rotated and once they get over 500MB or so they start to slow everything down.
        Signature

        Gary Smith

        PHP Developer and aging geek

        {{ DiscussionBoard.errors[1339372].message }}
  • Profile picture of the author shaddai
    Sigterm= signal terminate....it's a 'polite' request from the server to a process to quit, normally because it's doing nothing other than holding ram.

    Sighup= is the server going into kill the wabbit mode. The process hasn't responded to a polite suggestion, so the server kills the process.

    Given you've said:
    -I have a dedicated server
    -We have spam filters off. Mail scanning off. Tomcat off.
    BUT something is running a process or processes in Apache
    enough that it's shutting down our server.

    And reading this:
    -bandwidth monitoring enabled (mapping file: /etc/virtualhosting/mappings/apache.domainmap)
    -Oct 30 02:10:15 2009] [warn] child process 10749 still did not exit, sending a SIGTERM

    If I were given one guess, I would have to say that this log looks like it came from a VPS that was running into the end of it's available RAM. Apache is notorious for being a memory hog, so much so, many believe that Apache thinks a memory leak is a feature.

    To determine if it is truly an "Apache vs memory" issue, I would find a way to watch RAM usage over time. Some VPS accounts have a graph dealio that will show you. You should see it steadily climb or stairstep it's way up, hit a certain level in available ram, and then drop off like a rock. Check the time of that dropoff time against the server log & see if it matches your SIGHUP times, and that it's attempting to restart httpd.

    If you want to just watch it to learn it's behavior over an hour or so, SSH to the server & type 'top' at the prompt. This will help you understand how Apache is using memory in real time...along with the other processes on the machine.

    If it does start looking to be a memory issue, you can start with google & search for methods to reduce Apache's memory useage. Here's a couple to start with:
    figby.com Tuning Apache memory usage
    SOLVING APACHE MEMORY BOTTLENECK Sachin’s Weblog

    And the Apache page to help understand what options can change memory useage:
    prefork - Apache HTTP Server

    If tuning your Apache install to relax doesn't get it under control on your particular server, then there are 2 typical solutions to the problem. Get a machine with more RAM to work with, or move from Apache to Lighttpd. Lighttpd is like a naked version of Apache that's much easier on ram. You will have to make some minor tweaks to get Wordpress and Joomla (lighttpd rewrite rules, to make permalinks/SEF urls work right) to run right, but it's definitely worth the time.

    Anyway, again, don't be afraid to ask for help

    Todd

    Todd
    {{ DiscussionBoard.errors[1339982].message }}

Trending Topics