Massive Server Slowdown on PHP script - Can anyone help

by 7 replies
8
Hi,

I'm currently running a Linux based VPS, with 768MB of Ram and 30Gig of Disk space (1.13GHZ processor).

I have an application which collects details of domains and then connect to a service via cURL to retrieve details of the pagerank of these domains.

When I run a check on about 50 domains, it takes the remote page about 3 mins to load with all the results, before the script can parse the details and return it to my script. This causes a problem as nothing else seems to function until the script has finished executing, so users on the site will just get a timer / 'ball of death' while waiting for pages to load.

Can anyone tell me if I'm doing anything obviously wrong, or if there is a better way of doing it? (There can be anything between 10 and 10,000 domains queued, so I need a process that can run in the background without affecting the rest of the site)

Thanks
#programming #massive #php #script #server #slowdown
  • I guess the question is what's the bottle neck? Bandwidth? Disk? RAM/CPU? I would fire up your script to determine what the problem is first and address from there. I also see you're using cURL.. might want to look multi handling with curl (curl_multi_init()). This would allow multiple curl strings to fetch in one simultaneous call. I know when I started using it, it definitely sped up some of my scripts.

    Hope this helps.
    hanji
  • Thanks Hanji,

    It does.

    I cant seem to find the bottle neck. Strangley

    1) Images load ok but not pages parsed by PHP
    2) Other sites on the same VPS work ok.
  • If it's just this script my guess is that it's you making 50 calls to that remote service getting pagerank. It's probably hitting other services for each one -- so it's multiple domains returning the information. Obviously that's not going to be instant.
    • [1] reply
    • It's not just that script, it's the entire site, but only that site.

      I was researching the problem and noticed that a couple of other people have reported the same thing over the last few years, but no one came up with a possible solution.
      Using up too much memory, seemed to have been the consensus though. Bandwidth being limited was the next cause that was most mentioned.

      How are you running the script? Have you tried running it from a cron job instead?
      • [1] reply
  • You have two choices: first is to limit the number of domains that data is requested for, and attempt to speed the operations as much as possible, maybe by using curl_multi (How to use curl_multi() without blocking) or second, collect and store your own data, refreshing during low server load periods, then serving THAT data to your users.
    • [1] reply
    • For the bottleneck, I suggest you start by logging in to your VPS using SSH (terminal) and running the command 'top' while your program is running. If you can post back a screenshot we might be able to identify it.

      also, the command 'free' will show you your memory usage.

      /K
Join the discussion

Next Topics on Trending Feed