Offloading processing?

by 5 replies
7
I'm trying to set up a web application which includes an external client polling the server with new data which the server then uses to process the data and return a result.

Given the potential traffic I'm expecting the application to generate, I highly doubt that a shared hosting environment will be an appropriate place for this site to function.

I have looked at the possibilities of Amazon's EC2 micro instance, and for ~$5/month, it's hard to beat a dedicated alternative. I'm just not sure how to connect the shared hosting front-end to the processing back-end.

Thoughts?
#programming #offloading #processing
  • You probably want JSON to communicate between the two servers.
    • [1] reply
    • How much volume? A single micro instance can handle low volume sites.

      IF you are doing volume, you are not going to be paying $5 per month...

      You also can't load balance micro instances so you might want to consider stepping up if you plan on being able to scale.

      If you are running a database and web server along with PHP or something similar, you will bring a micro to it's knees.

      I found that a micro running MySQL and another running Apache will suffice for sites with some volume. You can also look at Amazons RDS as a database, they have a free trial

      The nice thing about it all is that you can try and it if doesn't work shut down your instance and go to the next one.
      • [ 1 ] Thanks
  • take a look at https://www.trestleapp.com/

    sounds like it might be what your after.
    • [ 1 ] Thanks
  • There is no such thing, actual traffic pays the the bills not hope of potential traffic.

    As jasong714 said it's all about the required volume (& frequency) of the data being intergrated from the donor site into yours scraping is such an ugly word especially if one has permission

    Without specifics about what data you are going after a definative or even rough outline of a solution is hard to provide. I've built a few experimental proof of concept job hunting scrapers recently on both professional shared & free hosted packages using external processing of the raw data without any problems. By web standards they are quite small with only around 500 visitors a day each but could quite happily be increased to meet additional demands without further expenditure.
    • [ 1 ] Thanks
  • Just like jasong714 told you, JSON, MySQL and apache is cheap and probably sufficient for your needs. You can always scale it up by using memcache to connect server to server.
    • [ 1 ] Thanks
Join the discussion

Next Topics on Trending Feed

  • 7

    Offloading processing?

    I'm trying to set up a web application which includes an external client polling the server with new data which the server then uses to process the data and return a result. Given the potential traffic I'm expecting the application to generate, I highly doubt that a shared hosting environment will be an appropriate place for this site to function.