Warriors I need your help - website traffic tanked all of a sudden

Ok, my head is spinning right now and I don't even have a logical explanation.

So my website's traffic has suddendly tanked.
It was going fine and as normal up until this afternoon/early evening. Since then I've been having 1 visitor every hour/hour and a half, or so.
Let me also say (and I don't know how relevant that may be) that we had been having a steady traffic growth, mainly thanks to quality link building we've been doing all this time. This past weekend we saw an increase of 150+ visitors and it had been going up every day since the weekend. Our rankings for our biggest pages also went up by a position or two.
And then today, in the middle of the day, the site suddendly tanks.
I checked the 3-4 biggest affiliate store pages on the site (which ranked between places 1 and 6 on google search - and that took A LOT of hard work) and they are all in the 40s and the 60s right now.
I just don't understand. I mean ALL of them???

Note that I have not bought any spammy links to the website, ever. I chose to do all link building manually and progressively rather than exponentially increasing the links to the website. I wanted to do everything by the book and this effort had been rewarded so far.

I did a little digging and found that for some reason the site experienced more slowness than usual loading yesterday and today according to our stats. My husband is talking to our hosting company right now to try and figure out why this is happening. Yet right now it's loading just fine for me.

Any suggestions/thoughts/ideas?? I am at a loss.
I get terrified at the idea that it might take a long time to get our good rankings back where they were. We simply can't afford to lose this income that we've been building gradually and steadily.

Please help.