I could really do with some emotional support right now - and some advice!!

Hi Warriors,

Apologies in advance for the long post. Please bear with me and give me your wisdom.

I'm living a bad dream right now that I'm sure many of you will have experienced at some point. So I really need your insights on how to overcome this.
As of last night, the affiliate (coupons) website I am running with my husband has been penalized by Google. Googling the name of the website returns loads of less relevant results and the actual website ranks at something like 70.
Our biggest pages that brought in the vast majority of traffic (they used to rank between 1st and 5th-6th for different search terms) are now between 40 and 60.
Other pages that were smaller but which brought occasional traffic (and conversions) are also gone and buried well into the 60s rankings. Of course our traffic is 98% gone too and so is our affiliate income.
So no doubt that the website has been penalized. What for? We have no clue!

Two of the explanations we are exploring are:
a) site slowness: according to google webmaster tools, there was a progressive (very small) increase in the time the site took to load as it became bigger, but recently there seemed to be a spike in that delay for no obvious reason. Despite what webmaster tools said, every time we tried to load the website it would work fine for us. On the 7th of Sep. we did notice that it became considerably slower (homepage took between 8 and 10 seconds to load). Same thing yesterday on and off, same thing today on and off. We've been talking to our hosting company who doesn't seem to think that there's anything wrong, apart from a temporary (rare as they said) problem with the server on the 7th. To relieve the slowness yesterday we got rid of a facebook widget we had on the homepage, as well as of OpenX ad server we had there. It didn't seem to make that much of a difference. We haven't made any other changes that could have affected the speed.
Hubby is now trying to look through the error log report from our website (wordpress) which is a whooping 1.4 GB big - makes no sense at all why it would be so huge!!! He's finding the same error everywhere which is "unable to load dynamic library" which we are now trying to find out what it means.

b) Link devaluation: although this is hard to believe, it could be a possibility. Of course we won't find out until the next webmaster tools update which could take several days. We deliberately chose to go by the book and build our links slowly and steadily and all link building was done manually and progressively. We never purchased spammy links en masse or had any spammy websites link to us, ever. Because of that, our link count is probably still quite low compared to many other websites. We don't believe any of our links are of bad quality, so it doesn't make much sense why google would stop giving us credit for links.

The most frustrating thing is that we were doing SO well before this happened. Our traffic was steadily going up and with some link building and page optimization we had managed to climb to the highest google rankings for some affiliate programs that converted well and the list of pages that brought us traffic was steadily growing, and so was our revenue. In July we made 4 times what we made in June, in August we made 3.5 times what we made in July and in September we were set to go well over that judging from our growing daily revenue. And then it crashed!
Last weekend we had our second big growth spurt ever since we started back in April, and this added about 100-150 daily unique visitors to our existing traffic. It was impressive! From Sunday the 4th till Wednesday the 7th our traffic was steadily going up, every single day.
Then Thursday started off well and as normal and around the afternoon our visitor numbers started to drop. I didn't think much of it at first (often times you hit some slowness in traffic for an hour or two for no apparent reason, and then it picks up again) but within a couple hours we would only get one visitor every hour or hour and a half. Then one visitor every two hours and so on, till we finally stopped getting visitors almost entirely. This is when we looked into it more and found out that our rankings had suddendly tanked.

Like I said, the most confusing thing is that we haven't done anything wrong (at least not to our knowledge) so we have absolutely no idea why this penalty might have come about. We've always tried to fix any problems google reported very promptly, and we've always purposefully tried to go by the book in our SEO.

I would greatly appreciate any insights, ideas, similar experiences and how you overcame them, advice etc. Or even some emotional support. I really need to find out what might have caused the penalty and fix it asap.
Right now it all seems so impossible to overcome, it seems like we'll never be able to get our rankings, traffic and income back.