Experts Opinion needed my site is dropping slowley after penguin 2.0

Hi guys before penguin my rank was rock solid i was 1st for the last 4 month's
my site was very stable links i used was using side bar links from relative websites and also web 2.0 links and press release. All hand written content.
After penguin update i went to 6st place then last week i was 8th now im 14 today. I have. (For a 2nd keyword i got hit from 1st position to 80th this was the day after penguin) however i did have this keyword on some side bar links i then changed these side bar links to naked domain name anchor text after prnguin, could this of really messed me up by changing it?
Whats the best thing i can do now it looks like my old way is just not working, i just continued to build lots of diluted non keyword links, and also done a press release.

I'm totally gutted this was my full income i also got 2 kids 15 week old baby and a 2 year old, its gutted me.

If any one has ranked higher afrer building linkd after penguin please share your information on what actually works, advice from i pro i needed i was in a highly competitive nich my site was pr4 and so was all my competitors thanks.