Help me choose

So Im working for a half year on a authority website, besides my niche websites. Currently it has a PR5 and OSE gives it like 40 authority and it has more then 200 pages content.

But.... organic visitors NONE. So about a month ago I changed the keywords I was targeting, and only 2 days, finally I started to get off around 100 visitors a day. But then after 2 days, back to zero again.

Im really losing my head over this, I have around 20 niche websites and more then 100k visitors on them every month. I know how to rank websites, but this one site is losing my mind.

if I search for my unique brand name(also the exact domain, it gets only listed on the 5th place) which is really strange because the websites above me only mention the name onces, so this makes me think something is really going on here.

I do have some old profile links, and crap links from the past, but they are older then 6 months. And I only did quality linkbuilding for the last 3 months. Just a few but quality.

So my question, is should I just wait more months and months. (I dont have a google webmaster msg ) or just start of clean with a new domain?

Or any other ideas?