The only things I can think of would be:
1. Site still uses wordpress 2011 theme and doesn't look terribly professional, although it does look pretty good.
2. The two articles of mine that draw most of the traffic are only about 400 words each. Maybe people just skim them and leave?
What are the best ways to fix this? Should I expand the articles that draw more traffic, or put more emphasis on clicking other pages? Or try a different theme and flush out the look of the site a bit more?
Also, how is this likely to effect adsense? Is it likely people will click ads when they leave quickly? I guess it's impossible to know for sure.