You don't need stats or examples. You've been told to do this by your host, so you've already got a very good reason.

It's also fairly obvious why the site is going to get faster, and you already seem to know the basics about that. Google likes fast sites a bit more, and they've been very forthcoming about that too.

Added: Caching a WordPress site is usually not problematic, because most of the sites are not too dynamic. It's one of the things that I'd recommend to almost anyone. It's possible some cache plugins wont work properly with your site or server, but there's alternatives.

I don't know who your host is, but usually they know what they are talking about.

Also, only for a small WP site would I recommend NOT using a plugin for this, and I too try to limit the number of plugins. W3TC is the most popular for this, probably.

Google does care (Page Speed). You can test page speed yourself at https://developers.google.com/speed/pagespeed/ .

A simple search for "Website Cache Plugin" on wordpress will give you a list of the best plugins to do the job. Consider the ones with good reviews stars. Your host advice could mean your pages have a lot of images or media which takes long to load from their servers. check your image size before you use the plugins. Cache plugins sometime can give you a headache by interfering with your site sometimes.My site once had problems loading and after removing the plugin everything went back to normal.

Yes use cache. Websites with sizable users should cache. It will greatly reduce server load because it eliminates the need to call the database and PHP functions all the time. With caching, it's almost like your serving your users a plain HTML site, definitely faster than without cache.

Using cache compresses your site size and make it load faster because it becomes smaller than the actual size.

And of course, Google loves sites that loads fast.
It's part of what they use to determine a site with good usability. The big G care about their customers and try to offer them the best browsing experience.

There are some good features in all caching plugins, but on shared hosting you can't achieve full potential. Anyway, you have to start somewhere, so take a look at here:
How to Install and Setup W3 Total Cache and MaxCDN for Beginners

Thank you all. All useful information. I will try out the plugin W3 Total Cache Plugin.

I was hesitant to just install the plugin under the hosts instructions because it sounded like a "default answer".

Should I use CloudFlare or MaxCDN? I noticed CloudFlare is free.

Ok strangely I installed W3 Total Cache Plugin and when enabling page cache, database cache, minify the pingdomc tests went from 69 to 78 however the page didn't display correct ie problem with CSS.

When I disabled minify it went from 78 down to 64.

Then i disabled the plugin and went back to 69.

I seem to get different results between Google Speed Page tests and Pingdom depending on the plugin used.

Is it better to have a higher score in Google Speed Page tests rather than Pingdom?

My advice based on this little exercise and extensive testing conducted today is to basically perform testing and figure out which plugin works best. Rather than just installing the plugin and assume that it helped, it is best to just measure the before and after stats.