Can I approve all submitted responses before they display on my site?

Absolutely.

Every response that is submitted appears in your account under Responses with the status Needs Review. You can also get an email alerting you that a response has been submitted. 

You can approve each individual response before it will appear on your website.