Can I approve all submitted responses before they display on my site?

Absolutely.

Every response that is submitted appears in your account under Responses waiting to be reviewed. You can also get an email alerting you that a response has been submitted. 

You can approve each individual response before it will appear on your website.