Yes, if the plugin is enabled, it will be protecting you since from the beginning.
You can set Outlook plugin settings or you can manually delete the infected mail.