AFAIK it deletes the infected attachments only…

The tray icon is in system while you are sending or recieving message, not all the time the Outlook/Exchange provider is running.