If you delete the mail from all places where it is in, the alerts will stop.
Ask your mail-provider to help you if needed.