Its alway good to clear your cache because if you have been on a infecting site then its might have been creating java cache if the site was requiring java.
Otherwise if you want you could clear your browser cache.