That many files on a whitelist will slow the process even more than the scanning.
Rather than that, if that many files are bypassed, when any of them get infected you’ll be in hot waters…
I want a better boot/logon speed, but I think you’ll be bypassing security acting the way you suggest.