In case anyone else was having a similar issue, turning off gaming mode fixed the problem completely.
Odd how gaming mode is specifically for gaming, and yet it has this behavior.
Thanks all.