It’s the firewall, somehow it causes high DPC latency. This effect indeed worsens over time, after multiple hours (Days) of system uptime without a reboot something seems to accumulate/clog up.