As mentioned many times here on this board:
Taskmanager is not a reliable source if you want to see how much memory a application/service is using.