As mentioned many times here on this board:

Taskmanager is not a reliable source if you want to see how much memory a application/service is using.