I have always found the download speeds the same whether avast is on or not.
Isnt this more a firewall issue rather than avast.?

Maybe the OP needs to look at his router or firewall settings first.