Windows firewall is not user friendly. You are not prompted for action (to allow/block) when a new unwanted connection is being made (outgoing especially).

Difference:
Private - suitable when using your computer as part of a home/private network. If this setting is selected, the firewall will allow all communication with the network.

Public - suitable when using your computer to connect to a public network and where you want to ensure the maximum level of security. This is the most secure setting and if selected, no incoming communication will be allowed, effectively making your computer completely invisible to others.

You can use Windows firewall and Avast FW at the same time though so when one misses something, other can stop it. There is no conflict of any kind.
But if you think you can live with creating rules manually for each program then windows firewall is all powerful.