Yes, you have to download the application.
As long as Avast does not provide OpenVPN config files, you have to use applications to use it.