Avast and Jumpshot

Vlk · 2015-05-30T03:50:28.000Z

Hi all,
We recently announced our investment in a marketing analytics platform called Jumpshot.

https://blog.avast.com/2015/05/29/avast-data-drives-new-analytics-engine/

It’s a very exciting investment for us for two reasons. First, because they create really interesting and unique market insights. Go to jumpshot.com if you want to see some of it – there’s a free trial that anyone can sign up for. And second, because they do it using a proprietary algorithm that strips all the PII out of the data they use. It’s the only data stripping tool we’ve seen that does that successfully.

Here’s how Jumpshot works:

Data is collected on computers and Android devices through the browser. Each record contains a set of fields that help Jumpshot algorithms assign the clickstream data appropriately. These fields include:

Installation identifiers (proprietary identifiers that do not contain any PII)
URL being visited
Referral URL (if this exists)
Window identifier
Tab identifier
Additional fields for processing purposes

In reality, the information Avast passes on to Jumpshot looks like this:

Identifier: 00002437-705b-4bc6-b062-54b7ea511c93
URL being visited: http://www.cnn.com/US/?hpt=sitenav
Referral URL: http://edition.cnn.com/
Window identifier: 3
Tab identifier: 42

Prior to processing, all records are automatically scanned for PII, and all PII parameter values are removed from the raw data. To strip PII, Jumpshot uses a proprietary algorithm that calculates multiple statistical features for parameters on all known websites. Based on these statistical values, only parameters that are proven not to be PII are whitelisted and their values are kept. All parameter values that are not whitelisted are stripped in the process, which leaves those parameter values overwritten by the word “REMOVED”. The stripping of PII is done on the Avast premises in Prague, to ensure that the PII never leaves our hands.

Let’s have a look at an example. With a shopping site like Amazon, the URL before stripping contains some PII:

https://www.amazon.com/gp/buy/addressselect/handlers/edit-address.html?ie=UTF8&addressID=jirptvmsnlp&addressIdToBeDeleted=&enableDeliveryPreferences=1&from=&isBillingAddress=&numberOfDistinctItems=1&showBackBar=0&skipFooter=0&skipHeader=0&hasWorkingJavascript=1

The algorithm automatically replaces the PII with the word REMOVED in order to protect our users’ privacy, like this:

https://www.amazon.com/gp/buy/addressselect/handlers/edit-address.html?ie=UTF8&addressID=REMOVED&addressIdToBeDeleted=&enableDeliveryPreferences=1&from=&isBillingAddress=&numberOfDistinctItems=1&showBackBar=0&skipFooter=0&skipHeader=0&hasWorkingJavascript=1

The stripping processes doesn’t end here, though. Next is aggregation. Data processing is performed once a day in a cascade of data transforming and aggregating map-reduce jobs. Aggregations are typically applied on a per-domain (website) and per-URL (web page) basis. To further protect our users‘ privacy, we only accept websites where we can observe at least 20 users. This ensures that no reverse engineering is possible on the aggregated data – there’s nothing that can lead back to a specific user. All aggregated data is then stored in an RDBMS (currently PostreSQL) database on a per-domain and keyword basis.

These aggregated results are the only thing that Avast makes available to Jumpshot customers and end users.

Now, the key is not only what information is actually collected and how it is processed, but also how transparent we are about the whole process. Avast is committed to protecting its customers on all fronts, which is why we inform our users, even beyond our EULA and Privacy policy, that their browsing information will be collected but stripped of personally identifiable information and will be used to help us better understand new and interesting trends. We actually tried to make this very, very explicit, and that’s why we have an extra step in the Avast installer which informs our users in a very straightforward way about what we’re doing.

Users can remove themselves from the system in two ways – by unchecking the “Statistics“ box in the Avast browser add-on settings (see attached picture), or by sending an email to customer support requesting to have their information deleted. If a user wants to be deleted, the system automatically blacklists their user ID from all data transforming activities.

By focusing on protecting our users, we ensure that the data Jumpshot customers get is accurate because the larger the data pool, the more statistically valid the data customers get to work with. So Jumpshot has a vested interest in protecting our users’ privacy.

If you have any questions, please don’t hesitate to ask.

Thanks,
Vlk

essexboy · 2015-05-30T10:29:42.000Z

Currently we do not make any money from this relationship but it is an experiment as to whether we can fund our security products indirectly instead of nagging our users to upgrade. As most people are aware, most all products we use every day—Chrome, Facebook, Firefox, WhatsApp, Gmail, etc.—are indirectly funded by advertisements. In most cases though, the products directly examine what users are doing and provide them targeted advertisements. Although we suspect some security companies are doing this, we do not believe it is the proper approach. Instead, we think that this anonymized, aggregated approach is much better to maintain the trust relationship that we think is so important between us and you, our loyal users.

So does this imply that if it works the ad popups will become history ?

bob3160 · 2015-05-30T13:16:42.000Z

essexboy post:2:

Currently we do not make any money from this relationship but it is an experiment as to whether we can fund our security products indirectly instead of nagging our users to upgrade. As most people are aware, most all products we use every day—Chrome, Facebook, Firefox, WhatsApp, Gmail, etc.—are indirectly funded by advertisements. In most cases though, the products directly examine what users are doing and provide them targeted advertisements. Although we suspect some security companies are doing this, we do not believe it is the proper approach. Instead, we think that this anonymized, aggregated approach is much better to maintain the trust relationship that we think is so important between us and you, our loyal users.
So does this imply that if it works the ad popups will become history ?

Thanks you, it was the exact question I was going to ask.
Now that everyone knows the information is being collected, will that impact the amount of data you receive ???
I’m sure more people will opt out now that they are aware that Avast is harvesting such information and will eventually use it as a money making endeavor.
Is Avast the only Security company doing this ???

essexboy · 2015-05-30T13:55:29.000Z

Now that everyone knows the information is being collected, will that impact the amount of data you receive ??? I'm sure more people will opt out now that they are aware that Avast is harvesting such information and will eventually use it as a money making endeavor. Is Avast the only Security company doing this ???

If I am reading this correctly then then it is just the same as counting the number of people going in and out of a train system, where you see a lot of people going to and from and you can see where they are going but you do not have the foggiest as to who they are. Total invisibility as far as you are concerned. If this then reduces or gets rid of the ad nags then everyone comes out on top

DavidR · 2015-05-30T13:58:43.000Z

Not sure how that would differ from the old Save Surf that was in the AOS browser add-on. Many people disliked that feature thinking that a security base product has no right to be offering this type of thing.

Personally I would probably opt-out of this as I actively block ads (targeted or otherwise) in my browser.

Even with the assurances in the information that personal data will be removed, I still see this as an area that the Privacy naysayers will jump on (excuse the cheap jump shot pun).

bob3160 · 2015-05-30T13:59:03.000Z

essexboy post:4:

Now that everyone knows the information is being collected, will that impact the amount of data you receive ??? I'm sure more people will opt out now that they are aware that Avast is harvesting such information and will eventually use it as a money making endeavor. Is Avast the only Security company doing this ???
If I am reading this correctly then then it is just the same as counting the number of people going in and out of a train system, where you see a lot of people going to and from and you can see where they are going but you do not have the foggiest as to who they are. Total invisibility as far as you are concerned. If this then reduces or gets rid of the ad nags then everyone comes out on top

If that’s the end result, then Avast can get back to growing and protecting since it will eliminate one of the biggest complaints on this forum.

essexboy · 2015-05-30T14:07:18.000Z

Personally I would probably opt-out of this as I actively block ads (targeted or otherwise) in my browser.
Even with the assurances in the information that personal data will be removed, I still see this as an area that the Privacy naysayers will jump on (excuse the cheap jump shot pun).

My reading of this is that it is similar to google analytics and has no relationship to ads but just seeing who goes where from where

bob3160 · 2015-05-30T14:30:47.000Z

@ David,
I’ve long since realized that it’s become too small a world and that hiding in it has become impossible.
I’ve also realized quite a while ago that Privacy as we knew it many years ago no longer exists.
I don’t block Avast or Google or Bing or Yahoo. I do block ads with uBlock.
I do realize that nothing in life is free and one way or another all things need to be paid for.
If this technology is a means of allowing Avast to earn the income they need to protect us and grow
their technology without the Popups we now complain about, then by all means, go for it.

DavidR · 2015-05-30T14:32:16.000Z

essexboy post:7:

Personally I would probably opt-out of this as I actively block ads (targeted or otherwise) in my browser.
Even with the assurances in the information that personal data will be removed, I still see this as an area that the Privacy naysayers will jump on (excuse the cheap jump shot pun).

My reading of this is that it is similar to google analytics and has no relationship to ads but just seeing who goes where from where

Yes, but I also block google-analytics or rather don’t allow it in NoScript and RequestPolicy add-ons.

CraigB · 2015-05-30T15:38:28.000Z

Be interesting to know if this will be the end of the advertising popups but I don’t think it will TBH, analytics will provide Avast with browsing habit information ( data mining ) while the popup Ad’s are for the other side of the marketing division which is to bring in $

Secondmineboy · 2015-05-31T13:22:12.000Z

This may answer your question about the end of Popups:

We have always strived to have an honest relationship with our users, and we will continue to do so. Currently we do not make any money from this relationship but it is an experiment as to whether we can fund our security products indirectly instead of nagging our users to upgrade. As most people are aware, most all products we use every day—Chrome, Facebook, Firefox, WhatsApp, Gmail, etc.—are indirectly funded by advertisements. In most cases though, the products directly examine what users are doing and provide them targeted advertisements. Although we suspect some security companies are doing this, we do not believe it is the proper approach. Instead, we think that this anonymized, aggregated approach is much better to maintain the trust relationship that we think is so important between us and you, our loyal users.

Vincent Steckler

Lisandro · 2015-06-04T17:55:09.000Z

@Vlk:

Vince said that around 100 million users have joined (kept) in the program. We’re 230 million users out there.
I suppose (just my feeling) that the majority of the ones who dropped did because it requires a browser addon. Others should have opt out.
Is it technology possible to do the same job using Web Shield instead of the browser addon?

lucD · 2015-06-07T07:48:15.000Z

Sorry to be contrarian but I think I’d prefer to receive the popups. They are a bit annoying but they’re surely less intrusive that my entire browsing history being sucked up on an ongoing basis. I’m not entirely comfortable even with anonymised data transfers and I’m certainly not convinced that any algorithm is fool-proof … and if that algorithm is proprietary then we’ll never really know for sure. That said … I do appreciate that opt-out is possible (I shall do so by continuing not to install AOS) and I hope that it stays that way.

Lisandro · 2015-06-07T19:21:39.000Z

lucD post:13:

I’m not entirely comfortable even with anonymised data transfers and I’m certainly not convinced that any algorithm is fool-proof … and if that algorithm is proprietary then we’ll never really know for sure.

All the data is processed in Avast servers. So, we DO certainly know for sure because any data delivered could be analysed by Avast.

lucD · 2015-06-09T12:14:12.000Z

I have a lot of trust in Avast, the anti-virus developer and its core program (I don’t install the bloat), its has protected my pc for years and I pay Avast all respect and gratitude. But Avast has another side, it’s the side of the company that introduced Safe Price (default on), a browser clean up product which changes users’ default search provider to its own sponsor and a software updater that directs users to a site that installs adware with the updates. Avast has every right (and need!) to try to earn money but if it follows paths that undermine trust then people may be less inclined to feel confidence in its screening of browsing data when Avast proposes to suck it up in its entirety.

Lisandro · 2015-06-09T12:30:06.000Z

lucD post:15:

A software updater that directs users to a site that installs adware with the updates.

Avast did not install the software first. It is only looking for the updates. The complain should be addressed to the software manufacturer imho.

system · 2015-06-23T14:44:19.000Z

So if the browser plug-in is disabled, these statistics are not collected? And how does this differ from the data that’s collected when “Participate in the Avast community” is checked in the main UI settings?

RejZoR · 2015-06-28T11:36:21.000Z

I’m not usng AOS, so this doesn’t affect me at all?

system · 2015-07-03T11:13:00.000Z

RejZoR post:18:

I’m not usng AOS, so this doesn’t affect me at all?

LackoDan post:17:

So if the browser plug-in is disabled, these statistics are not collected? And how does this differ from the data that’s collected when “Participate in the Avast community” is checked in the main UI settings?

I have just read the blog about this JumpShot stuff. Neither the blog or this forum topic was clear that this only affects those using Avast! Online Security. I assume it does only affect those using the plugin, but I would like these questions answered please, just to make it clear.

Thanks.

MartinZ · 2015-07-10T07:14:05.000Z

Hi,

if you are not using AOS it’s not affecting you. Changing the settings “Participate in the Avast community” in main UI isn’t related to this. If you would like to use AOS and opt-out from the data collection then uncheck this setting (Statistics) in the AOS Settings.

Announcements