QuoteStudy: 39% Of Google Search Referrers Now "Not Provided"

It is just over a year since Google began encrypting search by default for signed-in users. A new study finds that as a result, 39% search-related traffic from Google to web sites now has search terms withheld.

Problem is to see this data now you apparently have to use Google Analytics and attach your account to Google Webmaster Tools. They've got us by the short-hairs.

That's becoming a huge problem. On some sites where we have everything setup and connected to Google, we still see a significant number of not provided.

It pains me.

Most of my keyword discovery is based on this info as it's really the only factual data I can base my decisions on.
There rest is just guess work. And I hate wasting money on guess work

Quote from: bill on November 14, 2012, 02:24:52 AM


Problem is to see this data now you apparently have to use Google Analytics and attach your account to Google Webmaster Tools. They've got us by the short-hairs.

And even that doesn't help - they will still show as not provided

Now if, on the other hand, you used AdWords...
It's all a money milking scheme, no more no less

>Now if, on the other hand, you used AdWords...

Or pony up and get the Corp Analytics version and you're good to go.

But were you referring to the Corp version when you said on some sites you still get not provided?

sounds like - from the following post - GA premium doesn't have any more clarity on "not provided" keywords...

http://www.cloudshapes.co.uk/posts/not-provided-ga-premium-chrome-future/

I graphed this the other day - on average for our clients we see 17% not provided, but that's risen from less than 1% in a quite steady line over the past 7 months.  It also seems to be still climbing, no signs of any type of slowdown.......

Now, if I were a consultant like some here, I'd see this as an opportunity.  You are among the best-educated regarding se referral sources on the planet. Who better to provide sight for the soon-to-be blind?

I'm surprised it's that LOW actually. I swear last I looked most sites I know are already higher.

>> opportunity

Maybe for the short term, but this data will soon just be gone almost entirely. I don't see how leaving webmasters blind improves the user experience.

any idea how this affects services like Hitwise that get their data from ISPs?

Preumably ISPs will see see upstream secure traffic, something like

Request 1:  google.com

which redirects to

Request 2: https://www.google.com/

Request 3: https://www.google.com/search?q=keyword

Request 4: http://www.yoursite.com/


So, Hitwise should be able to see the secure URL containing the query string as an upstream "referrer", is my guess.

That said, if you're already paying Hitwise the chances are you have enough traffic such that a 40% hit to the data still leaves you with a clear enough picture of the traffic you're getting.

In the long term, doesn't this hurt Google most of all though? If they won't provide the referral data, webmasters (of all sizes) will find it harder and harder to provide quality sites to Google... which impacts on their (already shaky) results quality. People started using Google because they had noticably better results than anyone else. If they eventually become noticably worse, users will just go elsewhere. There's a certain amount of user inertia to overcome, but ask Y! or AV what happens to an SE when it's no longer the shiniest tool in the box...

QuotePreumably ISPs will see see upstream secure traffic

The encryption is happening at the transport layer. So the ISP won't be able to decrypt the URL so they can't extract the data any more than you can. The ISP doesn't know what your GET parameters are. The only thing an intervening service between the client and the server knows is the destination hostname and the port number.

http://en.wikipedia.org/wiki/File:IP_stack_connections.svg
http://answers.google.com/answers/threadview/id/758002.html#answer
https://blog.httpwatch.com/2009/02/20/how-secure-are-query-strings-over-https/

It's really frustrating not being able to see actual referrer data. What was google's spin on this as to why they now don't show referral data?

I haven't really researched this... so count this as hearsay, more or less....

They say that it's in the HTTP RFC (or the SSL RFC??)  that URLs not pass private data or some such thing and the search keywords are private data.

So in theory, if all your pages were served up as https and the connection between you and the user was full verified, it would be possible to pass the referrer and other data. Whether or not Google actually passes this data if it's a full https connection or not, I don't know. I also don't know whether even if it is https on your side whether it's really okay to pass private data from one domain to another.

http://googleblog.blogspot.com/2011/10/making-search-more-secure.html
http://www.wired.com/threatlevel/2011/10/google-search-https/

It's a bad change for webmasters and one can argue that over the long term it negatively impacts the user experience because we can't respond as well to what user's are looking for, but for the most part it is a good change for Google users.