There’s a lot of chatter on the wires here (ooh, I make it sound so glamorous and newsroom-y – it would be more accurate to say there are a lot of e-mails going back and forth) about Comscore’s press release about cookie deletion. It makes for somewhat alarming reading – according to the report, 31% of Internet users delete their first-party cookies at least once a month, with 7% deleting them more than four times. Comscore estimates that this means that a cookie-based count of unique users would be overstated by a factor of 2.5, or 150%.
Of course, Comscore is hardly likely to come out with a piece of research that provides a glowing endorsement of cookies, since their measurement methodology – panels (or, more accurately, sampling using a piece of client software that users install on their machines) competes directly with regular web analytics solutions, which rely on cookies for user counts and persistence. But is this study as alarming as it seems?
One thing that caught my eye about the study is that the first-party counts excluded log-in cookies. I’m not quite sure what they meant by this (i.e. whether those were just session cookies), but a lot of sites’ first-party cookies are login cookies. So the first-party cookies measured were ‘non-essential’ cookies; perhaps much more likely to be deleted.
Furthermore, if your site is issuing log-in cookies (assuming they’re persistent), you can use these cookies to generate UU numbers, even if not all users have them (assuming your web analytics solution is sophisticated enough to do this). The great thing about a log-in cookie is that, even if the user clears their cookie, when they come back and log in again, their log-in cookie looks identical, even if it’s not the same actual cookie. So you can have users delete their cookies 10 times a month with no problems, as long as the new cookie you give them looks the same as the old one.
Which leads me onto the point I made in my previous post – sites need to work with their web analytics vendor to implement strategies to limit the impact of cookie deletion on their numbers. It’s still very common to encounter a site which is issuing a high-quality cookie associated with a log-in, but is using a ‘junk’ first- (or even third-)party cookie for sessionizing, UU counts and persistence.
The other point I’d make is that debates about the absolute accuracy of web analytics have been raging for as long as the industry has existed, and the main answer to this kind of thing remains the same today as it always have – don’t rely on your web analytics solution for absolute numbers. Instead, focus on trends and comparisons – which have a much higher chance of being accurate if your measurement technique is consistent across your audience and over time. If you want absolute numbers, use a panel.
Hi Ian,
Great Post. The only thing I disagree about is that there is an absolute number at all. Panel based numbers aren’t absolute, just a different kind of estimation.
-Ian
Follow-up On ComScore Report
To follow up on my earlier post about the ComScore reportI want to point everyone over to a post by Ian Thomas on his blog Lies, Damned Lies that brings up an interesting point about the first-party cookies in the
Hi Ian
The first time I read the comScore report my reaction was “tell me something I don’t know”. As you say, the industry has been openly discussing the inaccuracies of web analytics tools for some years.
I remember joining the Web Analytics Demystified forum and having a heated debate with Eric Peterson (then at Jupiter) about his similar results from client interviews. What came out from that and subsequent research, was that when asked users simply say one thing (usually in good faith) but do another.
For example, if I was to be asked how often I delete cookies I would probably say monthly. The truth is however that I have cookies sat on my machine that are over a year old. I guess I just forget!
Trying to measure the impact of cookie deletion has been on my mind ever since – its so difficult to do! So whenever I end up in a conversation about tracking, I ask the person – how often do you delete yours?
Some of the answers I get are hilarious – “oh yes, weekly for sure”, when I know damn well the person hasn’t a clue of what a cookie is (sorry Fearghal).
The truth is that inaccuracies occur for many different reasons and cookie blocking/deletion is only one of them. For example, if I take my laptop home, often my wife will share my computer to check her email and do some quick online shopping. Thats 2 very different users with two very different internet needs. There is not a web analytics vendor in the world that is capable of differentiating this behaviour – they would simply be treated as the same person.
Describing the accuracy – or lack of it in web analytics is a bit of a passion of mine. Look out for an accuracy whitepaper from me in the near future at:
http://www.brianjclifton.com/blog