The Never Ending Saga…

Every story generally has an end. But, in the case of TAM (Television Audience Measurement) Media Research it seems more like a never ending saga.

TAM is yet again in news. I cannot remember a single moment in my 17 years of covering the Media industry when TAM was in news for some good news. It’s kind of etched now in people’s mind be it the broadcasters or the Aam Janta ( common man) that TAM - is this large devil which we all need to get rid off.
It’s been almost 15 years since TAM started and within four years of its existence the problems with TAM also started. The issue then was the sample size and issue now is not only that of the sample size but some broadcasters (largely one of the bigger GEC which has recently   also questioned TAM on its methodology having witnessed low ratings in the recently concluded IPL-6).

This is despite TAM having set up an independent ‘vigilance desk’ will be headed by a former top officer from the Maharashtra police department, the six member ‘transparency panel’ of ombudsmen - includes M. Damodaran, former Chairman of the Securities and Exchange Board of India (Sebi), broadcasting and ad industry veteran Chintamani Rao; former McDonald’s International Media Director Giovanni Fabris; UK television audience expert and BARB and ITV veteran Ivor Millman; former GroupM Insights Director Sheila Byfield; and former MRSI President and Hansa Consulting CEO Praveen Tripathi.

The idea of setting up the vigilance panel was to address all queries coming from broadcasters. Broadcasters can claim that this was set up too late in the day when they are already disillusioned by TAM’s approach and workings. But, the point is it’s better late than never. At least till the time BARC does not start operations there is something for broadcasters to approach and address all their issues.

I am surprised not one report has talked of the independent panel and if broadcasters in question who have pulled out of TAM did approach this independent panel before pulling out. It does not seem like even the reporters reporting the issue has bothered to ask these basic questions. Journalism today is all about only What? And not about  (5 Ws and a H).      

That’s not all TAM also updated the guidelines for all TAM data users only in early May 2013. The update guidelines clearly states that over the past decade, the data and its dimensions itself have vastly increased; more sample size, more geographies to analyze (28 reported markets in 2008 to 42 reported markets in 2013), platforms like Digital emerging, explosion in number of active channels 427 in 2009 to 509 in 2013 etc.

TAM in 1998 had started with a mere 1800 meters across 9 cities. Today it has close to 10000 peoplemeters representing 36,000 TV viewing respondents across 225 cities and towns that have a population of one lakh or more – the largest in the world!  TAM has increased the size purely through internal accruals and help from its parent company Kantar…whereas when it was set up fifteen years, the industry commitment was that it will fund TAM in all its expansion and upgradation endeavors!

The recently update guidelines prescribes what a TAM data user should keep in mind before analyzing and representing data or seeing the products of such efforts. It does not claim to be exhaustive but covers the most common areas on data (mis)representation. The guidelines which is available on TAM’s corporate website clearly states all the necessary dos and don’ts and also explains each aspect of sampling and methodology very clearly.

I recently, read in one report about a English news channel (Times Now) claiming it got zero ratings. Now the guideline note clearly states the meaning of Zero Ratings. Zero ratings does not mean that no one watched the channel all it means is that its below the actual threshold of sample size which is why it is categorized as zero rating.

The guidelines note clearly explains the above as take for instance, if the actual population TVR of a show was 0.5% and the analyst was analyzing data on a sample size of 50 individuals, there is ~80% chance that she will get a zero TVR in the output. But at a sample size of 1000 individuals, this chance of a zero rating dramatically reduces to just ~1%. Even a sample size of 200 individuals reduces the chance of a zero rating output (when the true population TVR is 0.5%) to ~37%.

Now assume that the analyst was looking at some estimates for a show whose true population TVR was 1%. The chances of her getting a zero TVR for this show at a sample size of 50, 200 and 1000 individuals is ~61%, ~13% and 0.004% - a large improvement over the above 0.5% case.

The bottom line therefore is to choose reasonable sample sizes for analysis and also treating relatively small TVR estimates with care while analyzing them.

The other aspect highlighted in the guidelines note is the Trend in data. We often have mailers from channels claiming they are no.1 channel this week in their respective genre.

According to TAM, the biggest advantage of the TAM panel is that one is measuring more or less the same people (20%-25% average yearly churn) over time. Due to this, changes in viewing behavior are reported more precisely than if we were to simply conduct two separate surveys at different points in time. But the power of the panel is not about just calculating a change over some two points in time.

We can see how gradually or suddenly this change is happening by also trending the data between the two points of time. And importantly, to judge whether this change was a ‘one-off’ or part of a systematic change.

Take for example a claim that states (only) that a channel’s reach “has increased by 30%”. Closer examination of the claim showed that two specific weeks were used to show this growth (Week 9 and 18). The claim as such is numerically correct. But the conclusion gets reversed completely when we look at the trend of the channel’s performance and find that reach is actually in a declining phase.

The aspects that the guidelines highlights is on the rankings, use of averages, estimates, use of right basket for analysis, use and/or interpret relative channel shares with discretion., use or interpret time spent per viewer with discretion, use of straight perpendicular graphs and not 3D graphs which  are generally misleading.

Global experts in the field on media audience measurement will tell you about many instances where the TV Broadcast Industry’s faith on its neutral & central audience measurement service has gone through fluctuations. Understandably so. As much as the measurement service provider wants the client/industry to focus on audience insights and analytics, the story for them, starts and ends with the bi-product of report card/rankings. What comes to the rescue and resolve of such situations is when media audience measurement provider work very closely with the Industry Body and if need be, individually, with its stakeholders.

Today, TAM India, is in a similar state. Certain sections of the TV Broadcast industry are reviewing their faith on the very service, that, if you come to think of it, actually helped them grow their own stature and business over the last many years. If you actually come to think of it, it was because of TAM in India that its TV AD market today stands at Rs 12000 crores. Do you think advertisers would have invested had there been no TAM?

If you think about it, whether it is the government, advertisers, broadcasters, agencies or even the academia, do you think they would have got to the know the following had there been no TAM?:
• That news genre has grown and moved out of the 1% share mark...today it stands at around 6%.
• The grand success and growth of Big B from big to small screen.
• Without measurement, we would not have learnt that IPL has become such a success.
• 5 years ago, a Hindi General Entertainment Channel was launched. Without measurement, would we have known of it as one of the established and leading channels?
• It is only with measurement that the industry got to understand how digitization made an impact on consumer/audience behaviour.
• In the year 2000, the industry had less than 100 TV channel getting reported. Today, the number touches close to 600. Would the growth of these channels be possible without the presence of measurement?
• All these years, the industry has seen a healthy (sometimes even double digit) growth of TV advertising. Would this advertising investment have happened in the absence of measurement?
• Would the industry have known how the women audience lapped up the 2003 and 2011 Cricket World Cup? It is only through measurement.
• Without measurement, none of us would have realized the potential and growth of regional TV Channels
• Measurement helped the industry understand the dynamics and difference in viewership between small and big towns.

The Industry never had a dialogue with TAM. For that, ofcourse, the first they would need to do is to prepare and arrive at a common consensus and agenda across ISA, IBF and AAAI on what they want from TAM system...what TAM’s KRAs and KPIs should be. Will someone or some industry body take the onus of creating that for TAM?
My conclusion to this whole issue is that Broadcasters will continue to call TAM a devil until they see lower ratings. So long as the ratings are good and is in their favor, all’s well with TAM! Unfortunately for TAM –they are caught between the devil and deep sea.