More thoughts on PPM
·Nov 30, 01:35 AM
Let’s continue to explore this PPM issue.
The primary problems that Arbitron is having with the PPM results are (1) that overall radio listening is down, and (2) that declines seem worst among stations serving African-Americans.
So, are the PPM results a perfect measure of reality and the diary has simply been wrong for 40 years, or somewhere in the PPM system is there either (1) a sample size problem, or (2) a response bias problem, or (3) both?
My belief is, it’s both.
Regarding sample size, I believe that there’s a huge flaw in the PPM system, at least as it’s been presented to us. Arbitron is putting PPMs in the hands of far fewer people than they used to put diaries in the hands of, arguing that the PPM is giving them 13 weeks’ of data rather than one, so they’ve got a lot more data to work with.
But, but, but… This is wrong!
Let me try to explain:
Suppose we used to survey 10 Chicagoans every week to find out what percentage are Chicago Blackhawks fans. That means over the course of a calendar quarter, we’ll talk to 130 people. That’s probably a decent sample size to answer a question like this. (I could do the math on standard deviation and margin of error, but let’s save that for another day.)
The analogy for Arbitron’s new approach is to survey maybe 30 Chicagoans — but survey them every week. “Hey, that’s 390 data points every quarter! Better than the old system’s 130 data points!”
No, it isn’t.
In week #1, 30 people is too small a sample size for the question — we could get a result that’s way off of reality. And surveying the same people every week for 12 more weeks is going to give us, basically, the same wrong answer every week.
Yes, the results* will be more stable than under the old, 130-person system, but that doesn’t mean it’s a better approach.
More to follow.
share: del.icio.us. Reddit Digg Yahoo Wink Windows Google Newsvine
Commenting is closed for this article.
Other blog entries
Spotify is an "online music service," but that doesn't make it "radio"
How Pandora could become a $70 stock (in a few years)
Late-night addendum to the Pandora analysis: "Enterprise value"
Memorial Day cliche'
Zune's failure proves the way onto devices, for radio, is apps
Great Moments in Business Journalism VXIII -- Pandora version
Great Moments in Business Journalism XVII
NAB is negotiating a VERY FAIR deal with musicFIRST
Simulcasts aren't working
Book your TUESDAY travel to Washington, DC!