Enjoy great online radio at AccuRadio




More thoughts on PPM
·Nov 30, 01:35 AM
Posted by: Kurt Hanson

Let’s continue to explore this PPM issue.

The primary problems that Arbitron is having with the PPM results are (1) that overall radio listening is down, and (2) that declines seem worst among stations serving African-Americans.

So, are the PPM results a perfect measure of reality and the diary has simply been wrong for 40 years, or somewhere in the PPM system is there either (1) a sample size problem, or (2) a response bias problem, or (3) both?

My belief is, it’s both.

Regarding sample size, I believe that there’s a huge flaw in the PPM system, at least as it’s been presented to us. Arbitron is putting PPMs in the hands of far fewer people than they used to put diaries in the hands of, arguing that the PPM is giving them 13 weeks’ of data rather than one, so they’ve got a lot more data to work with.

But, but, but… This is wrong!

Let me try to explain:

Suppose we used to survey 10 Chicagoans every week to find out what percentage are Chicago Blackhawks fans. That means over the course of a calendar quarter, we’ll talk to 130 people. That’s probably a decent sample size to answer a question like this. (I could do the math on standard deviation and margin of error, but let’s save that for another day.)

The analogy for Arbitron’s new approach is to survey maybe 30 Chicagoans — but survey them every week. “Hey, that’s 390 data points every quarter! Better than the old system’s 130 data points!”

No, it isn’t.

In week #1, 30 people is too small a sample size for the question — we could get a result that’s way off of reality. And surveying the same people every week for 12 more weeks is going to give us, basically, the same wrong answer every week.

Yes, the results* will be more stable than under the old, 130-person system, but that doesn’t mean it’s a better approach.

More to follow.



share:  del.icio.us.  post this at del.icio.us  Reddit  post this at Reddit  Digg  post this at Digg  Yahoo   post this at Yahoo! my web  Wink   post this at Wink  Windows   post this at Windows Live  Google  post this at Google Bookmarks  Newsvine  post this at Newsvine

Comment

  1. Kurt…
    Thank goodness someone has finally pointed to what I consider to be THE flaw in PPM. Thank-you for that.
    If you’ve grabbed 18 “wrong” respondents in your group of 30, they remain wrong for the entire 13-week panel period. At least with the old methodology those 18 were only a problem for one of the 13 weeks.
    I did not see very much discussion of the paneling approach until after the ridiculous cell sizes made headlines. No wonder; I feel this is a real biggie.

    Bob Harper

    Bob Harper · Nov 30, 07:49 AM · #

  2. Marbles in a swimming pool – eh Kurt.
    Welcome to my nightmare

    AEV · Nov 30, 09:31 AM · #

  3. How would you like to own a restaurant and not know for a month or so how you did? Your place looks busy…but you won’t know if you’re doing the right thing for a month. It just sucks now… I hope PPM will get it right for us on air staff.

    JoBo · Dec 2, 08:07 AM · #

  4. Maybe the reason the sample size is smaller is that to print and mail diaries is cheaper than manufacturing a piece of equipment that may or may not work. Also, how many stations truly serve the African American community anymore?

    AC · Dec 3, 05:51 AM · #

Commenting is closed for this article.


Blogroll
AccuRadio is powered by...
Conference schedules
Sep. 12 SF Music Tech Summit: San Francisco, CA
Sep. 13 RAIN Summit Chicago @ NAB/RAB RadioShow: Chicago, IL
Sep. 14-16 NAB/RAB RadioShow: Chicago, IL
Sep. 24 IBS Radio/Webcast Conf.: Chicago, IL
Oct. 6-7 Digital Music Forum West: Los Angeles, CA
Oct. 13-14 Digital Content Summit/Music: New York, NY
Oct. 18-22 CMJ Music Marathon: New York, NY
Oct. 27-30 CBI Fall Convention: Orlando, FL
Nov. 5 IBS Radio/Webcast Conf.: Boston, MA
Nov. 19 IBS Radio/Webcast Conf.: New York, NY
Dec. 3 IBS Radio/Webcast Conf.: Los Angeles, CA