CBC's Response to Violations of its Journalistic Policies

The previous post expressed concerns about how CBC has been presenting results of opinion polls and online surveys.  The post was sent to the CBC Ombudsman and the Executive Producer of Power and Politics was good enough to respond and explains that her program will take steps to improve how polls are presented.  Her response follows and afterwards are my comments to the producer: 

---

From: Amy Castle

Sent: Tuesday, December 24, 2013 11:56 AM
To: bkiefl
Cc: CBC Ombudsman
Subject: Letter to CBC Ombudsman

December 24, 2013
Dear Mr
Kiefl,

Thank you for sharing your recent blog post   with the CBC Ombudsman regarding Power & Politics.  As the Executive   Producer, I can tell you your feedback is essential to how we do our work. I must, however, disagree strongly with your   suggestion that our program is "ignoring a vital area of CBC   journalistic policy." On the contrary, we're very much living up to our   Journalistic Standards and Practices.

You raised a number of concerns in your blog post.
 Let me respond to each of them in turn.

In terms of our weekly segment “Political Traction”
with Jaime Watt, you say “CBC should not be implying that this is
representative of Canadians and therefore possibly leading viewers into
thinking that Navigator has conducted representative polling.”

At no point do we characterize Political Traction as a poll.
 Rather, we are clear on the show and on our website that Jaime Watt is
tracking the political conversation in Ottawa
and across Canada.
 The goal of the segment is to discuss which political issues are trending
in the Canadian conversation, and to highlight the differences - if any - in
the conversation being held in Ottawa
compared to the national conversation.

The Traction methodology is explained on our website here: http://www.cbc.ca/news/politics/political-traction-how-it-s-done-1.1031158.  
I am attaching to this note a detailed breakdown of that
methodology.

You say that “Power and Politics should be more forthright
about the methodology used by Navigator.”  Although we have the
methodology on our website, I will ensure that we remind our viewers on air
where to find details of that methodology.

Regarding our weekly feature, The Nanos Number, pollster Nik Nanos
draws on a number of polls from reputable sources and we are always clear where
the polls come from.

CBC’s policy states this: We report polls not commissioned by CBC as long as we can verify that the
methodology meets CBC standards.  The sample size, methodology and
interpretation of results of non–CBC polls should be reviewed by the CBC
research department. To help our audience place a poll in context, we provide
relevant information about the methodology and size of the sample along with
the results. Where applicable, we provide the margin of error.

We abide by these rules with the Nanos Number.  All polls
used in the Nanos Number meet CBC standards.  We also provide information
on the methodology, size of the sample and margin of error.  These details
are clearly stated on our website http://www.cbc.ca/news/politics/the-nanos-number-1.1134156

I agree with you, however, that the information is difficult to
read when we post it on television.  This is due to a new graphics programme
that we recently started using.  I am currently working with our graphics
designer to ensure that our graphics are upgraded to ensure that the font is
more easily readable on air.

You raise concerns that Mr Nanos is using crowdsourcing in his polling.
 In fact the term crowdsource is a Nanos tradename and should not be
confused with a process.  The Nanos polling methodology includes a sample
of random land and cell lines where people are randomly selected to do a study
online.  His methodology is robust and meets our strict CBC standards.

Regarding our daily Ballot Box segment, interaction with our
audience is an important part of our show.  We welcome comments from
viewers, and encourage participation in our daily political conversation.
 The Ballot Box is an important part of that interaction.  

At no point do we refer to the Ballot Box as a poll.  As you
mentioned, we are transparent in showing the number of votes on screen whenever
the Ballot Box appears.

I will, however, ensure that the number of votes gets posted
online along with the percentage results at the end of each day.

Thank you very much for your feedback. It is also my
responsibility to tell you that if you are not satisfied with this response,
you may wish to submit the matter for review by the CBC Ombudsman. The
Office of the Ombudsman, an independent and impartial body reporting
directly to the President, is responsible for evaluating program compliance
with the CBC's journalistic policies. The Ombudsman may be reached by
mail at Box 500,
Terminal A, Toronto, Ontario M5W 1E6,
or by fax at (416) 205-2825, or by e-mail at ombudsman@cbc.ca.

Sincerely,

Amy Castle
Executive Producer, Power and Politics

CBC News Network
www.cbc.ca/politics

My Response to the Executive Producer of Power and Politics:

Ms. Castle, thank you for the response.  Below are my comments and concerns:

1. Navigator or Political Traction: Your suggestion of “…ensur(ing) that we remind our viewers on air where to find details of th(e) methodology” would be beneficial and resolve my concern about Navigator. Putting that information on your web site is good but an on-air explanation is excellent.  I suspect only a handful of viewers have ever seen the web site reference.
2. Nanos Number:  I think it is good that you are “currently working with (y)our graphics designer to ensure that…graphics are upgraded to ensure that the font is more easily readable on air.”  However, while I recognize that this would improve the viewer’s understanding of the polling methodology, the spirit of the policy on reporting polls would be better reflected if you did more than include a graphic, legible or not, with sampling details, etc.  A half minute explanation of the methodology is warranted and would add credibility to what Mr. Nanos says about results.  Mr. Nanos also appeared recently with Evan Solomon on The House and discussed poll results dealing with the PM’s credibility; there was scant reference to polling methodology.  Poll results from an unspecified source dealing with CPP were reported on December 16, 2013 on CBC News Network and again, there was no reference to methodology. So, this appears to be a wider issue than the Nanos Number on Power and Politics and I would appreciate your alerting Jennifer McQuire to this concern.  
3. The Ballot Box:  the policy dealing with online surveys, which I had a hand in drafting, was to ensure that viewers/listeners understood specifically that such “surveys” are not scientific in any way.  The policy was to make that absolutely clear and so I suggest you include a reference to Ballot Box being unscientific each time you present the results in graphic form on air.  Moreover, my understanding of the policy dealing with online surveys is that only raw vote numbers will be presented; percentages will not be presented under any circumstances.  I appreciate your offer to “ensure that the number of votes gets posted online along with the percentage results at the end of each day.” But I believe this is a misinterpretation of the policy. Percentages should not be presented in any form.   To quote from the policy: “We report the results by giving the number of votes cast for each option. We do not give the results as a percentage, as we do with bona fide polls.”  So, including raw numbers on air and on your web site follows the policy but including percentages with or without the raw numbers violates the spirit and the letter of the policy.  I note that a number of CBC programs are using online surveys and all of them violate the policy by including just percentages on their web sites.  These include Day Six, Q and The National and I would appreciate your also bringing this to the attention of Jennifer McQuire.  (The online survey by The National gives percentages to two decimal places, giving the impression the results are especially accurate.)

While the above may seem like nitpicking to someone producing a show as fast moving and well produced as Power and Politics, I assure you that my concerns about CBC journalistic policy are much greater than how polls are presented.  CBC journalism has always occupied the high ground and relaxing journalistic standards only plays into the hands of critics who want to get the CBC on their ‘level playing field’. I firmly believe that the future of the CBC will be determined by the quality of its journalism and slippage in journalistic standards will spell the end of CBC TV and radio.

Barry Kiefl

CBC Journalistic Policy Benched?

CBC’s weekday two-hour program, Power and Politics, covers politicians and political events in more depth than any other program on TV.  It carefully balances the time devoted to the political parties, ensuring all the main parties and their views are represented on various panels, etc.  Its choice of topics tends toward political scandal and sins, making it no different than most other media but that’s another topic.  For reasons unexplained, Power and Politics is ignoring a vital area of CBC journalistic policy.  CBC carefully developed journalistic policies relating to opinion polling and posts them on its website. However, basic tenets of the policy are ignored in almost every edition of Power and Politics.
 
The CBC policy in question states: “We report polls not commissioned by CBC as long as we can verify that the methodology meets CBC standards. The sample size, methodology and interpretation of results of non-CBC polls should be reviewed by the CBC research department. To help our audience place a poll in context, we provide relevant information about the methodology and size of the sample along with the results. Where applicable, we provide the margin of error.”

Two different non-CBC “polls” or what seem to be polls are regularly featured on Power and Politics, The Nanos Number and Navigator.   

Navigator is an example of non-CBC research that seemingly violates CBC policy.  The avuncular Jamie Watt, chairman of Navigator, a successful political consulting firm, appears on his own segment of Power and Politics and presents data that both he and Evan Solomon, host of Power and Politics, imply are representative of “Canadians.” In fact, only after digging into Navigator’s website does one learn that the "tracking" percentages characterized as being representative of Canadians are based primarily on analysis of blogs, online comments, Twitter, and comments on political talk shows (presumably including comments by Jamie Watt and his staff who appear on such programs.)  

 

There is nothing wrong with someone analyzing tweets and online comments but CBC should not be implying that this is representative of Canadians and therefore possibly leading viewers into thinking that Navigator has conducted representative polling.  We know that relatively few people tweet and research, including some done by CBC, determined that those who bother to write newspapers or call talk shows, etc. are generally not representative of either the audience or the population. Power and Politics should be more forthright about the methodology used by Navigator.

The Nanos Number, a product of Nanos Research, is also a regular segment on Power and Politics but featuring actual poll results.  Survey results on some occasions appear to come from a conventional telephone poll, while on other occasions, results are based on what Nanos calls an “RDD Crowdsource random survey of 1,000 Canadians… recruited by telephone through the proprietary Nanos Crowdsource sample and administered a survey online.”   Crowdsourcing has been used to fund entrepreneurs but it is very unclear how it has been used here. The Nanos Number segment, featuring the amiable and professional Nik Nanos, usually provides little or no information about the methodologies, other than an illegible footnote partially obscured by program graphics. One must go to the Nanos website to learn how the surveys have (presumably) been conducted.  So, once again, this seems to be in violation of the CBC’s journalistic policy.  Power and Politics should be transparent about the Nanos surveys and “provide relevant information about the methodology.”

 

 

CBC should utilize its in-house experts in survey research.  In one recent Nik Nanos segment on CBC radio’s The House, Evan Solomon did not accurately read the survey question put to respondents, perhaps because the question, which dealt with the Prime Minister’s credibility, seemed awkwardly worded, something CBC researchers and journalists should pick up on.

The third poll-related segment that regularly appears on Power and Politics is called The Ballot Box.  Each night a different survey question is put to viewers and visitors to CBC’s website and they are encouraged to vote online.   The CBC journalistic policy regarding online surveys is exceptionally well thought out and clear: “Online surveys are a tool of audience engagement. Since it does not fulfill any of the criteria set out in polling policy, the questions and the results are not characterized as polls. We report the results by giving the number of votes cast for each option. We do not give the results as a percentage, as we normally do with bona fide polls. If programs refer to online questions, the results are reported in a way that clearly indicates it has no scientific validity and are not meant to represent the accurate range of either public opinion nor the opinion of our audience.”   Power and Politics, which could be re-christened Polls and Politics, is clearly violating the CBC standards with regards to how these online results are being presented.  cbc.ca only presents percentages and during the program percentages are presented along with raw results.  In my view this lends an air of polling legitimacy to The Ballot Box, which is not warranted, as any professional researcher and experienced journalist knows.

CBC TV is under greater duress than at any point in its history.  In recent years its role in our lives has been usurped by other broadcasters and media, the most recent example being a much reduced role in the broadcasting of NHL hockey.  Over time the funding that all Canadians provide to the CBC via Parliament has been reduced. Today we give our money, many times more than we ever provided to CBC, to private media conglomerates such as Rogers and Bell; for better or worse they have used those funds to push CBC out of many program areas. The one critical element that CBC TV and radio offer to Canadians still unmatched by private radio and TV is quality journalism.  That quality has been maintained because of the excellent journalists who work at CBC and CBC's journalistic standards.   Hopefully, Power and Politics will re-visit those standards.

While were at it, Power and Politics could you please stop running the same two or three commercials targeted to pensioners and the elderly day after day, hour after hour. Yes, I remember Paul Henderson's goal in 1972!  Does the sales department think CBC's older viewers have such impaired memory they can’t remember commercials aired every day for months and just minutes earlier?