Can unofficial statistics help to achieve the Sustainable Development Goals (SDG) Global Indicator Framework and how can official statistics play a role in this process?

Official Statistics goals on whiteboard

The first discussion will centre around statements taken from an article by MacFeeley and Nastav 'You say you want a (data) revolution: a proposal to use unofficial statistics for the SDG Global Indicator Framework’, published in the September 2019 issue (Vol 35, number 3). You are invited to discuss the main or detailed statement(s).

Unofficial Statistics for the SDG Global Indicator Framework

'You say you want a (data) revolution: a proposal to use unofficial statistics for the SDG Global Indicator Framework’ by MacFeeley and Nastav can be read here: http://bit.ly/2KXB21V

Main statement for discussion:

Official Statistics should consider switching from a purely production or manufacturing based model to a mixed business model: one combining the manufacture of official statistics with the franchising of production under license.

Other subsidiary statements to be discussed are:

On the role of National Statistical Organisations (NSO’s) and International Statistical Organizations (IO’s)

In this post-truth era, NSOs and IOs should assert their mandate and legitimate role as custodians of knowledge and protectors of deliberative public spaces. The statistics community is underestimating the changes underway in the world of data and statistics. The various networks and philanthropic partnerships established to strengthen data ecosystems and promote collaboration in recent years may be counterproductive, inadvertently undermining the UN. Therefore, Official Statistics must adapt in a way that allows it to take some control (or at least exert more influence) over a rapidly disintegrating information landscape. NSO’s and IO’s must act now to ensure the integrity of Official Statistics in a rapidly disintegrating information landscape.

On the control and ownership of statistical information

There will be ideological resistance in some countries to governments collecting more data – the fear of a Big Brother state. The neo-liberal agenda aims to minimize the role of the public sector - even in the data sphere. There will be resistance to expanded government oversight and accreditation. NSO’s and IO’s must act now to ensure this resistance does not erode the right to live in an informed society.

On the SDG Global Indicator Framework

It is highly improbable that based on current ‘Official Statistics’ by 2030 the SDG Global Indicator Framework will be fully, or even close to fully, populated. Currently less than half of the framework is populated. However, the UNSC could change into the body with the authority and competence to certify statistics as ‘fit for purpose’ - to review unofficial statistics and assess whether they can be certified as 'Official' for the purposes of populating the SDG Global Indicator Framework. It would be what the Académie des Sciences or the Royal Society was to the Victorian era, in terms of homologation.

iampatt0

Comment on:
22/08/2019 13h03

The advent of wide spread IOT data can be integrated and compared with official statistics to provide a more granular and broad spectrum of statistics. The short answer is yes they should be considered.

Comment

Volker

Comment on:
05/09/2019 09h25

I understood that the endeavour of collecting statistics on SDGs aims also at strengthening the National Statistical System (NSS) as producers of "Official Statitics". If this is so, then we should be careful with passing by the National Statistical Institutes (NSI) as central coordinators of the NSS. In my opinion NSIs should have a say on the use of any statistics considered not official.

In this sense - "you say you want a (data) revolution (...) All I can tell you brother is you have to wait."

Comment

Alphonse

Comment on:
06/09/2019 11h45

1. For many developing countries the business production model is irrelevant, as statistics are seen as a public good and governments have an obligation to provide the required statistical information.
2. In many countries governments are using data collected by non-governmental agencies (trade board, professional associations) and by doing so, these data become "official@"statistics.
3. A certification process is too heavy and would distract from the duty at hand. NSO are already using "non-official" statistical information, if they are of good quality. This means adherence to sound scientific and methodological principals.
4. Countries can choose the SDGs they would like to adopt and report on. These will be those of national interest. The total number of SDGs used will therefor be reduced,

Comment

Peter_de_Jong

Comment on:
06/09/2019 14h54

After reading “You say you want a [data] revolution” two questions come up:

Are there candidates for the unofficial official statistics, or are we discussing the empty set?

How complex would be the vetting process of the unofficial statistic (and the producing agency or company)?

Comment

steve.macfeely@un.org

Comment on:
06/09/2019 16h19

Thanks everyone for the first round of comments. Some preliminary responses:

Volker - Yes, we agree that the importance of NSSs is paramount. We propose 2 possibilities (1) national accreditation and (2) global accreditation. Option 1 automatically addresses your concern. For option 2, you raise an important point. One that we failed to address in the paper, but one we highlighted at the launch presentation at the WSC - what happens if even one country objects to a global indicator constructed from an unofficial statistic? Good question. We will need some sort of mechanism to deal with this. But glad to see the Beatles reference wasn't lost on you.

Alponse - We deliberately remind readers of the distinction between data and statistics in the paper, to make the very point you make. NSOs use unofficial data every day of the week, and by subsuming those data into their statistics, the output remains official. We are asking a different question - could NSOs or the IAEG-SDG use fully formed statistics (rather than input data)? Therefore, I would argue the production model is central to official statistics as a public good.

Perhaps our model is too heavy? Who knows - To the best of our knowledge we don't think it's ever been tried anywhere. The heaviness comes from our concerns over the risks of eroding trust in official statistics. I guess, the first thing is the establish whether the principle is agreeable? If yes, then the details could be fine tuned.

Peter - If we understand you question correctly, then yes, we are discussing the Tier 3 indicators (or perhaps Tier 2 if the statistics fail to materialise). The complexity of accreditation is unknown - as noted above, we don't think this has been tried before. Apart from the usual methodological concerns, our real concern is how to ensure the process is done with the aim of providing impartial information.

Thanks everyone for your comments. We will reflect on them further, while we await others.

Comment

dunnejo

Comment on:
07/09/2019 18h43

In terms of unofficial statistics produced by the non Government sector (profit and not for profit), I am wondering whats in it for these organisations.

If the organisations motivation is simply CSR etc then the health of the producing organisations, the management personality of the organisation (all liable to change) and other possible motivations become a significant challenge.

If the producing organisations have 'skin in the game' as the paper discusses, it becomes very difficult to certify and monitor such statistics even for National Statistical Offices, let alone an international authority.

Such organisations can simply continue to produce and market their statistics if certification doesn't suit them.

Comment

JamesWhitworth

Comment on:
09/09/2019 18h56

In this data revolution, the role of the NSOs and IOs already includes “vetting” of non-official data and indicators – if only to determine if they can be used in their own production. It might seem to be a small step to a more formal accreditation.

However, in reply to Alphonse’s comments, Steve admits that such an accreditation system will be definition be heavy. Whether this is done at a national or international level – and I would argue that comparability concerns would suggest the latter – this is going to be very expensive. Whilst  the UNSC should oversee the process, it would be too much for one organisation to take on, given the sheer number of indicators in the GIF.

Comment

steve.macfeely@un.org

Comment on:
11/09/2019 10h34

John asks what's in it for the statistical compilers. It's a good question. The answer will vary compiler by compiler. For some, the 'validation' or a gold star awarded by the NSS or the UN might be viewed as something attractive or valuable. In which case we have a win-win. If they are not interested, then no harm done. But peer review might in time bring more compilers into the fold (he says optimistically).

James has highlighted the issue of costs - this is a real issue and something we thought about quite a lot. While the cost of an accreditation system will not be trivial, this must be weighed against the of NSOs having to compile the actual indicators - this will not be inexpensive either, and arguably it will be more expensive than accreditation. So the question should, I think, be framed around comparative or opportunity costs, rather than absolute costs.

Comment

hstephen

Comment on:
13/09/2019 03h35

Accreditation aside, the numbers released by an agency become official by virtue of the charter of the agency. That charter in turn defines the ethos of statisticians working in agencies, or otherwise recognised as official statisticians, in the same way that the apparatus of effect measures defines a biometrician; or the location of ore bodies defines a geostatistician. This vocational imperative does not change when the information environment changes. Behind official releases are statements of inferential truth, and quality assurance in the passage from collecting data - by design or by bulk transfer - to assembling an analysable dataset. An official statisticans skill base extends from understanding the [principles of design, process control, inference, and release and interpreting and anticipating policy requirement. All agencies should have at core methodologists fluent in this language, and build in specialists that match the challenge. It maybe that national agencies ar enot well placed to take up the challenge of SDG program, but it should not be beyond the capacity of the community of official statistics in broad.to contribute through alternative auspices., I am not attracted to a franchising model, it is going if anything in the wrong direction.

Comment