Closed: Why should there still be a need for elaborate official statistics in the future?

The paper by Walter Radermacher (WR) is an inspiring paper that stimulates the reader’s reflections on official statistics and society. I have selected some of the points made by WR for comments.

Evidence based decision making. Do we expect a growing importance?
Evidence based decision making is an often used concept in discussions about the importance of official statistics. The vision of evidence-based decision making is however not built on a clear definition of what we really mean by evidence-based decision making.  Maybe it is a good start to consider what is the alternative – i.e.  decision making that are not evidence based.  Maybe populism or  even religious guidance for politics are examples of non-evidence-based decision making.  It is easy to see that there is no common agreement on the distinction between evidence based and non-evidence-based decisions.
There are   however some good and relevant examples of decisions where there is a rather precise use of statistics. One example may be how to compensate pensions and government paychecks for inflation. Consumer Price Indexes (CPI)   a key part of  official statistics in many countries  will be used.   It should be mentioned that the evidence taken from CPI may be criticized on scientific grounds, for example on how to consider quality changes in commodities. The Boskin report for USA "Toward A More Accurate Measure Of The Cost Of Living"  issued on December 4, 1996) concluded that as a result of no implemented method for including quality changes, there had been an overcompensation of pensions. This is an example of that even what is  believed to be evidence based compensation of pensions may be scientific disputed

There is an obvious gain for the discussion if we introduce a distinction between the cases where data is used at the individual level for decisions at the individual level and when data/statistics is used at the aggregate level – for example to adjust fiscal policy. There are many examples of individual decisions using data – like taxation. This may not however be examples of the use of statistics for evidence-based decision making. Statistics is used to design the taxation system – tax policy – and individual data are used to decide the correct tax for each individual.  

When it comes to official statistics it is important also to bring in what WR  adds that official statistics belongs to the core of inalienable public service. --- We are talking about information that is held as a public good in an infrastructure for utilization by citizens, entrepreneurs, teachers, researchers, politicians or in other words by everyone. With reference to Alain Desrosières and Theodore Porter the concept of enlightenment is included in the discussion. The full content of official statistics in a country  has to be set up based on listening to user demands but also to take initiatives and produce official statistics  that the statistician sees that will enlighten the society – that is to be  paternalistic. It has been said that National accounts had to be introduced   with a paternalistic attitude  and would never have been included if official statisticians were only listening to user demands.

Data, statistics, facts – the new fuel.
One interesting discussion of concepts in the WR paper is the distinction between evidence-based decision making is replaced  and data driven decision making. One statement in the WR paper is “Data is the new fuel”.  It is also a discussion of how  data  meet politics. Some paragraphs in the WR paper raises also  some provocative questions.

One questions raised is: “Why should a politician wait for official statistics when there was real time data for the most difficult decision”.

Another provocative question is “Why afford a statistical service to the cost of taxpayers when it seems so easy to analyze the data that all the machines mutually connected within the   Internet of Things are generating all the time?”

I observe that my answers to these question places me as a traditional defense attorney for classic  official statistics.  My general attitude is that official statistics is a system of various single statistics  that is put together in a system.  In addition  to characteristics of each statistics  the system has inbuilt  consistency and coherence.  International comparability  is also  an extremely  important part of the system of official statistics.  So may be there are examples of single statistics that can be produced cheaper  and with acceptable quality but we have to look at the full system and give value to consistency  coherence and  international comparability: he answers to the previous questions.

Some few more words on characteristics of the full system of official statistics. Well defined statistical units
•    Description and control of the statistical populations with full coverage and no double counting
•    Classifications – relevant for the statistical tables
•    Methods and standards, definitions.  Concepts may be simple as total number of units or more complex as CPI and GDP
•    Data and observations. Data may be new observations about the statistical units. Data will also be collected from archives in the statistical office, Microdata or previous statistical outputs.
•    Dissemination channels, web and paper.
•    Detailed documentation 
The system is constructed with components as primary data, archive data, classifications, methods and standards (as SNA).. It is my opinion that quick fix solutions with cheap big data will be far from delivering comparable quality statistics,  

How do WR define facts?

The triangle in the WR paper gives an intuitive description of the concepts. As I see it  one example of policy may be taxes, benefits, services produced for individuals as education, health service. So in this way I see how policy may generate observations that may be collected and stored as data.  These data may represent facts about individuals. The arrow from data I would have expected should point to  statistics -  which is not facts. Why facts produce data is not clear for me so I do not follow the arrow from facts to data. Statistics or facts in WR term will influence on policy    and that explains the arrow from facts to policy but why policy both influenced data and facts is for me unclear.
Data is a wide and vague concept.  Data is both observations and computed statistics that have been collected and stored. At the present storing means electronic storing.  Data may be used for various purposes. In some cases, data about individuals have been stored and delivered for use as background information- for a treatment It may be data from medical records. But many other examples can be given. If the decision – treatment – of the individual can be labelled policy is not obvious. May be governance could be instirduced.

Data is the new fuel
This is a relevant reminder about the importance of several types of data driven algorithms both in private and public sector. Artificial Intelligence AI is a concept used in many contexts with a rapid expanding importance.
It is obvious that WR is inspired of this trend. For official statistics two different aspects of these trends may be identified
•    How can the production of official statistics become more efficient by introducing AI inhouse in Statistical offices?
•    How can official statistics be used as input in other algorithms? A new user group for official statistics
•    Will widespread use of AI have effects on the demand for official statistics, eventually also on the surroundings for the production of official statistics.
•    AI will create a demand for improved rules and laws for data protection.

Another discussion in the WR paper is the so-called interrelation between “Governing by the numbers” and “Informational Governance”. My recommendation for bringing this discussion forward is to discuss concrete examples in detail.  
We have to bear in mind that statistics as a scientific discipline does not have a tradition for the use of the concept - fact.
Statisticians produce estimates of various parameters in statistical distributions. This is done based on observations (often labelled data). Each observation about a unit of the population may in theory be checked to find if it is true or false. Even if all observed individual data are true, the estimate of e.g. the total number of units may not be true.  If the task for the statistician is to estimate the average value for the total population   this may be done by a point estimate and the statistician will support the point estimate with an estimated confidence interval.

Official statistics is not only about estimation of single parameters like the average or mean but also to support theories and assumptions about relations or interactions between social phenomena. So, the question about facts may also include questions about true or false social theories. Some will also ask questions about causalities. These are difficult questions and we know that observed statistical correlations may be spurious or non-meaningful.

Evidence based decision making at the global level
Some of the serious political challenges may be national and may find national solutions based on national evidence or national official statistics. There are however several examples of challenges that are global by nature, the evidence and the policy have to be global,
Climate challenge is an example of such an extremely complex  and global problem. A lot of people are concerned and have good intentions, but good intentions is not sufficient to secure an effective global climate policy. One important aspect of the climate problem is that it is a global problem. The sum of emissions has a negative effect on the whole world i.e. all nations independent of the national contribution to a reduction in emissions. The climate problems all nations will face are a result not only of own emissions but a result of the total emissions from all countries.
The solutions have to be based on an efficient and strong international cooperation (supra national?)
It is not only the climate challenge that is global, but we see that economy and business have strong global components. There are extreme strong supra national capital powers.  There are multinational companies that are extreme powerful.

All the international business activities an international value chains have obvious negative effects on the possibility for all countries to produce high quality national economic statistics. The national accounts have in most countries’ severe quality problems and one of the main causes are all the cross-border transactions, Traditional international statistical cooperation may help but such a cooperation is not sufficient.
Two important aspects for a solution
•    For official statistics we need a strong and even perhaps supranational commitment 
•    Rules and practice that secure exchange of statistical microdata between countries strictly for statistical purpose.

The scope for international statistical cooperation has to be extended. One theoretical solution could have been one big super efficient  global statistical office with access to data for all countries and data for all kind of transactions between countries.  I see such a solution as complete unrealistic.  So how could we simulate a similar solution based on national statistical offices  with strong supranational institutions  conducting international statistical activities. A key element will be to  give national statistical offices access to statistical data from other countries. Strict rules for data protection have to be  internationally accepted. 

Without data – statistics stops.
Official statistics are faced with quality challenges. Some of these problems are results of data problems. In many cases statisticians observe reductions in response rates for statistical surveys. Data collection with traditional methods   are also resource demanding. It is natural that Big data may be regarded as a future solution.  There are however some prerequisites that should be fulfilled for full benefit from Big Data in official statistics. For official it is important to have a statistical population with well-defined statistical units and well-defined statistical population both for full count or sampling surveys.  For an efficient statistical production process, it is necessary to store data with the use of a well-defined ID number system. This is more or less necessary for linking of information from different sources and also for establishing of longitudinal data.  Some components from various data sources of electronic traces may fulfill such quality characteristics - e.g.  with a link to a general ID number system. A statistical system for a country will normal be based on institutional approach, that is that we define the institutes – statistical units – persons, family, households, companies, enterprises and establishments and so on. The we collect data for the units and produces national statistics.  Another approach will be to build statistics on functions. We observe and do counting of activities - functions. One example  will be  that we at  the border  observe all goods that cross the border, i.e. export and import. Road transport may be measured by observing traffic at a measuring point if we follow the functional approach.  The institutional approach  will be that  we ask all companies about their  transport work.  A smart strategy for data collection should balance between both institutional and functional approach – but for me it seems more secure for consistency and coherence and full coverage and no double counting if the statistical system  follow the  institutional approach. So, when moving into big data sphere how to secure control over the units and statistical population  which is fundamental for quality in official statistics.
 

The article is a useful taking of the temperature of official statistics as it is practised today, at least from the vantage of those in the inner circles of national (and international) agencies charged with its collection, processing and publication. Further I think that comments elicited draw out the strengths in the paper: reference to formation of official statistics as a discipline - 'beyond mathematics'; fluency with the policy world, not only at the top but as well built up from recruitment of statistical graduates to the national service. The challenge from new data sources appears to be overemphasised, and I would agree too that 'talking to customers' unless backed by the confidence of years of experience in the daily grind of the statistical factory, is perilous.

I did like the openness to the shift from national to global official statistics in Dr Rademacher's article, and found his remarks there approrpriate and useful. Official Statisitcians work closely with the policy process - a defining characteristic. Policy (not politics) relates to public governance; and there is indeed a fluidity in what is being governed for (and by) whom. Understanding this environment certainly justifies the call fo 'humanise' their education.

At the same time the data revolution - which seem almost to overbalance the account, is not foreign to the challenges that have propelled official statistics into the changing world, from the 18th centuiry forward. That they point to the engineering complexion of methods advances - at least since the time of mainframe computers, pioneered by people working on offical collections. The ethical bracing, taken up in the article, is tested given issues of ownership, privacy protectionm, quality standards, and scientific, political and managerial scruple.

As with past socio-technical change the fallacy is to believe that the system in place should guide the future. The eingineering side of OS should spring into action: what society wants in its shifting governance moods is responsiveness of design, and thinking to the informational challenges. Otherwise expressed what OS needs in practical terms is an information map,whether articulated in founding legislation of a national or supranational service, or more diffusely by emerging debates on governance - such as in the appearance of the SDGs; or calls for human rights statistics, or climate monitoring. From this we can build the bridge to statisitcally sound answers appearing when they are needed, not five years afterwards.

We have agreed on the principles (across the profession); yet we are coy about who we will trust to supervise; how we will certify the product. Just as the civilian air transport system incorporates a high standard of public safety (irrespective of jurisdiction) backed by the engineering profession, so the statistics needed for emerging world will be backed by the same professional tradition that created the national accounts, the systems of household social surveys, vulnerability indexes and so on. But we need to invest in this profession!

This is a thoughtful paper that addresses many challenges that official statistics face. I liked especially the double focus on the production and use of numbers/indicators, both related through the notion of a statistical chain. With regard to the use of indicators I fully agree that there should be more "market research". Probably that could be a fruitful field of cooperation between academia official statistical institutes.

Furthermore the triangle representing Data, Facts and Politics seems to work well for going through their relationships in a systematic way. Yet, their graphical depiction (showing reciprocal and symmetrical arrows going from one corner of the triangle to the other) could be misleading, in particular with regard to the relationship between data and policy. I guess - and this is rather in agreement with the text itself - the arrow should point from policy to data only.

The author bemoans that very often raw data and the facts resulting from them are both labelled "data", the triangle suggests that there could be a fruitful and direct relationship between data and policy. I agree that such a relationship is often imagined, such as in the catchphrase Data for Policy (D4P) that relates basically to new (digital) data sources and also to political expectations towards our assumed data wealth. However, such a direct relationship seems to be rather ficticious. Raw data typically requires laborious steps of refinement - among that I would also count what the text called 'experimental statistics' - before "facts" can be derived. Although it might not be impossible to fully automatize these stages of refinement, the automatization itself requires time, methodological decisions and the explication of correspondence rules that assure the validity of aspired "facts". When these correspondence rules stand the test of professional reflection and critique, results can legimitately be regarded as "facts".

Hence, different from what the figure of the triangle suggests, I would argue that "data" that seem to intuitively speak to laypeople as "facts", indeed invisibilise the professional and institutional work of sens-making that has been invested into them. I think that there is no shortcut from data to policy or politics without serious efforts of sense-making. But that is rather a call for more "reasearch-and-development" with regard to new "raw data" sources and does not preclude that the resulting facts might drive politics or policy at some point in the future.

There’s a lot in Walter’s paper that many of us have been thinking about and saying for years.

However I’m struggling with his point on the lack of textbooks and training material. On the one hand we all know that there is a general problem of statistical literacy on the user side, but is the same true on the producer side? Official statistics is something of a niche business and even within it there are some even more niche areas. It’s difficult to see how this could be done in an efficient manner. Who would be responsible? How much would it cost? What is the market for the proposed textbooks? Where does this fit into a very long priority list? Where do we start? And… where do we stop?

Yes, there is a skills shortage that continually needs to be addressed, but the official statistical community has worked well in this, given resources and priority constraints, through co-operation programmes, existing training initiatives, institutions like IOAS and a plethora of manuals and handbooks. In niche areas, where there are so few experts, one-on-one training though getting them together is an efficient solution and this approach has been particularly successful in national twinning schemes.

Congratulations to the author! Indeed a thought-provoking paper with a lot of issues of relevance for statistics in a very unstable, fragmented, digitised and globalising world with in some quarters an anti-science, and anti- (expert) knowledge attitude. I will give some general comments or better ideas, that were induced by reading the paper and then give some concrete reactions to the topics raised.
Of course, statistics and science in general cannot solve all the ills of society but at least we can try to keep our house (statistics) in order and keep it tidy. One of the main issues that emerge in the paper and also in the issues for discussion is the right focus. It is fashionable to see statistic as an a business enterprise with products that the customers (clients) need or want. The objective of business is to make a profit; statistics are common goods and should be freely available to all sectors of society. There is therefore an inherent conflict between these two views! I personally see the production of official statistics as part of public administration, which requires a special status (see Fundamental principles) to perform optimally!
On several occasions Walter mentions the need for epistemology bring some order in the chaos. I fully agree and will go further and state that what is needed is that all statisticians, or all academics for that matter, have a clear understanding on the nature and principles of science in general and of the main institutions and sectors of their society. What is required is a solid education in the principles of sciences (logic, ontology, epistemology and methodology) so that statisticians will be able to develop proper definitions, systems of reasoning, avoid tautologies and fallacies.
Whether we like it or not statistics has a fundamental mathematical underpinning and we can basically make a distinction between theoretical (mathematical) statistics and applied statistics, statistics used in a wide variety of disciplines. Official statistics are just statistics provided by governments, which are needed for society to function properly. The statistics required have to reflect the needs of the society as a whole and not just what the government of the day would like, and yes, it needs the input of all relevant sectors, or to use a fashionable term, stakeholders. There are several mechanisms in place to receive the input of society (or stakeholders) in the national statistical needs or national programmes. Now, there are criteria, which hopefully are explicit and known to stakeholders, what could and could not be part of a national statistical programme that will be carried out by the national statistical system. Some topics of interest to specific sectors or sub-groups in society may not qualify, and for which ad hic research projects could be developed by other professional organisations. Also, often vocal sectors of societies against the results of statistical exercises are often those that do not participate in the definition and shaping of the national statistical needs programmes.
There are two main issues we have to deal with, 1. Who is a statistician, and 2. What is the subject matter of statistics?
Until recently statisticians were persons with a specific training (academic or professional) in statistics, either as the general science of statistics or a specific type of application (remember actuarial science and actuaries) or in a specific discipline (astronomy, agriculture, health sociology, economics, etc.). Now, basically everybody who uses statistical techniques or data is considered a statistician, irrespective of their background or training. Remember some statistical packages claim that to apply some rather sophisticated statistical technique one does not need to have studied statistics!
There are myriads of definitions of what statistics as a profession or science is and also what statistics are. Not everybody may agree but statistics as a science (free to Fisher) may be considered as the application of mathematical principles to observational data, of populations (any collections of objects) with the aim to study (describer and explain) their variations, interrelations and causes and the reduction of the masses of data. Statistics are series of numerical values of objects (parts of populations, hence calculated values (facts) of characteristics of society or the universe affected by multiple causes (free to Yule)).
Based on the above, not every number is a statistic! Statistics are constructed or calculated numerical values of characteristics of a unit of observation that derive their meaning from a conceptual framework, that have probability distributions, and are time and location specific. Statistics that are confirmed over time and different location may become (scientific) facts.
Now some remarks on the items for discussion:
Although I like the article very much for the issues to present, I do not share the sales pitch of “'Governing-by-the-numbers”. Numbers have no intrinsic meaning; they acquire meaning only when they are linked with an object. The statement “You ow me 100” means nothing. It becomes interesting when we know that 100 could means dollars, euros or ounces of gold! It is better to state “Governing by statistics”.
Main item:
Indeed data are given facts are produced. Data are the raw material for statistics,. However it takes a long time for facts, scientific facts, to be established hence it is more prudent to start governing using good statistics!.
There is a need for authoritative statistics, and society needs “official statistics” and a system that produces them free from interference by governments and societal groups. What is needed is an autonomous national statistical system headed by an independent National Statistical Authority, and bringing together national statistical data producers, Government agencies, but also, (why not?), academic research institutions that produce the required data of an flexible but comprehensive programme established through dialogue and consultation of all sectors of society. One key issue for such a system is the necessary financial resources to ensure that it is truly autonomous.
Secondary items
Trust in official statistics.
The level of professionalism or adherence scientific principles of the data producers and transparent and comprehensive communication of the data and their limitation are key elements to create trust in the national statistical system.

New data sources
Statisticians should be weary of being swept away by the lure of new sources of data and should carefully study what these new sources are and how they could fit the requirements of the national statistical system and how well these new sources abide by the principles of science.

Closing the gap.
Official statistics is not a special field of statistics; trey can be of any areas of science provided that they are produced by the system that provides the “official” label. Up to now it is government produced statistics, but if the national system under a national statistical authority is created and operated they will be responsible for the official statistics.
Up to now official statistics is part of the public administration. In public administration as a domain of action, research and study there are a many different monographs, guidelines and studies dealing with all aspects of public administration. There is even a Handbook of Research Methods in Public Administration as part of a Comprehensive Publication Program; some of the publications in this series answer part of the concerns presented by Walther. See the Public Administration And Public Policy programme of the CRC Press of the Taylor & Francis Group.

A very wide ranging paper. There is so much to react to, it's hard to know where to start.

I found the remark that EBPM is a tautology in enlightened democratic societies fascinating. I had to pause and think about that. In the end, I guess it is probably is, but it wasn't always so. So, I guess that change in and of itself is a mini data revolution. Unfortunately, the enlightenment may be shorter lived than we had hoped.

But there are really 2 points I want to react to:

The first is regarding statistical literacy and closing the scientific gap. In the paper Walter notes that 'statistical competence and literacy must not be reduced to mathematical-technical skills. Rather, it also requires an understanding of the fundamentals of the humanities' (p. 523). I was so happy to read this. I could not agree more. I presented a paper at the ISI this year, effectively, but less eloquently and succinctly, making the same point. This is of profound importance to any debate on the topic.

The second issue is the role of civil society. This links directly to our paper posted on the first discussion platform. How do we co-design and co-produce? In general I agree with the thesis being proposed by Walter. I think in an era of rapidly globalizing data, the question posed 'whether the statistical governance structures dating back to the discussions of the 1990s, which focused on nationally organized public administrations of statistics...still meet today's challenges' (p.533) is a huge issue. In some recent papers, I have asked the same question. This is obviously a very sensitive issue, but it is one that surely needs to be discussed. Where I hesitate slightly, is the idea that the 'separation between the producers and the consumers of statistics needs to be removed' to create 'prosumers' (p.513). While I understand the sentiment and without question greater participation in design is the way forward, But I'm still inclined to think that some separation between production and consumption is safer for everyone.

These are my initial reactions. Congratulations on a very thought provoking paper.

Steve