Wednesday, December 21, 2011

Wither Mathematical Statistics?


Herbert Robbins wrote (or talked) a nice reflection on the state of mathematical statistics in 1975, published a paper under this title in Advances of Applied Probability, Vol.7. This was probably at the end of golden years of theoretical statistics, some of this history also captured nicely in the very entertaining book by E.L. Lehmann, Reminiscences of a Statistician: the Company I Kept, published by Springer 2008. Mathematical statistics (ms) probably came out of the need to provide the rigor and theory for data analysis and data collection in the period of agricultural boom in the hands of R.A. Fisher, E.S. Pearson, Jerzy Neyman, etc, and was probably well justified to boom right after demonstrating its usefulness during the second world war along with the birth of operation research. But this inward-looking and self-indulged developments, though productive, probably the golden years in statistics, but by 1975, the initial excitement seemed to have run out of steam and it is in need of new air and fresh ideas. It is interesting to the reflections  from one of the major figures in modern statistics on the future of  ms at the time. He is probably not alone, as John W. Tukey had said the same thing earlier many times, such as in the earth-shaking article: The Future of Data Analysis, published in 1962, Annals of Mathematical Statistics. Apparrently there had been a disconnect between the development of mathematical statistics and data analysis. That was bad, as mathematical statistics was supposed to help "doing data analysis", not being done as another branch of mathematics.  This was an unfortunate development, since its founding fathers like Harold Hotelling and Abraham Wald, among others, many of whom have both applied roots such as economics but would like to see statistics as an integral but independent component of the scientific discipline, not just a subdiscipline sitting within a mathematics department. Ironically, whether it is C. Stein's breakthroughs or systematic efforts such as the 1970s robustness studies, they all proved  that the optimality of statistical procedures is more an illusion than something can be achieved remotely in practice. Now in the new century, the tide has completely changed with the waves of problems in biology and medicine such as imaging and if statisticians cannot face up to the task of the new digital era, and many others such as engineers and computer scientists will gladly do the work instead. Today it will be surprising to find any modern statistician who is not involved in some kinds of real application areas, whether climate, environment or clinical trials. Statisticians nowdays need to know more than a few areas, including many statistical methodologies  and theories, so that,  when faced with a consulting problem from a scientist colleague, they do not have to take on Robbins' advice of following the Hippocratic Oath and do no harm, they can actually make inroads into some potentially new areas.

References:
1.J. W. Tukey (1962), The Future of Data Analysis, Annals of Mathematical Statistics.
2. 陈希孺: 数理统计学:世纪末的回顾与展望, 统计研究, 2000 年第2 期.

2 comments:

  1. Heya¡­my very first comment on your site. ,I have been reading your blog for a while and thought I would completely pop in and drop a friendly note. . It is great

    stuff indeed. I also wanted to ask..is there a way to subscribe to your site via email?





    Mathematics and Statistics

    ReplyDelete
    Replies
    1. Glad to see someone actually read my stuff. This blog thing is new to me and I don't know how you can can subscribe. Can you choose to follow someone?

      Delete