The standard error of regressions: A replication of McCloskey and Ziliak's evaluation

There is a substantial body of literature dealing with the improper use of statistical significance within economic analysis. Amongst the problems that have been identified are fundamental misunderstandings about the influence of sample design and size on statistical significance, an excessive focus on statistical significance to the exclusion of economic and policy significance and a harmful conflation of different types of significance within conclusions. A meta-analysis of 51 (out of 102) agricultural economics papers reviewed and accepted for an African conference in 2010 finds improper usage of statistical significance that is comparable in nature and extent to that found in a previous meta-analysis focussing on published articles from the American Economic Review. For instance, well over 50% of the papers employed ‘sign econometrics’ and ‘asterisk econometrics’.  Overall, the findings underline the need for greater rigour in the production and selection of papers that employ regression analysis, possibly through more widespread use of guides that promote an accurate interpretation of statistical significance and stronger focus on  substantive significance.