Reliability and Validity of Performance Measurement in the Public Sector: The Case of the National Emergency Management Agency’s 'Battle Against Fire'

MUNSEOK CHO; Jeon， Joo Sang; Chang-Won Lee

This research explores the reliability and validity of performance measurement of 'The Battle against Fire' policy of the National Emergency Management Agency. Despite the importance of performance measurement in public management, few studies have focused on performance measurement error. This research empirically analyzes the effectiveness of the policy and reliability and validity of its measurement system. The findings are as follows: First, the effects of 'The Battle against Fire' are unclear, as the threats to internal validity such as the effects of regression, maturation, and history cannot be rejected. Similarly, the inconsistent outcomes within the frontline sub-units of the National Emergency Management Agency also suggest that the effects of the policy are unclear. Second, present performance depends on past outcomes because of endogeneity problems in relation to the outcome indicator. Finally, the causal effects of the process indicator on the outcome were restricted. These findings raise the necessity both of controllability in performance measurement and of redesigning the performance measurement system based on testing the relationship between the outcome and the process.

Citation status

KCI Citation Counts (8)
열림/닫힘 버튼
REFERENCES (38)
열림/닫힘 버튼
* 2021년 이후 발행 논문의 참고문헌은 현재 구축 중입니다.

[book] 남궁근 / 2010 / 행정조사방법론 / 法文社

[other] 소방방재청 / 2010 / 연도별 화재통계연보(1997-2010)

[web] 소방방재청 / 국가화재정보센터 / http://nfds.go.kr

[journal] Ammons, David N. / 1995 / Overcoming the Inadequacies of Performance in Local Government: The Case of Libraries and Leisure Services / Public Administration Review 55(1) : 31~47

[journal] Anspach, Rhnee. R. / 1991 / Everyday Methods for Asessing Organizational Effectiveness / Social Problems 38(1) : 1~19

[journal] Au, Chor-fai / 1996 / Rethinking Organizational Effectiveness: Theoretical and Methodological Issues in the Study of Organizational Effectiveness in Social Welfare Organizations / Administration in Social Work 20(4) : 1~21

[journal] Behn, Robert D. / 2003 / Why Measure Performance? Different Purposes Require Different Measures / Public Administration Review 63(5) : 586~606

[journal] Boschken, Herman L. / 1994 / Organizational Performance and Multiple Constituencies / Public Administration Review 54(3) : 308~312

[journal] Brewer, Gene A. / 2002 / Why Elephants Gallop: Asessing and Predicting Organizational Performance in Federal Agencies / Journal of Public Administration Research and Theory 10(4) : 685~711

[book] Brunsson, Nils / 2006 / Mechanisms of Hope. Maintaining the Dream of the Rational Organization / Copenhagen Business School Press

[book] Brunsson, Nils / 1993 / The Reforming Organization / Routledge

[journal] Campbell, Donald T. / 1957 / Factors Relevant to the Validity of Experiments in Social Settings / Psychological Bulletin 54 : 297~312

[journal] Campbell, Donald T. / 1969 / Reforms as Experiments / American Psychologist 24 : 409~429

[journal] Campbell, Donald T. . / 1970 / Considering the Case Against Experimental Evaluations of Social Innovations / Administrative Science Quarterly 15 : 110~113

[book] Campbell, Donald T. / 1966 / Experimental and Quasi-Experimental Designs for Research / Rand-Mcnally

[book] Fitzgerald, Lin / 1996 / Performance Measurement in Service Industries: Making It Work / The Charted Institute of Management Accountants

[book] Hatry, Harry P. / 1999 / Performance Measurement: Getting Results / Urban Institute

[book] Hatry, Harry P. / 1990 / Service Efforts and Accomplishments Reporting: Its Time Has Come / Governmental Accounting and Standards Board

[journal] Heckman, James / 1977 / Assessing the Performance of Performance Standards in Public Bureaucracies / The American Economic Review 87(2) : 389~395

[book] Hoffmann, Olaf. / 1999 / Performance Management: Systeme and Implementierungsansӓze / Verlag Paul Haupt

[journal] Hood, Christopher / 2001 / Risk Regulation under Pressure: Poroblem Solving or Blame Shifting / Administration and Society 33(1) : 21~53

[journal] Kopczynski, Mary / 1999 / Comparative Performance Measurement: Insights and Lessons Learned from a Consortium Effort / Public Administration Review 59(2) : 124~134

[book] Long, Scott J. / 1997 / Regression Model for Categorical and Limited Dependent Variables / Sage Publications, Inc

[journal] Lewis-Beck, Michael S. / 1980 / Can Government Regulate Safety? The Coal Mine Example / The American Political Science Review 74(3) : 745~756

[journal] Example. / 1980 / / The American Political Science Review 74(3) : 745~756

[book] March, James G. / 1994 / A Primer on Decision Making: How Decisions Happen / The Free Press

[journal] Martin, Stephan / 2005 / Multiple Public Service Performance Indicators: Toward an Integrated Statistical Approach / Journal of Public Administration Research and Theory 15 : 599~613

[book] Petroski, Henry / 1985 / To Engineer is Human: The Role of Failure in Successful Design / St. Martin's Press

[book] Poister, Theodore H. / 2003 / Measuring Performance in Public and Nonprofit Organizations / Jossey-Bass

[book] Rashid, Noorzaman / 1999 / Managing Performance in Local Government / Kogan Page

[journal] Sitkin, Sim B. / 1992 / Learning through Failure: The Strategy of Small Losses / Research in Organizational Behavior 14 : 231~266

[journal] Smith, Peter / 1995 / On the Unintended Consequences of Publishing Performance Data in the Public Sector / International Journal of Public Administration 18 : 277~310

[book] Suchman, Edward A. / 1967 / Evaluative Research / Russel Sage

[journal] Thiel, Sandra Van / 2002 / The Performance Paradox in the Public Sector / Public Performance & Management Review 25(3) : 267~281

[journal] Theurer, Jim / 1998 / Seven Pitfalls to Avoid When Establishing Performance Measures / Public Management 8(7) : 21~24

[journal] Wholey, Joseph S. / 1992 / The Case for Performance Monitoring / Public Administration Review 52(6) : 604~610

[book] Wholey, Joseph S. / 1997 / Clarifying Goals, Reporting Results, In Using Performance Measurement to Improve Public and Nonprofit Programs / Jossey-Bass : 91~98

[journal] Yin, Robert K. / 1975 / Using the Case Survey Method to Analyze Policy Studies / Administrative Science Quarterly 20 : 371~381

KJC Korea
Journal Central

Statistics

Tools

Sharing

Issue List

Citation status