RMM Vol. 3, 2012, 178–184
Special Topic: Coevolving Relationships between Political Science and Economics
Edited by Herbert David, Hartmut Kliemt and Elinor Ostrom
http://www.rmm-journal.de/
Alexander Libman
Learning to Be Different: Quantitative
Research in Economics and Political
Science*
Comment on Achim Schlüter and Róger Madrigal,
“The SES Framework in a Marine Setting: Methodological
Lessons”
Abstract:
The comment addresses the subtle differences that exist between economics and political
science in terms of how the standards for the empirical quantitative research are set.
It shows that the common methodology is applied by the disciplines in a different fash-
ion. These differences could become obstacles for communication, but could also provide
fruitful background for discussion of disciplines, if one explicitly takes them into account.
1. Introduction
The spread of rational choice theories and quantitative methods in political sci-
ence and the advancement of political economics from the periphery to the core
of economics created numerous opportunities for learning and exchange between
economics and political science. Researchers, theories and empirical results
cross disciplinary borders more and more frequently. Achim Schlüter and Roger
Madrigal point out yet another possible area of fruitful exchange between disci-
plines: the use of phenomena-driven research.
I intend to use Schlüter’s and Madrigal’s instructive discussion of how eco-
nomics and political science can address similar topics with similar methods
to illustrate that despite topical and methodological overlap the two disciplines
may still be going different ways. In doing so I focus on the empirical quanti-
tative scholarship in political science and political economics and try to look at
more subtle differences in the way arguments are constructed and weighted by
the disciplines.
My comment does not aim to take a stance for or against any of the disci-
plinary approaches; it merely intends to describe the differences, which seem
* The author appreciates the very helpful comments and suggestions of Hartmut Kliemt. All mis-
takes remain my own.
Learning to Be Different 179
to exist between disciplines. In a world where both economists and political
scientists try to learn from empirical evidence produced by the other, under-
standing these differences will facilitate communication. Both the economics
and the political science view on how to do empirical research have advantages
and disadvantages.
2. What Do Regression Results Mean?
The starting point of empirical quantitative research in both economics and po-
litical science is similar. There is a research question, which is derived from
the theoretical debate in the literature, and typically asks for the existence of
a causal link between a number of variables (e.g. democracy and public spend-
ing; resource availability and likelihood of civil wars). To test the existence of
this causal link, some sort of econometric procedure is used: typically regres-
sion analysis, based either on an original or a secondary dataset. The way how
this statistical analysis is performed is itself subject to debate (see for critical
assessments of how econometrics is done in economics (Kraemer 2011), political
science (Schrodt 2010) and business administration (Kmetz 2011)). Yet my com-
ment is interested in what happens after statistics has been done: how can we
interpret our findings?
Within the perspective of economics, straightforward cross-sectional OLS re-
gressions are unlikely to provide accurate empirical evidence. The endogeneity
problem is pervasive. This has severe repercussions for most political economics
papers. In fact, endogeneity has been the central concern of empirical economics
in the last decades. Substantial advancements have been achieved. Angrist and
Pischke (2010) describe what they call a ‘credibility revolution’ in economics,
associated with better identification strategies and more active use of natural
experiments. Indeed, a lot of work in modern empirical political economics is as-
sociated with improvements of identification strategies, by searching for better
instruments, random treatments or panel data settings allowing to remove the
unobserved heterogeneity (e.g. Acemoglu et al. 2002; Miguel et al. 2004; Gorod-
nichenko and Roland 2010; Enikolopov et al. 2011). The central premise under-
lying this is that if the empirical design is sound and endogeneity problems are
taken care of then the outcomes of regressions can be viewed as evidence for or
against the theoretical conjecture the paper aims to test.
Political scientists are also cautious about simple cross-sectional OLS regres-
sions, and a lot of attention has been devoted to identification strategies in po-
litical science as well (see the survey in Sovey and Green 2011 and an example
in Kern and Hainmueller 2009). However, unlike economists, political scientists
often seem to be generally skeptical about the ability of econometrics and statis-
tics to prove the existence of causal links between variables and even if there
are no endogeneity problems in estimations. Instead, the researcher is required
to describe a possible causal mechanism, which would explain the link between
180 Alexander Libman
variables, and also provide at least partial evidence in favor of this mechanism.
This evidence can be derived from qualitative research (e.g. process tracing) or
be based on well-developed theory.
There is a striking difference between how economics and political science
deal with causality. In economics qualitative research is typically not accepted
as evidence; causality is inferred from econometric regressions, if the latter man-
age to deal with endogeneity problem. In political science on the contrary a lot of
work has recently been devoted to qualitative (Mahoney 2010) and mixed meth-
ods (Lieberman 2005) to provide background knowledge on causal mechanisms;
econometrics as such is considered as merely an indication of possible existence
of causal effects, but not as proof.1
As such, the task of identification in political science is in some regards easier
than in economics and more difficult in other. On the one hand, since statistical
evidence is not considered the ultimate criterion of truth, there is more space
for simpler models and estimations than in economics, even if endogeneity could
be present; the (often hopeless) quest for perfect instruments does not pose a
challenge comparable to that in economics. On the other hand, even the most
advanced statistical technique is unlikely to be accepted as compelling evidence,
as long as no mechanism is suggested linking the variables in a convincing and
systematic way and based (possibly) on thorough qualitative research, elabo-
rated theory and knowledge of the case.
3. Theory and Conceptualization
Both economics and political science share the idea that empirical research
should be guided by theoretical investigation. Yet the role of theory and the
way theory is constructed is different again. Economics clearly equates theory
to a formal model. This formal model, in turn, can either be used as a moti-
vation for the paper, providing a simplified description of the logic one intends
to test in the empirical case, or may directly generate equations to be estimated
empirically. Though the second option is by far less often present in political eco-
nomics (unlike other fields of economic research), it is considered superior since
it makes the link between the theory and the empirics easier to trace. Empirical
papers that do not use an explicit formal model, are at least expected to refer to
the established formal results when they formulate their hypotheses.
Political science is, generally speaking, more open to testing theories, which
have not originated from formal models. However, at the same time political
science is much more cautious in terms of conceptualization of key elements
of the model, i.e. definitions of concepts and of applicability of the model. While
discussing democracy, autocracy, federalism, decentralization, regionalism or in-
1 I am well aware that the notion of causality is treated much more carefully in theoretical econo-
metrics; my comments rather reflect the perception of (many) empiricists.
Learning to Be Different 181
tegration a political scientist is by far more likely to be asked about the precise
meaning of the concepts applied and their relation to each other. Conceptualiza-
tion is in fact more important than a formal model; the latter ‘makes sense’ only
if embedded in the general discussion of key concepts.
This difference could be explained by two reasons. On the one hand, eco-
nomics is a much more homogenous discipline, with a universal language pro-
vided by standard microeconomics and macroeconomics. The basic concepts of
that language are much less often subject to debate than basic concepts in po-
litical science (although, possibly, unjustly, see Green and Kotlikoff 2007). The
very idea of political economics and public choice was to use this already ex-
isting language and body of theories to describe and to explain political phe-
nomena. And, even researchers who are skeptical about the empirical validity
and explanatory power of basic theories like that of the model of rational eco-
nomic man still seem to be convinced of the value of a rather precise language
to express their ‘economic stories’ unambiguously. Political science, on the other
hand, contains multiple frameworks (for example, those described by Schlüter
and Madrigal): each of them offers its own vocabulary for theory-building. De-
veloping a new conceptual framework is considered an acceptable and necessary
part of research (e.g. Buzan and Waever 2011). And it is much more difficult to
select the appropriate framework based on any objective criteria—more so than
to confront different theories in an empirical study. Hence, it becomes crucial to
‘root’ one’s discussion in a particular literature or argument to signal its link to
a particular framework and to avoid confusion and misunderstanding.
The dominance of formal models in economics is also explained by the idea
that the same model can be applied to entirely different circumstances, just
by re-defining the notation (Schofield 2004). Political science is more open to
other ways of theorizing than formal models and to a multiplicity of domain
specific conceptual frameworks. Obviously in political science more attention to
terminology is required.
It is interesting to observe how these differences reflect themselves in a typ-
ical shape of an economics vs. political science paper. A political science article
typically starts with a thorough and detailed literature review, followed by the
discussion of possible causal links (‘theory’), which is then tested empirically. An
economics paper sets off with a brief and compact literature discussion (some-
times limited to introduction), and moves to the formal model or econometric
equations as soon as possible. These compositional differences are, in fact, of-
ten sufficient to confuse the researcher: an economist is lost in endless debates
on terminology and lengthy verbal discussion, while the political scientist lacks
the understanding of what actually the presented formal model and subsequent
econometric test aim to look at and is concerned about the sloppy use of termi-
nology.
182 Alexander Libman
4. Data
The very definition of the quantitative research in economics and political sci-
ence requires the use of sufficiently large datasets for econometric analysis.
Here, again, political science and economics diverge in terms of accepted prac-
tices. Traditionally, economists have been mostly using secondary statistical
data. Though this has changed to some extent by the ascent of experimental
economics the—in this regard—particularly relevant field experiments are still
limited in application and today an ‘ideal’ economics paper takes advantage of a
large (secondary) micro-dataset.2 Political scientists are much more likely to use
primary statistical data generated by researchers themselves.3 It is therefore
less surprising that political scientists are typically much more concerned with
the way the data is measured and the variables are coded and with the qual-
ity of data. The fact that political scientists are typically more knowledgeable
about institutional details of a particular setting leads to additional concerns
and questions about the data. At the same time, an innovative dataset is more
likely to be considered a benefit for the discipline as such, while for an economist
it is of merely instrumental value. The need to incentivize researchers to collect
novel data is strongly pointed out in political science (e.g. Gibson 1995) and less
so in economics (the experimental economics development notwithstanding).
Economists and political scientists may find it difficult to understand each
other, precisely because the methodologies they use appear to be so similar. For
an economist, an empirical political science paper may seem to be doing the
right thing by using econometric technique but be based on too simple econo-
metrics. The economist would expect more robustness checks and methodolog-
ical advancements to be present in this respect. A political scientist would be
concerned about the economists’ use of inadequate or questionable data and typ-
ically feel that an econometric sledge-hammer is used to crack a conceptually
soft problem.
2 Typically, political economists and political scientists describe as ‘natural experiments’ political
changes, which created circumstances similar to randomized treatment and not experiments in
the sense that they have been designed and implemented by economists themselves. For example,
a natural experiment can be a consequence of unexpected policy changes and reforms (e.g. Ger-
man reunification, see Alesina and Fuchs-Schündeln 2007 or, with some caveats, post-Communist
transformation, see Gehlbach and Malesky 2011; Frye 2012) or of the influence of geographical
factors orthogonal to policy variables (e.g. Yanagizawa-Drott 2012). True ‘field experiments’ are
popular in some areas of economics as well (e.g. development economics, see Duflo and Kremer
2005), but rare and difficult to implement in political economics and political science (although
there are some examples of natural experiments in this area as well, see Banerjee et al. 2012;
Beath et al. 2012).
3 These datasets are often based on coding qualitative information, e.g. legal texts and acts, political
events (wars, elections, rebellions, etc.), party programs, biographies of politicians etc., or on
surveys implemented by researchers.
Learning to Be Different 183
4. Concluding Remarks
This comment probably provides a biased picture of both economics and politi-
cal science—more of the latter, since political science is more heterogeneous than
economics. Thus, it should not be considered an empirically accurate analysis—
it merely offers some observations for further discussion. The stylized facts pre-
sented here should serve as a reminder how essentially the same topics and
methods can be applied with different emphasis by different disciplines. The
construction of inter-disciplinary bridges, therefore, requires not only attention
to the grand methodological debates, but also to more subtle differences.
References
Acemoglu, D./S. Johnson and J. A. Robinson (2002), Reversal of Fortune: Geography and
Institutions in the Making of the Modern World Income, Quarterly Journal of Eco-
nomics 117(4), 1231–1294.
Alesina, A. and N. Fuchs-Schündeln (2007), Good-Buy Lenin (Or Not?): The Effect of
Communism on People’s Preferences, American Economic Review 97(4), 1507–1528.
Angrist, J. D. and J.-S. Pischke (2010), The Credibility Revolution in Empirical Eco-
nomics: How Better Research Design Is Taking the Con out of Econometrics, Journal
of Economic Perspectives 24(2), 3–30.
Banerjee, A./R. Chattopadhyay/E. Duflo/D. Keniston and N. Singh (2012), Can Institu-
tions Be Reformed from Within? Evidence from a Randomized Experiment with the
Rajasthan Police, NBER Working Paper No. 17912.
Beath, A./C. Fotini and R. Enikolopov (2012), Winning Hearts and Minds through De-
velopment: Evidence from a Field Experiment in Afghanistan, MIT Political Science
Department Research Paper No. 2011–14.
Buzan, B. and O. Waever (2011), Regions and Powers: The Structure of International Se-
curity, Cambridge: Cambridge University Press.
Enikolopov, R./M. Petrova and E. Zhuravskaya (2011), Media and Political Persuasion:
Evidence from Russia, American Economic Review 111(7), 3253âĂŞ-3285.
Frye, T. (2012), In from the Cold: Institutions and Causal Inference in Postcommunist
Studies, Annual Review of Political Science 15, 245–263.
Gehlbach, S. and E. J. Malesky (2011), The Grand Experiment That Wasn’t? New Insti-
tutional Economics and the Postcommunist Experience, mimeo.
Gibson, J. L. (2005), Cautious Reflections on a Data-Archiving Policy for Political Sci-
ence, PS: Political Science and Politics 28(3), 473–476.
Gorodnichenko, Y. and G. Roland (2010), Culture, Institutions and the Wealth of Nations,
mimeo.
Green, J. and L. J. Kotlikoff (2007), On the General Relativity of Fiscal Language, mi-
meo.
Kern, H. L. and J. Hainmueller (2009), Opium for the Masses: How Foreign Media Can
Stabilize Authoritarian Regimes, Political Analysis 17(4), 377–399.
184 Alexander Libman
Kmetz, J. L. (2011), Fifty Lost Years: Why International Business Scholars Must Not
Emulate the US Social Science Research Model, mimeo.
Kraemer, W. (2011), The Cult of Statistical Significance—What Economists Should and
Should Not Do to Make Their Data Talk, Schmollers Jahrbuch 131, 455–468.
Kremer, M. and E. Duflo (2005), Use of Randomization in the Evaluation of Develop-
ment Effectiveness, in: Pitman, G./O. Feinstein and G. Ingram (eds.), Evaluating
Development Effectiveness, New Brunswick: Transaction Publishers, 205–231.
Lieberman, E. S. (2005), Nested Analysis as a Mixed-Method Strategy for Comparative
Research, American Political Science Review 99(3), 435–452.
Mahoney, J. (2010), After KKV: The New Methodology of Qualitative Research, World
Politics 62(1), 120–147.
Miguel, E./S. Shanker and E. Sergenti (2004), Economic Shocks and Civil Conflicts: An
Instrumental Variables Approach, Journal of Political Economy 112(4), 725–753.
Schofield, N. (2004), Mathematical Methods in Economics and Social Choice, Heidel-
berg: Springer.
Schrodt, P. A. (2010), Seven Deadly Sins of Contemporary Quantitative Political Analy-
sis, mimeo.
Sovey, A. J. and D. P. Green (2011), Instrumental Variables Estimation in Political Sci-
ence: A Readers’ Guide, American Journal of Political Science 55(1), 188–200.
Yanagizawa-Drott, D. (2012), Propaganda and Conflict: Theory and Evidence from the
Rwandan Genocide, mimeo.