Every Model is Wrong
We have been discussing Austrians and their relationship to “mainstream” economics here lately and that topic raises a lot of issues. I want to go into some of those issues in depth but I want to address them in a very broad way that isn’t really about Austrian economics. I want to address some deep philosophical questions surrounding the nature of models and their role in economics and the mindset of many people in various “heterodox” schools including Austrians but also Marxists, post-Keynesians and so on.
In general, models serve one or both of two purposes. They can be explanatory and exploratory. By the former I mean that a model can be used to explain some concept that the creator has in mind in a (relatively) simple way to someone who does not necessarily fully see or understand it to begin with. By the latter I mean that it can be used to explore a question which the creator of the model does not know the answer to. Now clearly, a model which is originally exploratory, typically becomes an explanatory tool once it has answered the questions it is designed to answer and the function of explanation works in essentially the same way in the mind of the “student,” which is to say that it walks them through a series of logical steps which take them from a set of premises that they already “knew” to a set of conclusions which they previously did not know. So every model is in some sense explanatory and exploratory, the distinction is in the mind of the beholder.
But the important thing to consider is the mindset of the person developing/working with the model. Are they designing the model in order to explore some questions that they don’t know or to explain some concept that (they at least think) they know? In my view, either one is fine. Probably, the best models are created for the purpose of explaining a concept that is already, at least partially, understood by their creators. However, what I think is imperative is that, when developing and working with a model, the modeler remain at all times an explorer. By this I mean that he must be open to the possibility that the model will reveal something he had not previously understood.
This is the line between a scientist and a rhetorician. It is also analogous to the (easier to understand) process of observation. If a rhetorician thinks that an increase in X leads to an increase in Y, their goal is to convince you of that and they select data that reinforces their argument and try to ignore, downplay or obfuscate information that contradicts it. (For an example of this turn on any cable news network at any time.) If a scientist thinks that an increase in X leads to an increase in Y, they go through a careful process of gathering and analyzing data that can either confirm or refute that hypothesis and if the data refutes it, they acknowledge that outcome.
In modeling, we have a similar situation. You can try to make a model that shows that an increase in X leads to an increase in Y but if you get in there and suddenly discover that this is not the case in your model, do you say “surprisingly, the relationship between X and Y in this model is not that which I previously speculated,” or do you say “this model isn’t working” and tweak it to get what you want or throw it out all together? (For the record, it’s not necessarily bad to tweak it but you should notice that you have to do that because something didn’t work exactly the way you thought and this should increase your understanding.)
Science vs. Rhetoric
The same dichotomy applies when evaluating an existing model. We all approach a new model with some set of preexisting beliefs (though the beliefs may be of varying intensities). The question then is: how willing are we to be convinced that our beliefs are incorrect? If someone answers “not willing at all under any circumstances,” then they are not acting as a scientist. This may be okay in certain circumstances but it must be acknowledged. And if you are, for instance, trying to learn economics from the internet (or from a fiery heterodox professor) you will do well to identify who you are dealing with.
Now if you encounter a model and you are willing to be persuaded that your previous belief is wrong, that doesn’t mean that you must be persuaded by the conclusions of the model. The purpose of a mode is that it meticulously identifies which assumptions or axioms lead to which conclusions in a context in which the viewer has complete knowledge about the processes under examination. So if the conclusions of a model are “wrong,” meaning that they are not consistent with real-world relationships, then it must be either because there is a logical flaw in the model or the assumptions of the model do not hold in the real world. (Usually with established models, it is the latter, a case of the former is “the law of diminishing marginal utility” which is why that one bothers me so much.)
But if you are looking at a model and you don’t agree with the conclusions because you don’t agree with the assumptions, there is a productive response and a non-productive response. The productive response is to address exactly how the supposedly faulty assumption is driving the supposedly faulty result in the context of the model and show how it would be different with different assumptions. The non-productive approach is to just say “well they assumed X, and that’s a silly assumption so we don’t need to pay attention to what they are doing.”
The former approach is how science evolves. You try to adopt the most useful framework for thinking about issues and then you get in there and see what happens under different sets of assumptions. If there are issues that you think are important which are difficult or impossible to address under the existing framework, you try to create an alternate framework to address them. The latter approach is just a way of giving yourself and others a justification for ignoring a conclusion you don’t like.
And this is the problem with most (not all) “heterodox” thinkers. They devote their energy to tearing down systems that exist because they have shown themselves to be the most useful for understanding something rather than building up a better system of understanding. This is also why they become “heterodox” because people working within the mainstream framework see nothing of value in what they are saying and their frequently misguided attacks on the mainstream framework provide ample license to dismiss them if you truly understand the way that framework works and why it is the way it is.
The reason many (not all) heterodox thinkers go down this path is that they are rhetoricians more than scientists. This means that they are looking for the quickest, simplest way to dismiss people who disagree with them and to inoculate themselves and their followers from any type of thinking that might lead them to these alternate views. In short, they are trying to win an argument rather than find the truth. I like to think of it in terms of an electrical analogy.
When confronted with a conclusion which is different from our preexisting belief, our mind is put into a situation similar to an electrical charge suspended in midair, we are in a kind of intellectual disequilibrium (my apologies to people who actually understand electricity, it’s just an analogy). Our mind has to find a path to ground, and it prefers the quickest, easiest path. The important distinction is in what we consider ground. For most people “ground” is our original beliefs, so we look for the quickest path back to them which can usually be supplied by saying “well this assumption is probably wrong, so the model is probably ‘wrong,’ so I don’t need to pay attention to it, I will just keep thinking what I already though, and by the way, so should you.”
What makes you a scientist is that your “ground” is not whatever you previously thought but rather it is the truth. So whenever this happens, you have to question whether what you thought before is the truth or whether it is something different that the model is trying to direct you to. In order to figure this out, you have to really understand what the model is doing, not just dismiss it at the first opportunity. This path is has a lot more resistance.
The path of the scientist is never really in “equilibrium” because we never know the whole truth. And every model is wrong. At least this is true (and especially so) in economics. This is how you can tell whether someone is a scientist or a rhetorician. The scientist knows that his knowledge is incomplete. He is not looking for a model that is “the correct model,” one in which all assumptions are true and nothing that is relevant in the real world is missing. This model is not possible. If you had that model, you would be God (at least the omniscient part, if not the omnipotence).
This fact is not nearly as clear in the harder sciences where they have the luxury of creating models which precisely predict outcomes in the real world. A physicist can employ a model that will tell you exactly where a cannonball will land when fired from a given cannon with a given load and given wind/temperature/atmospheric pressure/etc. conditions. If you fire the cannon, the outcome will most likely be a little off because those conditions will not be exactly what was hypothesized but they have the ability to do experiments where they control such variables and they can zero in pretty close on it. For the same reason, a chemist can tell you (more or less) exactly what will happen if you mix certain chemicals and this can be incorporated into an industrial process the produces a certain chemical incredibly consistently.
In economics, we seldom are able to achieve this degree of precision in the real world because the starting point of our analysis is not relationships between things that are easily observable, controllable and quantifiable but is rather the subjective feelings inside the minds of countless individuals which cannot be independently observed, controlled or quantified and these interact in incredibly complex ways. The purpose of an economic model is to put some structure to the way we think about these interactions so that specific (usually qualitative) conclusions can be reached. But, again–and I can’t stress this enough–those conclusions are always “wrong” in some sense because the model is a highly abstract analogue for the real world. This doesn’t mean the models are useless.
If you hire a physicist to predict where your cannonball will land and then you fire it and it lands an inch to the left, you wouldn’t say “clearly, the model is wrong, you’re ‘physics’ is all nonsense.” And yet, that’s what many people do with economics (although, admittedly, economic models usually don’t get that close.) The model helped you get a lot closer than you could have without it. If you think you can make a better model that will get closer, then by all means do that, in this case you are a scientist. But if you are trying to make a model that says the cannonball will go where you want it to go, or a model that says you can’t know where it will go–if you are just looking for a way to invalidate the model with no better alternative, then you are just anti-physics. In that case, don’t be surprised if physicists don’t want to talk to you.
Beware the rhetoricians in long robes.
Here are some examples of attacks on economics that get us nowhere. If you are reading criticisms like this, you may be dealing with a rhetorician and not a scientist. Exercise caution.
1. Economists assume people have complete knowledge but they don’t.
We all know that. The question is what are you going to do about it? This doesn’t mean that a model showing what would happen if they did have perfect knowledge is useless. It is actually a very useful jumping-off point for considering what might happen if they don’t. Many “mainstream” economists have tried to create models in which people don’t have perfect knowledge but you have to somehow explicitly define what they know and don’t know and what they believe about what they don’t know in order to get any useful implications out of it. If someone just say “people don’t know everything so we can’t do anything” then they are not being productive, they are just trying to tear down the existing framework.
2. You can’t use probability to represent peoples’ beliefs.
This follows on from number 1 and I may do a more complete post on probability in the future but, as I said above, if you are going to have a model in which people don’t know everything, you have to somehow define what they believe. If someone just says “they have no idea” then there is no way to logically infer what makes sense for them to do. In other words, they are just saying that it is impossible to make any sense of anything so we shouldn’t bother, we’ll just keep believing whatever we believe. If you can think of a better way of defining what people believe, there’s probably a Nobel prize in it for you.
3. Markets aren’t always in equilibrium.
Again, there are lots of “mainstream” models in which markets are not in equilibrium. The key point is that such markets always try to give a specific reason why they are not in equilibrium. Instances of this abound even in introductory econ and include price controls, taxes, money illusion, sticky wages, adverse selection and moral hazard. But these models all have some concept of “equilibrium” in the model which may or may not always correspond to the competitive, Walrasian market equilibrium because without such a concept it is impossible to reach any kind of conclusion about what would happen. If you have no equilibrium concept, then you are just making arbitrary conjectures or else you are just saying “anything could happen” so I will presume that the thing I already thought would happen is the thing that will. This approach is not productive scientifically.
4. You can’t aggregate individual preferences/beliefs/behaviors.
Yes you can. The question is how best to aggregate them. If their criticism amounts to “people aren’t all the same,” then yes that’s true but nobody is saying they are, they are probably just imagining that they are in order to illuminate some other principle. There are legitimate “fallacies of composition” but in those cases, it is necessary to show how what is being done specifically is actually a fallacy and how specifically it is driving the supposedly erroneous result. If they are just saying “we can’t aggregate anything so we can’t do macroeconomics” then they are not being productive.
5. Money is endogenous.
Have been over this ad nauseam, so won’t say too much more about it but if this is their argument (and to some extent it is my argument), then it is incumbent upon them to explain why the process of money determination gives a meaningfully different result than a model which takes it as exogenous, and not to just say “it’s not exogenous so your model is wrong.” So far I haven’t seen that model. I also haven’t created it yet but am working on it (it’s not all that easy).
6. There’s too much math.
If in other sciences we should arrive at certainty without doubt and truth without error, it behooves us to place the foundations of knowledge in mathematics.
I happen to agree, generally, that there is too much focus on mathematical rigor in economics and not enough on understanding the meaning of what we are doing. But there is a good reason to place the foundations of our knowledge in mathematics. Mathematics is pure logic. The more you can reduce your logic to mathematics, the harder it is to argue with it. This highlights the assumptions and the conclusions of the model as the important things rather than having to wonder if each logical step we made along the way from one to the other was really entirely logical (see the law of diminishing marginal utility). If you are only trying to convince someone who already agrees with your conclusion, this may not be necessary but if you hope to change anyone’s mind about anything, it helps to work within an agreed upon logical framework. And again, if they are just saying “it’s too mathematical, it can’t be right,” they are probably just looking for an excuse to ignore it without really understanding it.
This criticism is general, it is not Austrian-specific. I just tend to focus on Austrians (or at least a certain type of person typically considers themself “Austrian”) because I have more experience with them and because I am ideologically close to them. This is my general perception of this type of person (call them “Rothbardians,” or “pop-Austrians,” or “internet Austrians,” or “Mises.org acolytes,” or whatever).
Their primary motivation, the reason they got into economics, is that they are suspicious of government and think most forms of government intervention are bad. They see a lot of “mainstream” economists making economic arguments for the types of interventions that they think are bad. They find these arguments outrageous. They gravitate toward ways of thinking that offer the easiest path to dismissal of all of those arguments which they find outrageous. They discover Mises.org and other blogs or commentators that offer them a plausible way to believe that they understand what the “mainstream” is doing without actually understanding it and dismiss it all with a few waves of hand and go on believing what they already believe.
Now please keep in mind that I agree entirely with those beliefs. I just think that they are throwing the proverbial baby out with the bathwater and that this is not a scientifically productive approach, even though it is having some rhetorical success. And what’s more, you don’t need to do this! The neoclassical framework offers plenty of room to support free markets. The same phenomenon can be observed in more leftists heterodox ideologies, post-Keynesians for example, but we have to clean up our own house first. Rhetorically, It’s easy to convince people who already agree with you that you are right. To convince people who don’t, we have to be scientists.
P.S. Some disclaimers:
1. There are, apparently, different breeds of “Austrian” out there. This does not necessarily apply to all “Austrians.” If you fancy yourself an Austrian and you aren’t doing these things, I have no beef with you. In fact, I would love to have a few beers with you and discuss how great free markets and entrepreneurs are and how to save more souls from Mises.org.
2. An argument could be made that many like Noah Smith are doing the same thing to the reasonable Austrians by pointing to crazy things that some self-proclaimed Austrians say and using it to convince people to dismiss anything any Austrian says. That is not my goal here.