Abstract

The transition for researchers from course-taking to paper-writing is the most difficult step for aspiring professors. Here are some related thoughts that I have accumulated over the years to provide some advice to budding PhD students.

Advance warning: Not everything in this document is politically correct or what one may want to hear. If you are the sensitive type, maybe you are better off not reading it. And nothing in this document is guaranteed to be true. On the contrary—smart people may disagree with me.

Talents and Skills

When I was a third-year PhD student at the University of Chicago in 1987, having just passed my qualifying exams, I was suddenly confronted with the stark reality that I had to write a research paper within three months or lose my stipend. Chicago was famous for its lack of support in the process at the time (which is no longer the case). How was I ever going to succeed? I even thought about dropping out. Does it come as a surprise that an economic who ultimately becamse as successful as myself started out troubled? It should not. It is pretty much standard fare. (And students who don't acknowledge that they are worried are either just faking it or too stupid to recognize the issues.)

To succeed in a PhD program requires the intellectual capability to pass qualifying exams, which, by virtue of strong selection at the admissions stage, four out of five incoming PhD students already have. This is rarely the stumbling block. Combined with hard work, only about one in eight students should end up failing in this way. In contrast, three out of the eight will fail because they do not end up writing a good research paper.

Unfortunately, it is difficult to predict who will succeed at this transitional step and who will not—even for students who have been in a PhD program for two years. The students themselves do not know. The professors do not know. Some of the smartest test-takers in PhD programs never succeed in writing good research papers, despite early indications that they would morph into superstars. Some of the superstar research faculty today were just mediocre test-takers. They passed but did not excel. Just after they had passed their qualifying exam, no one would have picked them as likely winners of the game—as authors of some of the most influential papers in economics today.

For students from foreign countries, culture and language are often additional hurdles. This applies especially to students from Asian countries. The U.S. academic system is not Confucian. There is an appropriate level of respect for the faculty—and it is “not too much” (more rarely, the problem is too little). Don't have too much respect for what I am writing here, either. Make up your own mind.

Students have to learn how to “sell themselves,” too. This is partly an acting job. They need to come across as potentially good teachers and colleagues–confident, but not too confident. Fun. Intellectually interesting. Capable of surviving a horde of hungry MBA students. This is perceived to be more difficult for some foreign (and sometimes foreign women) students. Holding these qualities constant, I believe there is no racial, ethnic, orientation, religious gender, etc., preferences in academic finance and economics today.¹ Besides, such discrimination would be so odious and so contrary to the entire spirit of the academic enterprise that I would hope that the faculty collegium will stop any such behavior in the bud.

There is (potentially unfair) discrimination by university, PhD advisors, the existing faculty networks, and work areas (rational expectations modeling, behavioral finance, etc.). C'est la vis.

Qualities of a good job-market paper

But let's get back to the key problem: how to write a good job market paper. Ideally, a good job market paper has two qualities:

It answers an interesting question.
It shows off technical sophistication, good training, and smarts.

A good number of candidates have focused more on the latter than the former. It is not clear whether this is wise.

This is not necessarily because focusing on the modeling is better, but because it is and has become more difficult to find interesting novel economic insights.² There is also more randomness here: math is more predictable than empirical findingss. Sadly, in my mind at least, more than half of the job-market papers nowadays seem to be “interesting papers about models,” not “interesting papers about behavior” in the real world (which survive the smell test—i.e., where we believe they are likely true)..

Moreover, it is not uncommon for students with uninteresting but technically impressive papers to receive good job offers. It is uncommon for students with interesting but technically trivial papers to receive good job offers. It is also easier to dismiss a job market paper with “any undergraduate student could have written this” when the main cleverness in the paper was about finding a good question and spending a year collecting data. It is harder to dismiss a job market paper when it “solves a difficult modeling problem,” even if no one ultimately believes this to matter. After all, it could at some point in the future.

Having said all this, almost all the top stars in our profession are famous for having written interesting papers, not for papers that show off interesting techniques. (In fact, it is difficult to point to any star in our profession who is know primarily for technique and not for an interesting finding.) Many of them have come to the market with less tech-y papers, too. Thus, a techy job market paper is often not the wisest choice, either. Your mileage may vary.

Incidentally, in our profession, it is also easy and common to confuse smarts with mathematical skills. They are not the same! For example, there is is my (ex-)colleague Judy Chavelier, one of the best and smartest economists of her generation—but her papers are not mathematical, at all. (I am in awe of her.) I will spare everyone holding forth on the opposite cases.

John Cochrane, one of my best teachers over many years (even though I do not agree with many of his views), believes that I am overstating the case. In his view, interesting results with simple technique get jobs, technique overload does not. Chicago has hired many young professors who have not written highly technical work. Their papers showed an understanding of the literature, previous results, pitfalls, and (wisely) use a range of available techniques most suited to answering the question at hand. On the other hand, maybe there are just few such papers on the job market, and Chicago has managed to skim off a good number of them over the years. You should then conclude that the payoff to writing such papers is very high, because they are relatively scarce.

Lowering the barrier

From the side of the faculty, the noise factor in the success of our PhD students is unsettling.

How can we identify good students that will suceed on the job market? It's hard. When I look at our PhD program applications, the candidates look incredibly good. I would not admit myself into a PhD program today, even though I would end up among the students writing a good job market paper. I would not even recognize my own abilities at the end of the qualifying exams. Heck, when Milton Harris wrote a good job recommendation letter for me after 3.5 years in a PhD program, I still thought I did not deserve it. He saw something in me that I did not see in myself. And yet, I have succeeded beyond my wildest dreams in what matters—writing some interesting papers in my career.

A PhD is an intergenerational contract. One owes it to one's own students what one has received from one's advisor. (If you succeed, so will you! Don't you forget this!) How can we faculty help? And, yes, most faculty will be happy to help, even if they are very busy.

The truth is that we are all guessing. I don't know the right recipe, either.

I try to give classes where I force students to write papers. Writing critiques of existing papers is an especially good way to go about writing one's first paper, even if this is usually not a good path to a job market paper. (The eisting path is already set out, and simply replicating a paper often makes one recognize the many little decisions that inevitably had to be made along the way.) Unfortunately, in reality, it often turns out that 3 months are not enough to replicate a paper, and students get distracted on other tasks. Students often spend a lot of time preparing for their second-year qualifying exams, or “tooling up.” The irony on tooling up is that the tools in most economics publications are much less than what students end up acquiring. It is eye-opening to open a recent finance or economics journal, read the papers, and then realize that only one in thirty papers actually uses anything beyond basic calculus and linear algebra.

I try to ask second-year PhD students to be research assistants on a paper that I am working on, so that they can see the many dead alleys that one has to explore to come up with the final product—to see that the final product, even if it looks elegant and simple, was not how this neat when it started; to see that even if the paper looks easy in its final form, chances are that it was a mess in its intermediate form. No one can set out to write path-breaking much less seminal papers. More likely, a small paper accidentally develops into something bigger. Finding a good question that is solvable is the key to writing a good paper. Sometimes, I have just pretended to the student that (s)he works for instead of with me, and then I made students coauthors along the way. (The problem with promising this up-front is what to do if the student becomes uninterested or unmotivated.)

Finding a good first-paper question

How long should it take to write an academic paper? Roughly speaking, finding the question, data, design, and analysis of the paper typically takes about one quarter of the time. Writing the first draft takes another quarter. And presenting it, revising it, polishing it, submitting it, re-revising it takes the final half. A normal paper takes about 2 months of data and analysis, 2 months of writing, and 4-6 months of (interrupted) handling time. A good faculty member publishes one paper per year and abandons one paper once every two years. Thus, (s)he works on about three papers at a time.

For the first paper, the process can often take twice as long. It is a little like riding a bicycle (or driving a car or flying an airplane or having sex or...)—you can read about it, but until you have done it, you really do not understand it. Instead, you may wonder how anyone can manage to do this so seemingly easily. It seems overwhelming. But here is the truth: The first time is awkward for almost everyone. The second time will feel a lot more natural. Eventually, writing papers becomes second nature. It is only then that you can focus on broader things and multiple tasks at the same time.³

When I started out, at the end of my second year, I thought I would never find a good topic. The scarce resource for most faculty researchers is attention/focus and time (and in this order). It is not finding interesting and good research questions—this is only the scarce resource for PhD students and only the first one or two times. It is a passing phase. Don't let it trouble you too much.

I keep a file with interesting questions and answers that I come across. Sometimes, I have them in seminars, sometimes in classes, sometimes while reading the WSJ or the Economist, sometimes in the shower, sometimes in the middle of the night. A few days later, when I reread my ideas, for 9 out of 10 research ideas, I wonder how I could have found this stupid an idea any good and interesting. Only 1 out of 10 of my crazy ideas survive for more than a few reads over subsequent months. It is those that I eventually end up working on.

The Paper Outline

Academic papers typically have a pretty standard outline. For empirical work, it is introduction, data (incl descriptive statistics), methods, empirical findings, robustness checks, and conclusion. For theoretical work, it is introduction, the setting, the equilibrium, the solution, implications, robustness to alternative assumptions, and conclusion.

The best way to write the empirical paper, given the results, is to proceed in steps:

Create your exhibits (set of tables and figures). Each exhibit should contain the description (what the data definitions and sample period are, what is dependent and what is independent, and what the key methods are. At the end of the table, have one or two sentences what the reader is supposed to learn from this table. (In fact, at the CFR, I am almost mandating this final sentence for my authors.)
One good exhibit is a reference table of key variables, regardless of whether your paper is theoretical or empirical. The memory of your readers is more limited than your own.

The exhibits should be telling the story of the paper without the need for any writing. Giving them to another PhD student in the area should allow this person to tell you the key points of the paper. The writeup is primarily clarification now, and primarily for non-experts.
Start with the aforementioned outline. Shoot for a paper length of 20 pages. (Papers always have a natural tendency to expand for longer than they should.) Do not write the abstract and introduction yet. They come last. For each section, write the set of important bullet points, one for each paragraph. For example:
- Introduction:
  - Dependent variables: Horseshit.
  - Independent variables from the data source “farm”:
  - Independent variables from the data source “weather service”:
  - Independent variables from the data source “veterinarians”:
  - Intersection of available data sets.
  - Insert and describle table: descriptive statistics.
  - Other warnings and issues.
- Methods:
  - Typical dependent variable. Transformations.
  - Typical independent variable. Transformations.
  - Identification strategy.
- Preliminary Analysis
  - Groups
  - Insert grouping table.
  - Explain findings with groups
  - Interpretation: when the horse eats more, it produces more horseshit.
- More Sophisticated Analysis
  - The issue is survival. Dead horses neither eat nor shit.
  - Insert more sophisticated table...
Concrete numerical examples are always helpful, be it about the construction of variables, or the solution to a model.

If you do this well, you are not just writing the outline of your paper, you are also writing the outline of your first paper presentation!

It is not unusual that during the process, you will realize that you have to go back and rerun some of your analysis. C'est la vis.
After this bullet-point outline is complete, take one day off and think about whether your bullet points cover every paragraph that you need/want to write. Revise the bullet points. Then take a few days off (to forget what you wrote, so that you have a fresher eye reading it again). Come back. Do your bullet points and your organization still make sense? Did you have a point for each paragraph that you want to write? This process should take about 7-10 days. Share this with your faculty advisor before you write the paper. Then discuss it over coffee, so that your advisor can ask questions along the way. In response, you may need to revise the bullet points again. And again.
Ok, now that you are done with the bullet point outline, write the first draft of your paper. Write about one section per day. After the first draft, rewrite the paper (for English grammar and syntax and flow!) about five times. Each rewrite should take about two days. In total, the writing stage should take about 3-4 weeks.
Somewhere in the middle of this process, begin to write the introduction.and abstract. It should be written only after the key wrteup of the paper has been completed. The introduction should start with one paragraph what your question is and why the question is interesting. Then it should have about two to four paragraphs with your paper's answer. At this point, it can cover some immediate objections and/or robustness check. Then, just before the end, your introduction should provide context (no more than one paragraph) within the literature. Then a description of how the paper will proceed. This is not (necessarily) the common paragraph “In section 1, I do X” but an outline that will make it easier for the reader to understand the context of what follows.

Interpretation, speculation, implications, etc., can go into your conclusion. You are allowed to take some liberty here, but be clear about what is opinion and what is not. Don't piss off your readers by offering too much speculation that is too far away from what you have actually found. You have to remain reasonable.

You are also not Eugene Fama, You cannot give the profession advice about where it should go. Individuals like Gene are allowed to speculate in writing about this, especially in journals that indulge more liberally in this sort of fare. (Unfortunately, Gene usually does not—I tried to convince him to write a paper to suggest directions, but he refused politely, telling me he would think about it. He told me that he prefers to work on smaller and clearer questions where he can contribute specific answers.) You are just a Ph.D. student. Chances are that answers to broader questions have occurred to many others. Your best chance is to become an expert at a well-defined but smaller question, where you can offer a truly new perspective and answer.⁴ And, if you think the profession should do more research of type X or in area Y, then you should do so yourself.

I realize that there is a certain hypocrisy in me writing what I am writing here. I just told you that it is not up to mere mortals to write “what to do” pieces. But please realize that mine is not an academic paper intended to be submitted to an academic journal. It is just a piece of musings. Nevertheless, chances are that even at this level, my advice here is still opinionated enough that it will piss off a number of researchers. If you must muse, please do not do this in a paper that you plan to submit to a top economics or finance journal.

John thinks I am all wrong here. He suggests the following outline instead:

Introduction

What is the basic point / most important finding

Explain the central fact behind that finding

Connect it to previous literature (in a separate section)

Explain data and method

Present the finding in detail

Answer some objections.

(Appendix: answer more objections)

He suggests that one should skip the “preliminary analysis” and make sure the central finding is in a table with number no greater than two. Do not present more than four tables.

Other Thoughts

If you are suffering through writing a paper (and are getting a PhD) because you want it to help you get somewhere and not because you like doing it, then you are in the wrong area. The path itself is the reward. And it can be a very frustrating path along the way. You will experience plenty of rejections. You will need a thick skin. And, at the end, your chance of getting a job at the UoChicago is not much higher than the chance of the average waiter in Los Angeles to star in a movie.
The PhD qualifying exam is a combination of an IQ test, a test of skills (and usually a test of specific course material), and hazing. After the qualifying exams are over, unlike the research parts of the program, this will not become the job. In fact, exams will be over, probably forever.
It is an open secret that almost all faculty would fail the qualifying comprehensive exams in their own schools. The name is a misnomer. These are never basic knowledge and background exams. Faculty can usually answer the questions that they themselves are asking, and occasionally the questions in their own area. Beyond this, they would all flunk. (They did pass such exams in their own days, though.)
It is the job of PhD students to doubt that generations of economists before them have done.
If you think the faculty are smarter than you are in seminars, realize that they have probably seen similar papers many dozens of time. It is easier for them to follow and understand what is happening. Just learn from their questions (in seminars). Over time, you will be able to follow other researchers' papers more easily, too.
I prefer to put long descriptions of related literature into its own section just before the conclusion. Your readers want to learn early on what you are doing, not what others have done. Yes, in the introduction, they need a little bit of motivating context to understand the novelty of what you bring to the table, but this is it.
Don't work on more than three papers at a time. One is risky—it may not pan out. It may also get stuck occasionally, in which case having an alternate paper to work on is good. You cannot do a good job on more than three projects at the same time.
Don't be so risk-averse that you cannot start on any paper. If you don't shoot at a target, you cannot hit it. Most research projects morph into something else as time goes by, anyway. This is normal. Start working on some topic that interests you and be open-minded in where it leads you.
If you cannot think of any new ideas, then just start by replicating the most interesting paper in the literature by yourself. That is, take the main idea, and implement it yourself while not looking at the implementation details of the paper. Then, when done, look at what you did and what the original paper did. Anything interestingly different?
Make friends with your student colleagues. They are your peers. They will accompany you throughout your professional career. Your co-students are your first and best source of help. Think of your co-students as your colleagues. Have seminars. Help them and expect them to help you. (If you are not willing to help others, why do you think faculty owe you help?) And remember that they are not your competition—the PhD job market is international, not local.
It is lonely at the top—what I mean is that it is lonely being a faculty member. Usually, as a PhD student, you have 2-5 colleagues per year and about 10-20 that are in roughly the same shoes you are. For young faculty, it gets sparser. It is more like 1 colleague, and 1-5 in similar shoes. For senior faculty, it is more like 1 colleague for every few years of spacing.
Don't be afraid of idea theft by faculty. They are usually too busy to have time to steal your idea and compete with you. And you would often be lucky if a faculty member would take the time to take over the parts of the process that you are not yet good at, even if the paper then becomes coauthored. The time you are saving that allows you to write another paper is usually more than worth it. And presumably your coauthor would then let everyone know that this was your idea to begin with.
(It is a matter of ethics that faculty do not push themselves onto a paper, or (worse yet) begin to work on the same area. Yes, it has happened, but it is very rare. Of course, there have been exceptions. Advice: ask fellow and past students about the reputation of a faculty member that you want to work with. If this person has the reputation of being a psychopath, then avoid this person, no matter how nice this person seems to be and how good this person is. The outcome in such cases is inevitably bad.)
Don't think it's all about talent and hard work. It's 90% luck—and 90% talent and hard work.
Finance is a subarea of economics. It is not a separate discipline. It is the largest subarea of economics, roughly one third in numbers. If you want to be read by finance professors, even if you think that the area journals (such as the Journal of Finance) are worse than the field journals (such as The Review of Economic Studies), you may still be better off with a publication in the former. The fact that finance professors often send their best work to the top finance journals first makes finance different from other fields of economics, where researchers typically send their best work to the general economics journals first.⁵
By the way, all journals are noisy. This gets worse in the referee process. See my forthcoming RFS publication about the editorial review process. All journals publish many bad papers and reject many good papers. It is a depressing situation for junior faculty, but one that you cannot change and that will not change for many years to come.
It is much better if your job-market paper is single-authored. It is not unusual nowadays to have a second paper when coming out. Your other paper(s) may be coauthored.
It used to be that five years in the program carried a stigma. This is no longer true. Five years is now the normal period of time. There is a stigma for seven-year students, though. Four years is a good idea only if you have a real solid portfolio.
In 2014, IMHO, the most interesting areas in economics are
1. Truly-experimental work (e.g., Internet experiments, behavioral studies, etc). We live in a new world where data can flow like crazy. We live in a world where academic researchers have convinced many policy-makers to try out experiments.
2. Identification-related work (e.g., regression discontinuities on interesting questions). Yet, watch out not to go too far here. IV's with diff-in-diffs about why farmers in Grafenrheinfeld purchased more when they received credit cards may be well identified, but they are uninteresting. You need a good quasi-experiment here for which there is reason to believe that it is generalizable enough. If not, you are answering precisely nothing. This is as uninteresting as answering nothing precisely.
  By the way, please don't become too dogmatic: There are questions for which there is no good identification, and yet we can still learn interesting things about them. It is true that correlation does not imply causation. However, no correlation implies no causation. Knowing whether there is correlation or not narrows down the set of valid explanations for a phenomenon.
3. Critiques that point out errors and mistakes (or simply over-interpretation) in past work. But, critiques rarely make good job market papers. This is an area that is interesting to me.
Other researchers would argue that extending more structural papers that make our paradigm models fit the data better are more exciting.

Fortunately, there is a diversity in taste in economics. I mean this. It would be terrible if everyone did what I liked. The only aspect of our profession that I lament is that many of us are too defensive and not tolerant enough of other approaches. Of course, given my CFR critique of structural work, I am often accused of being intolerant, too. However, please realize that my own paper made an argument. It does not mean that I think all or even most structural work is bad. In fact, I think there are entire areas, such as fixed-income and derivative pricing, where even I think it is great. And I am glad that Chris, Toni, and Ilya defended their own viewpoints. Don't just adopt mine.
I am showing my age with some of my views. Prove me wrong!

John Cochrane has often written good writing advice for PhD students. Look on his website.

I occasionally revise this document. It is a living document. It is not fixed in stone. If you disagree with me, good! If you find an error, please let ivo.welch@gmail.com me know.