ChatGPT guide - ganong-noel/lab_manual GitHub Wiki

Works well

Debugging

• Debugging code
  • Simply copy and paste the code into the textbox and write after it: "debug this"
• Interpreting errors, specifically for coding inside restricted data environments
  • Since we can't copy out large chunks of code into GPT if working in restricted data environments, we can hand-type error messages. This has been helpful for pyspark errors in builds. Here is one example.

ggplot

• Drafting plots
  • Do this by a) describing the data b) describing the plot. It is important here to take the extra time to be precise, such as asking questions as if you were asking another human being on e.g. StackOverflow
• Explaining how to add features you are unfamiliar with
  • For example, you can ask "how do I place and arrow inside of a ggplot bar plot?"
  • Alternatively, you can paste code in and write "add an arrow to this plot"

Issues with Git

• Explaining and understanding GitHub messaging
  • Prompt: "how do I do xxx in git?" or "what does xxx mean in git?" or "how do I fix xxx error in git?"
    • Example here for error

LaTeX tables

• Translating tables on paper to a passable LaTeX table
  • Performs poorly at the final polishing step

Brainstorming

• Proposing explanations for economic phenomena
  • GPT performs better than expected:example here

Answering basic questions

• Programming questions
  • How to translate log changes into percent changes (here)
• Optimizing code for speed (fine)
  • Pasting in code and prompting GPT to "make this run faster"
  • Strong at data.table optimization, which will increase speed a lot for slow code
• Functional questions
  • Example here about understanding dentistry: this was a much more useful/organized response than Google gave
• Economics questions
  • This is relevant for a referee report or understanding something we are doing. For example, here is a prompt about understanding the difference between a notch and a kink in bunching estimators. Be careful here because it might hallucinate, but in this case it does a very nice job.
• For all other and more complicated questions, you will have more success searching on Google and looking for StackOverflow results.

Cross-Referencing Studies

How to Do:

1. Break the work into small steps
   • ChatGPT can do a very good job at matching things you might do well.
2. Save things in files often and never work with pdfs
   • Txt and csv seem to do fine. Here, for example, I take the works cited and ask to get a txt back with one line per entry:

    Aaronson D, Mazumder B, Schechter S. 2010. What is behind the rise in long-term unemployment? Econ. Perspect. 34(2):28–51
    Altmann S, Falk A, Ja ̈ger S, Zimmermann F. 2015. Learning about job search: a field experiment with job seekers in Germany. Discuss. Pap. 
    9040, IZA, Bonn
    Amaral P, Ice J. 2014. Reassessing the effects of extending unemployment insurance benefits. Econ. Comment. 2014-23, Fed. Reserve Bank 
    Cleveland
    Arranz JM, Muoz-Bulln F, Muro J. 2008. Do unemployment benefit legislative changes affect job finding? Unpub- lished manuscript, Univ. 
    Alcala ́
    Baily MN. 1978. Some aspects of optimal unemployment insurance. J. Public Econ. 10:379–402
    This is a list of citations. Can you create a file "svwreferences.txt" file where each line has one entry from this works cited

3. Use Chunks
   • By trial and error, 19 is usually the best chunk size. If you do more, GPT will often screw something up as below:

    Error analyzing
    It seems that the text for the eighth chunk of citations was not completed, leading to a syntax error. Could you please provide the 
    complete list for chunk 8, including all 23 citations? This will enable me to compile them correctly into the text file.

4. Always provide example output when you have complex output. As below:

    Ok, just a problem with the last names...you have done the first initial of the last author. Names go last then first. For example, with 

    Aaronson D, Mazumder B, Schechter S. 2010. What is behind the rise in long-term unemployment? Econ. Perspect. 34(2):28–51

    you should get Aaronson as first author last name. Please do the same thing again with this change

5. Use those chunks all at once later: image below

• Click the arrow next to analyzing to see the code GPT is writing; very useful to catch small mistakes
• Full conversation here: https://chat.openai.com/share/b5f61029-3e0e-443d-9add-ecd5d91522a7

Works poorly

Explaining econometrics

• GPT can explain a basic difference-in-difference model, but will struggle with anything more complex, often making mistakes.

Coding in SQL Athena, Pyspark

• GPT returns code that does not run
  • PN addendum: I have found that it is quite helpful in explaining and remediating errors from Pyspark code though.

Understanding data structure

• GPT cannot compute something with lags and first differences in a simple way

Debugging inside Chase VDI

• The environment is proprietary so GPT is unable to answer basic questions about where things are and how things work

Translating from one coding language to another

• Often GPT will say the file is too long or that it needs more information because the code types do not easily map

Answering involved coding questions

• GPT struggles to answer longer questions involving larger segments of code, and it's very hit-or-miss whether it produces correct output.
• If it produces a long block of code on its own, you must check every single line. It has a high error rate on long blocks of code.

Data documentation

• Can't answer questions about IPUMS or other public data sources

Plot digitization

• ChatGPT is not able to look at an image of a plot and give you the numbers that correspond to the data points.