Large language models (e.g., ChatGPT) as research assistants – Daniel Lemire's blog

🌈 Abstract

The article discusses how software and artificial intelligence, particularly large language models like GPT-4, can assist and potentially replace academics in various tasks such as querying documents, improving text, generating ideas, writing grant applications, coding, and finding reviewers and journals. It also explores the potential impact of these technologies on academic productivity and progress, noting both optimistic and cautious perspectives.

🙋 Q&A

[01] Software can beat humans at most games

1. What are some examples of games where software can beat humans?

• The article mentions that software can beat human beings at games such as Chess, Go, and poker.

2. What other tasks can large language models like GPT-4 excel at?

• The article states that GPT-4 can beat 90% of human beings at the bar exam, and that artificial intelligence can match math Olympians.

[02] Academics' primary skills and how AI can assist

1. What are the primary skills of academics?

• The article states that the primary skills of academics are language-related, such as synthesis, analogy, extrapolation, and analyzing the literature, identifying gaps, and formulating research questions.

2. How can AI help academics?

• The article suggests that AI can help academics in tasks such as:
  • Querying documents
  • Improving text
  • Generating ideas
  • Helping with grant applications
  • Writing code
  • Finding reviewers and journals

3. How prevalent is the use of AI in academic writing?

• The article cites a study that found an increasing number of research papers (up to 18% in some fields) are written with tools like GPT-4, and the author suspects that a majority of all research papers will be written with the help of AI in the near future.

[03] Potential impact of AI on academic work

1. What are the potential benefits of AI in academic work?

• The article suggests that AI could allow a small group of researchers to be highly productive and free them to explore further with less funding, potentially leading to a new era of scientific progress.

2. What are the potential drawbacks of AI in academic work?

• The article cautions that AI tools in science risk introducing a phase where more is produced but less is understood, and that AI may simply ease scientific misconduct, leading to a "sea of automatically generated documents."

3. How does the impact of AI vary across different types of academic work?

• The article suggests that theoretical work is likely more impacted by AI than more applied, concrete work, such as conducting actual experiments, which may be harder to automate.

[04] Comparison to other professions

1. How does the impact of AI on academics compare to its impact on other professions?

• The article notes that plumbers and electricians may not be as easily replaced by AI, citing the Moravec paradox, and that some tasks within academia, such as conducting experiments, may also be harder to automate than more theoretical work.