As a starting place, here are initial recommendations of questions to be asked when considering the use of an LLM as part of your scientific research workflow.
- Where in your research workflow do you plan to use an LLM (e.g., literature review, data collection, data preparation, data analysis, writing)?
- Based on the research tasks for the LLM, which LLM (e.g., ChatGPT, BARD, Claude) is most appropriate? What characteristics weigh into this decision?
- How will the LLM complement and/or supplement other research tasks in workflow?
- Are you planning to use a website LLM (e.g., ChatGPT) or use an API to connect to the LLM?
- What completion parameters do you plan to manipulate when using the LLM?
- How will LLM responses be evaluated for accuracy, bias, and other potential limitations?
- How will you provide for data security and privacy when using the LLM?
- What methods will you apply to mitigate the risk of inaccuracies, biases, and/or plagiarism in LLM-generated results?
- How will you comply with applicable institutional and/or regulatory guidelines for using LLMs in research?
- Will you pre-register the study?