As a starting place, here are initial recommendations of questions to be asked when reviewing research in which an LLM was used as part of their scientific research workflow.
- Were training data for embedding(s) acquired in a transparent and ethical manner?
- Were proper steps for data privacy and protections taken?
- Did the research methods mitigate the risk of inaccuracies, biases, and/or plagiarism in LLM-generated results?
- Did the researcher(s) disclose any conflicts of interest related to the use of LLMs?
- Did the researcher(s) comply with applicable institutional and/or regulatory guidelines?
- Were proper citations and credit given?
- To the extent possible are the LLM methods done in a manner that is reproducible and transparent?
- Were LLM outputs described in a non-anthropomorphic manner?