As a starting place, here are initial recommendations of questions to be asked when reviewing research in which an LLM was used as part of their scientific research workflow.

  • Were training data for embedding(s) acquired in a transparent and ethical manner?
  • Were proper steps for data privacy and protections taken?
  • Did the research methods mitigate the risk of inaccuracies, biases, and/or plagiarism in LLM-generated results?
  • Did the researcher(s) disclose any conflicts of interest related to the use of LLMs?
  • Did the researcher(s) comply with applicable institutional and/or regulatory guidelines?
  • Were proper citations and credit given?
  • To the extent possible are the LLM methods done in a manner that is reproducible and transparent?
  • Were LLM outputs described in a non-anthropomorphic manner?