As a starting place, here are initial recommendations of questions to be asked when considering the use of an LLM as part of your scientific research workflow.
- Have you documented the model, date, and time for each LLM session?
- Have you documented all of the prompts used in the research?
- Have you documented all of the packages (requirements.txt) that you have used (e.g., DSPy, RAGAS, etc.)
- Are you going to make the LLM data (in full) available on an open science service (e.g., OSF, add other data services)?
- In your writings about the research are you included all the parameters used? If these changed for different prompts, is that documented as well?
- How will you acknowledge and document the use of LLMs in any subsequent reports or articles?
- Have you checked professional association guidelines in your discipline for the norms/standards for acknowledging the use of LLMs ?