Embeddings

As a starting place, here are initial recommendations of questions to be asked when reviewing research in which an LLM was used as part of their scientific research workflow.

• Were embedding(s) (i.e., RAG) used in the research?
• Is the tool used to create the embedding model provided and described?
• Were multiple embeddings created, tested, or used (i.e., chained)?
• Is the size of chunks used in preparing the data provided?
• Were different sizes of chunks tested for influence on LLM performance?
• Is the size of overlap permitted when creating chunks provided?
• Is the tool used for similarity matching (i.e., vector database) provided and described (e.g., FAISS)?
• What retrieval tools and/or techniques were used (e.g., reranking)?
• Is the code available?
  

* Fine Tuning vs. Embedding

Recommend Additions/Subtractions/Edits Citable Article on LLM Norms and Review Criteria