In qualitative data analysis, codebooks offer a systematic framework for establishing shared interpretations of themes and patterns. While the utility of codebooks is well-established in educational research, the manual process of developing and refining codes that emerge bottom-up from data presents a challenge in terms of time, effort, and potential for human error. This paper […]
Continue ReadingOne of the grand challenges of artificial intelligence is developing agents capable of conducting scientific research and discovering new knowledge. While frontier models have already been used to aid human scientists, e.g. for brainstorming ideas or writing code, they still require extensive manual supervision or are heavily constrained to a specific task. We’re excited to […]
Continue ReadingMilton Pividori spent a year and a half studying how best to use ChatGPT in research. Here, he highlights three key lessons.
Continue ReadingThe European Commission has put forward a set of guidelines to support the European research community in their responsible use of generative artificial intelligence (AI). As the adoption of generative AI technology continues to expand across various fields, including science, these recommendations address both the promising opportunities and potential challenges associated with its proliferation. Rooted […]
Continue ReadingWe present an approach for automatically generating and testing, in silico, social scientific hypotheses. This automation is made possible by recent advances in large language models (LLM), but the key feature of the approach is the use of structural causal models. Structural causal models provide a language to state hypotheses, a blueprint for constructing LLM-based […]
Continue ReadingFighting reviewer fatigue or amplifying bias? Considerations and recommendations for use of ChatGPT and other large language models in scholarly peer review Abstract: The emergence of systems based on large language models (LLMs) such as OpenAI’s ChatGPT has created a range of discussions in scholarly circles. Since LLMs generate grammatically correct and mostly relevant (yet […]
Continue ReadingThe recent surge in open-source Large Language Models (LLMs), such as LLaMA, Falcon, and Mistral, provides diverse options for AI practitioners and researchers. However, most LLMs have only released partial artifacts, such as the final model weights or inference code, and technical reports increasingly limit their scope to high-level design choices and surface statistics. These […]
Continue ReadingWe investigate the potential for Large Language Models (LLMs) to enhance scientific practice within experimentation by identifying key areas, directions, and implications. First, we discuss how these models can improve experimental design, including improving the elicitation wording, coding experiments, and producing documentation. Second, we discuss the implementation of experiments using LLMs, focusing on enhancing causal […]
Continue ReadingAs we are witnessing a fundamental transformation of organizations, societies, and economies through the rapid growth of data and development of digital technology (George, Osinga, Lavie, & Scott, 2016), artificial intelligence (AI) has the potential to transform the management field. With the power to automatize, provide predictions of outcomes, and discover patterns in massive amounts […]
Continue ReadingThe advent of ChatGPT by OpenAI has prompted extensive discourse on its potential implications for science and higher education. While the impact on education has been a primary focus, there is limited empirical research on the effects of large language models (LLMs) and LLM-based chatbots on science and scientific practice. To investigate this further, we […]
Continue Reading