Principles to Guide Artificial Intelligence in Education Research: Three Considerations

In the last few years, artificial intelligence (AI) tools have become common fixtures in our daily lives, from virtual assistants to search engine auto-suggestions to customer service platforms. The use of these tools will continue to grow in all aspects of our lives. The education field, including education research, is no exception, with AI use spurred by new funding opportunities and advancements in the technology itself. 

Education researchers, including those at AIR, often use AI in their work. They also generate evidence about the efficacy of AI-enhanced applications that educators, students, and others use in teaching, learning, and assessment.

While AI has great potential to improve efficiency in both education research and education practices, there is reason to be cautious. Researchers at AIR are working toward developing a set of evolving principles for how to address key issues related to AI in education research, and we are inviting others to join us.

As we begin this work, we are keeping three considerations in mind.
 

1. Mitigate bias, improve equity, and build on opportunity

The promise of AI to address educational needs must be balanced by mitigating its risks for bias and inequity. For instance, generative AI models are trained using vast amounts of data from a range of sources. Researchers have noted the importance of addressing biases in these training data and embedding ethical principles within AI algorithms to reduce the likelihood that AI models produce biased outputs that reinforce inequities. Widely publicized examples of AI producing inaccurate images or biased narratives illustrate these risks.

Further, an education researcher or developer may take an “off the shelf” generative AI model (e.g., ChatGPT) and then incorporate additional data to refine or customize the model. While the ability to customize is a strength of AI, education researchers should be aware of how that process can introduce bias and shift the model’s original features. This bias could then transfer into the research methods using the customized model, or into the application that students interact with in a learning program. Both of these instances then have a risk of exacerbating inequalities.

Researchers have also pointed out that the speed at which AI is being taken up in education has left little room for vetting, which typically occurs as technologies are adopted. One casualty of this speed is likely to be equity, which demands a more thoughtful, nuanced, and, ultimately, slower approach.
 

2. Be curious about AI-enabled tools and methods, but remain aware of quality concerns

Researchers and educators now can access a large and growing set of AI-enabled tools. These include tools for scoring essays, generating new questionnaire or test items, and synthesizing research evidence. By approaching the use of these tools with curiosity and creativity, researchers and educators alike can take advantage of AI’s benefits, such improving efficiency and timeliness.

At the same time, researchers should take a critical approach when it comes to AI. The quality of output from AI-enabled applications varies, as evidenced by instances of inaccurate and/or inappropriate responses. Human experts must be involved in both evaluating and using AI tools. Education researchers should carefully gauge the quality of AI tools’ output. Additionally, researchers should rigorously and gather evidence of a tool’s quality or validity, including the potential for bias.
 

3. Keep transparency and replicability as the research standard

AI presents education researchers with a challenge when it comes to transparency and replicability, which is encouraged by the Standards for Excellence in Education Research. AI models, such as large language models (LLMs), have incredible computational power, but they are not transparent. In fact, the construction of LLMs is opaque to the point that even their developers cannot, or will not, say how these models generate their output. As data are entered into an AI tool, they may be sent to a variety of subprocessing entities, and this "supply chain" of data processing isn't always disclosed. Moreover, as the models themselves evolve with increasing amounts of input, their output also changes, raising questions about how to ensure replicability.

Researchers, funders, and educators should agree to a set of basic norms around how to handle these issues. This may start with committing to reporting norms for research studies, such describing the steps researchers took in using AI (e.g., prompts used in an LLM) and/or acknowledging what is known or unknown about how data in an AI tool were processed.
 

Looking Ahead

AI has immense potential to shift the paradigm of teaching and learning to ensure equity and access—and to support the field of education with rapidly conducted research. Education researchers and the education community will be best served by determining collectively how to capitalize on this opportunity. Developing vetted uses of AI in education research methods and rigorous research evidence for AI-enabled teaching and learning applications are key to ensuring AI is a tool for positive impact. Principles must guide this work, to ensure that we are able to realize a positive impact of AI and not fall prey to its potential follies.

It is critical that we, as a research and education community, get this right. That is why we know that we cannot do this in isolation. We invite others to join us in broad discussion about the vital role of research to help realize the potential of AI in education.