What inspired or motivated you to use this tool/resource?
Many students struggle with critical analysis. Generative AI skills have been identified by the QAA as a new key graduate attribute. Students believe AI skills will be important in their degree and in their future careers. ChatGPT is the most widely used and well-known generative AI tool, with a majority of students having used it; however, most students do not understand their university’s AI policies and fear unintentional academic misconduct. Many academics have also recognised the threats posed by GenAI to academic integrity and students’ development as scholars, communicators, and thinkers.
I wanted to use GenAI in class, firstly to fulfil a genuine need, which was to help them to develop their critical analysis skills. Secondly, I wanted to demonstrate to students how to use AI ethically so that they could be more confident in the ethical use of these tools in future. Thirdly, I wanted to offer an introduction to GenAI skills to improve the students’ employability.
What was your aim in using this tool/resource?
I wanted students to practice critical analysis and incrementally improve their work using ChatGPT as an assistant, pointing out areas that require development as well as elements that have been accomplished successfully.
What did you use the tool/resource for?
This was a GI-led session intended to give first-year psychology and sport science students a) a tool to improve their critical analysis skills and b) some generative AI prompting skills. GIs had been given the worksheet in advance and asked to lead their groups through it, giving a little context at the beginning about employability and the potential importance of having some AI skills in their future careers, and also why critical analysis is important in science and to their degree.
How did the tool/resource impact your teaching?
Some students said they had never used ChatGPT because they were scared of it. Given the likelihood that GenAI will, in some form, be important to them in the future, exposing them to the possibilities was a useful exercise. It was also useful to chat to the students about their current awareness of AI tools, do some mythbusting, and offer guidance. For example, one student disclosed that they used a popular AI-driven paraphrasing tool, which is highly unethical in most use-cases. I was able to explain why it was unethical and suggest different (and ethical) ways to work on their ability to paraphrase, which would make them learn the material more effectively.
How well did the tool/resource perform, would you recommend it?
Issue 1. I gave an example prompt and clear instructions. However, most students did NOT copy the example prompt, and instead only wrote poor prompts that did not achieve the goals I was hoping for. This is one of the exercises on the sheet (using a poor prompt and then a good one), but many students just sort of stopped there and didn't then move on to the good prompt. It felt like they'd just skim-read the sheet.
- Lesson 1: Perhaps the instructions need to be even clearer, maybe bolding the part that asks them to copy the entire good prompt. Perhaps bullet points could be used to indicate actions of steps in a process.
- Lesson 2: It was a really useful teaching exercise to go around and sit with students and look at their prompts and have a chat about the difference between the good and bad prompts. Including a good and bad prompt is a useful exercise that I would take forward.
- Lesson 3: In future I would give my GIs clearer instructions about how to interact with the students - they were walking around and answering questions, but not sitting with students and asking the questions themselves.
Issue 2. Some students slightly amended both the good and bad prompts, saying "improve this" instead of "evaluate this". ChatGPT then rewrote the work for them without giving explicit guidance. There is a limit to how good ChatGPT can be in this situation and it could encourage misuse (or legitimise misuse if students thought that having ChatGPT rewrite their work was the purpose of the session).
- Lesson 4: Going around and looking at prompts and what came out of it was crucial to spot these issues and correct them.
How well was the tool/resource received by students?
Students had varied prior experience. Some had not used ChatGPT, some had. Some enjoyed the exercise and took a lot from it. Some didn't see the point. There were opportunities to have conversations about the value of AI skills, how to use it ethically, and the value of doing your work yourself. I would run the session again in the future, incorporating my lessons learned.
Share a ‘Top Tip’ for a colleague new to the tool/resource
It took some students half of the one-hour seminar to log in because of the security protocols. I would recommend asking students to create an account before the seminar to make the best use of their time.
I would also recommend actively engaging with the students to identify poor use and make suggestions, to avoid inadvertently legitimising unethical use.
How would I summarise the experience in 3 words?
Efficient, illuminating, scalable
Recommended reading and References:
Thomson, Pickard-Jones, Baines, and Otermans (in press).
Contact for more information:
Beverley Pickard-Jones: b.pickard-jones@bangor.ac.uk