Findings

Findings from the PAR phase of the project

• Reconfirmation of the importance of kaiakos’ digital and AI literacies as being essential to ensuring that AI integration aligns with learning activity outcomes.
• Level of learning (i.e. level of ākonga literacy) must be considered when using AI tools. Both prompt engineering and evaluation of AI outputs require ‘intermediate’ literacy skills.
• Multimodal interaction with Gen AI tools is now possible with ChatGPT4o and Google Gemini. However, communication using voice and then trying to remember the replies, places a high cognitive load on ākonga.
• Gen AI tools which allow ākonga to archive their interactions with the tool were found to be useful for later referencing.
• Contextualisation of learning in foundation/bridging programmes is challenging. There are often difficulties in bringing authentic disciplinary-based learning activities into a class when individual ākonga have different focuses for their learning journey.
• Very few Gen AI tools/apps support Te Reo Māori. Those that do are inaccurate. At the moment it is prudent not to use Gen AI tools/apps with ākonga, especially at foundation/beginner level.
• A similar premise may be made with English as Other Language (ESOL) ākonga. Text-based interaction with Gen AI tools/chatbots require sufficient language to be able to type in prompts which the AI will understand and respond to. Using the voice feature is possible, however for non-native English speakers both the accent of the chatbot and the speed of its responses can be difficult.
• For kaiako, Gen AI presents many opportunities to support the agile development of relevant, contextualised, contemporary learning resources. Kaiako will benefit from attaining AI literacies to support their ako. 

Findings from the Q-method analysis – informing the development of the chatbot

Digital literacy and English competency a pre-requisite

As Gen AI access is through digital hardware, digital competencies are a pre-requisite to its use. This is especially important for ‘second-chance’ or adult ākonga who may not have acquired digital competencies at school or use desk-based hardware at work. Additionally, some younger ākonga did not have access to desk-based hardware or software at home. Their main digital device would be a mobile phone.

Identifying and matching the tool to the learning outcome

A variety of Gen AI tools and ways to use them to support ako and the development of academic literacies was introduced, utilised and evaluated.

Across the sub-projects, we used these tools to support ako with:

• Idea generation – all the sub-projects introduced Gen AI to help generate ideas for writing.
• Revision of study content – Ākonga were guided to use Gen AI to help them revise the content from their other courses. Ākonga appreciated the use of Gen AI to generate quizzes and questions, to help them revise for upcoming assessments.

Academic and information literacies supported included:

• Using AI to help summarise ‘readings’. Ākonga appreciated these capabilities but struggled with phrasing prompts that would interrogate readings to give them the right information. Scaffolding of prompts is an essential skill set when using AI tools or apps.
• Triangulation or cross-checking of the AI-generated responses produced by Gen AI. Ākonga were shown by their kaiako how to evaluate the responses from Gen AI. Gen responses could be checked against search results from Google Scholar, other Gen AI tools or the readings provided by the kaiako.
• Checking their writing for grammar, spelling and structure of their writing.

Gen AI tool preferences

Workplace-based ākonga and their kaiako, tested out the main Gen AI tool/apps. These were ChatGPT3.5, ChatGPT4o (4 omni), Google Gemini, Claude 3.5 Sonnet and Microsoft CoPilot. There were functions within each Gen AI tool/app that were preferred as a support for learning English.

Findings incorporated into chatbot

From the data analysis, the following principles were formulated to advise two Bachelor of Information and Communication Technologies (BICT) ākonga to provide direction for the AI app/tool or chatbot could support foundation/bridging ākonga.

1. The app/tool or chatbot would be founded on principles of Universal Design for Learning (UDL). UDL seeks to provide ākonga with opportunities to engage with resources which are accessible through principles of good design.
2. The level of English expected for inputs and outputs from the app/tool or chatbot should match the literacy skills of the ākonga who will be using it.
3. The main purpose of the app/tool or chatbot would be to help ākonga check their writing and coach them towards improving their writing based on appropriate criteria.
4. The app/tool or chatbot would be evaluated by ākonga and kaiako to ensure that the user experience (UX) is fit for purpose.

Recommendations

The integration of Gen AI into the ako of foundation/bridging programmes, requires careful learning design. The main goal of these programmes is to ensure ākonga attain the academic literacy skills necessary for further study or work.

There is a need to ensure that Gen AI supports academic literacies without compromising ākonga engagement with the deliberate practice required to attain the literacy skills.

Addressing issues of digital literacy and equity

• Equitable access to digital hardware and infrastructure, along with individuals’ digital skills to utilise PCs or mobile devices are requirements for engagement and participation in an increasingly digital and AI world.

Clear institutional guidelines and policies on AI use must be available

• Clear direction from institutions is required on how, when, who and what Gen AI is used.
• Many educational institutions, focus on the misuse of AI, especially about the assessment process and issues of plagiarism. However, aiming to find, and share relevant, authentic and useful ways to integrate Gen AI into learning activities could support the development and practice of academic reading and writing.
• Consistency across programmes and clear institutional policies were requested by ākonga to eliminate any confusion.

AI literacies are essential

• AI Literacies are different from digital literacies. Learning how to use Generative AI (GenAI) tools requires digital literacies, as digital access is required to operate AI tools, but evaluating and refining the information the tools provide requires AI literacy skills.
• Critical thinking and information literacies are required to be able to construct queries for GenAI tools, evaluate the responses, and articulate any refinements needed by the AI model.
• The project data shows that an introduction to AI is an important first step for all kaiako and ākonga BEFORE they begin using AI to support learning.  Kaiako and ākonga can begin to know the capabilities of the AI tools – not merely as an ‘enhanced search engine’ - and understand the differences between using a Gen AI tool, searching an academic database and using a browser.

AI literacies for foundation/bridging ākonga are specific

• AI literacies for foundation/bridging ākonga are specific due to the need for these ākonga to attain academic literacies.
• Reliance on Gen AI to provide answers, does not encourage ākonga to undertake the deliberate practice and cognitive training to attain essential academic skills. Hence, bespoke or customised AI is recommended to target the learning outcomes that are AI supported and provide learning opportunities instead of provisioning ‘answers’.

AI literacies must be scaffolded

• Scaffolding AI literacies enable ākonga to reap the main benefits from Gen AI by using AI tools to support learning.
• Obtaining the best results from AI tools require the learning of skills specific to AI utilisation. These include prompt ‘engineering’ and critical evaluation and judgement of AI responses.

AI tools must be matched to ākonga level of learning

• Matching AI tools to ākonga levels of learning, learning goals/outcomes and digital capabilities is required. There are many Gen AI tools, each with specific advantages and disadvantages for various learning needs. The right tools at the right time can enhance ākonga understanding/achievement.