CXC on alert for cheats

As the buzz about ChatGPT intensifies, the Caribbean Examinations Council (CXC) has added its voice to the conversation, underscoring that “any form of cheating is a big concern” for the regional exam body.

The artificial intelligence chatbot, which was launched in November 2022, is capable of generating impressively detailed humanlike written text, and has passed law and business school exams.

But CXC Director of Operations Dr Nicole Manning says, based on the design of the School Based Assessment (SBA), a candidate’s competence should not be determined from one assignment.

“The SBA has both a formative and summative dimension. In relation to the formative dimension, it is expected that the teacher’s focus would really be on assessment for learning, as such, facilitating the assessment of skills and competencies not otherwise easily assessed in the final examinations administered in our January or May/June sittings,” Manning told The Sunday Gleaner.

She explained that the SBA process is expected to assess skills such as complex decision making, communication, collaboration, creativity and innovation.

Further, Manning said there are many other assessment strategies that should be used besides a written assignment to assess students’ skills, such as peer group critique or oral presentations with structured questioning.

“We believe there should be no difference between the level of concern with a student utilising the services of a subject matter expert to complete an assignment on their behalf and one who uses ChatGPT,” the director of operations said.

SCRUTINY OF SBAs

In the past, teachers and students have come under scrutiny regarding the execution and grading of SBAs.

The Barbados-based CXC began moderating all SBAs instead of samples from respective examination centres in 2021.

In a 46-page report on the administration of the July-August 2020 Caribbean Secondary Education Certificate (CSEC) and Caribbean Advanced Proficiency Examinations (CAPE), the chief examiners attributed the higher marks that teachers tended to award for the SBA to a number of factors.

Among them were the lack of thoroughness and vigilance by teachers while marking SBAs.

This led to the award of full marks, in some instances, for areas that students did not even attempt, as well as “... the misguided belief that the CXC moderators would be inclined to reduce teachers’ marks and that some element of mark inflation would protect their students from falling below their expected grade”, the report noted.

Manning said CXC will continue to engage teachers about the effective management of the SBA process.

“We believe teachers are resilient and, being the professionals that they are, we expect that the threat to SBA content will be effectively managed. Additionally, we will also continue our work with ministries of education and all parties, to share information, guide and resolve challenges which may arise. Alternative Chatbot engines such as GPT-2 Output detector, Writer AI Content Detector and Content at Scale are possible tools which may be used to assist in the detection of plagiarism,” she advised.

FORCE STUDENTS TO THINK

Principal of Campion College, Grace Baston, reasoned that, until a sophisticated AI detector is developed, ChatGPT will pose a challenge in determining the authenticity of students’ work and the validity of their grades, as a result of their access to the technology.

In January, OpenAI launched a classifier trained to distinguish between AI-written and human-written text but cautioned that it was “not fully reliable”.

“I teach at the sixth-form level and my solution to that is a very traditional response. My students do their work under my supervision with pen and paper. They are not on devices and the key thing is for teachers to set the type of questions that forces students to think rather than regurgitate answers,” Baston told The Sunday Gleaner.

Further, the principal shared that it is “crystal clear” when ChatGPT has done a students’ work, because the average student does not have that level of competence with English to produce that kind of writing.

“One of the weaknesses of Jamaican students across the board is their writing, even those who have grade ones and straight A profiles in English Language. When they have to do extended writing, invariably you come across errors because children just don’t read as much as they used to and they write as they speak. So, one of the clear giveaways would be the quality of the English,” Baston said, adding that, historically, teachers have done a very good job of being able to detect heavy plagiarism.

When asked about the strength of its plagiarism-detection mechanism, CXC said it has consistently been able to address the issue.

“Preventing AI-assisted plagiarism is no easy feat. A lot truly depends on one’s integrity and values,” Manning said.

She explained that there are two levels of review for plagiarism in SBAs – at the centre level and at the CXC level.

Manning shared data with The Sunday Gleaner representative of plagiarism that may have been missed at the centre level but was identified at the CXC level.

Between 2019 and 2022, twenty cases of plagiarism were detected, with the majority being among CSEC students and only one among CAPE students.

CXC’s penalty for plagiarism in SBA, artwork and other submissions is the cancellation of the scores on the question(s) or paper(s).

BECOMING INTELLECTUALLY LAZY

Principal of William Knibb Memorial High School, Linvern Wright, argued that ChatGPT will make plagiarism harder to detect.

He said students who use the tool are likely to become intellectually lazy and certainly not focused on experiencing learning.

“ChatGPT is more about profit than it is about promoting critical and independent thinking skills. Educators will need to become more alert and focused on becoming better at enabling learners to appreciate the development of their minds rather than just earning a grade with minimal, if any, effort,” Wright said.

Meanwhile, Cornwall College principal, Michael Ellis, told The Sunday Gleaner that he is not a fan of any tool that is likely to stifle originality.

On the contrary, he said, he likes the fact that it can be used to create authentic assessments which are creative learning experiences to test students’ skills and knowledge in realistic situations.

“It’s a great way to develop an idea and produce large amounts of texts in quick time. In my view, it should be seen as a tool that enhances teaching and learning and not a replacement for people or initiative,” Ellis said.

DECEPTIVELY GOOD WORK

Nicola Davis Kerr and Kevin Jackson, who are teachers at Excelsior High School, shared that there are concerns regarding students’ use of ChatGPT, since it could mean that they would not be performing the tasks that are necessary for aiding and facilitating the mastery of objectives.

“In some cases, too, the work they produce may be deceptively good and cause teachers to wrongly assume that learning objectives have been mastered,” Davis Kerr said.

But the educators reasoned that the concern is limited because most teachers quickly develop an idea regarding students’ ability to manipulate subject content, as well as their ability to express themselves.

Therefore, work that is significantly superior to what a student normally does may be easily identified and the student interrogated to determine their true knowledge.

They recommended a mixture of assignments – independent and supervised – to reduce the opportunity for students to short circuit the learning process.

Next month, CXC will host a panel discussion on ChatGPT with experts during its meetings with the national committees across the Caribbean region.

Manning said these committees have representation from various groups such as student bodies, school administration and ministries of education.

Additionally, CXC will host the 48th International Association for Educational Assessment (IAEA) annual conference in September, under the theme ‘Digitalizing Assessments and Credentials’, and it is their intention to place some focus on ChatGPT and other artificial intelligence tools.

judana.murphy@gleanerjm.com