Online supervised exams allowed students to sit assessments anywhere, monitored by webcams and microphones. Photo: 123RF
- Massey could not initially contact its technology service provider during an online exam failure
- The overseas-based company initially did not understand the severity of the situation
- It then said the problem was fixed, but it was not
- Massey is now reviewing the use of online exam technology
As technology failures had students frantically trying to log on to their online supervised exams at Massey University, the overseas-based system provider was not immediately contactable.
And when the provider PSI did respond on the evening of 10 June, it failed to understand the severity of the situation, and then told Massey the system was safe to continue using.
But two days later the same bugs returned, again locking students out of their exams, and Massey stopped using the technology. It will not return this academic year.
More details about the June exam technology failures have been released to RNZ by Massey under the Official Information Act.
Included in the often heavily redacted documents was an executive summary of an external report into the incident, by PWC. It found Massey did not have an adequate process in place about the level of service PSI should provide and that the university's communications process about the incident was slow.
Online supervised exams allowed students to sit assessments anywhere, monitored by webcams and microphones. Three out of Massey's five colleges used them.
But problems during June's examinations affected about 400 students, who could not log on to their tests, had delays in doing so, or could see names, IDs and course codes for other students.
A Massey briefing document written shortly after the outage said students started reporting problems with the system about 2.40pm on 10 June. IT found it was not a Massey issue so contacted PSI 10 minutes later, sending a 'severity 1' message, followed by phone calls.
"At 4.26pm on Monday [10 June], PSI responded to Massey, though the messaging from PSI was confused and they did not appear to understand the severity of the issue for Massey and our students," the briefing said. "It took PSI until 5.36pm to resolve issues. Massey was then assured the system was safe to continue to use for the remainder of the semester 1 examination period."
No problems were reported on 11 June, but they returned on 12 June.
"Massey immediately alerted PSI, with no response."
The university suspended the online supervised exams, and alternative arrangements were made for students, such as reverting to online, open-book assessments.
Photo: RNZ / Samuel Rillstone
The PWC review praised aspects of Massey's response, including the way it handled privacy concerns, helped students at the time, and had engaged positively with PSI.
"Massey continues to request more information from the vendor to better understand the incident and their processes."
But the review found shortcomings in other areas, such as the university's assessment services team deciding the failures were a 'severity 1' incident, but having no required response for this. There was no 'business continuity plan', which would lay out alternative options should such a failure happen.
Other findings included that Massey did not "set adequate service requirements with the vendor" about how to support Massey during an incident, and that communications about the incident took hours to write due to the number of people involved.
The report recommended improvements in these areas.
Massey University provost, professor Giselle Byrnes, said the university's "future of online examinations working group" was considering the report's recommendations and its long-term plans for online supervised exams.
"We would like to assure our stakeholders that Massey is committed to ensuring these recommendations are implemented to rebuild confidence in our examinations and assessments processes, as well as streamlining our internal procedures."
Byrnes said that would include contingency plans.
Alex Murray, a student affected by the outages, said he was disappointed that the student body at large was not given a chance to have a say on online supervised exams.
He organised an online petition against the use of the technology, which was signed by more than 1000 people, and he said he had not spoken to anyone who liked the system.
The university recently held a planning event to gather ideas on revitalising its Manawatū campus, and could do something similar to garner opinions about online supervised exams, he said.
PSI, which has its headquarters in the United States, was contacted for comment.
