Tips for A.I.-Assisted Exam Integrity Reviews

Your institution might be set up to use Artificial Intelligence (A.I.) to assist with Exam Integrity reviews. After your exam-takers complete an exam with ExamMonitor, you'll need to review the A.I. report and the recording to determine if any breaches occurred.

The A.I. report provides you with valuable assistance by calling attention to unusual behaviors, such as multiple people, unexpected objects, speaking, and off-screen gazing. However, you should think of the A.I. as an assistant. It's the people at your institution who are ultimately responsible for investigating the findings, reviewing the recordings, and determining if any breaches occurred.

This article provides tips to help you to succeed with the A.I.-assisted review process.

 

Before You Begin

  1. Log in to your institution's ExamSoft portal.
  2. Select the Exam Integrity tab. 

    Note: For full details about Exam Integrity and the various Exam Integrity UI pages, see: Exam Integrity: Review Results from ExamID and ExamMonitor (Enterprise and Legacy)

  3. On the Assessments page, find the assessment that you want to review, and then select the Assessment Name.

    Note: This link goes to the Review Queue, which is described in the next section of this article.

 

Review Queue

Use the Review Queue page to monitor the status of the A.I. review and your own progress in issuing dispositions for each exam-taker's results.

 

ExamMonitor Status

In the ExamMonitor column of the table, you might see the following symbols.
Symbols Description
rtaImage1.png The A.I. review is completed. This process might take up to 24 hours after the exam is submitted.

Note: If you have conducted assessments with human proctoring in ExamSoft, you might be expecting to see a green checkmark as the indicator of the final "success" status. However, for A.I.-assisted reviews, this eye icon is the final "success" status.
rtaImage1.png The A.I. review is in progress. If you visit the Review Queue page less than 24 hours after the exam deadline, it's likely that some exams will still be in progress.
rtaImage1.png Either the files have not been uploaded yet or ExamMonitor was not enabled for this assessment. It could indicate that a student has not taken the exam, or, if you are checking this page a short time after the exam deadline, it could indicate that the files are still uploading.
rtaImage1.png Due to this exam-taker's user account settings, the assessment was delivered in non-secure mode. 

 

Disposition Status

When the A.I. review is completed, the disposition status is set to Pending Review. Someone at your institution needs to review the A.I. report and the recording to determine if any integrity breaches occurred.
IMPORTANT

If you have conducted assessments with human proctoring in ExamSoft, you might be expecting to see red triangles indicating incidents that require investigation. However, those indicators are provided during human proctoring only. For A.I.-assisted reviews, you'll need to review the A.I. report for every exam-taker in order to determine whether the detected actions are "incidents" or acceptable behaviors.


As you work through the list of students on the Exam-Taker Details page, we recommend that you enter a disposition there. The Disposition column on the Review Queue page will serve as a status list and a to-do list, indicating which exams require review by your team.
Disposition Option Description
Pending Review The default status issued by the A.I.. Someone at your institution needs to review the results for this exam-taker, and enter a Disposition on the Exam-Taker Details page.
Resolved Someone at your institution reviewed the A.I. report and the recording and determined that no breach occurred.
Escalated for Review Someone at your institution reviewed the A.I. report and the recording and determined that there was a need for further investigation.
Example: A junior member of your team might be authorized to review the findings and make recommendations but not to make an ultimate determination.
In this scenario, a senior member of the team needs to review the A.I. report, make a final determination, and enter a final disposition (Resolved or Confirmed Breach) on the Exam-Taker Details page.
Confirmed Breach Someone at your institution has reviewed the A.I. report and the recording and determined that an integrity breach occurred.

 

Exam-Taker Details Page

  1. To open this page: elect an exam-taker's name on the Review Queue page.
  2. To navigate between the exam-takers' results: Use the Prev and Next links near the top right corner of the page.
  3. Select the ExamMonitor tab to see the A.I. report.

    rtaImage1.png

    Note: If there are any Notes in the top left corner of the page, this would mean that someone else at your institution has reviewed this exam and entered information.

  4. Read the notes to see what has been done so far.

    rtaImage1.png

  5. Review the video closely to identify any suspicious behaviors. You can view the video in full-screen mode to see more detail.

    Note: It can be helpful to view the first few minutes the video to get a baseline impression of the exam-taker and the work environment. For example, you can take note of any lighting issues, background noise, or incidental movement. You can notice any apparel, such as a mask or a scarf, that might result in false indications of "test taker missing" (because the A.I. cannot detect expected features of the face).

  6. Scroll down to the bottom of the page to see the Incident Text and Incident Graph from the A.I. review.

    Notes:

    • In the Incident Graph, use the controls (a) to zoom in or out. Select a colored bar (b) to jump to that portion of the video. Then play the video to view the indicated behavior.

      rtaImage1.png

    • In the Incident Text, see a description of each behavior. The A.I. also adds a label to indicate the potential severity of the action, such as Low, Medium, or High, and indicates the length of time for each action.

      Types of incidents: Variations from expected exam-taking behaviors will be detected and should be reviewed to determine if they constitute integrity breaches.

      • Unnatural hand gesture: The exam-taker might be fidgeting or leafing through a textbook or using an unauthorized device.
      • Off screen gaze: Perhaps the exam-taker is gazing into space while thinking about the question, or perhaps the exam-taker is looking at a cheat sheet posted on the wall. 
      • Speaking: The exam-taker could be reading a question aloud, or perhaps some background noise is being picked up by the microphone. However, you'll need to watch these sections of the recording to see if the exam-taker is talking with someone to get help with an answer.
      • Suspicious objects: Watching the few minutes of the recording can familiarize you with any incidental objects that might be noticed by the A.I.. Check these sections to see if the exam-taker is using unauthorized resources.
      • Multiple people: Possibly the exam-taker is in a library, cafe, or another busy place. Perhaps the exam-taker is at home with other family members. You'll need to review these sections to see if other people are passing by in the background or if they are helping the exam-taker to cheat.
      • Test taker missing: Possibly the exam-taker is wearing a hat, scarf, mask, or other apparel that obscures the face. Possibly the exam-taker is slouching back against her chair or resting his chin on his hand. However, it's also possible that the exam-taker has stepped away from the computer for an extended time.
  7. Enter Notes near the top left corner of the page after reviewing the video.

    IMPORTANT

    • Enter your comments carefully because after you submit them, they cannot be edited or deleted.
    • Also be aware that individuals at your institution might need to share these notes during academic integrity hearings that result from this process.
    • Remember that the notes do not indicate a final determination and do not change the status.

     

  8. Enter a Disposition near the top right corner of the page when you finish reviewing the A.I. report and the recording for an exam-taker. This way, the Disposition column on the Review Queue page will reflect your progress through this set of A.I. reports. 

    Note: For descriptions of the various disposition options, see the Disposition Status section of this article.

 

Was this article helpful?
1 out of 1 found this helpful

Articles in this section