How to retrieve complete quiz log data, i.e. session information and responses, using API calls?

j_strohhecker

Dear all,

I am a instructor and I am searching for a possibility to retrieve the complete quiz log data including the responses. I have found a way to obtain the session information, but I am missing a solution for the responses (as explained  https://community.canvaslms.com/t5/Instructor-Guide/How-do-I-view-a-quiz-log-for-a-student/ta-p/580). Any help (or hint to an alternative way of exporting the complete log data) would be highly appreciated. Many thanks in advance.

Best, Jürgen

James

Jürgen ( @j_strohhecker )

I am answering this for Classic Quizzes, not New Quizzes.

There is an API for retrieving quiz logs with some responses, before I get to that, let me make a few comments.

After you click View Log, you get the action table. At the top is a list of the attempts where you can select an attempt.

If there are multiple attempts, it is possible that an attempt has no responses. I guess it's possible even without multiple attempts, but students will open the quiz another time and then decide not to take it. This is encouraged by those quizzes with unlimited attempts and keeping the highest grade. Others open it, forget they opened it, and then it automatically submits when the quiz closes. This means that the last attempt might not have any responses.

Notably, this can happen even if the log says a student answered questions.

For example, I'm looking at a quiz that says the session started at 00:02 and at 00:05, the student answered questions 1, 2, 8, 12, and 13.  This is documented and expected behavior. See the View Autosave Answers section of the document you linked to.

The follow-up to that is that once you click on any of the links to questions, you may need to scroll down past the first few questions to check.

In other cases, certain types of questions do not show the responses. I have a matching question that the student got completely correct. When I look at the log, it says that question was answered once, but then shows "No Answer" for the response.

Pulling the information from the API is unlikely to get you much that isn't already in the logs. Canvas makes the API call when it shows the log information to you.

Log files sometimes have expirations on them. I tried pulling a log from a quiz taken in 2016 and finished in 2020 and there was no information available for it. If Canvas is showing you the log information, this likely isn't the issue or Canvas wouldn't be able to show it to you, either.

 

Now on to the API. We're using the Retrieve captured events endpoint of the Quiz Submission Events API.

GET /api/v1/courses/:course_id/quizzes/:quiz_id/submissions/:id/events

Note that :id is the quiz submission ID, not the quiz ID. To get this ID, you will need to first fetch the Quiz Submissions.

GET /api/v1/courses/:course_id/quizzes/:quiz_id/submissions

The id property from the Quiz Submissions matches up with the :id for the retrieve captured endpoints.

You can also get the Quiz Submission ID ( the :id ) from the URL when viewing the log.

Your URL will look something like /courses/1234/quizzess/258/submissions/987/log. The 987 is the quiz submission ID.

Note that if there are multiple attempts and not viewing the default attempt, you will need to specify the attempt as a query parameter when you get the events. That is also contained in the URL for the log. For example, for my student, I'm using attempt 8. You may also want to specify a larger per_page query parameter. The default is 10, meaning you only get 10 events at a time. The maximum is 100 and you will need to use pagination after that.

GET /api/v1/courses/:course_id/quizzes/:quiz_id/submissions/:id/events?attempt=8&per_page=100&page=1

This gives you an object quiz_submission_events that is an array. The first item should be the submission_created, which contains a copy of the quiz. This is not displayed in the quiz log. The second one is session_started and this is the first entry contained in the log you see in the browser.

Now we get to the interesting stuff. My third item has event_type="question_answered" and contains an event_data array. It contains the quiz_question_id and the answer. This event contains a lot of nothing. By that, I mean it was the one that went through and initialized the answers. Everything is either null or an empty array. Every event also has a timestamp that can be used to determine the times. These events are dated with UTC dates, not relative timestamps since the quiz began.

Then I have lots of page_blurred, paged_focused, and question_viewed entries.

I now look at the next question_answered event. It contains a quiz_question_id and an answer code. I go back to the first event for the submission_created. I find that quiz_question_id and see it's question 3, it's a multiple choice question. I then take the answer code and look it up to see that they gave the answer "1 is dropped as soon as there are 2" and the weight of that question is 100, meaning it was the correct response.

Multiple choice questions show up properly in the log, so I'm going to hunt down my Matching question that didn't show up.

From the first submission_created event that contains the quiz, I search for question_type='matching_question'. It didn't take long as it was question 1 and at the top. I grab the id for that question and then search for it.

It's not there. How can that be? I check the Link headers for pagination issues. It shows that I have all of the events on a single page, I do not need to go back and check for additional pages.

I have a second question that is matching and it shows up in the events, so I know it's not an issue of me using the wrong IDs. Canvas just doesn't have an event for the student answering that question. Accordingly, the event log shows it as No Answer since there is no event record for it being answered. But the student answered it and got it correct.

That second matching question shows up one additional time beyond that initial submission_created event. That's for a question_viewed event. There is no event for the question being answered. The student answered the question and missed one part, but there is no event to back that up. The Canvas log in the browser just says the question was answered once but there is No Answer.

The case of No Answer also appears for multiple answers questions. My question 7 was one of those that the log shows was answered once but there is No Answer. The student answered it correctly, but there is no event for it. The only time it showed up in the event log was a question_viewed event that happened 8 seconds into the quiz.

 

Now I decided to see what would happen if I took a quiz in Canvas that had a multiple fill in the blank question. There's just one question on the quiz.

The only events being written were page_blurred and page_focused. There were no events written for me answering the question.

There were, however, non-API calls made to backup my responses every time I changed an answer. It sends the question text and what my questions were. Again, that's a non-API call, which means that it is only accessible from within Canvas. It's also used for saving responses, not reading them at a specific point in time.

 

In short, there is an API, but I'm not sure how helpful it will be. Some answering of questions is just not recorded as an event. I do not know whether it is not sent, not received, or filtered out in the process and not recorded.

That brings me to the question of what you really need out of this?

If you are wanting the log for cheating detection, I don't think there's any extra information to be obtained through the API than through the web interface.

If you want the responses and think this is the only place to find them, then there is actually a more reliable method. You need to use the submissions API  and specify a parameter of include[]=submission_history to get the quiz responses. Multiple attempts require additional parameters, but I'll hold off on that here because that may not be what you're after at all.