Classic Quiz statistics issue

rlbrown21

In the statistics report from quizzes, you can see how many students missed a question.

The issue is that if a student gets multiple attempts, only the attempt kept is showing in the statistics.  

So if a student misses the same question twice, but gets it correct on a third attempt (which ends up being the graded attempt), then the stats only show that the question was answered correctly.  more accurately it should show two incorrect and one correct attempt so faculty know where students are struggling.

am I missing something?  is there a setting or a different report that is more accurate?

James

@rlbrown21 

For Classic Quizzes, you have Analysis Reports (Student and Item) links at the top of the Quiz Statistics page.

Neither one will exactly give you what you want -- statistics for all attempts.

• The Item Analysis uses the first attempt but has several limitations.
• The Student Analysis gives you all attempts, but does not do any kind of statistics on them.

To get the multiple attempt situation, you would need to use the Student Analysis and then do your own review of the data.

 

On the back end of Canvas, not available through the web interface, there is a Create a quiz report endpoint in the Canvas REST API. This allows the user to request that all submissions should be used. That sounds promising until you read the whole description of what it does: "Whether the report should consider all submissions or only the most recent. Defaults to false, ignored for item_analysis." This makes it totally useless for what you need.

• This is kind of technical and cannot be done through the Canvas web interface.
• It defaults to false, but if you get the student analysis through the web interface, it does include all of the submissions. There's no need to use the API if you're a teacher just looking at the data for your class. It is helpful for people who have to download a lot of these reports (say there are 20 sections of a course that gave this quiz).
• The most important part is that it is "ignored for item analysis". In other words, even if you told the item analysis to include all of the submissions, it's not going to do that.

 

That brings us back to the Student Analysis report and creating your own statistics.

Analyzing the Student Analysis isn't the easiest thing to do, especially depending on the question and whether it's part of a question group. I can go in and create a Pivot Table, put the student names as the rows and then pick a question for the column. I then take the name or ID and put it in the values and change it to be Count (if it defaults to Sum). It will then show how many times each student gave that particular answer.

This image shows that the last student answered "a minimum of 12 hours" three times and did not get the question on one of their four attempts. Getting results are not reliable. I use question groups so there are a lot of blank entries for me. You also cannot tell the order in which the responses came in.

You would have to change the columns for each question, so it's a slow process.

You could make a different table. This time use the numeric column that follows the question that contains how many points the student achieved. Those columns don't have useful names in the CSV file, so you may want to rename them before creating the pivot table.  Use the student name for the rows, the attempt for the column, and the sum of the point column for that specific question as the values.

In the image, the question was worth 0.5 points, so that means the student got it correct. I can see that my second student didn't get the question on the first or second attempt, they missed it on the third, but got it right on the fourth attempt.

This technique does not show me what choices the student gave. I would have to look at the data in the file to find that -- the pivot table is meant to summarize.