How is "NA" a permissible value in the "criterion_use_range" field?

Question

I recently received a data dump of ALL rubric data from our Canvas instance so that I could analyse how our rubrics are being set up.

One field I was interested to investigate was the ratio of rubrics using point vs using ranges. This field seems to be captured by "criterion_use_range" and contains values TRUE (i.e. is using ranges) and FALSE (i.e. is using points)

However, when I look at the data nearly half the cells have "NA". I thought this might be because the rubrics weren't scored or if the setting "Use this rubric for assignment grading" wasn't selected but that is not the case.

Does anyone know what the NA signifies?

Thanks

KenJohnson1 · Answer

Thanks for the detailed response, @Jeff_F.

I'm looking forward to the day when I can ask AI to do the analysis and it spits out the answer! :)

Jeff_F · Answer

Yes. NA appears to indicate that the range versus point distinction is not applicable at the criterion level, not that the value is unknown or missing. Canvas only sets TRUE or FALSE when a criterion’s structure supports that evaluation, which is why a single rubric can contain a mix of TRUE and NA values

A useful next step is to treat criterion_use_range as a criterion level attribute and design analysis paths that explicitly account for NA as “not applicable” rather than missing.

Some possible approaches (I wrote scenario prompt to save me time writing this out…. )  :

Option 1. Criterion level distribution analysis

Start by counting criteria by value TRUE, FALSE, and NA. This gives a baseline view of how often each modeling state occurs and helps confirm that NA is common and expected. This also avoids prematurely collapsing the data to the rubric level where meaning is lost.

Option 2. Exclude NA for range vs point comparisons

When the goal is to compare point based versus range-based criteria, explicitly filter to rows where criterion_use_range is TRUE or FALSE. Treat NA criteria as out of scope for that specific question rather than assigning them to either group. This produces cleaner ratios and avoids misclassification.

Option 3. Rubric level aggregation with context

Instead of a single rubric level flag, aggregate up from criteria. For example, count how many criteria in a rubric are TRUE, how many are FALSE, and how many are NA. This allows you to identify mixed rubrics and analyze patterns such as rubrics that combine range based and fixed-point criteria.

Option 4. Structural follow up analysis on NA criteria

For criteria with NA, shift the analysis focus from range usage to structure. Look at characteristics such as number of ratings per criterion, whether all ratings share the same point value, or whether the criterion is effectively binary or informational. This can explain why NA appears and may reveal common rubric design patterns.

Option 5. Rating level analysis after filtering

Once NA criteria are excluded, further analysis can drill into rating level design. Examples include number of rating levels, spread of point values, or consistency of ranges across criteria that do support range usage. This keeps each analysis aligned with what the data can meaningfully represent.

Option 6. Document analytic assumptions explicitly

Regardless of approach, clearly document that NA means “not applicable at the criterion level.” This helps future analysts avoid treating NA as missing data and preserves alignment with Canvas’s internal modeling.

KenJohnson1 · Answer

Thanks for the reply @Jeff_F . That makes sense but I can't work out the criteria for NA and almost half of the rubrics in my data dump have an NA value.

For example, I have several rubrics where two of the criteria in the rubric are TRUE and the others are NA. D