Canvas Data 2 schema version errors

@sgergely It seems that schema versions for a number of tables in the the canvas namespace have reverted. For example, "rubrics" had been at version 2 in mid February, but shows version 1 when downloaded today from DAP. The following tables seem to have issues:

access_tokens
assignment_overrides
assignments
attachments
conversation_messages
developer_key_account_bindings
discussion_topics
grading_standards
learning_outcomes
master_courses_master_content_tags
rubrics
submission_comments
submissions

We aren't using the DAP client to load the tables, but we do check versions numbers as part of our process.

Find more posts tagged with

cd2

Canvas Data 2

Accepted answers

sgergely

Hello everyone! Thanks for reporting the issues!

There was an issue while we were deploying a new version of our application in the background without the intention of making any changes to the schema. The released version got corrupted and unfortunately published "version 1" for the tables reported above. Once I have seen @mclark19 mention I instantly went after this and notified the team and they have quickly fixed it.

Sorry for the issue we have caused! We are implementing additional tests to catch such issues before they go live to prevent these in the future.

All comments

phanley

I'm also seeing errors ('cannot drop values in an enumeration: pending' ) with the same tables this morning

Sample error log:

Running command: ['/usr/bin/bash', '-c', 'set -e; dap syncdb  --namespace canvas --table access_tokens || dap initdb  --namespace canvas --table access_tokens ']
Output:
Python version: 3.11.11 (main, Feb 25 2025, 02:38:55) [GCC 12.2.0]
Platform: uname_result(system='Linux', node='canvas-data-sync-dap-init-or-sync-db-i6kk27gj', release='5.15.0-131-generic', version='#141-Ubuntu SMP Fri Jan 10 21:18:28 UTC 2025', machine='x86_64')
Package versions: {'pysqlsync': '0.8.2', 'json_strong_typing': '0.3.7', 'instructure-dap-client': '1.3.1', 'aiohttp-retry': '2.9.1', 'types-aiofiles': '24.1.0.20241221', 'tsv2py': '0.7.1', 'aiofiles': '23.2.1', 'PyJWT': '2.10.1', 'aiohttp': '3.11.11'}
pysqlsync.postgres:connecting to cd@dbserver:5432/canvas
pysqlsync.postgres:PostgreSQL version 16.0.6 final
Database version: PostgreSQL 16.6 (Debian 16.6-1.pgdg120+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 12.2.0-14) 12.2.0, 64-bit
INFO: pysqlsync.postgres:connecting to cd@dbserver:5432/canvas
ERROR - operation not permitted; cannot drop values in an enumeration: pending
INFO - Python version: 3.11.11 (main, Feb 25 2025, 02:38:55) [GCC 12.2.0]
INFO - Platform: uname_result(system='Linux', node='canvas-data-sync-dap-init-or-sync-db-i6kk27gj', release='5.15.0-131-generic', version='#141-Ubuntu SMP Fri Jan 10 21:18:28 UTC 2025', machine='x86_64')
INFO - Package versions: {'pysqlsync': '0.8.2', 'json_strong_typing': '0.3.7', 'instructure-dap-client': '1.3.1', 'aiohttp-retry': '2.9.1', 'types-aiofiles': '24.1.0.20241221', 'tsv2py': '0.7.1', 'aiofiles': '23.2.1', 'PyJWT': '2.10.1', 'aiohttp': '3.11.11'}
INFO:pysqlsync.postgres:connecting to airflow_user@canvas-data-rw:5432/canvas
ERROR - operation not permitted; cannot drop values in an enumeration: pending
INFO - Command exited with return code 5
ERROR - Task failed with exception

reynlds

I'm seeing something very similar on these tables:

access_tokens
assignment_overrides
assignments
attachments
conversation_messages
custom_gradebook_columns
developer_key_account_bindings
discussion_topics
grading_standards
learning_outcomes
lti_resource_links
master_courses_child_subscriptions
master_courses_master_content_tags
rubrics
submission_comments
submissions
submission_comments
submissions
web_conference_participants

Here are the errors I'm seeing...everyt weekend things go to heck...:

2025-03-03 06:05:27,773 - ERROR - operation not permitted; cannot drop values in an enumeration: pending

2025-03-03 06:08:21,369 - ERROR - operation not permitted; cannot drop values in an enumeration: Course

2025-03-03 06:08:38,755 - ERROR - operation not permitted; cannot drop values in an enumeration: outcome_alignment_cloningfailed_to_clone_outcome_alignment

2025-03-03 06:09:26,514 - ERROR - operation not permitted; cannot drop values in an enumeration: public

2025-03-03 06:16:03,345 - ERROR - operation not permitted; cannot drop values in an enumeration: audio/aacaudio/amraudio/flacaudio/mp4audio/mpegaudio/oggaudio/vnd.dlna.adtsaudio/wavaudio/webmaudio/x-m4aaudio/x-ms-wmaaudio/x-wavvideo/3gppvideo/avivideo/mp4video/mpegvideo/oggvideo/quicktimevideo/webmvideo/x-m4vvideo/x-matroskavideo/x-ms-asfvideo/x-ms-wmvvideo/x-msvideo

2025-03-03 06:19:40,419 - ERROR - operation not permitted; cannot drop values in an enumeration: deleted

2025-03-03 06:21:46,117 - ERROR - operation not permitted; cannot drop values in an enumeration: not_threaded

2025-03-03 06:25:41,398 - ERROR - operation not permitted; cannot drop values in an enumeration: archived

2025-03-03 06:28:34,728 - ERROR - operation not permitted; cannot drop values in an enumeration: weighted_averagestandard_decaying_average

2025-03-03 06:29:29,780 - ERROR - malformed HTTP response:

ERROR: The request could not be satisfied

504 ERROR

2025-03-03 06:29:29,799 - ERROR - malformed HTTP response

2025-03-03 06:30:46,728 - ERROR - malformed HTTP response:

2025-03-03 06:30:46,747 - ERROR - malformed HTTP response

2025-03-03 06:31:02,721 - ERROR - operation not permitted; cannot drop values in an enumeration: MediaTrack

2025-03-03 06:37:35,275 - ERROR - operation not permitted; cannot drop values in an enumeration: archiveddraft

2025-03-03 06:38:51,185 - ERROR - operation not permitted; cannot drop values in an enumeration: audio/aacaudio/amraudio/flacaudio/mp4audio/mpegaudio/oggaudio/vnd.dlna.adtsaudio/wavaudio/webmaudio/x-m4aaudio/x-ms-wmaaudio/x-wavvideo/3gppvideo/avivideo/mp4video/mpegvideo/oggvideo/quicktimevideo/webmvideo/x-m4vvideo/x-matroskavideo/x-ms-asfvideo/x-ms-wmvvideo/x-msvideo

2025-03-03 06:39:32,565 - ERROR - operation not permitted; cannot drop values in an enumeration: audio/aacaudio/amraudio/flacaudio/mp4audio/mpegaudio/oggaudio/vnd.dlna.adtsaudio/wavaudio/webmaudio/x-m4aaudio/x-ms-wmaaudio/x-wavvideo/3gppvideo/avivideo/mp4video/mpegvideo/oggvideo/quicktimevideo/webmvideo/x-m4vvideo/x-matroskavideo/x-ms-asfvideo/x-ms-wmvvideo/x-msvideo

jwals

Hi @mclark19, we had most of our tables fail yesterday morning (Monday, March 3). Around 80 tables failed with some version of "operation not permitted; cannot drop values in an enumeration" and a couple failed with "malformed HTTP response." We ran into the "cannot drop values in an enumeration" issue almost exactly a year ago when Instructure made significant changes with DAP 1.0. By yesterday afternoon all of the tables were syncing successfully again without us having to drop and reinit any tables or types. Rather than a regional issue, I'm wondering if Instructure (accidentally?) made some breaking changes that were quietly rolled back.

StevenWalls

We had multiple table syncs fail yesterday with those error too, none have failed today except one with "malformed http." Haven't heard an explanation.

sgergely

Hello everyone! Thanks for reporting the issues!

Sorry for the issue we have caused! We are implementing additional tests to catch such issues before they go live to prevent these in the future.

Unanswered Questions

EDUCAUSE Insights: Data and Decision-Making
EDUCAUSE is full of inspiration, whether you’re onsite or following from afar. Join the evolution by sharing what sparks your curiosity, the questions you have, and the insights you discover. We'll learn from each other, expand our networks, and strengthen our Community. Your perspective matters, and the more we share,…
Problems with GraphQL in User Pageviews reports
Prompted by the most recent Canvas release notes, we looked into the Page Views feature, and we were happy with the new documented changes. However, we notice that more user actions are done by GraphQL API, which does not provide details in the page view report, either in the GUI or in the CSV file download. We and most…
Metrics Easy Button
I am currently teaching five courses this semester with more than 70 students. To track their progress, I currently have to review each course and section individually, which is very time-consuming. Ideally, I would like to consolidate key information for all students into a single Excel file—showing class attendance…
Analytics API / Metrics calculation logic
Hello, We are currently using the Analytics API provided in the Instructure Developer Documentation Portal to retrieve certain metrics-related data. We would like to know if it’s possible to replicate these datasets using the Canvas Data Platform (CDP), instead of relying on API calls. For example, could you please advise…
CD2: Enhanced Rubrics
Is there a way to tell via CD2 if a course is using Enhanced Rubrics? I didn't see it in the entity diagram for Rubrics table, but I might have missed something... Thanks Ian