Canvas Data to Azure??

Question

I've got Canvas Data coming in to my AWS environment by following this guide:

Build a Canvas Data Warehouse on AWS in 30 minutes... - Canvas Community (canvaslms.com)

However, I'd like to do something similar in Azure. Without building it from scratch, are there any institutions out there that are sending Canvas Data into Azure like this?

jsimon3 · Answer

https://github.com/uvadev/PullCanvasData2/pkgs/nuget/PullCanvasData2
This will give you a sense on how to use it just configure the table writing to point to azure
https://github.com/uvadev/CD2-Cron-or-DB-Source

warren_goetzel · Answer

We have CD2 in Azure now, and are looking to add Live Events next.

bliszewski · Answer

I'm working on this now, but we're pretty new to Azure so there's a lot of discovery happening along the way and I'm not sure yet what the final result will be as our institution works out the Azure tools and processes they want to support.

So far I've got an Azure Timer Function dumping (a selection of) object downloads to Blob Storage, and was going to start looking into Data Factory to see if that can handle the CD2 metadata or if we need something more custom to process it first. It currently requires a database to store query history as I'm not sure if there's a better way to do that.

For anyone interested in this, what kinds of things are you looking for? Are there other Azure services/tools you like to use or think would be useful in the process? Is your endgame a (fully-structured?) database, a data lake, or just dumping the files so you can figure out what you want to do with them from there?

There's a lot of refinement I still need to do with regards to how objects are queried and downloaded, but--especially as I'm pretty new to Azure--I'd appreciate some feedback about how to make this a more robust and re-usable resource that I'd be happy to share.