How to get all data from Live API

Hi All,

I was trying to get User Page View from within Canvas of a particular student and the CSV only show 300 lines of data. I was told by Canvas to use Live API, I ran the /v1/users/{user_id}/page_views but the data I got was even shorter, only 9 instances.

I know there is a way to get more than 300 instances but can't find that info any where.

Thanks in advance if anyone have any idea how to get this.

Find more posts tagged with

Accepted answers

All comments

James

@qnguyen

The Live API is just a user interface around the REST API to make it easy for people to test. What you're dealing with is the issue of pagination.

By default, most requests limit themselves to return 10 responses (perhaps you miscounted -- most software that displays it starts counting with 0 rather than 1). You can specify a query parameter of per_page=100 to get 100 at a time, but most requests won't support more than 100 at a time, even if you specify a bigger number.

There is a link header returned in the request that includes a link to the next page. It uses a bookmark for the page views (not the page=2 style used by other calls). You will need to make one request to get the link for the next request. You will need to take the link header and use that to request the next 100 entries and so forth.

The purpose of the Live API is to test and debug and pagination isn't really a part of that. You should use a programming language to make the API calls. Some people use Postman or another REST client, but you will still need to follow the headers to get additional pages.

James

@qnguyen

Some people hear the word programming and they give up because they don't have those skills. I thought of a few ways that you can get this information without any programming. None of these are efficient and not something you would want to do on a regular basis, but it can be helpful in a pinch.

Live API

Under the Users API is the List user page views endpoint. It contains two parameters that may be more useful than forcing yourself to go back more than 300 pages. That is the start_time and end_time. What you could do is take the time from the last page view returned and use it as the start_time for your second request. Then take the time from the last page view returned and use it as the start_time for your third request.

The problem with the Live API is the limitation of 10 pages per request and that might cover as little as a few seconds of page views.

Postman

You can also use the start_time and end_time parameters in Postman. Very rarely do I want to know the last 300 page requests for a user. It's usually some specific time period where I suspect something is going on. I can use the start and end times to narrow in on that time. By using the approach above but with Postman and the per_page=100, you can avoid the Link header.

This is faster than the Live API because you can get 100 pages at a time. If you're going to use Postman, then using the link header is probably quicker than changing the start time.

Browser Developer Tools

In most browsers, I can press F12 to bring up the developer tools, but I'm going to use Chrome to illustrate.

By going to the Network tab and choosing Fetch/XHR, you can monitor the traffic that Canvas sends and then copy/paste it into a website that will convert JSON to CSV. You can then copy/paste that into Excel and quickly get more than 300 rows. This approach is good when you do want the most recent page views but you want to go back more than 300 rows.

I'm not advocating a site in particular, but a quick Google search found csvjson.com/json2csv that will allow you use tab as a separator and copy the contents to the clipboard.

Here is a video I made that illustrates the process.

Unanswered Questions

Concluded Course and Canvas Data Services
Canvas Data Services sent out an alert that a course was concluded (because the course was concluded) and the status of each user in the UI is set to complete, however, the enrollments API endpoint shows the status of each user in the course as active. Is this a common issue? Have I misinterpreted how the individual…
Canvas Media Question
How can one track Canvas Media use without having the reports available from Canvas Studio? Checking API endpoints and the Data 2 schema, there do not seem to be dedicated Canvas Media endpoints. Am I missing some? A method I've considered is using the body field of the wiki_pages table, as this contains the html code of…
Canvas Catalog API - Completed Certificates
Hello, I'm trying to utilize an API call to pull completed certificates and was hoping to download and store the certs as a backup. I can successfully run the API call but it will not allow me to view the certificate unless I access the URL while signed into an account where it's "my" cert. Is there a permissions setting…
UI for weekly progression idea
UI for weekly progression idea that rest in the Schedule tab of the **K-12 UI** of Canvas it's incomplete and I really would not take it and push it into instance but I am going to post it here because in the past people like @James && @robotcars || @Steve_25 have had terrific input in the past... maybe w/their eyes and…
Canvas integration with React + Flask webapp - initially for authenticaiton
I have a new React + Flask webapp. It has its own login for users to start using the app. I have added this as an external tool in Canvas. What I want initially is to see of canvas users can start using the webapp without having to login if they click on the external tool link in a canvas course. I have 2 methods…