Canvas Data 2 - User Time Zone Code

Question

Hi Instructure!

Is there a way to access the time zone code for the user's set time zone?

Example:

* Right now, a user can set their time zone in the LMS to  Eastern Time (US & Canada) (-06:00/-04:00)at /profile/settings.
* The time zone value for this user via Canvas Data 2 looks like this: Eastern Time (US & Canada)

Problem

The current format, to my knowledge, requires that I manually map every possible time zone selection to the conversion code if I want to analyze user activity in their set time zone.

Possible Solutions

To convert the time zone (I'm using redshift), I need either the offset which is being stripped, or one of the following code formats: [US/Eastern, EDT]

To me the solutions are either

* Add a new column to the users table with the code or offset
* Alter the format of existing time zone field in the users table so it includes the code or offset. If it is easier to include the offset via the format that appears in the LMS I can use regex to grab it.

Alternatively, let me know if ya'll know of a better solution! Thank you!

Jo-lCollins · Answer

Hi @James

Thank you for this information, but I do not think your response satisfies my question. To address some of your main points...

"trying to report times or include specialized time zone offsets is a bad idea and not something that Canvas should do"

When a student sets their time zone, this changes the time reporting a student interacts with within the Canvas LMS. I would like to be able to run analyses that reflect what the student is experiencing and how that relates to average student behavior. That is not something I consider a "bad idea", and it is something Canvas does do.

Different database languages require different conversion codes

This is a good flag, and something that is relevant for Canvas Data 2 devs to consider. However, I do not think this blocks their ability to provide a machine translatable data point for converting UTC timestamps to the student's set local time.

Timestamps should be stored as UTC

I agree. I am not asking timestamps to be stored in local time.

Time zones in canvas are unlikely to change, you should just map them yourself

I am requesting that mapping to be provided by Canvas Data 2.

"One downside is that people may change their time zones. I doubt that's a very big issue."

If a user changes their time zone, that change should be reflected in the users table when Canvas Data 2 syncs. If the time zone mappings were provided by CD2, the problem you describe here would not exist.

"Not all users set their time zone"

This is an institution level problem, not an issue with the data. 86% of users at my institution have set their time zone. While I appreciate you sharing your experience with these datasets, anecdotal examples like this are irrelevant.

A student may travel outside their set time zone.

This is irrelevant. I would like to be able to build datasets that show the time a student interacts with within the LMS. Additionally, this example describes rare behavior that does not reflect the vast majority of students.

Because of outlier student behavior, there would be error in time zone conversions, so you might as well just use Live Events, which also has error. Bad data is bad data so what's the difference?

The difference is in the type of error. Live events experiences missing records due to (what appears to be) machine error. Engineering teams I've worked with or communicated with who use the live events service have to supplement the service with API calls to ensure their data is complete. The Live Events error is more egregious when analyzing student engagement than the "error" caused by students traveling or failing to set their time zone, which (in my view) represents variance within student behavior that statistical tooling and data cleaning techniques are built to address.

There is a difference between time zone reporting in Live Events vs Canvas Data 2

While this is an interesting observation I do not see how this is relevant to my question "Can time zone mappings be provided within Canvas Data 2?". Though, I suppose it would be relevant if time zone was added to the return result from the rest api's page views endpoint

tagging @sgergely, because you liked James' response, and based on this post I think you work on the DAP development project. It's an excellent tool! 👋

Jo-lCollins · Answer

@robotcars @James

Sorry I should have been more clear. I am requesting instructure provide the timezone code. I'm aware adding columns to the synchronized DAP data would cause problems, and that is not what I was proposing.

I previously used the assets live events stream, and that provided a time zone code that could be used to convert UTC to the user's time zone, but the live events stream experiences random gaps in data coverage that have proven problematic for analysis so I am instead collecting user page views via the rest api, which is providing 100% data coverage. The page views endpoint does not provide the user's time zone. If this data could be added to the endpoint's return result that would also work. I could activate the assets live stream just to pull the most recent time zone code for a user, but given the size of assets data, and its redundancy with page views, this solution  feels a bit wasteful of resources.

The purpose of this post is to ask instructure's dev team if there is a way for them to provide information via Canvas Data 2 so analytic teams can analyze student activity according to the student's time zone. I am not asking for solutions for manually mapping UTC to student time zones myself.

James · Answer

@Jo-lCollins

Building off what Robert ( @robotcars ) said, rather than adding the local time to the users table, I would create a separate table -- not one managed by Canvas Data 2-- that contains the conversion. Then you can join the data with that table to get the zone information that you need.

I would further discourage using the offset as they change based on whether it's daylight saving time or not. So the user isn't really setting their time zone to use Eastern Time (US & Canada) (-06:00/-04:00), that's what is being displayed to the user to help them understand what the time zone is. Different programs have different ways of representing the time zones.

In Moment JS, which Canvas uses for web-based time management, it's called "US/Eastern". In Ruby, which is the language the Canvas server uses, "Eastern Time (US & Canada)". The IANA calls it America/New_York (although there are other cities listed). Some call it EST or EDT, but that's confusing because it depends on whether daylight saving time is in effect, so it's a bad way to store it.

PostgreSQL, the database that Canvas runs is what Redshift is based on, so the time zone support should be similar. You can issue pg_timezone_names(); to get a list of supported names. I don't have PostgreSQL installed to test it, but it looks like US/Eastern, America/New_York, and EST/EDT are supported. I would avoid the EST/EDT approach.

Because there are so many ways that it could be stored, you're likely going to need to map it yourself. There is a mapping from Ruby on Rails to the IANA values provided by Ruby on Rails. That converts the "Eastern Time (US & Canada) that is contained in Canvas Data 2 with "America/New_York" that should work with PostgreSQL / Redshift. (I say should because I don't use PostgreSQL or Redshift).