Canvas Data 2 - Weblogs

Question

Hello,

Is there a DAP command or python code I can use to extract CD2 weblogs data for 1 day instead of 30 days? I need to create a job which extracts data on daily basis so that I can build history of the weblogs data over time. I tried using the "incremental query since last seen" for the weblogs to extract data for one day but it looks like the "incremental query since last seen " is only working for the core data tables,  It returns zero records when I use it for the weblogs data.

stimme · Accepted Answer

@BalwinderSingh A DAP command to run an incremental query and return about 1 day's worth of web_logs records could be the following, if you wanted to store compressed CSVs:

dap incremental --table web_logs --format csv --namespace canvas_logs --since 2023-11-21T05:00:00Z

For me this command returned a plausible set of records. I also got a new until date to use as the since date tomorrow. There's a section of the documentation about chaining incremental queries that might be helpful.