Doc/DEPRECATION Copy Section

Canvas API Deprecation

API Method Deprecation Copy Chapter

Deprecate a method with a replacement Copy Section

@deprecated_method NOTICE YYYY-MM-DD EFFECTIVE YYYY-MM-DD

  A description of the deprecated method and why we're deprecating it.

  Use {api:FooController#bar_action Foo#bar_action} instead.

def foo_action
end
def bar_action
end

Deprecate a method without a replacement Copy Section

# @deprecated_method NOTICE YYYY-MM-DD EFFECTIVE YYYY-MM-DD
  #   A description of the deprecated method and why we're deprecating it.
  def foo_action
  end

API Model Deprecation Copy Chapter

Deprecate an entire API model Copy Section

@model Foo

    {

      "id": "Foo",

      "description": "A description.",

      "deprecated": true,

      "deprecation_notice": "YYYY-MM-DD",

      "deprecation_effective": "YYYY-MM-DD",

      "deprecation_description": "A description of the deprecation.",

      "properties": {

        "bar": {

          "description": "A property.",

          "example": "baz",

          "type": "string"

        }

      }

    }

Deprecate an API model property Copy Section

@model Foo

    {

      "id": "Foo",

      "description": "A description.",

      "properties": {

        "bar": {

          "deprecated": true,

          "deprecation_notice": "YYYY-MM-DD",

          "deprecation_effective": "YYYY-MM-DD",

          "deprecation_description": "A description of the deprecation.",

          "description": "A property.",

          "example": "baz",

          "type": "string"

        }

      }

    }

API Argument Deprecation Copy Chapter

Before:

@argument foo [Required, String]

  A description of the argument.

After:

@deprecated_argument foo [Required, String] NOTICE YYYY-MM-DD EFFECTIVE YYYY-MM-DD

  A description of the argument, along with a description of the deprecation.

API Response Field Deprecation Copy Chapter

Before:

@response_field foo

  A description of the response field.

After:

@deprecated_response_field foo NOTICE YYYY-MM-DD EFFECTIVE YYYY-MM-DD

  A description of the response field, along with a description of the

  deprecation.

---

Doc/App And Jobs Gem Groups Copy Section

App and Jobs Gem Groups

Canvas runs two distinct server types from the same codebase: app servers (handling HTTP requests via Apache/Passenger in Production) and jobs servers (processing background jobs via script/delayed_job). The :app_server and :jobs_server Bundler groups (located in Gemfile.d/jobs_server.rb and Gemfile.d/app_server.rb) allow each server type to load only the gems it needs, reducing memory usage.

How It Works Copy Chapter

In config/application.rb, Bundler loads gems using the Bundler.require(*Rails.groups) statement. By default, Rails.groups includes:

- :default, which means all gems in the default group (i.e. when no group is specified) are included
- The environment, e.g. "development" in a dev environment, or "production" in a production environment. This means all gems in the group corresponding with the current environment will be included.

For example, in a development rails console you would see:

Rails.groups

=> [:default, "development"]

In addition, the RAILS_GROUPS environment variable can be set to include additional groups. For example, to include the :jobs_server gem group:

RAILS_GROUPS=jobs_server bundle exec rails console
Rails.groups

=> [:default, "development", "jobs_server"]

This RAILS_GROUPS env var is how we conditionally load these app and jobs groups.

Automatic Group Inference Copy Chapter

Regardless of environment, the correct gem group is automatically inferred based on the server type. In config/boot.rb we set RAILS_GROUPS to "jobs_server" for job servers, and to "app_server" for app servers. We identify jobs servers using the RUNNING_AS_DAEMON env var that is set in script/delayed_job. We identify app servers using the RUNNING_IN_RACK env var that is set in config.ru.

If RAILS_GROUPS is set in the environment before starting, it is used as-is and the automatic inference is skipped. This allows you to override the default behavior when needed.

When to Add a Gem to a Group Copy Chapter

Add to :jobs_server if every call site for a gem is reachable only via a background job mechanism:

- .delay(...).<method>
- handle_asynchronously :<method>
- Delayed::Job.enqueue(...)
- Delayed::Periodic.cron(...)
- A worker class implementing perform

Add to :app_server if every call site is reachable only via a synchronous HTTP request.

Leave in the default group if:

- It is used by both server types
- You are unsure — the default is always safe
- It is Rails infrastructure or a shared utility called from both contexts

When in doubt, trace each call site back to its entry point before deciding. Also note that calls to delay_if_production(...).<method> should be treated as synchronous for this purpose; this code will run on app servers in development environments, and therefore any gems used within these calls need to be available on both app and jobs servers.

Verifying a Change Copy Chapter

You can verify that a gem is loaded in one env, and not loaded in another, by executing a rails runner command with the RAILS_GROUPS env var set. For example, after adding the mimemagic gem to Gemfile.d/jobs_server.rb, you can run:

Prints nil — not loaded on app server

DISABLE_SPRING=1 RAILS_GROUPS=app_server bundle exec rails runner "puts defined?(MimeMagic).inspect"
nil
Prints "constant" — loaded on jobs server

DISABLE_SPRING=1 RAILS_GROUPS=jobs_server bundle exec rails runner "puts defined?(MimeMagic).inspect"
"constant"

Alternatively, you can run canvas locally, and put a puts or debugger statement within app & jobs code and then check for the gem's existence.

---

Doc/Canvas Operations Library Copy Section

CanvasOperations

A library for running common operations in deployed Canvas environments, with consistent logging, metric emission, progress tracking, and error handling, and more.

See lib/canvas_operations

Features Copy Chapter

Usage Copy Chapter

Creating a New Operation Copy Section

Subclass CanvasOperations::BaseOperation and override the execute method:

class EnableCoolFeatures < CanvasOperations::BaseOperation
  # define callbacks for your operation
  before_run :validate_feature_prerequisites
  after_run :notify_stakeholders
  after_failure :notify_engineering_team
  # define settings that can be changed on-the-fly
  setting :feature_list, default: "feature_one"
  setting :stakeholder_notification_channel, default: "#releases"
  # defaults to false. If true, the operation will create a Progress
  # record when run and update that progress when a completion or
  # failure state for the operation is reached.
  self.progress_tracking = true
  def execute
    log_message("Enabling features #{feature_list}!")
  end
  def validate_feature_prerequisites
    raise "Prerequisites not met!" unless ...
  end
  def notify_stakeholders
    SlackClient.post_message(
      channel: stakeholder_notification_channel,
      text: "The following features have been enabled: #{feature_list}"
    )
  end
  def notify_engineering_team
    SlackClient.post_message(
      channel: "#engineering-alerts",
      text: "#{name} failed! Please investigate."
    )
  end
end

Run the operation:

MyOperation.new.run_later

Operations are run in an async job via the inst-jobs gem.

See lib/canvas_operations/base_operation.rb for more details on available methods and features.

See lib/canvas_operations/base_concerns/settings.rb for more details on using operation settings.

See lib/canvas_operations/base_concerns/callbacks.rb for more details on using operation callbacks.

DataFixup Operations Copy Section

As described above, the base operation class can be subclassed for specific use cases. The DataFixup operation class serves as an example of this pattern, but is also a useful tool in its own right.

The DataFixup operation provides a standard framework for efficiently performing data fixups that require processing large numbers of records, either individually or in batches.

To use this operation, create a subclass of CanvasOperations::DataFixup:

module DataFixup
  module InstructureIdentity
    class UnsetAuthlogicAttributesOnInstPseudonyms < CanvasOperations::DataFixup
      # Optionally override setting defaults (more details below)
      setting :range_batch_size, default: 10_000, type_cast: :to_i
      # Should records be yielded one at a time, or in batches? (more details below)
      self.mode = :batch
      # If set to true, the return value of process_record or process_batch will be
      # recorded in an auditable Attachment associated with the operation's context.
      self.record_changes = true
      # Define the scope of records to process (more details below)
      scope do
        Pseudonym.instructure_identity.where(
          "ABS(EXTRACT(EPOCH FROM (pseudonyms.last_request_at - pseudonyms.created_at))) <= 1"
        ).where(
          login_count: 1
        ).where.not(
          last_request_at: nil
        )
      end
      # Define how to process a batch of records (more details below)
      def process_batch(pseudonym_batch)
        pseudonym_batch.update_all(last_request_at: nil, current_login_at: nil, current_login_ip: nil)
      end
    end
  end
end

and then instantiate and call run_later on your fixup from a migration:

db/migrate/20250820214915_unset_authlogic_attributes_on_inst_pseudonyms.rb

class UnsetAuthlogicAttributesOnInstPseudonyms < ActiveRecord::Migration[7.2]
  tag :postdeploy
  def up
    DataFixup::InstructureIdentity::UnsetAuthlogicAttributesOnInstPseudonyms.new.run_later
  end
end

#### DataFixup Properties

| Property | Description |
|-------------------------|-------------|
| mode | Controls how records are yielded to your processing logic. <br> - :individual_record: Each record is yielded one at a time to the process_record method.<br> - :batch: Records are yielded in batches to the process_batch method. <br> No matter which you choose, records are loaded efficiently in batches. |
| record_changes | Whether to record changes made by the datafixup in Attachment logs associated with the context. Returns from process_record or process_batch are written to chunked text files and uploaded as Attachments. Defaults to false and is always disabled in test environments. |
| scope | The ActiveRecord scope that defines the set of records to be processed by the fixup. |
| process_record(record) | (For :individual_record mode) Define this method to specify how to process each individual record. |
| process_batch(records) | (For :batch mode) Define this method to specify how to process a batch of records. |
| run_on_default_shard | If true, the data fixup will run on the default shard. If false, the default shard is skipped. Defaults to true. |

#### DataFixup Settings

| Setting | Description |
|---------------------------|-------------|
| range_batch_size | How many IDs per chunk find_ids_in_ranges should yield. Larger numbers result in your scope query being run less often, but over a larger set of rows. |
| job_scheduled_sleep_time| How long, in seconds, to sleep between scheduling an async batch of work. Increasing this value can help if the jobs cluster primary is getting hit too hard. |
| processing_sleep_time | How long, in seconds, to sleep between processing batches or individual records. Increasing this value can help if the cluster's primary node is getting hit too hard. |

These settings can be changed on-the-fly; just be sure to send SIGHUP to job hosts to ensure configuration is reloaded.

A data fixup operation and associated files can be generated with rails g data_fixup <OperationName>.

See lib/canvas_operations/base_concerns/settings.rb for more details on using operation settings.

#### DataFixup Additional Properties
batch_strategy

Defaults to :pluck_ids, which performs a pluck on the scope (after range filtering), and loads records into memory in batches based on those IDs.

In some cases, it may be appropriate to change this strategy. To do so, set the batch_strategy class instance variable:

class UnsetAuthlogicAttributesOnInstPseudonyms < CanvasOperations::DataFixup
...
  self.batch_strategy = :id
...
end

See /usr/src/app/config/initializers/active_record.rb for additional details on available batch strategies.

RootAccountOperation Copy Section

The RootAccountOperation class is designed for operations that need to run per root account. This is essential when you need to execute the same operation across multiple accounts with proper isolation and context.

Key features:
- Automatic shard binding based on the root account's shard
- PluginSetting context wrapping via PluginSetting.with_account (when available)
- Unique singleton job keys per root account to allow concurrent execution across different accounts
- Progress tracking scoped to each root account

To use this operation, create a subclass of CanvasOperations::RootAccountOperation:

class NotifyAccountAdmins < CanvasOperations::RootAccountOperation
  # define callbacks for your operation
  before_run :validate_feature_prerequisites
  after_run :notify_stakeholders
  after_failure :notify_engineering_team
  def execute
    admin_count = 0
    root_account.account_users.active.each do |account_user|
      send_notification(account_user.user)
      admin_count += 1
    end
    results[:admin_count] = admin_count
    results[:root_account_id] = root_account.global_id
    log_message("Notified #{admin_count} admins for account #{root_account.global_id}")
  end
  def validate_feature_prerequisites
    raise "Prerequisites not met!" unless ...
  end
  def notify_stakeholders
    SlackClient.post_message(
      channel: stakeholder_notification_channel,
      text: "The following features have been enabled: #{feature_list}"
    )
  end
  def notify_engineering_team
    SlackClient.post_message(
      channel: "#engineering-alerts",
      text: "#{name} failed! Please investigate."
    )
  end
  private
  def send_notification(user)
    # Send notification logic here
  end
end

Run the operation for a specific account:

NotifyAccountAdmins.new(
  root_account: Account.find(123),
).run_later

Run the operation for all active accounts:

Account.root_accounts.active.find_each do |account|
  NotifyAccountAdmins.new(
    root_account: account,
  ).run_later
end

Important characteristics:

- Each account gets its own delayed job with a unique singleton key: operations/{operation_name}/shards/{shard_id}/accounts/{account_global_id}
- Operations for different accounts can run concurrently
- Operations for the same account are deduplicated (only one pending/running job per account)
- The delayed job's shard and account attributes are automatically set correctly
- Progress records are associated with the root account (not the cluster primary)

See lib/canvas_operations/root_account_operation.rb for implementation details.

---

Doc/Copyright Copy Section

A new file should have the following at the top:

/*
 * Copyright (C) 2025 - present Instructure, Inc.
 *
 * This file is part of Canvas.
 *
 * Canvas is free software: you can redistribute it and/or modify it under
 * the terms of the GNU Affero General Public License as published by the Free
 * Software Foundation, version 3 of the License.
 *
 * Canvas is distributed in the hope that it will be useful, but WITHOUT ANY
 * WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR
 * A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
 * details.
 *
 * You should have received a copy of the GNU Affero General Public License along
 * with this program. If not, see <http://www.gnu.org/licenses/>.
 */

---

Doc/Detect N Plus One Queries Copy Section

Detect N+1 Queries

Canvas uses the prosopite gem to detect N+1 query problems and prints information about them to log/development.log. It also prints this information to its own dedicated log file, log/prosopite.log when in development. Here's an example report:

N+1 queries detected:
  SELECT "context_external_tools".* FROM "public"."context_external_tools" WHERE "context_external_tools"."id" = 1 LIMIT 1
  SELECT "context_external_tools".* FROM "public"."context_external_tools" WHERE "context_external_tools"."id" = 1 LIMIT 1
  SELECT "context_external_tools".* FROM "public"."context_external_tools" WHERE "context_external_tools"."id" = 1 LIMIT 1
  SELECT "context_external_tools".* FROM "public"."context_external_tools" WHERE "context_external_tools"."id" = 1 LIMIT 1
Call stack:
  config/initializers/postgresql_adapter.rb:315:in exec_query'
  app/models/content_tag.rb:283:in content'
  app/models/assignment.rb:3504:in quiz_lti?'
  app/models/assignment.rb:394:in can_duplicate?'
  lib/api/v1/assignment.rb:193:in assignment_json'
  lib/api/v1/assignment_group.rb:82:in block in assignment_group_json'
  lib/api/v1/assignment_group.rb:75:in map'
  lib/api/v1/assignment_group.rb:75:in assignment_group_json'
  ...<more stack trace>

Prosopite.scan Copy Chapter

You can pass a block to Prosopite.scan to have it check for N+1 queries:

Prosopite.scan do
  Course.where(id: 1..5).each { |course| course.assignments.first }
end
N+1 queries detected:
  SELECT "assignments".* FROM "public"."assignments" WHERE "assignments"."context_id" = 4 AND "assignments"."context_type" = 'Course' ORDER BY assignments.created_at LIMIT 1
  SELECT "assignments".* FROM "public"."assignments" WHERE "assignments"."context_id" = 1 AND "assignments"."context_type" = 'Course' ORDER BY assignments.created_at LIMIT 1
  SELECT "assignments".* FROM "public"."assignments" WHERE "assignments"."context_id" = 2 AND "assignments"."context_type" = 'Course' ORDER BY assignments.created_at LIMIT 1
  SELECT "assignments".* FROM "public"."assignments" WHERE "assignments"."context_id" = 3 AND "assignments"."context_type" = 'Course' ORDER BY assignments.created_at LIMIT 1
Call stack:
  config/initializers/postgresql_adapter.rb:315:in exec_query'
  (irb):5:in block (2 levels) in <main>'
  (irb):5:in block in <main>'
  (irb):4:in <main>'

If you don't want to pass a block, you can use Prosopite.scan along with Prosopite.finish:

Prosopite.scan
Course.where(id: 1..5).each { |course| course.assignments.first }
foo = "bar"
Prosopite.finish
N+1 queries detected:
  SELECT "assignments".* FROM "public"."assignments" WHERE "assignments"."context_id" = 4 AND "assignments"."context_type" = 'Course' ORDER BY assignments.created_at LIMIT 1
  SELECT "assignments".* FROM "public"."assignments" WHERE "assignments"."context_id" = 1 AND "assignments"."context_type" = 'Course' ORDER BY assignments.created_at LIMIT 1
  SELECT "assignments".* FROM "public"."assignments" WHERE "assignments"."context_id" = 2 AND "assignments"."context_type" = 'Course' ORDER BY assignments.created_at LIMIT 1
  SELECT "assignments".* FROM "public"."assignments" WHERE "assignments"."context_id" = 3 AND "assignments"."context_type" = 'Course' ORDER BY assignments.created_at LIMIT 1
Call stack:
  config/initializers/postgresql_adapter.rb:315:in exec_query'
  (irb):5:in block in <main>'
  (irb):5:in <main>'

Enabling Detection Copy Chapter

In Development & Test Copy Section

Automatic N+1 detection for requests is on by default (development and test env only). All controller actions are wrapped in a Prosopite.scan while in development or test.

To disable N+1 detection, set the DISABLE_N_PLUS_ONE_DETECTION environment variable to 'true'.

You can manually invoke Prosopite.scan in any environment.

In Production Copy Section

You can enable N+1 detection on a per-request basis by passing a
n_plus_one_detection=true parameter to the request. A report will be saved to
your users files (/files). The format of the filename is:
n_plus_one_detection-<controller>#<action>-<iso8601 timestamp>, but you can
also specify a custom name by passing n_plus_one_name=<name> which will make
the filename n_plus_one_detection-<name>-<iso8601 timestamp>.

Note that you must be logged in as a Site Admin user to enable this
functionality. This is to prevent abuse of the feature by regular users. However, a Site Admin user can masquerade as any non-site-admin user and generate an N+1 report (the resulting report will be in the Site Admin user's Canvas Files).

Limitations Copy Section

If you try to ask for both a flamegraph and an N+1 report, the N+1 report will not be generated. Flamegraphs take precedence over N+1 reports. This is to prevent the N+1 report from obfuscating the flamegraph data. Additionally, if the flamegraph grows too large, a stack overflow error may occur, which defeats the purpose of the reports.

---

Doc/Flamegraphs Copy Section

Flamegraphs

TL;DR Copy Chapter

Add the flamegraph=true&flamename=my-custom-name query params to any request, and view/download flamegraphs in your Canvas Files.

Why Flamegraphs? Copy Chapter

Flamegraphs can help to identify performance bottlenecks in our Canvas Ruby code (it can help identify problems rooted in inefficient database queries AND problems rooted in too-much-processing-time-spent-in-ruby).

If you're not familiar with flamegraphs and don't know how to analyze them, I encourage you to take an hour or two researching the topic; it's not difficult to analyze a flamegraph file, and it can be immensely helpful in tracking down performance issues!

Generating an HTML Flamegraph File Copy Chapter

Users with Site Admin access can have an interactive HTML flamegraph generated for them, for any request, and have it delivered to their Canvas Files. Site Admin users can also masquerade as any non-site-admin user and generate a flamegraph report (the resulting report will be in the Site Admin user's Canvas Files).

In order to generate a flamegraph for a given request, simply add the

flamegraph=true

query parameter to the request. That's it! It works for both HTML and JSON requests.

Viewing the Generated HTML Flamegraph File Copy Chapter

Next, go to your Canvas files (/files) and notice there's a flamegraphs folder with your newly-generated flamegraph file is in there!

The format of the filename is: flamegraph-<controller>#<action>-<iso8601 timestamp>, but you can also pass the

flamename=my-kewl-stuff

query parameter, which will make the filename flamegraph-my-kewl-stuff-<controller>#<action>-<iso8601 timestamp>.

You can preview the file within Canvas and interact with it (try clicking things and using the search bar, it's interactive!). You can also download the HTML file and view it locally in your browser.

More Info Copy Chapter

Flamegraphs are generated by the awesome ruby stackprof library. See app/services/flamegraphs/flamegraph_service.rb for implementation details.

See git show b860ae73 for the commit that added this functionality.

If you have any questions, feel free to reach out to @solson in Slack.

---

Doc/I18n Copy Section

Internationalization (i18n) in Canvas LMS

Canvas LMS uses a internationalization (i18n) system that handles translations for both backend (Ruby) and frontend (JavaScript) code.

Overview Copy Chapter

The i18n system in Canvas LMS is built on:
- Ruby i18nliner for backend translations
- @instructure/i18nliner-canvas for frontend translations
- format-message for specific packages like canvas-rce

The system supports translations in:
- Ruby files
- JavaScript files
- TypeScript files
- Handlebars templates

Translation Files Copy Chapter

Main translation files are located in several places:

Source and Generated Files Copy Section

1. @instructure/translations Package
- Central package containing translations for all Canvas packages
- Located in packages/translations/
- Contains language files for each supported locale
- Packages can access their translations through this package

2. Individual Package Setup
Example using canvas-media:


// scripts/installTranslations.js
   const {getTranslationList, readTranslationFile} = require('@instructure/translations')

3. Automated Updates
- Packages use scripts like commitTranslations.sh to:
- Create a new git branch
- Commit translation updates
- Push changes to Gerrit
- Notify teams via Slack when updates are needed

Using Translations Copy Chapter

In React Components Copy Section

import {useScope as createI18nScope} from '@canvas/i18n'
const I18n = createI18nScope('ComponentName')
// Usage
I18n.t('string to translate')

In Ruby Code Copy Section

t('string to translate')

Special Cases Copy Chapter

Canvas Rich Content Editor (RCE) Copy Section

The RCE package (packages/canvas-rce) uses format-message instead of i18nliner for translations:

import formatMessage from 'format-message'
formatMessage('string to translate')

Development Workflow Copy Chapter

1. Extract Translations Copy Section

To extract translatable strings from the codebase:

bundle exec rake canvas:compile_assets i18n:extract

This command:
- Scans Ruby, JavaScript, and TypeScript files for translatable strings
- Generates/updates translation files in config/locales/generated/

2. Available Tasks Copy Section

- i18n:check: Validates translation calls in Ruby and JS code
- i18n:extract: Extracts strings from source code into YAML
- i18n:generate: Creates runtime translation files
- i18n:export: Prepares files for translators
- i18n:import: Imports new translations
- i18n:generate_js: Generates JavaScript translation files
- i18n:generate_lolz: Generates LOLZ pseudo-translations (useful for testing)
- i18n:lock: Locks specific translation keys

3. LOLZ Translations Copy Section

Canvas includes a special "LOLZ" locale for testing and development:
- Generated using rake i18n:generate_lolz
- Creates pseudo-translations
- Stored in config/locales/lolz.yml

Translation Management Copy Chapter

Available Locales Copy Section

Locales are configured in config/locales/locales.yml:
- Defines which languages are available
- Specifies if a locale is crowdsourced

Best Practices Copy Chapter

1. Use Scopes
- Always use translation scopes to organize strings by component/feature
- Choose meaningful scope names that reflect the component's purpose

2. Avoid String Concatenation
- Use interpolation instead of concatenating strings
- Example: I18n.t('Welcome %{name}', { name: userName })

3. Keep Translations Updated
- Run i18n:extract when adding new strings
- Verify translations with i18n:check

Testing Copy Chapter

When writing tests that involve translations:

1. Don't Mock Unless Necessary
- Avoid jest.mock('@canvas/i18n')
- Use real translation calls when possible

2. Test User Experience
- Focus on testing how translated content appears to users
- Verify that translated strings are properly displayed
- Consider using LOLZ translations for visual testing

Common Issues and Solutions Copy Chapter

1. Missing Translations
- Run i18n:extract to ensure all strings are captured
- Check that the scope is correctly defined

2. Runtime Errors
- Verify that translation files are properly generated
- Check for missing interpolation variables

Future Considerations Copy Chapter

Moving to i18next Copy Section

For new standalone packages (similar to Canvas RCE or Canvas Meteor), consider using i18next instead of format-message.

Additional Resources Copy Chapter

- i18nliner Documentation
- format-message Documentation
- i18next Documentation
- Canvas LMS translation tasks: gems/i18n_tasks/lib/tasks/i18n.rake

---

Doc/Live Events Copy Section

Live Events

Canvas includes the ability to push a subset of real-time events to a
Kinesis stream, which can then be consumed for various analytics
purposes. This is not a full-fidelity feed of all changes to the
database, but a targeted set of interesting actions such as
grade_change, login, etc.

Development and Testing Copy Chapter

There are two components to local development:
- the kinesis stream (which can hook into the live-events-publish lambda)
- the subscription service and its UI (live-events-subscriptions, live-events-lti)

Kinesis Stream Copy Section

If using the docker-compose dev setup, there is a "fake kinesis" available in
docker-compose/kinesis.override.yml available for use. To start this kinesis
container run docker compose up -d kinesis. Once it's up, make sure you have
the aws cli installed, and run the following command to create a stream (with
canvas running). Keep in mind that we are running this locally so actual AWS
credentials are not needed, run the following command as you see it here:

AWS_ACCESS_KEY_ID=key AWS_SECRET_ACCESS_KEY=secret aws --endpoint-url http://kinesis.docker/ kinesis create-stream --stream-name=live-events --shard-count=1 --region=us-east-1

Once the stream is created, configure your Canvas to use it in your
config/dynamic_settings.yml. This file is a local shim for Consul. If you have
copied the example file at config/dynamic_settings.yml.example recently, you
should already see a live_events block and it should already be configured properly.
If you don't see a live_events block, check the example file or copy this block:

live_events.yml: |-
        aws_endpoint: http://kinesis:4567
        kinesis_stream_name: live-events
        aws_access_key_id: key
        aws_secret_access_key_dec: secret

Depending on your docker networking setup, you may need to substitute either
http://kinesis:4567, http://kinesis.docker, or http://kinesis.canvaslms.docker
for the aws_endpoint (the first two should be equivalent).

Restart Canvas, and events should start flowing to your kinesis stream.
You can view the stream with the tail_kinesis tool:

docker compose run --rm web script/tail_kinesis http://kinesis:4567 live-events

#### Stubbing Kinesis

Instead of viewing events in the kinesis stream, you can add the stub_kinesis
attribute to the dynamic_settings live_events block that you configured above,
with a value of true. This will print live events to stdout instead of sending
them to a kinesis stream.

An easy way of accessing stdout when using dockerized Canvas is this:

docker compose logs -f --tail=100 <jobs|web> # whichever container you need

#### Connecting to local Publisher Lambda

The live-events-publish repo should be checked out and running locally.
This contains the publisher lambda, and other infrastructure including a local
kinesis stream. Note the url of that kinesis stream, which may look like
http://kinesis.live-events-publish.docker:4567.

There should already be a stream created in that container, with the name
found in docker-compose.yml, in the KINESIS_LOCAL_STREAM_NAME environment
variable. If that stream doesn't exist, create it with this aws command:

AWS_ACCESS_KEY_ID=ACCESS_KEY AWS_SECRET_ACCESS_KEY=SECRET_KEY aws --endpoint-url http://kinesis.live-events-publish.docker/ kinesis create-stream --stream-name=live-events-local-test-stream --shard-count=1 --region=us-east-1

Once the stream is created, configure your Canvas to use it in your config/dynamic_settings.yml.
This file is a local shim for Consul. If you have copied the example file at
config/dynamic_settings.yml.example recently, you should already see a live_events block.
Note that these settings differ from the example block above. If you don't see a live_events
block, check the example file or copy this block:

live_events.yml: |-
        aws_endpoint: http://kinesis.live-events-publish.docker
        kinesis_stream_name: live-events-local-test-stream
        aws_access_key_id: ACCESS_KEY
        aws_secret_access_key_dec: SECRET_KEY

Restart Canvas, and events should start flowing to the kinesis stream, and to
the publisher lambda itself. You can view the stream and publisher lambda
activity by looking at the output of docker compose up in the live-events-publish
repo.

Subscription Management Copy Section

#### Connecting to local Subscription Service

The live-events-subscriptions repo should be checked out and running locally.
This contains the subscriptions for live events, which the publisher uses when
propagating events.

To connect Canvas with the subscription service, open config/dynamic_settings.yml
and make sure that the live-events-subscription-service prefix contains the
proper app-host value, which should be the url where your local subscription
service is running. Instructions for connecting on the subscription service side
are found in the live-events-subscriptions repo, in README.md.

#### Connecting to local Live Events LTI Tool

The live-events-lti repo should also be checked out and running locally. This
is an LTI tool which provides a UI for managing the subscriptions contained in
the subscription service. Instructions for configuring this LTI tool are
contained in the live-events-lti repo, in README.md.

Canvas LMS Live Events Consumers Copy Chapter

Canvas LMS emits live events to multiple subscribers including Quiz LTI and
Gauge, among others.

---

Doc/Profiling Ruby Copy Section

Profiling Ruby

If you've got ruby code you're concerned is behaving badly (especially slowly),
there are some tools installed that can help you figure out why.

Stackprof Copy Chapter

stackprof is in the test bundle, and is a
nice choice because it's comparatively lightweight. It uses sampling of the stack
at various intervals rather than complete instrumentation, so it can be statistically noisy,
but should still accumulate into a reasonable sense of where your time is going.

Apply it to a chunk of code like so:

StackProf.run(mode: :wall, out: 'tmp/stackprof-canvas-test.dump', interval: 1000) do
  #...the code you want to profile, often the body of a test
end

This will sample every 1000 microseconds (1 millisecond), and write the
output to the specificed "out" file. See the stackprof docs linked above for more
details on configuration options

Once you've produced a dumpfile, you can produce a report on the results
by using the stackprof command directly:

bundle exec stackprof tmp/stackprof-canvas-test.dump --limit 20

That should give you a report like:

/ Detailed source-code truncated for AI context efficiency. /

Which can show you which stack frames are frequently at the top of the stack.

---

Doc/Testing Javascript Copy Section

Testing JavaScript

The process of testing JavaScript sometimes confuses people. This document's goal
is to alleviate that confusion and establish how to run JavaScript tests.

Jest Copy Chapter

Whenever possible, which for now means when you are testing something that only imports
stuff that does not use AMD imports (eg, it only requires
stuff from app/jsx or node_modules), you should write your js tests for
Jest going forward. It is faster and the testing experience will be better.

Jest is a Node-based runner. This means that the tests always run in a Node environment and not in a real browser. This lets us enable fast iteration speed and prevent flakiness.

While Jest provides browser globals such as window thanks to jsdom, they are only approximations of the real browser behavior. Jest is intended to be used for unit tests of your logic and your components rather than the DOM quirks. Use a QUnit Karma test or Selenium for browser end-to-end tests if you need them.

Filename Conventions Copy Section

Put the test files (or __tests__ folders) next to the code they are testing so that relative imports appear shorter. For example, if App.test.js and App.js are in the same folder, the test just needs to import App from './App' instead of a long relative path. Colocation also helps find tests more quickly in larger projects.

Writing Jest Tests Copy Section

To create tests, add it() (or test()) blocks with the name of the test and its code. You may optionally wrap them in describe() blocks for logical grouping but this is neither required nor recommended.

Jest provides a built-in expect() global function for making assertions. A basic test could look like this:

import sum from './sum'
it('sums numbers', () => {
  expect(sum(1, 2)).toEqual(3)
  expect(sum(2, 2)).toEqual(4)
})

All expect() matchers supported by Jest are extensively documented here.<br>
You can also use jest.fn() and expect(fn).toBeCalled() to create "spies" or mock functions and in jest tests you should probably use that instead of sinon mocks/spies/stubs like we use in our QUnit tests.

NEVER use getByRole queries in tests. Use of byRole queries (getByRole, findByRole, queryByRole, getAllByRole, findAllByRole, queryAllByRole) is discouraged due to performance issues. Please consider using a different query such as byText or byLabelText instead.

\*NOTE: You cannot run jest if there is anything with AMD, CoffeeScript, or some of the Webpack aliases (which lead to AMD or CoffeeScript).

Testing Components Copy Section

There is a broad spectrum of component testing techniques. They range from a “smoke test” using a jest snapshot, to shallow rendering and testing some of the output, to full rendering and testing component lifecycle and state changes.

An example of a simple smoke test for your components:

import React from 'react'
import ReactDOM from 'react-dom'
import SomeComponent from './SomeComponent'
it('renders without crashing', () => {
  const div = document.createElement('div')
  ReactDOM.render(<App />, div)
})

This test mounts a component and makes sure that it didn’t throw during rendering. Tests like this provide a lot value with very little effort so they are great as a starting point.

When you encounter bugs caused by changing components, you will gain a deeper insight into which parts of them are worth testing in your application. This might be a good time to introduce more specific tests asserting specific expected output or behavior.

All Jest matchers are extensively documented here.

Focusing and Excluding Tests Copy Section

You can replace it() with xit() to temporarily exclude a test from being executed.<br>
Similarly, fit() lets you focus on a specific test without running any other tests.

Running Tests Copy Section

To run all tests:

yarn test:jest

To run a subset of files or directories:

yarn test:jest path/to/components/__tests__/spec.js path/to/other_component/ ...

To rerun tests on a file change and/or debug remotely:

yarn test:jest:debug path/to/components/__tests__/spec.js

Coverage Reporting Copy Section

Jest has an integrated coverage reporter that works well with ES6 and requires no configuration.<br>
Run npm test -- --coverage (note extra -- in the middle) to include a coverage report like this:

Note that tests run much slower with coverage so it is recommended to run it separately from your normal workflow.

Snapshot Testing Copy Section

Snapshot testing is a feature of Jest that automatically generates text snapshots of
your components and saves them on the disk so if the UI output changes, you get
notified without manually writing any assertions on the component output.
Read more about snapshot testing.

Running QUnit Karma Tests Copy Chapter

A lot of the older stuff is still QUnit. For more info on running those older tests, see the "Running js tests with webpack" section of working_with_webpack.md.

Tl;dr: run a single test in watch mode like:

yarn jspec-watch spec/coffeescripts/util/deparamSpec.js

Running Tests in Docker Copy Chapter

See the "Running javascript tests" section of developing_with_docker.md.

Javascript Test Coverage Copy Chapter

You can generate code coverage locally by having webpack
set up( touch config/WEBPACK) then running COVERAGE=1 yarn test.
You should then have a folder in your root directory called coverage-js
in which contains an index.html which if you open it will show you
the test coverage for all javascript (js, jsx)

---

Doc/Testing With Selenium Copy Section

Testing with Selenium

You may run the Selenium tests either natively or in docker.

Running Selenium Tests Natively (Mac) Copy Chapter

We're making a few assumptions here:
- you're using an Apple computer
- you've already installed Homebrew
- you've already installed Postgres (postgresapp.com is an excellent option),
and Postgres is running on your computer
- you've already installed Node.js

0. Install yarn if you haven't already:

brew install yarn

If you find you need an older version of yarn, follow these instructions to
install the older version and switch to it:

https://stackoverflow.com/a/52525732/3038677

1. Follow the instructions in script/prepare/README.md to setup the prepare
script.

You'll use the prepare script later to automate installing and updating Canvas
on your computer.

Note: some features of prepare only work if you have access to Instructure's
Gerrit host. See the README for details.

2. Install a web browser driver on your computer for the browser you wish to run
the tests in. Homebrew is the easiest way to go:

brew install chromedriver --cask # necessary for running tests in Chrome
brew install geckodriver # necessary for running tests in Firefox

Now let's get Canvas ready to run the tests.

3. Copy the Selenium and database configuration files:

cp config/selenium.yml.example config/selenium.yml
cp config/database.yml.example config/database.yml

4. Use prepare to install Canvas plugins and dependencies, create databases,
run database migrations, etc:

prepare

You might encounter problems with some Ruby dependencies. The "Dependency
Installation" section
in the public Canvas LMS Github wiki has some useful tips.

4.a. Optional. Run delayed jobs in the foreground (not all Selenium tests need
this but some do):

script/delayed_job run

or run it in the background:

script/delayed_job run &

5. Run the Selenium tests:

bundle exec rspec spec/selenium

or run a specific Selenium test:

bundle exec rspec spec/selenium/accounts_spec.rb:36

Running Tests against Headless Chrome Copy Section

Selenium tests can be run against headless Chrome by changing a few properties in
config/selenium.yml. Specifically, you'll need to set headless to true and
window_size to something that makes sense, like so:

headless: true
  window_size: "1237,974"

This can be useful when you don't need to see what your test is doing, since it
can run in the background without stealing focus or interrupting other work. It's
especially useful when running specs many times to check for flakiness.

Running Selenium Tests in Docker Copy Chapter

See the Selenium section
of the doc/docker/developing_with_docker.md instructions.

Selenium Testing Best Practices Copy Chapter

Using Helper Methods for Waiting Copy Section

For clarity and reliability, prefer using built-in helper methods instead of relying on numerous expect statements to handle asynchronous operations. Canvas provides many useful helper methods in spec/selenium/test_setup/common_helper_methods/ including:

- custom_wait_methods.rb - Contains wait_for, wait_for_new_page_load, and other waiting utilities
- custom_page_loaders.rb - Page loading and navigation helpers
- custom_selenium_actions.rb - Common Selenium actions
- custom_validators.rb - Validation helpers for Selenium tests

Using these helpers makes tests more readable and reliable than chains of expect statements.

Selenium-Specific Guidelines Copy Section

- Use expect syntax: Use the expect(value).to matcher syntax consistently
- Organize with contexts: Use context blocks to describe different test scenarios
- Use let for test data: Use let to define reusable test data and objects

---

Doc/Trace Database Queries Copy Section

Trace Database Queries

Canvas includes the "active_record_query_trace" gem when running in development or test mode. This gem
prints a stack trace of every ActiveRecord query the Rails application makes.

The query trace logging is off by default, but can be enabled and configured by setting
environment variables. Note that traces cannot be enabled in production mode.

Environment variable overview Copy Chapter

| ENV VAR | Description |
|---------| ----------- |
| AR_QUERY_TRACE | traces are enabled if this variable is set. Traces disabled otherwise. |
| AR_QUERY_TRACE_TYPE | Controls what kind of queries print traces. Valid values are "all", "write", and "read". |
| AR_QUERY_TRACE_LINES | Controls how many lines of the trace are printed. Defaults to 10. |
| AR_QUERY_TRACE_LEVEL | Filters the backtrace to "app", "rails", or "full" |

---

Doc/Using Guard Rail In Development Copy Section

Using GuardRail in Development

GuardRail allows activating different database configurations for specific blocks of code. This is frequently done to offload read queries to a replica (secondary) database in Canvas.

When offloading read queries to a replica, it's important to have the ability to test the change locally and verify no writes are occurring by accident.

This guide shows how to configure a read-only user to allow testing these kind of changes during development.

For more information on GuardRail see The Canvas Manual.

1. Open config/database.yml ')" title="Copy chapter prompt for LLMs"> Copy Chapter

2. Add the following to the common YML section: YML section:')" title="Copy chapter prompt for LLMs"> Copy Chapter

secondary:
    replica: true
    username: canvas_read_only

This should result in a common section that looks something like this:


common: &common
  adapter: postgresql
  host: <%= ENV.fetch('CANVAS_DATABASE_HOST', 'postgres') %>
  ...
  secondary:
    replica: true
    username: canvas_read_only

3. Create a new user and grant read-only access to databases: Copy Chapter

docker compose run --rm web psql -h postgres -U postgres -c "CREATE USER canvas_read_only WITH PASSWORD 'sekret'"

When prompted for a password, use the Canvas default postgres password (sekret at the time of writing),

Next, grant the user read-only privileges to all tables in each database.

For each database (development, test, etc.) run the following, substituting the correct name for <database name>`:

docker compose run --rm web psql -h postgres -U postgres -d <database name> -c 'GRANT SELECT ON ALL TABLES IN SCHEMA public TO canvas_read_only'

4. That's it! Copy Chapter

=> GuardRail.activate(:secondary) { DeveloperKey.create! }

This should result in the following error:

ActiveRecord::StatementInvalid (PG::InsufficientPrivilege: ERROR:  permission denied for table developer_keys)

Activating the primary DB configuration, however, should allow inserting the new row:

=> GuardRail.activate(:primary) { DeveloperKey.create! }
...
SQL  (1.3ms)  COMMIT  [development:1 primary]

---