### Doc/DEPRECATION

# Canvas API Deprecation
In the examples below, the deprecation dates should follow these rules:
  * The `NOTICE` date should be the date that the deprecation warning
    will first be visible in production.
  * To determine the `EFFECTIVE` date, add 90 days to the `NOTICE`
    date. If that day is a production release date, use that date. If that
    date is _not_ a production release date, use the next production release
    date after that date.
  * Both dates should be formatted as YYYY-MM-DD.

## API Method Deprecation
To deprecate an API method, use the `@deprecated_method` tag. You must provide
a `NOTICE` date and an `EFFECTIVE` date for the deprecation, along with a
description for the deprecation.

### Deprecate a method with a replacement
```ruby
# @deprecated_method NOTICE YYYY-MM-DD EFFECTIVE YYYY-MM-DD
#   A description of the deprecated method and why we're deprecating it.
#   Use {api:FooController#bar_action Foo#bar_action} instead.
def foo_action
end

def bar_action
end
```

### Deprecate a method without a replacement
```ruby
  # @deprecated_method NOTICE YYYY-MM-DD EFFECTIVE YYYY-MM-DD
  #   A description of the deprecated method and why we're deprecating it.
  def foo_action
  end

```

## API Model Deprecation
To deprecate an API model, add the `deprecated`, `deprecation_notice`,
`deprecation_effective`, and `deprecation_description` keys. These keys can be
applied at the base model level to deprecate the entire model, or they can be
applied at the property level to individually deprecate model properties.

### Deprecate an entire API model
```ruby
# @model Foo
#     {
#       "id": "Foo",
#       "description": "A description.",
#       "deprecated": true,
#       "deprecation_notice": "YYYY-MM-DD",
#       "deprecation_effective": "YYYY-MM-DD",
#       "deprecation_description": "A description of the deprecation.",
#       "properties": {
#         "bar": {
#           "description": "A property.",
#           "example": "baz",
#           "type": "string"
#         }
#       }
#     }
```

### Deprecate an API model property
```ruby
# @model Foo
#     {
#       "id": "Foo",
#       "description": "A description.",
#       "properties": {
#         "bar": {
#           "deprecated": true,
#           "deprecation_notice": "YYYY-MM-DD",
#           "deprecation_effective": "YYYY-MM-DD",
#           "deprecation_description": "A description of the deprecation.",
#           "description": "A property.",
#           "example": "baz",
#           "type": "string"
#         }
#       }
#     }
```

## API Argument Deprecation
To deprecate an API argument, rename the `@argument` tag to
`@deprecated_argument`. You must provide a `NOTICE` date and an `EFFECTIVE` date
for the deprecation, along with a description for the deprecation.

Before:
```ruby
# @argument foo [Required, String]
#   A description of the argument.
```

After:
```ruby
# @deprecated_argument foo [Required, String] NOTICE YYYY-MM-DD EFFECTIVE YYYY-MM-DD
#   A description of the argument, along with a description of the deprecation.
```

## API Response Field Deprecation
To deprecate an API response field, rename the `@response_field` tag to
`@deprecated_response_field`. You must provide a `NOTICE` date and an
`EFFECTIVE` date for the deprecation, along with a description for the
deprecation.

Before:
```ruby
# @response_field foo
#   A description of the response field.
```

After:
```ruby
# @deprecated_response_field foo NOTICE YYYY-MM-DD EFFECTIVE YYYY-MM-DD
#   A description of the response field, along with a description of the
#   deprecation.
```

---

### Doc/App And Jobs Gem Groups

# App and Jobs Gem Groups

Canvas runs two distinct server types from the same codebase: **app servers** (handling HTTP requests via Apache/Passenger in Production) and **jobs servers** (processing background jobs via `script/delayed_job`). The `:app_server` and `:jobs_server` Bundler groups (located in Gemfile.d/jobs_server.rb and Gemfile.d/app_server.rb) allow each server type to load only the gems it needs, reducing memory usage.

## How It Works

In `config/application.rb`, Bundler loads gems using the `Bundler.require(*Rails.groups)` statement. By default, [`Rails.groups`](https://api.rubyonrails.org/classes/Rails.html#method-c-groups) includes:

- `:default`, which means all gems in the default group (i.e. when no group is specified) are included
- The environment, e.g. `"development"` in a dev environment, or `"production"` in a production environment. This means all gems in the group corresponding with the current environment will be included.

For example, in a development rails console you would see:

```ruby
> Rails.groups
=> [:default, "development"]
```

In addition, the `RAILS_GROUPS` environment variable can be set to include additional groups. For example, to include the :jobs_server gem group:

```bash
RAILS_GROUPS=jobs_server bundle exec rails console
> Rails.groups
=> [:default, "development", "jobs_server"]
```

This `RAILS_GROUPS` env var is how we conditionally load these app and jobs groups.

## Automatic Group Inference

Regardless of environment, the correct gem group is automatically inferred based on the server type. In config/boot.rb we set `RAILS_GROUPS` to `"jobs_server"` for job servers, and to `"app_server"` for app servers. We identify jobs servers using the `RUNNING_AS_DAEMON` env var that is set in `script/delayed_job`. We identify app servers using the `RUNNING_IN_RACK` env var that is set in config.ru.

If `RAILS_GROUPS` is set in the environment before starting, it is used as-is and the automatic inference is skipped. This allows you to override the default behavior when needed.

## When to Add a Gem to a Group

**Add to `:jobs_server`** if every call site for a gem is reachable only via a background job mechanism:

- `.delay(...).<method>`
- `handle_asynchronously :<method>`
- `Delayed::Job.enqueue(...)`
- `Delayed::Periodic.cron(...)`
- A worker class implementing `perform`

**Add to `:app_server`** if every call site is reachable only via a synchronous HTTP request.

**Leave in the default group if:**

- It is used by both server types
- You are unsure — the default is always safe
- It is Rails infrastructure or a shared utility called from both contexts

When in doubt, trace each call site back to its entry point before deciding. Also note that calls to `delay_if_production(...).<method>` should be treated as synchronous for this purpose; this code will run on app servers in development environments, and therefore any gems used within these calls need to be available on both app and jobs servers.

## Verifying a Change

You can verify that a gem is loaded in one env, and not loaded in another, by executing a `rails runner` command with the `RAILS_GROUPS` env var set. For example, after adding the `mimemagic` gem to `Gemfile.d/jobs_server.rb`, you can run:

```sh
# Prints nil — not loaded on app server
DISABLE_SPRING=1 RAILS_GROUPS=app_server bundle exec rails runner "puts defined?(MimeMagic).inspect"
nil

# Prints "constant" — loaded on jobs server
DISABLE_SPRING=1 RAILS_GROUPS=jobs_server bundle exec rails runner "puts defined?(MimeMagic).inspect"
"constant"
```

Alternatively, you can run canvas locally, and put a `puts` or `debugger` statement within app & jobs code and then check for the gem's existence.

---

### Doc/Canvas Operations Library

# CanvasOperations

A library for running common operations in deployed Canvas environments, with consistent logging, metric emission, progress tracking, and error handling, and more.

See [`lib/canvas_operations`](../lib/canvas_operations)

## Features
- **Operation Base Class**: All operations inherit from a common base, ensuring consistent behavior and features.
- **Shard Binding**: Operations are bound to a single Switchman shard for data consistency.
- **Progress Tracking**: Integrated with a `Progress` model for tracking and reporting (opt-in).
- **Metric Emission**: Emits events to InstStatsd for monitoring operation lifecycle.
- **Configurable Settings**: Per-operation, per-cluster settings.
- **Callbacks**: Lifecycle hooks for before/after/around run and failure events.
- **Error Handling**: Standardized error classes for shard and mode validation.

## Usage

### Creating a New Operation

Subclass `CanvasOperations::BaseOperation` and override the `execute` method:

```ruby
class EnableCoolFeatures < CanvasOperations::BaseOperation
  # define callbacks for your operation
  before_run :validate_feature_prerequisites
  after_run :notify_stakeholders
  after_failure :notify_engineering_team

  # define settings that can be changed on-the-fly
  setting :feature_list, default: "feature_one"
  setting :stakeholder_notification_channel, default: "#releases"

  # defaults to false. If true, the operation will create a Progress
  # record when run and update that progress when a completion or
  # failure state for the operation is reached.
  self.progress_tracking = true

  def execute
    log_message("Enabling features #{feature_list}!")
  end

  def validate_feature_prerequisites
    raise "Prerequisites not met!" unless ...
  end

  def notify_stakeholders
    SlackClient.post_message(
      channel: stakeholder_notification_channel,
      text: "The following features have been enabled: #{feature_list}"
    )
  end

  def notify_engineering_team
    SlackClient.post_message(
      channel: "#engineering-alerts",
      text: "#{name} failed! Please investigate."
    )
  end
end
```

Run the operation:

```ruby
MyOperation.new.run_later
```

Operations are run in an async job via the inst-jobs gem.

See [`lib/canvas_operations/base_operation.rb`](../lib/canvas_operations/base_operation.rb) for more details on available methods and features.

See [`lib/canvas_operations/base_concerns/settings.rb`](../lib/canvas_operations/base_concerns/settings.rb) for more details on using operation settings.

See [`lib/canvas_operations/base_concerns/callbacks.rb`](../lib/canvas_operations/base_concerns/callbacks.rb) for more details on using operation callbacks.

### DataFixup Operations

As described above, the base operation class can be subclassed for specific use cases. The `DataFixup` operation class serves as an example of this pattern, but is also a useful tool in its own right.

The `DataFixup` operation provides a standard framework for efficiently performing data fixups that require processing large numbers of records, either individually or in batches.

To use this operation, create a subclass of `CanvasOperations::DataFixup`:

```ruby
module DataFixup
  module InstructureIdentity
    class UnsetAuthlogicAttributesOnInstPseudonyms < CanvasOperations::DataFixup
      # Optionally override setting defaults (more details below)
      setting :range_batch_size, default: 10_000, type_cast: :to_i

      # Should records be yielded one at a time, or in batches? (more details below)
      self.mode = :batch

      # If set to true, the return value of `process_record` or `process_batch` will be
      # recorded in an auditable Attachment associated with the operation's context.
      self.record_changes = true

      # Define the scope of records to process (more details below)
      scope do
        Pseudonym.instructure_identity.where(
          "ABS(EXTRACT(EPOCH FROM (pseudonyms.last_request_at - pseudonyms.created_at))) <= 1"
        ).where(
          login_count: 1
        ).where.not(
          last_request_at: nil
        )
      end

      # Define how to process a batch of records (more details below)
      def process_batch(pseudonym_batch)
        pseudonym_batch.update_all(last_request_at: nil, current_login_at: nil, current_login_ip: nil)
      end
    end
  end
end
```

and then instantiate and call `run_later` on your fixup from a migration:

```ruby
# db/migrate/20250820214915_unset_authlogic_attributes_on_inst_pseudonyms.rb
class UnsetAuthlogicAttributesOnInstPseudonyms < ActiveRecord::Migration[7.2]
  tag :postdeploy

  def up
    DataFixup::InstructureIdentity::UnsetAuthlogicAttributesOnInstPseudonyms.new.run_later
  end
end
```


#### DataFixup Properties

| Property                | Description |
|-------------------------|-------------|
| `mode`                  | Controls how records are yielded to your processing logic. <br> - `:individual_record`: Each record is yielded one at a time to the `process_record` method.<br> - `:batch`: Records are yielded in batches to the `process_batch` method. <br> No matter which you choose, records are loaded efficiently in batches. |
| `record_changes`        | Whether to record changes made by the datafixup in Attachment logs associated with the context. Returns from `process_record` or `process_batch` are written to chunked text files and uploaded as Attachments. Defaults to `false` and is always disabled in test environments. |
| `scope`                 | The ActiveRecord scope that defines the set of records to be processed by the fixup. |
| `process_record(record)` | (For `:individual_record` mode) Define this method to specify how to process each individual record. |
| `process_batch(records)` | (For `:batch` mode) Define this method to specify how to process a batch of records. |
| `run_on_default_shard` | If true, the data fixup will run on the default shard. If false, the default shard is skipped. Defaults to true. |

#### DataFixup Settings

| Setting                   | Description |
|---------------------------|-------------|
| `range_batch_size`        | How many IDs per chunk `find_ids_in_ranges` should yield. Larger numbers result in your `scope` query being run less often, but over a larger set of rows. |
| `job_scheduled_sleep_time`| How long, in seconds, to sleep between scheduling an async batch of work. Increasing this value can help if the jobs cluster primary is getting hit too hard. |
| `processing_sleep_time`   | How long, in seconds, to sleep between processing batches or individual records. Increasing this value can help if the cluster's primary node is getting hit too hard. |

These settings can be changed on-the-fly; just be sure to send SIGHUP to job hosts to ensure configuration is reloaded.

A data fixup operation and associated files can be generated with `rails g data_fixup <OperationName>`.

See [`lib/canvas_operations/base_concerns/settings.rb`](lib/canvas_operations/base_concerns/settings.rb) for more details on using operation settings.

#### DataFixup Additional Properties
**batch_strategy**

Defaults to `:pluck_ids`, which performs a pluck on the scope (after range filtering), and loads records into memory in batches based on those IDs.

In some cases, it may be appropriate to change this strategy. To do so, set the `batch_strategy` class instance variable:

```ruby
class UnsetAuthlogicAttributesOnInstPseudonyms < CanvasOperations::DataFixup
...
  self.batch_strategy = :id
...
end
```

See `/usr/src/app/config/initializers/active_record.rb` for additional details on available batch strategies.


### RootAccountOperation

The `RootAccountOperation` class is designed for operations that need to run per root account. This is essential when you need to execute the same operation across multiple accounts with proper isolation and context.

Key features:
- **Automatic shard binding** based on the root account's shard
- **PluginSetting context wrapping** via `PluginSetting.with_account` (when available)
- **Unique singleton job keys** per root account to allow concurrent execution across different accounts
- **Progress tracking** scoped to each root account

To use this operation, create a subclass of `CanvasOperations::RootAccountOperation`:

```ruby
class NotifyAccountAdmins < CanvasOperations::RootAccountOperation
  # define callbacks for your operation
  before_run :validate_feature_prerequisites
  after_run :notify_stakeholders
  after_failure :notify_engineering_team

  def execute
    admin_count = 0
    root_account.account_users.active.each do |account_user|
      send_notification(account_user.user)
      admin_count += 1
    end

    results[:admin_count] = admin_count
    results[:root_account_id] = root_account.global_id
    log_message("Notified #{admin_count} admins for account #{root_account.global_id}")
  end

  def validate_feature_prerequisites
    raise "Prerequisites not met!" unless ...
  end

  def notify_stakeholders
    SlackClient.post_message(
      channel: stakeholder_notification_channel,
      text: "The following features have been enabled: #{feature_list}"
    )
  end

  def notify_engineering_team
    SlackClient.post_message(
      channel: "#engineering-alerts",
      text: "#{name} failed! Please investigate."
    )
  end

  private

  def send_notification(user)
    # Send notification logic here
  end
end
```

Run the operation for a specific account:

```ruby
NotifyAccountAdmins.new(
  root_account: Account.find(123),
).run_later
```

Run the operation for all active accounts:

```ruby
Account.root_accounts.active.find_each do |account|
  NotifyAccountAdmins.new(
    root_account: account,
  ).run_later
end
```

**Important characteristics:**

- Each account gets its own delayed job with a unique singleton key: `operations/{operation_name}/shards/{shard_id}/accounts/{account_global_id}`
- Operations for different accounts can run concurrently
- Operations for the same account are deduplicated (only one pending/running job per account)
- The delayed job's `shard` and `account` attributes are automatically set correctly
- Progress records are associated with the root account (not the cluster primary)

See [`lib/canvas_operations/root_account_operation.rb`](../lib/canvas_operations/root_account_operation.rb) for implementation details.

---

### Doc/Copyright

A new file should have the following at the top:

```
/*
 * Copyright (C) 2025 - present Instructure, Inc.
 *
 * This file is part of Canvas.
 *
 * Canvas is free software: you can redistribute it and/or modify it under
 * the terms of the GNU Affero General Public License as published by the Free
 * Software Foundation, version 3 of the License.
 *
 * Canvas is distributed in the hope that it will be useful, but WITHOUT ANY
 * WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR
 * A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
 * details.
 *
 * You should have received a copy of the GNU Affero General Public License along
 * with this program. If not, see <http://www.gnu.org/licenses/>.
 */
 ```

---

### Doc/Detect N Plus One Queries

# Detect N+1 Queries

Canvas uses the [prosopite](https://github.com/charkost/prosopite) gem to detect N+1 query problems and prints information about them to `log/development.log`. It also prints this information to its own dedicated log file, `log/prosopite.log` when in development. Here's an example report:

```ruby
N+1 queries detected:
  SELECT "context_external_tools".* FROM "public"."context_external_tools" WHERE "context_external_tools"."id" = 1 LIMIT 1
  SELECT "context_external_tools".* FROM "public"."context_external_tools" WHERE "context_external_tools"."id" = 1 LIMIT 1
  SELECT "context_external_tools".* FROM "public"."context_external_tools" WHERE "context_external_tools"."id" = 1 LIMIT 1
  SELECT "context_external_tools".* FROM "public"."context_external_tools" WHERE "context_external_tools"."id" = 1 LIMIT 1
Call stack:
  config/initializers/postgresql_adapter.rb:315:in `exec_query'
  app/models/content_tag.rb:283:in `content'
  app/models/assignment.rb:3504:in `quiz_lti?'
  app/models/assignment.rb:394:in `can_duplicate?'
  lib/api/v1/assignment.rb:193:in `assignment_json'
  lib/api/v1/assignment_group.rb:82:in `block in assignment_group_json'
  lib/api/v1/assignment_group.rb:75:in `map'
  lib/api/v1/assignment_group.rb:75:in `assignment_group_json'
  ...<more stack trace>
```

## Prosopite.scan

You can pass a block to `Prosopite.scan` to have it check for N+1 queries:

```ruby
Prosopite.scan do
  Course.where(id: 1..5).each { |course| course.assignments.first }
end

N+1 queries detected:
  SELECT "assignments".* FROM "public"."assignments" WHERE "assignments"."context_id" = 4 AND "assignments"."context_type" = 'Course' ORDER BY assignments.created_at LIMIT 1
  SELECT "assignments".* FROM "public"."assignments" WHERE "assignments"."context_id" = 1 AND "assignments"."context_type" = 'Course' ORDER BY assignments.created_at LIMIT 1
  SELECT "assignments".* FROM "public"."assignments" WHERE "assignments"."context_id" = 2 AND "assignments"."context_type" = 'Course' ORDER BY assignments.created_at LIMIT 1
  SELECT "assignments".* FROM "public"."assignments" WHERE "assignments"."context_id" = 3 AND "assignments"."context_type" = 'Course' ORDER BY assignments.created_at LIMIT 1
Call stack:
  config/initializers/postgresql_adapter.rb:315:in `exec_query'
  (irb):5:in `block (2 levels) in <main>'
  (irb):5:in `block in <main>'
  (irb):4:in `<main>'

```

If you don't want to pass a block, you can use `Prosopite.scan` along with `Prosopite.finish`:

```ruby
Prosopite.scan
Course.where(id: 1..5).each { |course| course.assignments.first }
foo = "bar"
Prosopite.finish
N+1 queries detected:
  SELECT "assignments".* FROM "public"."assignments" WHERE "assignments"."context_id" = 4 AND "assignments"."context_type" = 'Course' ORDER BY assignments.created_at LIMIT 1
  SELECT "assignments".* FROM "public"."assignments" WHERE "assignments"."context_id" = 1 AND "assignments"."context_type" = 'Course' ORDER BY assignments.created_at LIMIT 1
  SELECT "assignments".* FROM "public"."assignments" WHERE "assignments"."context_id" = 2 AND "assignments"."context_type" = 'Course' ORDER BY assignments.created_at LIMIT 1
  SELECT "assignments".* FROM "public"."assignments" WHERE "assignments"."context_id" = 3 AND "assignments"."context_type" = 'Course' ORDER BY assignments.created_at LIMIT 1
Call stack:
  config/initializers/postgresql_adapter.rb:315:in `exec_query'
  (irb):5:in `block in <main>'
  (irb):5:in `<main>'
```

## Enabling Detection

### In Development & Test

Automatic N+1 detection for requests is on by default (`development` and `test` env only). All controller actions are wrapped in a `Prosopite.scan` while in `development` or `test`.

To disable N+1 detection, set the DISABLE_N_PLUS_ONE_DETECTION environment variable to 'true'.

You can manually invoke `Prosopite.scan` in any environment.

### In Production

You can enable N+1 detection on a per-request basis by passing a
`n_plus_one_detection=true` parameter to the request. A report will be saved to
your users files (`/files`). The format of the filename is:
`n_plus_one_detection-<controller>#<action>-<iso8601 timestamp>`, but you can
also specify a custom name by passing `n_plus_one_name=<name>` which will make
the filename `n_plus_one_detection-<name>-<iso8601 timestamp>`.

Note that you must be logged in as a Site Admin user to enable this
functionality. This is to prevent abuse of the feature by regular users. However, a Site Admin user can masquerade as any non-site-admin user and generate an N+1 report (the resulting report will be in the Site Admin user's Canvas Files).

### Limitations

If you try to ask for both a flamegraph and an N+1 report, the N+1 report will not be generated. Flamegraphs take precedence over N+1 reports. This is to prevent the N+1 report from obfuscating the flamegraph data. Additionally, if the flamegraph grows too large, a stack overflow error may occur, which defeats the purpose of the reports.

---

### Doc/Flamegraphs

# Flamegraphs

## TL;DR

Add the `flamegraph=true&flamename=my-custom-name` query params to any request, and view/download flamegraphs in your Canvas Files.

## Why Flamegraphs?

Flamegraphs can help to identify performance bottlenecks in our Canvas Ruby code (it can help identify problems rooted in inefficient database queries AND problems rooted in too-much-processing-time-spent-in-ruby).

If you're not familiar with flamegraphs and don't know how to analyze them, I encourage you to take an hour or two researching the topic; it's not difficult to analyze a flamegraph file, and it can be immensely helpful in tracking down performance issues!

## Generating an HTML Flamegraph File

Users with Site Admin access can have an interactive HTML flamegraph generated for them, for any request, and have it delivered to their Canvas Files. Site Admin users can also masquerade as any non-site-admin user and generate a flamegraph report (the resulting report will be in the Site Admin user's Canvas Files).

In order to generate a flamegraph for a given request, simply add the

```text
flamegraph=true
```

query parameter to the request. That's it! It works for both HTML and JSON requests.

## Viewing the Generated HTML Flamegraph File

Next, go to your Canvas files (`/files`) and notice there's a flamegraphs folder with your newly-generated flamegraph file is in there!

The format of the filename is: `flamegraph-<controller>#<action>-<iso8601 timestamp>`, but you can also pass the

```text
flamename=my-kewl-stuff
```

query parameter, which will make the filename `flamegraph-my-kewl-stuff-<controller>#<action>-<iso8601 timestamp>`.

You can preview the file within Canvas and interact with it (try clicking things and using the search bar, it's interactive!). You can also download the HTML file and view it locally in your browser.

## More Info

Flamegraphs are generated by the awesome ruby [stackprof](https://github.com/tmm1/stackprof) library. See `app/services/flamegraphs/flamegraph_service.rb` for implementation details.

See `git show b860ae73` for the commit that added this functionality.

If you have any questions, feel free to reach out to @solson in Slack.

---

### Doc/I18n

# Internationalization (i18n) in Canvas LMS

Canvas LMS uses a internationalization (i18n) system that handles translations for both backend (Ruby) and frontend (JavaScript) code.

## Overview

The i18n system in Canvas LMS is built on:
- Ruby i18nliner for backend translations
- @instructure/i18nliner-canvas for frontend translations
- format-message for specific packages like canvas-rce

The system supports translations in:
- Ruby files
- JavaScript files
- TypeScript files
- Handlebars templates

## Translation Files

Main translation files are located in several places:

### Source and Generated Files
- `config/locales/generated/en.yml`: Contains all extracted strings from source code
- `config/locales/generated/en-js.json`: Frontend-specific translations
- `config/locales/generated/en-js-index.json`: Used to generate JS modules for runtime
- `config/locales/locales.yml`: Configuration file defining available locales

### Runtime Files
- `public/javascripts/translations/`: Contains compiled JavaScript translation bundles used at runtime
- `config/locales/*.yml`: Language-specific translation files (e.g., `hy.yml` for Armenian)
  - These files are typically updated by automated translation processes

### Package Translations

Some packages (e.g. canvas-media, canvas-rce) are published separately on npm. These packages handle translations differently:

1. **@instructure/translations Package**
   - Central package containing translations for all Canvas packages
   - Located in `packages/translations/`
   - Contains language files for each supported locale
   - Packages can access their translations through this package

2. **Individual Package Setup**
   Example using canvas-media:
   ```javascript
   // scripts/installTranslations.js
   const {getTranslationList, readTranslationFile} = require('@instructure/translations')
   ```
   - Uses `installTranslations.js` to:
     - Fetch translations from @instructure/translations
     - Generate locale-specific files
     - Set up code-splitting for translations
     - Create a getTranslations() function for runtime loading

3. **Automated Updates**
   - Packages use scripts like `commitTranslations.sh` to:
     - Create a new git branch
     - Commit translation updates
     - Push changes to Gerrit
     - Notify teams via Slack when updates are needed

## Using Translations

### In React Components

```javascript
import {useScope as createI18nScope} from '@canvas/i18n'
const I18n = createI18nScope('ComponentName')

// Usage
I18n.t('string to translate')
```

### In Ruby Code

```ruby
t('string to translate')
```

## Special Cases

### Canvas Rich Content Editor (RCE)

The RCE package (`packages/canvas-rce`) uses `format-message` instead of i18nliner for translations:

```javascript
import formatMessage from 'format-message'

formatMessage('string to translate')
```

## Development Workflow

### 1. Extract Translations

To extract translatable strings from the codebase:

```bash
bundle exec rake canvas:compile_assets i18n:extract
```

This command:
- Scans Ruby, JavaScript, and TypeScript files for translatable strings
- Generates/updates translation files in `config/locales/generated/`

### 2. Available Tasks

- `i18n:check`: Validates translation calls in Ruby and JS code
- `i18n:extract`: Extracts strings from source code into YAML
- `i18n:generate`: Creates runtime translation files
- `i18n:export`: Prepares files for translators
- `i18n:import`: Imports new translations
- `i18n:generate_js`: Generates JavaScript translation files
- `i18n:generate_lolz`: Generates LOLZ pseudo-translations (useful for testing)
- `i18n:lock`: Locks specific translation keys

### 3. LOLZ Translations

Canvas includes a special "LOLZ" locale for testing and development:
- Generated using `rake i18n:generate_lolz`
- Creates pseudo-translations
- Stored in `config/locales/lolz.yml`

## Translation Management

### Available Locales

Locales are configured in `config/locales/locales.yml`:
- Defines which languages are available
- Specifies if a locale is crowdsourced

## Best Practices

1. **Use Scopes**
   - Always use translation scopes to organize strings by component/feature
   - Choose meaningful scope names that reflect the component's purpose

2. **Avoid String Concatenation**
   - Use interpolation instead of concatenating strings
   - Example: `I18n.t('Welcome %{name}', { name: userName })`

3. **Keep Translations Updated**
   - Run `i18n:extract` when adding new strings
   - Verify translations with `i18n:check`

## Testing

When writing tests that involve translations:

1. **Don't Mock Unless Necessary**
   - Avoid `jest.mock('@canvas/i18n')`
   - Use real translation calls when possible

2. **Test User Experience**
   - Focus on testing how translated content appears to users
   - Verify that translated strings are properly displayed
   - Consider using LOLZ translations for visual testing

## Common Issues and Solutions

1. **Missing Translations**
   - Run `i18n:extract` to ensure all strings are captured
   - Check that the scope is correctly defined

2. **Runtime Errors**
   - Verify that translation files are properly generated
   - Check for missing interpolation variables

## Future Considerations

### Moving to i18next

For new standalone packages (similar to Canvas RCE or Canvas Meteor), consider using i18next instead of format-message.

## Additional Resources

- [i18nliner Documentation](https://github.com/jenseng/i18nliner)
- [format-message Documentation](https://github.com/format-message/format-message)
- [i18next Documentation](https://www.i18next.com/)
- Canvas LMS translation tasks: `gems/i18n_tasks/lib/tasks/i18n.rake`

---

### Doc/Live Events

# Live Events

Canvas includes the ability to push a subset of real-time events to a
Kinesis stream, which can then be consumed for various analytics
purposes. This is not a full-fidelity feed of all changes to the
database, but a targeted set of interesting actions such as
`grade_change`, `login`, etc.

## Development and Testing

There are two components to local development:
- the kinesis stream (which can hook into the `live-events-publish` lambda)
- the subscription service and its UI (`live-events-subscriptions`, `live-events-lti`)

### Kinesis Stream

If using the docker-compose dev setup, there is a "fake kinesis" available in
docker-compose/kinesis.override.yml available for use. To start this kinesis
container run `docker compose up -d kinesis`. Once it's up, make sure you have
the `aws` cli installed, and run the following command to create a stream (with
canvas running). Keep in mind that we are running this locally so actual AWS
credentials are not needed, run the following command as you see it here:

```bash
AWS_ACCESS_KEY_ID=key AWS_SECRET_ACCESS_KEY=secret aws --endpoint-url http://kinesis.docker/ kinesis create-stream --stream-name=live-events --shard-count=1 --region=us-east-1
```

Once the stream is created, configure your Canvas to use it in your
`config/dynamic_settings.yml`. This file is a local shim for Consul. If you have
copied the example file at `config/dynamic_settings.yml.example` recently, you
should already see a live_events block and it should already be configured properly.
If you don't see a live_events block, check the example file or copy this block:

```yml
      live_events.yml: |-
        aws_endpoint: http://kinesis:4567
        kinesis_stream_name: live-events
        aws_access_key_id: key
        aws_secret_access_key_dec: secret
```

Depending on your docker networking setup, you may need to substitute either
`http://kinesis:4567`, `http://kinesis.docker`, or `http://kinesis.canvaslms.docker`
for the aws_endpoint (the first two should be equivalent).

Restart Canvas, and events should start flowing to your kinesis stream.
You can view the stream with the `tail_kinesis` tool:

```bash
docker compose run --rm web script/tail_kinesis http://kinesis:4567 live-events
```

#### Stubbing Kinesis

Instead of viewing events in the kinesis stream, you can add the `stub_kinesis`
attribute to the dynamic_settings live_events block that you configured above,
with a value of `true`. This will print live events to stdout instead of sending
them to a kinesis stream.

An easy way of accessing stdout when using dockerized Canvas is this:

```
docker compose logs -f --tail=100 <jobs|web> # whichever container you need
```

#### Connecting to local Publisher Lambda

The `live-events-publish` repo should be checked out and running locally.
This contains the publisher lambda, and other infrastructure including a local
kinesis stream. Note the url of that kinesis stream, which may look like
`http://kinesis.live-events-publish.docker:4567`.

There should already be a stream created in that container, with the name
found in `docker-compose.yml`, in the `KINESIS_LOCAL_STREAM_NAME` environment
variable. If that stream doesn't exist, create it with this `aws` command:

```bash
AWS_ACCESS_KEY_ID=ACCESS_KEY AWS_SECRET_ACCESS_KEY=SECRET_KEY aws --endpoint-url http://kinesis.live-events-publish.docker/ kinesis create-stream --stream-name=live-events-local-test-stream --shard-count=1 --region=us-east-1
```

Once the stream is created, configure your Canvas to use it in your `config/dynamic_settings.yml`.
This file is a local shim for Consul. If you have copied the example file at
`config/dynamic_settings.yml.example` recently, you should already see a live_events block.
Note that these settings differ from the example block above. If you don't see a live_events
block, check the example file or copy this block:

```yml
      live_events.yml: |-
        aws_endpoint: http://kinesis.live-events-publish.docker
        kinesis_stream_name: live-events-local-test-stream
        aws_access_key_id: ACCESS_KEY
        aws_secret_access_key_dec: SECRET_KEY
```

Restart Canvas, and events should start flowing to the kinesis stream, and to
the publisher lambda itself. You can view the stream and publisher lambda
activity by looking at the output of `docker compose up` in the `live-events-publish`
repo.

### Subscription Management

#### Connecting to local Subscription Service

The `live-events-subscriptions` repo should be checked out and running locally.
This contains the subscriptions for live events, which the publisher uses when
propagating events.

To connect Canvas with the subscription service, open `config/dynamic_settings.yml`
and make sure that the `live-events-subscription-service` prefix contains the
proper `app-host` value, which should be the url where your local subscription
service is running. Instructions for connecting on the subscription service side
are found in the `live-events-subscriptions` repo, in `README.md`.

#### Connecting to local Live Events LTI Tool

The `live-events-lti` repo should also be checked out and running locally. This
is an LTI tool which provides a UI for managing the subscriptions contained in
the subscription service. Instructions for configuring this LTI tool are
contained in the `live-events-lti` repo, in `README.md`.

## Canvas LMS Live Events Consumers

Canvas LMS emits live events to multiple subscribers including Quiz LTI and
Gauge, among others.

---

### Doc/Profiling Ruby

# Profiling Ruby

If you've got ruby code you're concerned is behaving badly (especially slowly),
there are some tools installed that can help you figure out why.

## Stackprof

[stackprof](https://github.com/tmm1/stackprof) is in the test bundle, and is a
nice choice because it's comparatively lightweight.  It uses sampling of the stack
at various intervals rather than complete instrumentation, so it can be statistically noisy,
but should still accumulate into a reasonable sense of where your time is going.

Apply it to a chunk of code like so:

```ruby
StackProf.run(mode: :wall, out: 'tmp/stackprof-canvas-test.dump', interval: 1000) do
  #...the code you want to profile, often the body of a test
end
```

This will sample every 1000 microseconds (1 millisecond), and write the
output to the specificed "out" file.  See the stackprof docs linked above for more
details on configuration options

Once you've produced a dumpfile, you can produce a report on the results
by using the stackprof command directly:

```bash
bundle exec stackprof tmp/stackprof-canvas-test.dump --limit 20
```

That should give you a report like:

```
/* Detailed source-code truncated for AI context efficiency. */
```

Which can show you which stack frames are frequently at the top of the stack.

---

### Doc/Testing Javascript

# Testing JavaScript

The process of testing JavaScript sometimes confuses people. This document's goal
is to alleviate that confusion and establish how to run JavaScript tests.

## Jest

Whenever possible, which for now means when you are testing something that only imports
stuff that does not use AMD imports (eg, it only requires
stuff from app/jsx or node_modules), you should write your js tests for
[Jest](https://facebook.github.io/jest/) going forward. It is faster and the testing experience will be better.

Jest is a Node-based runner. This means that the tests always run in a Node environment and not in a real browser. This lets us enable fast iteration speed and prevent flakiness.

While Jest provides browser globals such as `window` thanks to [jsdom](https://github.com/tmpvar/jsdom), they are only approximations of the real browser behavior. Jest is intended to be used for unit tests of your logic and your components rather than the DOM quirks. Use a QUnit Karma test or Selenium for browser end-to-end tests if you need them.

### Filename Conventions

Put the test files (or `__tests__` folders) next to the code they are testing so that relative imports appear shorter. For example, if `App.test.js` and `App.js` are in the same folder, the test just needs to `import App from './App'` instead of a long relative path. Colocation also helps find tests more quickly in larger projects.

### Writing Jest Tests

To create tests, add `it()` (or `test()`) blocks with the name of the test and its code. You may optionally wrap them in `describe()` blocks for logical grouping but this is neither required nor recommended.

Jest provides a built-in `expect()` global function for making assertions. A basic test could look like this:

```js
import sum from './sum'

it('sums numbers', () => {
  expect(sum(1, 2)).toEqual(3)
  expect(sum(2, 2)).toEqual(4)
})
```

All `expect()` matchers supported by Jest are [extensively documented here](http://facebook.github.io/jest/docs/api.html#expect-value).<br>
You can also use [`jest.fn()` and `expect(fn).toBeCalled()`](http://facebook.github.io/jest/docs/api.html#tobecalled) to create "spies" or mock functions and in jest tests you should probably use that instead of sinon mocks/spies/stubs like we use in our QUnit tests.

**NEVER use `getByRole` queries in tests.** Use of `byRole` queries (`getByRole`, `findByRole`, `queryByRole`, `getAllByRole`, `findAllByRole`, `queryAllByRole`) is discouraged due to performance issues. Please consider using a different query such as `byText` or `byLabelText` instead.

\*NOTE: You cannot run jest if there is anything with AMD, CoffeeScript, or some of the Webpack aliases (which lead to AMD or CoffeeScript).

### Testing Components

There is a broad spectrum of component testing techniques. They range from a “smoke test” using a jest snapshot, to shallow rendering and testing some of the output, to full rendering and testing component lifecycle and state changes.

An example of a simple smoke test for your components:

```js
import React from 'react'
import ReactDOM from 'react-dom'
import SomeComponent from './SomeComponent'

it('renders without crashing', () => {
  const div = document.createElement('div')
  ReactDOM.render(<App />, div)
})
```

This test mounts a component and makes sure that it didn’t throw during rendering. Tests like this provide a lot value with very little effort so they are great as a starting point.

When you encounter bugs caused by changing components, you will gain a deeper insight into which parts of them are worth testing in your application. This might be a good time to introduce more specific tests asserting specific expected output or behavior.

All Jest matchers are [extensively documented here](http://facebook.github.io/jest/docs/api.html#expect-value).

### Focusing and Excluding Tests

You can replace `it()` with `xit()` to temporarily exclude a test from being executed.<br>
Similarly, `fit()` lets you focus on a specific test without running any other tests.

### Running Tests

To run all tests:

```
yarn test:jest
```

To run a subset of files or directories:

```
yarn test:jest path/to/components/__tests__/spec.js path/to/other_component/ ...
```

To rerun tests on a file change and/or debug remotely:

```
yarn test:jest:debug path/to/components/__tests__/spec.js
```

### Coverage Reporting

Jest has an integrated coverage reporter that works well with ES6 and requires no configuration.<br>
Run `npm test -- --coverage` (note extra `--` in the middle) to include a coverage report like this:



Note that tests run much slower with coverage so it is recommended to run it separately from your normal workflow.

### Snapshot Testing

Snapshot testing is a feature of Jest that automatically generates text snapshots of
your components and saves them on the disk so if the UI output changes, you get
notified without manually writing any assertions on the component output.
[Read more about snapshot testing.](http://facebook.github.io/jest/blog/2016/07/27/jest-14.html)

## Running QUnit Karma Tests

A lot of the older stuff is still QUnit. For more info on running those older tests, see the "Running js tests with webpack" section of [working_with_webpack.md](https://github.com/instructure/canvas-lms/blob/master/doc/working_with_webpack.md).

Tl;dr: run a single test in watch mode like:

```
yarn jspec-watch spec/coffeescripts/util/deparamSpec.js
```

## Running Tests in Docker

See the "Running javascript tests" section of [developing_with_docker.md](https://github.com/instructure/canvas-lms/blob/master/doc/docker/developing_with_docker.md).

## Javascript Test Coverage

You can generate code coverage locally by having webpack
set up( `touch config/WEBPACK`) then running `COVERAGE=1 yarn test`.
You should then have a folder in your root directory called `coverage-js`
in which contains an `index.html` which if you open it will show you
the test coverage for all javascript (js, jsx)

---

### Doc/Testing With Selenium

# Testing with Selenium

You may run the Selenium tests either natively or in docker.

## Running Selenium Tests Natively (Mac)

We're making a few assumptions here:
  - you're using an Apple computer
  - you've already installed Homebrew
  - you've already installed Postgres (postgresapp.com is an excellent option),
    and Postgres is running on your computer
  - you've already installed Node.js

0. Install `yarn` if you haven't already:

```sh
brew install yarn
```

If you find you need an older version of yarn, follow these instructions to
install the older version and switch to it:

[https://stackoverflow.com/a/52525732/3038677](https://stackoverflow.com/a/52525732/3038677)

1. Follow the instructions in `script/prepare/README.md` to setup the `prepare`
script.

You'll use the `prepare` script later to automate installing and updating Canvas
on your computer.

Note: some features of `prepare` only work if you have access to Instructure's
Gerrit host. See the README for details.

2. Install a web browser driver on your computer for the browser you wish to run
the tests in. Homebrew is the easiest way to go:

```sh
brew install chromedriver --cask # necessary for running tests in Chrome
brew install geckodriver # necessary for running tests in Firefox
```

Now let's get Canvas ready to run the tests.

3. Copy the Selenium and database configuration files:

```sh
cp config/selenium.yml.example config/selenium.yml
cp config/database.yml.example config/database.yml
```

4. Use `prepare` to install Canvas plugins and dependencies, create databases,
run database migrations, etc:

```sh
prepare
```

You might encounter problems with some Ruby dependencies. The ["Dependency
Installation" section](https://github.com/instructure/canvas-lms/wiki/Quick-Start#dependency-installation)
in the public Canvas LMS Github wiki has some useful tips.

4.a. Optional. Run delayed jobs in the foreground (not all Selenium tests need
this but some do):

```sh
script/delayed_job run
```

or run it in the background:

```sh
script/delayed_job run &
```

5. Run the Selenium tests:

```sh
bundle exec rspec spec/selenium
```

or run a specific Selenium test:

```sh
bundle exec rspec spec/selenium/accounts_spec.rb:36
```

### Running Tests against Headless Chrome

Selenium tests can be run against headless Chrome by changing a few properties in
`config/selenium.yml`. Specifically, you'll need to set `headless` to `true` and
`window_size` to something that makes sense, like so:

```yaml
  headless: true
  window_size: "1237,974"
```

This can be useful when you don't need to see what your test is doing, since it
can run in the background without stealing focus or interrupting other work. It's
especially useful when running specs many times to check for flakiness.

## Running Selenium Tests in Docker

See the [Selenium section](https://github.com/instructure/canvas-lms/blob/master/doc/docker/developing_with_docker.md#selenium)
of the `doc/docker/developing_with_docker.md` instructions.

## Selenium Testing Best Practices

### Using Helper Methods for Waiting

For clarity and reliability, prefer using built-in helper methods instead of relying on numerous `expect` statements to handle asynchronous operations. Canvas provides many useful helper methods in `spec/selenium/test_setup/common_helper_methods/` including:

- `custom_wait_methods.rb` - Contains `wait_for`, `wait_for_new_page_load`, and other waiting utilities
- `custom_page_loaders.rb` - Page loading and navigation helpers
- `custom_selenium_actions.rb` - Common Selenium actions
- `custom_validators.rb` - Validation helpers for Selenium tests

Using these helpers makes tests more readable and reliable than chains of expect statements.

### Selenium-Specific Guidelines

- **Use `expect` syntax:** Use the `expect(value).to matcher` syntax consistently
- **Organize with contexts:** Use `context` blocks to describe different test scenarios
- **Use `let` for test data:** Use `let` to define reusable test data and objects

---

### Doc/Trace Database Queries

# Trace Database Queries

Canvas includes the "active_record_query_trace" gem when running in development or test mode. This gem
prints a stack trace of every ActiveRecord query the Rails application makes.

The query trace logging is off by default, but can be enabled and configured by setting
environment variables. Note that traces cannot be enabled in production mode.

## Environment variable overview

| ENV VAR | Description |
|---------| ----------- |
| AR_QUERY_TRACE | traces are enabled if this variable is set. Traces disabled otherwise. |
| AR_QUERY_TRACE_TYPE | Controls what kind of queries print traces. Valid values are "all", "write", and "read". |
| AR_QUERY_TRACE_LINES | Controls how many lines of the trace are printed. Defaults to 10. |
| AR_QUERY_TRACE_LEVEL | Filters the backtrace to "app", "rails", or "full" |

---

### Doc/Using Guard Rail In Development

# Using GuardRail in Development

GuardRail allows activating different database configurations for specific blocks of code. This is frequently done to offload read queries to a replica (secondary) database in Canvas.

When offloading read queries to a replica, it's important to have the ability to test the change locally and verify no writes are occurring by accident.

This guide shows how to configure a read-only user to allow testing these kind of changes during development.

For more information on GuardRail see [The Canvas Manual](https://instructure.atlassian.net/wiki/spaces/CE/pages/1214382120/Canvas+ActiveRecord+Extensions#DATABASE-ENVIRONMENT-NUANCED-CONFIGURATION-WITH-GUARDRAIL).

## 1. Open `config/database.yml`
## 2. Add the following to the `common` YML section:
```
secondary:
    replica: true
    username: canvas_read_only
```

This should result in a `common` section that looks something like this:
```yml
common: &common
  adapter: postgresql
  host: <%= ENV.fetch('CANVAS_DATABASE_HOST', 'postgres') %>
  ...
  secondary:
    replica: true
    username: canvas_read_only
```

## 3. Create a new user and grant read-only access to databases:
First, create the new user
```bash
docker compose run --rm web psql -h postgres -U postgres -c "CREATE USER canvas_read_only WITH PASSWORD 'sekret'"
```

When prompted for a password, use the Canvas default postgres password (`sekret` at the time of writing),

Next, grant the user read-only privileges to all tables in each database.

For each database (development, test, etc.) run the following, substituting the correct name for `<database name>`:
```bash
docker compose run --rm web psql -h postgres -U postgres -d <database name> -c 'GRANT SELECT ON ALL TABLES IN SCHEMA public TO canvas_read_only'
```

## 4. That's it!
To validate that the new user has read-only access try activating the read-only DB configuration (using GuardRail) and try creating a row in a Canvas Rails console:
```ruby
=> GuardRail.activate(:secondary) { DeveloperKey.create! }
```

This should result in the following error:

```
ActiveRecord::StatementInvalid (PG::InsufficientPrivilege: ERROR:  permission denied for table developer_keys)
```

Activating the primary DB configuration, however, should allow inserting the new row:
```ruby
=> GuardRail.activate(:primary) { DeveloperKey.create! }
...
SQL  (1.3ms)  COMMIT  [development:1 primary]
```

---