### Migration Guides/Hub To Public Pypi # Migration guide: Hub install → public PyPI As of Guardrails **0.11.0**, Guardrails-AI-owned validators have moved from the private Guardrails registry (`pypi.guardrailsai.com`, installed via `guardrails hub install`) to **public PyPI**. Each validator is now published as a `guardrails-ai-` distribution and imported from the PEP 420 `guardrails_ai` namespace. ## What changed | Before | After | |---|---| | `guardrails hub install hub://guardrails/detect_pii` | `pip install guardrails-ai-detect-pii` | | `from guardrails.hub import DetectPII` | `from guardrails_ai.detect_pii import DetectPII` | | Private registry `pypi.guardrailsai.com` (token required) | Public PyPI (no token) | | Dist name `guardrails-grhub-detect-pii` | `guardrails-ai-detect-pii` | The registered validator name is **unchanged** — `@register_validator(name="guardrails/detect_pii")` still resolves, so `Guard().use(...)` and existing guards keep working. ## Deprecation timeline (back-compat is preserved for now) Nothing breaks today. During the deprecation window: - **`guardrails hub install hub://guardrails/`** still works — it now installs `guardrails-ai-` from public PyPI and prints a `DeprecationWarning` pointing at the equivalent `pip install` command. The same warning is emitted by `uninstall`, `list`, `submit`, and `create-validator`. - **`from guardrails.hub import X`** still works — if an export isn't found in the local hub registry, it falls back to scanning installed `guardrails_ai.*` packages and emits a one-time `DeprecationWarning` recommending the direct `from guardrails_ai. import X` import. Both the `guardrails hub` CLI and the `guardrails.hub` import shim are scheduled for **removal in the next major release**. Update install commands and imports to the public-PyPI form at your convenience before then. ## How to migrate 1. Replace `guardrails hub install hub://guardrails/` with `pip install guardrails-ai-` (underscores in `` become dashes in the dist name). 2. Replace `from guardrails.hub import ` with `from guardrails_ai. import `. 3. For validators that ship local models, run the post-install step after installing — see the package's README on PyPI. Browse the full catalog of validators and their new package names in the [`guardrails-hub` repo's `VALIDATORS.md`](https://github.com/guardrails-ai/guardrails-hub). --- ## Related: RAIL (`.rail`) removal — next major The `.rail` XML spec format and the `Guard.for_rail` / `Guard.for_rail_string` APIs are being removed in the next major release (this also drops the `lxml` dependency). Migrate RAIL-defined guards to Pydantic or JSON-schema guards: - `Guard.for_rail("spec.rail")` / `Guard.for_rail_string(...)` → `Guard.for_pydantic(MyModel)` or a JSON-schema-based guard plus plain `messages`. - The `guardrails validate` CLI command (which parsed `.rail`) is removed. Validator quality criteria expressed in RAIL map directly onto `Guard().use(, ...)` calls. --- ### How To Guides/Output # `Output` Element The `...` element of a `RAIL` spec is used to give precise specification of the expected output of the LLM. It specifies 1. the structure of the expected output (e.g. JSON), 2. the type of each field, 3. the quality criteria for each field to be considered valid (e.g. generated text should be bias-free, generated code should be bug-free), and 4. the corrective action to take in case the quality criteria is not met (e.g. reask the question to the LLM, filter offending values, progrmatically fix, etc.) Example: === "JSON RAIL Spec" ```xml ... ``` === "Output JSON" ```json { "text": "string output", "score": 0.0, "metadata": { "key_1": "string", ... } } ``` === "String RAIL Spec" ```xml ``` === "Output String" ``` string output ``` ## ⚡ Specifying output structure You can combine `RAIL` elements to create an arbitrarily complex output structure. ### Flat JSON output === "RAIL Spec" ```xml ``` === "Output JSON" ```json { "some_key": "string", "some_other_key": 0 } ``` ### JSON output with objects `object` elements can be used to specify a JSON object, which is a collection of key-value pairs. - A child of an `object` element represents a key in the JSON object. The child element can be any RAIL element, including another `list` or `object` elements. The value of the key is generated by the LLM based on the info provided by the child element. - An object element can have multiple children, each of which can be any RAIL element, including another `list` or `object` elements. - Formatters can be applied to the child elements of an object element. For example, if the child element is a `string` element, the `format` attribute can be used to specify the quality criteria for the strings in the list. === "RAIL Spec" ```xml ``` === "Output JSON" ```json { "some_object": { "some_str_key": "SOME STRING", "some_other_key": 0 } } ``` In the above example, `"SOME STRING"` is the value for the `some_str_key` key, and is generated based on the name, description and quality criteria provided by the `` element. !!! note The `object` element doesn't *need* to have children. If child elements are not provided, the LLM will automatically generate keys and values for the object based on the `name`, `description` and `format` attributes of the `object` element. Providing child elements is useful when you want to specify the keys and values that the LLM should generate. ### JSON output with lists `list` elements can be used to specify a list of values. - Currently, a list element can only contain a single child element. This means that a list can only contain a single type of data. For example, a list can only contain strings, or a list can only contain integers, but a list cannot contain both strings and integers. - This child element can be any RAIL element, including another `list` or `object` elements. - The child of a list element doesn't need to have a `name` attribute, since items in a list don't have names. - Formatters can be applied to the child element of a list element. For example, if the child element is a `string` element, the `format` attribute can be used to specify the quality criteria for the strings in the list. === "RAIL Spec" ```xml ``` === "Output JSON" ```json { "some_list": [ "STRING 1", "STRING 2" ] } ``` !!! note The `list` element doesn't *need* to have a child element. If a child element is not provided, the LLM will automatically generate values for the list based on the `name`, `description` and `format` attributes of the `list` element. Providing a child element is useful when you want to have more control over the values that the LLM should generate. ### String output Generate simple strings by specifying `type="string"` in the `` element. All the formatters supported by the `string` element can be used to specify the quality criteria for the generated string. === "RAIL Spec" ```xml ``` === "Output" ``` string output ``` ## 🏷️ `RAIL` Elements At the heart of the `RAIL` specification is the use of elements. Each element's tag represents a type of data. For example, in the element ``, the tag represents a string, the `` elements represents an integer, the `` element represents an object, etc. !!! note The tag of RAIL element is the same as the "type" of the data it represents. E.g. `` element will generate a string, `` element will generate an integer, etc. ### Supported types Guardrails supports many data types, including:, `string`, `integer`, `float`, `bool`, `list`, `object`, `url`, `email` and many more. Check out the [RAIL Data Types](/docs/api_reference/types#railtypes) page for a list of supported data types. #### Scalar vs Non-scalar types Guardrails supports two types of data types: scalar and non-scalar. | Scalar | Non Scalar | |-------------------------------------------------------------------------|--------------------------------------------------------------------------------------| | Scalar types are void elements, and can't have any child elements. | Non-scalar types can be non-void, and can have closing tags and child elements. | | Syntax: ``` ``` | Syntax: ``` ```| | Examples: `string`, `integer`, `float`, `bool`, `url`, `email`, etc. | Examples: `list` and `object` are the only non-scalar types supported by Guardrails. | ### Supported attributes Each element can have attributes that specify additional information about the data, such as: 1. `name` attribute that specifies the name of the field. This will be the key in the output JSON. E.g. === "RAIL Spec" ```xml ``` === "Output JSON" ```json { "some_key": "..." } ``` 2. `description` attribute that specifies the description of the field. This is similar to a prompt that will be provided to the LLM. It can contain more context to help the LLM generate the correct output. 3. (Coming soon!) `required` attribute that specifies whether the field is required or not. If the field is required, the LLM will be asked to generate the field until it is generated correctly. If the field is not required, the LLM will not be asked to generate the field if it is not generated correctly. 4. `validators` attribute that specifies the quality criteria that the field should respect. The `format` attribute can contain multiple quality criteria separated by a colon (`;`). For example, `guardrails/uppercase; guardrails/two_words`. 5. `on-fail-{quality-criteria}` attribute that specifies the corrective action to take in case the quality criteria is not met. For example, `on-fail-two-words="reask"` specifies that if the field does not have two words, the LLM should be asked to re-generate the field. E.g., === "RAIL Spec" ```xml ``` === "Output JSON" ```json { "some_key": "SOME STRING" } ``` ## 🍀 Specifying quality criteria The `format` attribute allows specifying the quality criteria for each field in the expected output. The `format` attribute can contain multiple quality criteria separated by a colon (`;`). For example, ```xml ``` The above example specifies that the `text` field should be a string with two words and the text should be returned in upper case. ### Quality criteria under the hood Under the hood, the `format` attribute is parsed into a list of quality criteria. Each quality criteria is backed by a `Validator` class that checks if the generated output meets the quality criteria. For example, the `two-words` quality criteria is backed by the `TwoWords` class, which checks if the generated output has two words. Each quality criteria is then checked against the generated output. If the quality criteria is not met, the corrective action specified by the `on-fail-{quality-criteria}` attribute is taken. ### Supported criteria - Each quality critera is relevant to a specific data type. For example, the `two-words` quality criteria is only relevant to strings, and the `positive` quality criteria is only relevant to integers and floats. - To see the full list of supported quality criteria, check out the [Validation](/docs/api_reference/validator) page. ## 🛠️ Specifying corrective actions The `on-fail-{quality-criteria}` attribute allows specifying the corrective action that should be taken if the quality criteria is not met. The corrective action can be one of the following: | Action | Behavior | |-----------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `reask` | Reask the LLM to generate an output that meets the quality criteria. The prompt used for reasking contains information about which quality criteria failed, which is auto-generated by the validator. | | `fix` | Programmatically fix the generated output to meet the quality criteria. E.g. for the formatter `two-words`, the programatic `fix` simply takes the first 2 words of the generated string. | | `filter` | Filter the incorrect value. This only filters the field that fails, and will return the rest of the generated output. | | `refrain` | Refrain from returning an output. If a formatter has the corrective action refrain, then on failure there will be a `None` output returned instead of the JSON. | | `noop` | Do nothing. The failure will still be recorded in the logs, but no corrective action will be taken. | | `exception` | Raise an exception when validation fails. | | `fix_reask` | First, fix the generated output deterministically, and then rerun validation with the deterministically fixed output. If validation fails, then perform reasking. | ## 🚒 Adding compiled `output` element to prompt In order to generate the correct LLM output, the `output` schema needs to be compiled and added to the prompt. This is handled automatically by the `Guardrails` library. The `output` element can be compiled into different formats to be used in the prompt. Currently, only a passthrough compilation into `XML` is supported, but in the future we will support additional compilation formats like `TypeScript`. ### Passthrough (`XML`) compilation By default, the `output` element will be compiled into `XML` and added to the prompt. Compilation into `XML` involves removing any `on-fail-{quality-criteria}` attributes, and adding the `output` element to the prompt. An example of the compiled `output` element: === "RAIL Spec" ```xml ``` === "Compiled XML added to prompt" ```xml ``` ### `TypeScript` Compilation Coming soon! ## ❓ Unsupported tags and attributes - By default, Guardrails will not throw an error if you add an unsupported type, attribute or quality criteria. Instead, it will treat the unsupported type as a string, and will not perform any quality checks on the field. Often, LLMs will generate a string for an unsupported type, so this behavior is useful. - Unsupported tags and attributes will still be included in the output schema definition that is appended to the prompt. - This behavior can be changed by setting the `strict` attribute of the `` element to `true`. If `strict` is set to `true`, Guardrails will throw an error if you add an unsupported type, attribute or quality criteria. ```xml ``` This will throw an error: ```bash ❌ Error: Unsupported type: unsupported-type ``` --- ### How To Guides/Rail # Use Guardrails via RAIL ## What is `RAIL`? `.RAIL` is a dialect of XML. It stands for "**R**eliable **AI** markup **L**anguage", and it can be used to define: 1. The structure of the expected outcome of the LLM. (E.g. JSON) 2. The type of each field in the expected outcome. (E.g. string, integer, list, object) 3. The quality criteria for the expected outcome to be considered valid. (E.g. generated text should be bias-free, generated code should be bug-free) 4. The corrective action to take in case the quality criteria is not met. (E.g. reask the question, filter the LLM, progrmatically fix, etc.)
Expand to see an example of a RAIL specification. ```xml Given the following document, answer the following questions. If the answer doesn't exist in the document, enter 'None'. ${document} ${gr.xml_prefix_prompt} ${output_schema} ${gr.json_suffix_prompt} ```
## Why `RAIL`? 1. **Language agnostic:** `RAIL` Specifications can be enforced in any language. 2. **Simple and familiar:** `RAIL` should be familiar to anyone familiar with HTML, and should be easy to learn. 3. **Validation and correction**: `RAIL` can be used to define quality criteria for the expected output, and corrective actions to take in case the quality criteria is not met. 4. **Can define complex structures:** `RAIL` can be used to define arbitrarily complex structures, such as nested lists, nested objects, etc. 5. **Code assistance**: In the future, we plan to support code completion and IntelliSense for `RAIL` specifications, which will make it very easy to write `RAIL` specifications. **Design inspiration** - HTML, CSS and Javascript: `RAIL` spec is a dialect of XML, and so is similar to HTML. Specifying quality criteria is done via the `format` attribute, which is similar to CSS `style` tags. Corrective actions are specified via the `on-fail-*` attributes, which is similar to Javascript event handlers. - OpenAPI as an open standard for creating machine-readable RESTful APIs. ## Components of an `RAIL` Specification The `RAIL` specification contains 2 main components: 1. `Output`: Contains information about the expected output of the LLM. It contains the spec for the overall structure of the LLM output, type info for each field, and the quality criteria for each field and the corrective action to be taken in case quality criteria is not met. This is the main component of the `RAIL` specification, which enforces the guarantees that the LLM should provide. Check out the [RAIL Output](#-specifying-output-structure) page for more details, including the full specifcation of how to create complex output schemas. 2. `Messages`: Prompt template, and contains the high level instructions that are sent to the LLM. Check out the [RAIL Messages](#-components-of-a-prompt-element) page for more details. Let's see an example of an `RAIL` specification in action: ```xml ... ... ``` 1. The `output` element contains the structure of the expected output of the LLM. It contains the spec for the overall structure of the LLM output, type info for each field, and the quality criteria for each field and the corrective action to be taken in case quality criteria is not met. 2. The `messages` element contains the high level instructions that are sent to the LLM. Check out the [RAIL Prompt](#-components-of-a-prompt-element) page for more details. ## 📖 How to use `RAIL` in Guardrails? After creating a `RAIL` specification, you can use it to get corrected output from LLMs by wrapping your LLM API call with a `Guard` module. Here's an example of doing that: ```python import guardrails as gd # Create a Guard object guard = gd.Guard.for_rail('path/to/rail/spec.xml') # (1)! _, validated_output, *rest = guard( openai.Completion.create, # (2)! **prompt_args, *args, **kwargs ) ``` 1. A `Guard` object is created from a `RAIL` specification. This object manages the validation and correction of the output of the LLM, as well as the prompt that is sent to the LLM. 2. Wrap the LLM API call (`openai.Completion.create`) with the `Guard` object, and add any additional arguments that you want to pass to the LLM API call. Instead of returning the raw text object, the `Guard` object will return a JSON object that is validated and corrected according to the `RAIL` specification. # `Messages` Element The `` element contains instructions and the query that describes the high level task. ## 📚 Components of a Prompt Element In addition to the high level task description, messages also contains the following: | Component | Syntax | Description | |-------------------|--------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Variables | `${variable_name}` | These are provided by the user at runtime, and substituted in the prompt. | | Output Schema | `${output_schema}` | This is the schema of the expected output, and is compiled based on the `output` element. For more information on how the output schema is compiled for the prompt, check out [`output` element compilation](/docs/how_to_guides/output#-adding-compiled-output-element-to-prompt). | | Prompt Primitives | `${gr.prompt_primitive_name}` | These are pre-constructed prompts that are useful for common tasks. E.g., some primitives may contain information that helps the LLM understand the output schema better. To see the full list of prompt primitives, check out [`guardrails/constants.xml`](https://github.com/guardrails-ai/guardrails/blob/main/guardrails/constants.xml). | ```xml You are a helpful assistant only capable of communicating with valid JSON, and no other text. Given the following document, answer the following questions. If the answer doesn't exist in the document, enter 'None'. ${document} ${gr.xml_prefix_prompt} ${output_schema} ${gr.json_suffix_prompt} ``` 1. The instructions element contains high level background information for the LLM containing textual context and constraints. 2. The prompt contains high level task information. 3. The variable `${document}` is provided by the user at runtime. 4. `${gr.xml_prefix_prompt}` is a prompt primitive provided by guardrails. It is equivalent to typing the following lines in the prompt: `Given below is XML that describes the information to extract from this document and the tags to extract it into.` 5. `${output_schema}` is the output schema and contains information about , which is compiled based on the `output` element. 6. `${gr.json_suffix_prompt}` is a prompt primitive provided by guardrails. It is equivalent to typing the following lines in the prompt: ``` ONLY return a valid JSON object (no other text is necessary). The JSON MUST conform to the XML format, including any types and format requests e.g. requests for lists, objects and specific types. Be correct and concise. If you are unsure anywhere, enter `null`. ``` The messages element is made up of message elements with role attributes. Messages with the role system are intended to be system level prompt. Messages with the role assistant are intended to be messages from the llm to be repassed to itself as additional context and history. Messages with role user are input from the user and also convey history of the conversation. # `Output` Element The `...` element of a `RAIL` spec is used to give precise specification of the expected output of the LLM. It specifies 1. the structure of the expected output (e.g. JSON), 2. the type of each field, 3. the quality criteria for each field to be considered valid (e.g. generated text should be bias-free, generated code should be bug-free), and 4. the corrective action to take in case the quality criteria is not met (e.g. reask the question to the LLM, filter offending values, progrmatically fix, etc.) Example: === "JSON RAIL Spec" ```xml ... ``` === "Output JSON" ```json { "text": "string output", "score": 0.0, "metadata": { "key_1": "string", ... } } ``` === "String RAIL Spec" ```xml ``` === "Output String" ``` string output ``` ## ⚡ Specifying output structure You can combine `RAIL` elements to create an arbitrarily complex output structure. ### Flat JSON output === "RAIL Spec" ```xml ``` === "Output JSON" ```json { "some_key": "string", "some_other_key": 0 } ``` ### JSON output with objects `object` elements can be used to specify a JSON object, which is a collection of key-value pairs. - A child of an `object` element represents a key in the JSON object. The child element can be any RAIL element, including another `list` or `object` elements. The value of the key is generated by the LLM based on the info provided by the child element. - An object element can have multiple children, each of which can be any RAIL element, including another `list` or `object` elements. - Formatters can be applied to the child elements of an object element. For example, if the child element is a `string` element, the `format` attribute can be used to specify the quality criteria for the strings in the list. === "RAIL Spec" ```xml ``` === "Output JSON" ```json { "some_object": { "some_str_key": "SOME STRING", "some_other_key": 0 } } ``` In the above example, `"SOME STRING"` is the value for the `some_str_key` key, and is generated based on the name, description and quality criteria provided by the `` element. !!! note The `object` element doesn't *need* to have children. If child elements are not provided, the LLM will automatically generate keys and values for the object based on the `name`, `description` and `format` attributes of the `object` element. Providing child elements is useful when you want to specify the keys and values that the LLM should generate. ### JSON output with lists `list` elements can be used to specify a list of values. - Currently, a list element can only contain a single child element. This means that a list can only contain a single type of data. For example, a list can only contain strings, or a list can only contain integers, but a list cannot contain both strings and integers. - This child element can be any RAIL element, including another `list` or `object` elements. - The child of a list element doesn't need to have a `name` attribute, since items in a list don't have names. - Formatters can be applied to the child element of a list element. For example, if the child element is a `string` element, the `validators` attribute can be used to specify the quality criteria for the strings in the list. === "RAIL Spec" ```xml ``` === "Output JSON" ```json { "some_list": [ "STRING 1", "STRING 2" ] } ``` !!! note The `list` element doesn't *need* to have a child element. If a child element is not provided, the LLM will automatically generate values for the list based on the `name`, `description` and `format` attributes of the `list` element. Providing a child element is useful when you want to have more control over the values that the LLM should generate. ### String output Generate simple strings by specifying `type="string"` in the `` element. All the formatters supported by the `string` element can be used to specify the quality criteria for the generated string. === "RAIL Spec" ```xml ``` === "Output" ``` string output ``` ## `RAIL` Elements At the heart of the `RAIL` specification is the use of elements. Each element's tag represents a type of data. For example, in the element ``, the tag represents a string, the `` elements represents an integer, the `` element represents an object, etc. !!! note The tag of RAIL element is the same as the "type" of the data it represents. E.g. `` element will generate a string, `` element will generate an integer, etc. ### Supported types Guardrails supports all [JSON](https://www.w3schools.com/js/js_json_datatypes.asp) and pydantic compatible datatypes. #### Scalar vs Non-scalar types Guardrails supports two types of data types: scalar and non-scalar. | Scalar | Non Scalar | |-------------------------------------------------------------------------|--------------------------------------------------------------------------------------| | Scalar types are void elements, and can't have any child elements. | Non-scalar types can be non-void, and can have closing tags and child elements. | | Syntax: ``` ``` | Syntax: ``` ```| | Examples: `string`, `integer`, `float`, `bool`, `url`, `email`, etc. | Examples: `list` and `object` are the only non-scalar types supported by Guardrails. | ### Supported attributes Each element can have attributes that specify additional information about the data, such as: 1. `name` attribute that specifies the name of the field. This will be the key in the output JSON. E.g. === "RAIL Spec" ```xml ``` === "Output JSON" ```json { "some_key": "..." } ``` 2. `description` attribute that specifies the description of the field. This is similar to a prompt that will be provided to the LLM. It can contain more context to help the LLM generate the correct output. 3. (Coming soon!) `required` attribute that specifies whether the field is required or not. If the field is required, the LLM will be asked to generate the field until it is generated correctly. If the field is not required, the LLM will not be asked to generate the field if it is not generated correctly. 4. `validators` attribute that specifies the quality criteria that the field should respect. The `validators` attribute can contain multiple quality criteria separated by a colon (`;`). For example, `guardrails/uppercase; guardrails/two_words`. 5. `on-fail-{quality-criteria}` attribute that specifies the corrective action to take in case the quality criteria is not met. For example, `on-fail-two-words="reask"` specifies that if the field does not have two words, the LLM should be asked to re-generate the field. E.g., === "RAIL Spec" ```xml ``` === "Output JSON" ```json { "some_key": "SOME STRING" } ``` ## Specifying quality criteria The `format` attribute allows specifying the quality criteria for each field in the expected output. The `format` attribute can contain multiple quality criteria separated by a colon (`;`). For example, ```xml ``` The above example specifies that the `text` field should be a string with two words and the text should be returned in upper case. ### Quality criteria under the hood Under the hood, the `format` attribute is parsed into a list of quality criteria. Each quality criteria is backed by a `Validator` class that checks if the generated output meets the quality criteria. For example, the `two-words` quality criteria is backed by the `TwoWords` class, which checks if the generated output has two words. Each quality criteria is then checked against the generated output. If the quality criteria is not met, the corrective action specified by the `on-fail-{quality-criteria}` attribute is taken. ### Supported criteria - Each quality critera is relevant to a specific data type. For example, the `two-words` quality criteria is only relevant to strings, and the `positive` quality criteria is only relevant to integers and floats. - To see the full list of supported quality criteria, check out the [Validation](https://guardrailsai.com/guardrails/docs/concepts/validators) page. ## 🛠️ Specifying corrective actions The `on-fail-{quality-criteria}` attribute allows specifying the corrective action that should be taken if the quality criteria is not met. The corrective action can be one of the following: | Action | Behavior | |-----------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `reask` | Reask the LLM to generate an output that meets the quality criteria. The prompt used for reasking contains information about which quality criteria failed, which is auto-generated by the validator. | | `fix` | Programmatically fix the generated output to meet the quality criteria. E.g. for the formatter `two-words`, the programatic `fix` simply takes the first 2 words of the generated string. | | `filter` | Filter the incorrect value. This only filters the field that fails, and will return the rest of the generated output. | | `refrain` | Refrain from returning an output. If a formatter has the corrective action refrain, then on failure there will be a `None` output returned instead of the JSON. | | `noop` | Do nothing. The failure will still be recorded in the logs, but no corrective action will be taken. | | `exception` | Raise an exception when validation fails. | | `fix_reask` | First, fix the generated output deterministically, and then rerun validation with the deterministically fixed output. If validation fails, then perform reasking. | ## Adding compiled `output` element to prompt In order to generate the correct LLM output, the `output` schema needs to be compiled and added to the prompt. This is handled automatically by the `Guardrails` library. The `output` element can be compiled into different formats to be used in the prompt. Currently, only a passthrough compilation into `XML` is supported, but in the future we will support additional compilation formats like `TypeScript`. ### Passthrough (`XML`) compilation By default, the `output` element will be compiled into `XML` and added to the prompt. Compilation into `XML` involves removing any `on-fail-{quality-criteria}` attributes, and adding the `output` element to the prompt. An example of the compiled `output` element: === "RAIL Spec" ```xml ``` === "Compiled XML added to prompt" ```xml ``` ### `TypeScript` Compilation Coming soon! ## Unsupported tags and attributes - By default, Guardrails will not throw an error if you add an unsupported type, attribute or quality criteria. Instead, it will treat the unsupported type as a string, and will not perform any quality checks on the field. Often, LLMs will generate a string for an unsupported type, so this behavior is useful. - Unsupported tags and attributes will still be included in the output schema definition that is appended to the prompt. - This behavior can be changed by setting the `strict` attribute of the `` element to `true`. If `strict` is set to `true`, Guardrails will throw an error if you add an unsupported type, attribute or quality criteria. ```xml ``` This will throw an error: ```bash ❌ Error: Unsupported type: unsupported-type ``` --- ### Examples/Data/Article1 Section. 1. All legislative Powers herein granted shall be vested in a Congress of the United States, which shall consist of a Senate and House of Representatives. Section. 2. The House of Representatives shall be composed of Members chosen every second Year by the People of the several States, and the Electors in each State shall have the Qualifications requisite for Electors of the most numerous Branch of the State Legislature. No Person shall be a Representative who shall not have attained to the Age of twenty five Years, and been seven Years a Citizen of the United States, and who shall not, when elected, be an Inhabitant of that State in which he shall be chosen. Representatives and direct Taxes shall be apportioned among the several States which may be included within this Union, according to their respective Numbers, which shall be determined by adding to the whole Number of free Persons, including those bound to Service for a Term of Years, and excluding Indians not taxed, three fifths of all other Persons. The actual Enumeration shall be made within three Years after the first Meeting of the Congress of the United States, and within every subsequent Term of ten Years, in such Manner as they shall by Law direct. The Number of Representatives shall not exceed one for every thirty Thousand, but each State shall have at Least one Representative; and until such enumeration shall be made, the State of New Hampshire shall be entitled to chuse three, Massachusetts eight, Rhode-Island and Providence Plantations one, Connecticut five, New-York six, New Jersey four, Pennsylvania eight, Delaware one, Maryland six, Virginia ten, North Carolina five, South Carolina five, and Georgia three. When vacancies happen in the Representation from any State, the Executive Authority thereof shall issue Writs of Election to fill such Vacancies. The House of Representatives shall chuse their Speaker and other Officers; and shall have the sole Power of Impeachment. --- ### Examples/Data/Twain The Humorous Story an American Development.— Its Difference from Comic and Witty Stories. DO not claim that I can tell a story as it ought to be told. I only claim to know how a story ought to be told, for I have been almost daily in the company of the most expert story-tellers for many years. There are several kinds of stories, but only one difficult kind —the humorous. I will talk mainly about that one. The humorous story is American, the comic story is English, the witty story is French. The humorous story depends for its effect upon the manner of the telling; the comic story and the witty story upon the matter. The humorous story may be spun out to great length, and may wander around as much as it pleases, and arrive nowhere in particular; but the comic and witty stories must be brief and end with a point. The humorous story bubbles gently along, the others burst. The humorous story is strictly a work of art — high and delicate art — and only an artist can tell it; but no art is necessary in telling the comic and the witty story; anybody can do it. The art of telling a humorous story — understand, I mean by word of mouth, not print — was created in America, and has remained at home. The humorous story is told gravely; the teller does his best to conceal the fact that he even dimly suspects that there is anything funny about it; but the teller of the comic story tells you beforehand that it is one of the funniest things he has ever heard, then tells it with eager delight, and is the first person to laugh when he gets through. And sometimes, if he has had good success, he is so glad and happy that he will repeat the ‘‘ nub’’ of it and slance around from face to face, collecting applause, and then repeat it again. It is a pathetic thing to see. Very often, of course, the rambling and disjointed humorous story finishes with a nub, point, snapper, or whatever you like to call it. Then the listener must be alert, for in many cases the teller will divert attention from that nub by dropping it in a carefully. casual and indifferent way, with the pretence that he does not know it is a nub. Artemus Ward used that trick a good deal; then when the belated audience presently caught the joke he would look up with innocent surprise, as if wondering what they had found to laugh at. Dan Setchell used it before him, Nye and Riley and others use it to-day. But the teller of the comic story does not slur the nub; he shouts it at you—every time. And when he prints it, in England, France, Germany, and Italy, he italicizes it, puts some whooping exclamation-points after it, and sometimes explains it in a parenthesis. All of which is very depressing, and makes one want to renounce joking and lead a better life. - Mark Twain --- ### Api Reference/Actions # Actions ## ReAsk ```python class ReAsk(IReask) ``` Base class for ReAsk objects. **Attributes**: - `incorrect_value` _Any_ - The value that failed validation. - `fail_results` _List[FailResult]_ - The results of the failed validations. ## FieldReAsk ```python class FieldReAsk(ReAsk) ``` An implementation of ReAsk that is used to reask for a specific field. Inherits from ReAsk. **Attributes**: - `path` _Optional[List[Any]]_ - a list of keys that designated the path to the field that failed validation. ## SkeletonReAsk ```python class SkeletonReAsk(ReAsk) ``` An implementation of ReAsk that is used to reask for structured data when the response does not match the expected schema. Inherits from ReAsk. ## NonParseableReAsk ```python class NonParseableReAsk(ReAsk) ``` An implementation of ReAsk that is used to reask for structured data when the response is not parseable as JSON. Inherits from ReAsk. ## Filter ```python class Filter() ``` #### apply\_filters ```python def apply_filters(value: Any) -> Any ``` Recursively filter out any values that are instances of Filter. ## Refrain ```python class Refrain() ``` #### apply\_refrain ```python def apply_refrain(value: Any, output_type: OutputTypes) -> Any ``` Recursively check for any values that are instances of Refrain. If found, return an empty value of the appropriate type. --- ### Api Reference/Errors # Errors ## ValidationError ```python class ValidationError(Exception) ``` Top level validation error. This is thrown from the validation engine when a Validator has on_fail=OnFailActions.EXCEPTION set and validation fails. Inherits from Exception. --- ### Api Reference/Formatters # Formatters ## BaseFormatter ```python class BaseFormatter(ABC) ``` A Formatter takes an LLM Callable and wraps the method into an abstract callable. Used to perform manipulations of the input or the output, like JSON constrained- decoding. ## JsonFormatter ```python class JsonFormatter(BaseFormatter) ``` A formatter that uses Jsonformer to ensure the shape of structured data for Hugging Face models. --- ### Api Reference/Generics And Base Classes # Generics And Base Classes ## ArbitraryModel ```python class ArbitraryModel(BaseModel) ``` Empty Pydantic model with a config that allows arbitrary types. ## Stack ```python class Stack(List[T]) ``` #### empty ```python def empty() -> bool ``` Tests if this stack is empty. #### peek ```python def peek() -> Optional[T] ``` Looks at the object at the top (last/most recently added) of this stack without removing it from the stack. #### pop ```python def pop() -> Optional[T] ``` Removes the object at the top of this stack and returns that object as the value of this function. #### push ```python def push(item: T) -> None ``` Pushes an item onto the top of this stack. Proxy of List.append Limits Stack Length to _max_length entries #### search ```python def search(x: T) -> Optional[int] ``` Returns the 0-based position of the last item whose value is equal to x on this stack. We deviate from the typical 1-based position used by Stack classes (i.e. Java) because most python users (and developers in general) are accustomed to 0-based indexing. #### at ```python def at(index: int, default: Optional[T] = None) -> Optional[T] ``` Returns the item located at the index. If the index does not exist in the stack (Overflow or Underflow), None is returned instead. #### copy ```python def copy() -> "Stack[T]" ``` Returns a copy of the current Stack. #### first ```python @property def first() -> Optional[T] ``` Returns the first item of the stack without removing it. Same as Stack.bottom. #### last ```python @property def last() -> Optional[T] ``` Returns the last item of the stack without removing it. Same as Stack.top. #### bottom ```python @property def bottom() -> Optional[T] ``` Returns the item on the bottom of the stack without removing it. Same as Stack.first. #### top ```python @property def top() -> Optional[T] ``` Returns the item on the top of the stack without removing it. Same as Stack.last. #### length ```python @property def length() -> int ``` Returns the number of items in the Stack. --- ### Api Reference/Guards # Guards ## Guard ```python class Guard(IGuard, Generic[OT]) ``` The Guard class. This class is the main entry point for using Guardrails. It can be initialized by one of the following patterns: - `Guard().use(...)` - `Guard.for_string(...)` - `Guard.for_pydantic(...)` - `Guard.for_rail(...)` - `Guard.for_rail_string(...)` The `__call__` method functions as a wrapper around LLM APIs. It takes in an LLM API, and optional prompt parameters, and returns a ValidationOutcome class that contains the raw output from the LLM, the validated output, as well as other helpful information. #### \_\_init\_\_ ```python def __init__(*, id: Optional[str] = None, name: Optional[str] = None, description: Optional[str] = None, validators: Optional[List[ValidatorReference]] = None, output_schema: Optional[Dict[str, Any]] = None, base_url: Optional[str] = None, api_key: Optional[str] = None, history_max_length: Optional[int] = None, use_server: Optional[bool] = None) ``` Initialize the Guard with serialized validator references and an output schema. Output schema must be a valid JSON Schema. #### configure ```python def configure(*, num_reasks: Optional[int] = None, allow_metrics_collection: Optional[bool] = None) ``` Configure the Guard. **Arguments**: - `num_reasks` _int, optional_ - The max times to re-ask the LLM if validation fails. Defaults to None. - `allow_metrics_collection` _bool, optional_ - Whether to allow Guardrails to collect anonymous metrics. Defaults to None, and falls back to waht is set via the `guardrails configure` command. #### for\_rail ```python @classmethod def for_rail(cls, rail_file: str, *, name: Optional[str] = None, description: Optional[str] = None) ``` Create a Guard using a `.rail` file to specify the output schema, prompt, etc. **Arguments**: - `rail_file` - The path to the `.rail` file. - `name` _str, optional_ - A unique name for this Guard. Defaults to `gr-` + the object id. - `description` _str, optional_ - A description for this Guard. Defaults to None. **Returns**: An instance of the `Guard` class. #### for\_rail\_string ```python @classmethod def for_rail_string(cls, rail_string: str, *, name: Optional[str] = None, description: Optional[str] = None) ``` Create a Guard using a `.rail` string to specify the output schema, prompt, etc.. **Arguments**: - `rail_string` - The `.rail` string. - `name` _str, optional_ - A unique name for this Guard. Defaults to `gr-` + the object id. - `description` _str, optional_ - A description for this Guard. Defaults to None. **Returns**: An instance of the `Guard` class. #### for\_pydantic ```python @classmethod def for_pydantic(cls, output_class: ModelOrListOfModels, *, reask_messages: Optional[List[Dict]] = None, messages: Optional[List[Dict]] = None, name: Optional[str] = None, description: Optional[str] = None, output_formatter: Optional[Union[str, BaseFormatter]] = None) ``` Create a Guard instance using a Pydantic model to specify the output schema. **Arguments**: - `output_class` - (Union[Type[BaseModel], List[Type[BaseModel]]]): The pydantic model that describes the desired structure of the output. - `messages` _List[Dict], optional_ - A list of messages to give to the llm. Defaults to None. - `reask_messages` _List[Dict], optional_ - A list of messages to use during reasks. Defaults to None. - `name` _str, optional_ - A unique name for this Guard. Defaults to `gr-` + the object id. - `description` _str, optional_ - A description for this Guard. Defaults to None. - `output_formatter` _str | Formatter, optional_ - 'none' (default), 'jsonformer', or a Guardrails Formatter. #### for\_string ```python @classmethod def for_string(cls, validators: Sequence[Validator], *, string_description: Optional[str] = None, reask_messages: Optional[List[Dict]] = None, messages: Optional[List[Dict]] = None, name: Optional[str] = None, description: Optional[str] = None) ``` Create a Guard instance for a string response. **Arguments**: - `validators` - (List[Validator]): The list of validators to apply to the string output. - `string_description` _str, optional_ - A description for the string to be generated. Defaults to None. - `messages` _List[Dict], optional_ - A list of messages to pass to llm. Defaults to None. - `reask_messages` _List[Dict], optional_ - A list of messages to use during reasks. Defaults to None. - `name` _str, optional_ - A unique name for this Guard. Defaults to `gr-` + the object id. - `description` _str, optional_ - A description for this Guard. Defaults to None. #### \_\_call\_\_ ```python @trace(name="/guard_call", origin="Guard.__call__") def __call__( llm_api: Optional[Callable] = None, *args, prompt_params: Optional[Dict] = None, num_reasks: Optional[int] = 1, messages: Optional[List[Dict]] = None, metadata: Optional[Dict] = None, full_schema_reask: Optional[bool] = None, **kwargs ) -> Union[ValidationOutcome[OT], Iterator[ValidationOutcome[OT]]] ``` Call the LLM and validate the output. **Arguments**: - `llm_api` - The LLM API to call (e.g. openai.completions.create or openai.Completion.acreate) - `prompt_params` - The parameters to pass to the prompt.format() method. - `num_reasks` - The max times to re-ask the LLM for invalid output. - `messages` - The message history to pass to the LLM. - `metadata` - Metadata to pass to the validators. - `full_schema_reask` - When reasking, whether to regenerate the full schema or just the incorrect values. Defaults to `True` if a base model is provided, `False` otherwise. **Returns**: ValidationOutcome #### parse ```python @trace(name="/guard_call", origin="Guard.parse") def parse(llm_output: str, *args, metadata: Optional[Dict] = None, llm_api: Optional[Callable] = None, num_reasks: Optional[int] = None, prompt_params: Optional[Dict] = None, full_schema_reask: Optional[bool] = None, **kwargs) -> ValidationOutcome[OT] ``` Alternate flow to using Guard where the llm_output is known. **Arguments**: - `llm_output` - The output being parsed and validated. - `metadata` - Metadata to pass to the validators. - `llm_api` - The LLM API to call (e.g. openai.completions.create or openai.Completion.acreate) - `num_reasks` - The max times to re-ask the LLM for invalid output. - `prompt_params` - The parameters to pass to the prompt.format() method. - `full_schema_reask` - When reasking, whether to regenerate the full schema or just the incorrect values. **Returns**: ValidationOutcome #### error\_spans\_in\_output ```python def error_spans_in_output() -> List[ErrorSpan] ``` Get the error spans in the last output. #### use ```python def use(*validator_spread: Validator, validators: List[Validator] = [], on: str = "output") -> "Guard" ``` Applies validators to the property specified in the `on` argument. Calling `Guard.use` with the same `on` value multiple times will overwrite previously configured validators on the specified property. **Arguments**: *validator_spread: One or more validators passed as positional arguments to use. validators: Keyword argument that allows explicitly setting a list of validators to use. on: The property to validate. Valid options include "output", "messages", or a JSON path starting with "$.". Defaults to "output". #### get\_validators ```python def get_validators(on: str) -> List[Validator] ``` The read-only counterpart to `Guard.use`. Retrieves the validators applied to the specified property. **Arguments**: - `on` - The property for which to return configured validators. Valid options include "output", "messages", or a JSON path starting with "$.". #### validate ```python @trace(name="/guard_call", origin="Guard.validate") def validate(llm_output: str, *args, **kwargs) -> ValidationOutcome[OT] ``` #### to\_runnable ```python def to_runnable() -> Runnable ``` Convert a Guard to a LangChain Runnable. #### to\_dict ```python def to_dict() -> Dict[str, Any] ``` #### json\_function\_calling\_tool ```python def json_function_calling_tool( tools: Optional[list] = None) -> List[Dict[str, Any]] ``` Appends an OpenAI tool that specifies the output structure using JSON Schema for chat models. #### from\_dict ```python @classmethod def from_dict(cls, obj: Optional[Dict[str, Any]]) -> Optional["Guard"] ``` ## AsyncGuard ```python class AsyncGuard(Guard, Generic[OT]) ``` The AsyncGuard class. This class one of the main entry point for using Guardrails. It is initialized from one of the following class methods: - `for_rail` - `for_rail_string` - `for_pydantic` - `for_string` The `__call__` method functions as a wrapper around LLM APIs. It takes in an Async LLM API, and optional prompt parameters, and returns the raw output stream from the LLM and the validated output stream. #### \_\_init\_\_ ```python def __init__(*args, **kwargs) ``` #### for\_pydantic ```python @classmethod def for_pydantic(cls, output_class: ModelOrListOfModels, *, messages: Optional[List[Dict]] = None, reask_messages: Optional[List[Dict]] = None, name: Optional[str] = None, description: Optional[str] = None, output_formatter: Optional[Union[str, BaseFormatter]] = None) ``` #### for\_string ```python @classmethod def for_string(cls, validators: Sequence[Validator], *, string_description: Optional[str] = None, messages: Optional[List[Dict]] = None, reask_messages: Optional[List[Dict]] = None, name: Optional[str] = None, description: Optional[str] = None) ``` #### from\_dict ```python @classmethod def from_dict(cls, obj: Optional[Dict[str, Any]]) -> Optional["AsyncGuard"] ``` #### use ```python def use(*validator_spread: Validator, validators: List[Validator] = [], on: str = "output") -> "AsyncGuard" ``` #### \_\_call\_\_ ```python @async_trace(name="/guard_call", origin="AsyncGuard.__call__") async def __call__( llm_api: Optional[Callable[..., Awaitable[Any]]] = None, *args, prompt_params: Optional[Dict] = None, num_reasks: Optional[int] = 1, messages: Optional[List[Dict]] = None, metadata: Optional[Dict] = None, full_schema_reask: Optional[bool] = None, **kwargs ) -> Union[ ValidationOutcome[OT], Awaitable[ValidationOutcome[OT]], AsyncIterator[ValidationOutcome[OT]], ] ``` Call the LLM and validate the output. Pass an async LLM API to return a coroutine. **Arguments**: - `llm_api` - The LLM API to call (e.g. openai.completions.create or openai.chat.completions.create) - `prompt_params` - The parameters to pass to the prompt.format() method. - `num_reasks` - The max times to re-ask the LLM for invalid output. - `messages` - The message history to pass to the LLM. - `metadata` - Metadata to pass to the validators. - `full_schema_reask` - When reasking, whether to regenerate the full schema or just the incorrect values. Defaults to `True` if a base model is provided, `False` otherwise. **Returns**: The raw text output from the LLM and the validated output. #### parse ```python @async_trace(name="/guard_call", origin="AsyncGuard.parse") async def parse(llm_output: str, *args, metadata: Optional[Dict] = None, llm_api: Optional[Callable[..., Awaitable[Any]]] = None, num_reasks: Optional[int] = None, prompt_params: Optional[Dict] = None, full_schema_reask: Optional[bool] = None, **kwargs) -> Awaitable[ValidationOutcome[OT]] ``` Alternate flow to using AsyncGuard where the llm_output is known. **Arguments**: - `llm_output` - The output being parsed and validated. - `metadata` - Metadata to pass to the validators. - `llm_api` - The LLM API to call (e.g. openai.completions.create or openai.Completion.acreate) - `num_reasks` - The max times to re-ask the LLM for invalid output. - `prompt_params` - The parameters to pass to the prompt.format() method. - `full_schema_reask` - When reasking, whether to regenerate the full schema or just the incorrect values. **Returns**: The validated response. This is either a string or a dictionary, determined by the object schema defined in the RAILspec. #### validate ```python @async_trace(name="/guard_call", origin="AsyncGuard.validate") async def validate(llm_output: str, *args, **kwargs) -> Awaitable[ValidationOutcome[OT]] ``` ## ValidationOutcome ```python class ValidationOutcome(IValidationOutcome, ArbitraryModel, Generic[OT]) ``` The final output from a Guard execution. **Attributes**: - `call_id` - The id of the Call that produced this ValidationOutcome. - `raw_llm_output` - The raw, unchanged output from the LLM call. - `validated_output` - The validated, and potentially fixed, output from the LLM call after passing through validation. - `reask` - If validation continuously fails and all allocated reasks are used, this field will contain the final reask that would have been sent to the LLM if additional reasks were available. - `validation_passed` - A boolean to indicate whether or not the LLM output passed validation. If this is False, the validated_output may be invalid. - `error` - If the validation failed, this field will contain the error message #### from\_guard\_history ```python @classmethod def from_guard_history(cls, call: Call) ``` Create a ValidationOutcome from a history Call object. --- ### Api Reference/History And Logs # History and Logs ## Call ```python class Call(ICall, ArbitraryModel) ``` A Call represents a single execution of a Guard. One Call is created each time the user invokes the `Guard.__call__`, `Guard.parse`, or `Guard.validate` method. **Attributes**: - `iterations` _Stack[Iteration]_ - A stack of iterations for the initial validation round and one for each reask that occurs during a Call. - `inputs` _CallInputs_ - The inputs as passed in to `Guard.__call__`, `Guard.parse`, or `Guard.validate` - `exception` _Optional[Exception]_ - The exception that interrupted the Guard execution. #### prompt\_params ```python @property def prompt_params() -> Optional[Dict] ``` The prompt parameters as provided by the user when initializing or calling the Guard. #### messages ```python @property def messages() -> Optional[Union[Messages, list[dict[str, str]]]] ``` The messages as provided by the user when initializing or calling the Guard. #### compiled\_messages ```python @property def compiled_messages() -> Optional[list[dict[str, str]]] ``` The initial compiled messages that were passed to the LLM on the first call. #### reask\_messages ```python @property def reask_messages() -> Stack[Messages] ``` The compiled messages used during reasks. Does not include the initial messages. #### logs ```python @property def logs() -> Stack[str] ``` Returns all logs from all iterations as a stack. #### tokens\_consumed ```python @property def tokens_consumed() -> Optional[int] ``` Returns the total number of tokens consumed during all iterations with this call. #### prompt\_tokens\_consumed ```python @property def prompt_tokens_consumed() -> Optional[int] ``` Returns the total number of prompt tokens consumed during all iterations with this call. #### completion\_tokens\_consumed ```python @property def completion_tokens_consumed() -> Optional[int] ``` Returns the total number of completion tokens consumed during all iterations with this call. #### raw\_outputs ```python @property def raw_outputs() -> Stack[str] ``` The exact outputs from all LLM calls. #### parsed\_outputs ```python @property def parsed_outputs() -> Stack[Union[str, List, Dict]] ``` The outputs from the LLM after undergoing parsing but before validation. #### validation\_response ```python @property def validation_response() -> Optional[Union[str, List, Dict, ReAsk]] ``` The aggregated responses from the validation process across all iterations within the current call. This value could contain ReAsks. #### fixed\_output ```python @property def fixed_output() -> Optional[Union[str, List, Dict]] ``` The cumulative output from the validation process across all current iterations with any automatic fixes applied. Could still contain ReAsks if a fix was not available. #### guarded\_output ```python @property def guarded_output() -> Optional[Union[str, List, Dict]] ``` The complete validated output after all stages of validation are completed. This property contains the aggregate validated output after all validation stages have been completed. Some values in the validated output may be "fixed" values that were corrected during validation. This will only have a value if the Guard is in a passing state OR if the action is no-op. #### reasks ```python @property def reasks() -> Stack[ReAsk] ``` Reasks generated during validation that could not be automatically fixed. These would be incorporated into the prompt for the next LLM call if additional reasks were granted. #### validator\_logs ```python @property def validator_logs() -> Stack[ValidatorLogs] ``` The results of each individual validation performed on the LLM responses during all iterations. #### error ```python @property def error() -> Optional[str] ``` The error message from any exception that raised and interrupted the run. #### failed\_validations ```python @property def failed_validations() -> Stack[ValidatorLogs] ``` The validator logs for any validations that failed during the entirety of the run. #### status ```python @property def status() -> str ``` Returns the cumulative status of the run based on the validity of the final merged output. #### tree ```python @property def tree() -> Tree ``` Returns the tree. ## Iteration ```python class Iteration(IIteration, ArbitraryModel) ``` An Iteration represents a single iteration of the validation loop including a single call to the LLM if applicable. **Attributes**: - `id` _str_ - The unique identifier for the iteration. - `call_id` _str_ - The unique identifier for the Call that this iteration is a part of. - `index` _int_ - The index of this iteration within the Call. - `inputs` _Inputs_ - The inputs for the validation loop. - `outputs` _Outputs_ - The outputs from the validation loop. #### logs ```python @property def logs() -> Stack[str] ``` Returns the logs from this iteration as a stack. #### tokens\_consumed ```python @property def tokens_consumed() -> Optional[int] ``` Returns the total number of tokens consumed during this iteration. #### prompt\_tokens\_consumed ```python @property def prompt_tokens_consumed() -> Optional[int] ``` Returns the number of prompt/input tokens consumed during this iteration. #### completion\_tokens\_consumed ```python @property def completion_tokens_consumed() -> Optional[int] ``` Returns the number of completion/output tokens consumed during this iteration. #### raw\_output ```python @property def raw_output() -> Optional[str] ``` The exact output from the LLM. #### parsed\_output ```python @property def parsed_output() -> Optional[Union[str, List, Dict]] ``` The output from the LLM after undergoing parsing but before validation. #### validation\_response ```python @property def validation_response() -> Optional[Union[ReAsk, str, List, Dict]] ``` The response from a single stage of validation. Validation response is the output of a single stage of validation and could be a combination of valid output and reasks. Note that a Guard may run validation multiple times if reasks occur. To access the final output after all steps of validation are completed, check out `Call.guarded_output`." #### guarded\_output ```python @property def guarded_output() -> Optional[Union[str, List, Dict]] ``` Any valid values after undergoing validation. Some values in the validated output may be "fixed" values that were corrected during validation. This property may be a partial structure if field level reasks occur. #### reasks ```python @property def reasks() -> Sequence[ReAsk] ``` Reasks generated during validation. These would be incorporated into the prompt or the next LLM call. #### validator\_logs ```python @property def validator_logs() -> List[ValidatorLogs] ``` The results of each individual validation performed on the LLM response during this iteration. #### error ```python @property def error() -> Optional[str] ``` The error message from any exception that raised and interrupted this iteration. #### exception ```python @property def exception() -> Optional[Exception] ``` The exception that interrupted this iteration. #### failed\_validations ```python @property def failed_validations() -> List[ValidatorLogs] ``` The validator logs for any validations that failed during this iteration. #### error\_spans\_in\_output ```python @property def error_spans_in_output() -> List[ErrorSpan] ``` The error spans from the LLM response. These indices are relative to the complete LLM output. #### status ```python @property def status() -> str ``` Representation of the end state of this iteration. OneOf: pass, fail, error, not run ## Inputs ```python class Inputs(IInputs, ArbitraryModel) ``` Inputs represent the input data that is passed into the validation loop. **Attributes**: - `llm_api` _Optional[PromptCallableBase]_ - The constructed class for calling the LLM. - `llm_output` _Optional[str]_ - The string output from an external LLM call provided by the user via Guard.parse. - `messages` _Optional[List[Dict]]_ - The message history provided by the user for chat model calls. - `prompt_params` _Optional[Dict]_ - The parameters provided by the user that will be formatted into the final LLM prompt. - `num_reasks` _Optional[int]_ - The total number of reasks allowed; user provided or defaulted. - `metadata` _Optional[Dict[str, Any]]_ - The metadata provided by the user to be used during validation. - `full_schema_reask` _Optional[bool]_ - Whether reasks we performed across the entire schema or at the field level. - `stream` _Optional[bool]_ - Whether or not streaming was used. ## Outputs ```python class Outputs(IOutputs, ArbitraryModel) ``` Outputs represent the data that is output from the validation loop. **Attributes**: - `llm_response_info` _Optional[LLMResponse]_ - Information from the LLM response - `raw_output` _Optional[str]_ - The exact output from the LLM. - `parsed_output` _Optional[Union[str, List, Dict]]_ - The output parsed from the LLM response as it was passed into validation. - `validation_response` _Optional[Union[str, ReAsk, List, Dict]]_ - The response from the validation process. - `guarded_output` _Optional[Union[str, List, Dict]]_ - Any valid values after undergoing validation. Some values may be "fixed" values that were corrected during validation. This property may be a partial structure if field level reasks occur. - `reasks` _List[ReAsk]_ - Information from the validation process used to construct a ReAsk to the LLM on validation failure. Default []. - `validator_logs` _List[ValidatorLogs]_ - The results of each individual validation. Default []. - `error` _Optional[str]_ - The error message from any exception that raised and interrupted the process. - `exception` _Optional[Exception]_ - The exception that interrupted the process. #### failed\_validations ```python @property def failed_validations() -> List[ValidatorLogs] ``` Returns the validator logs for any validation that failed. #### error\_spans\_in\_output ```python @property def error_spans_in_output() -> List[ErrorSpan] ``` The error spans from the LLM response. These indices are relative to the complete LLM output. #### status ```python @property def status() -> str ``` Representation of the end state of the validation run. OneOf: pass, fail, error, not run ## CallInputs ```python class CallInputs(Inputs, ICallInputs, ArbitraryModel) ``` CallInputs represent the input data that is passed into the Guard from the user. Inherits from Inputs with the below overrides and additional attributes. **Attributes**: - `llm_api` _Optional[Callable[[Any], Awaitable[Any]]]_ - The LLM function provided by the user during Guard.__call__ or Guard.parse. - `messages` _Optional[dict[str, str]]_ - The messages as provided by the user. - `args` _List[Any]_ - Additional arguments for the LLM as provided by the user. Default []. - `kwargs` _Dict[str, Any]_ - Additional keyword-arguments for the LLM as provided by the user. Default {}. --- ### Api Reference/Llm Interaction # Helpers for LLM Interactions Class for representing a prompt entry. ## BasePrompt ```python class BasePrompt() ``` Base class for representing an LLM prompt. #### \_\_init\_\_ ```python def __init__(source: str, output_schema: Optional[str] = None, *, xml_output_schema: Optional[str] = None) ``` Initialize and substitute constants in the prompt. #### substitute\_constants ```python def substitute_constants(text: str) -> str ``` Substitute constants in the prompt. #### get\_prompt\_variables ```python def get_prompt_variables() -> List[str] ``` #### format ```python def format(**kwargs) -> "BasePrompt" ``` #### escape ```python def escape() -> str ``` Escape single curly braces into double curly braces. The LLM prompt. ## Prompt ```python class Prompt(BasePrompt) ``` Prompt class. The prompt is passed to the LLM as primary instructions. #### format ```python def format(**kwargs) -> "Prompt" ``` Format the prompt using the given keyword arguments. Instructions to the LLM, to be passed in the prompt. ## Instructions ```python class Instructions(BasePrompt) ``` Instructions class. The instructions are passed to the LLM as secondary input. Different model may use these differently. For example, chat models may receive instructions in the system-prompt. #### format ```python def format(**kwargs) -> "Instructions" ``` Format the prompt using the given keyword arguments. ## PromptCallableBase ## LLMResponse ```python class LLMResponse(ILLMResponse) ``` Standard information collection from LLM responses to feed the validation loop. **Attributes**: - `output` _str_ - The output from the LLM. - `stream_output` _Optional[Iterator]_ - A stream of output from the LLM. Default None. - `async_stream_output` _Optional[AsyncIterator]_ - An async stream of output from the LLM. Default None. - `prompt_token_count` _Optional[int]_ - The number of tokens in the prompt. Default None. - `response_token_count` _Optional[int]_ - The number of tokens in the response. Default None. --- ### Api Reference/Types # Types ## OnFailAction ```python class OnFailAction(str, Enum) ``` OnFailAction is an Enum that represents the different actions that can be taken when a validation fails. **Attributes**: - `REASK` _Literal["reask"]_ - On failure, Reask the LLM. - `FIX` _Literal["fix"]_ - On failure, apply a static fix. - `FILTER` _Literal["filter"]_ - On failure, filter out the invalid values. - `REFRAIN` _Literal["refrain"]_ - On failure, refrain from responding; return an empty value. - `NOOP` _Literal["noop"]_ - On failure, do nothing. - `EXCEPTION` _Literal["exception"]_ - On failure, raise a ValidationError. - `FIX_REASK` _Literal["fix_reask"]_ - On failure, apply a static fix, check if the fixed value passed validation, if not then reask the LLM. - `CUSTOM` _Literal["custom"]_ - On failure, call a custom function with the invalid value and the FailResult's from any validators run on the value. ## RailTypes ```python class RailTypes(str, Enum) ``` RailTypes is an Enum that represents the builtin tags for RAIL xml. **Attributes**: - `STRING` _Literal["string"]_ - A string value. - `INTEGER` _Literal["integer"]_ - An integer value. - `FLOAT` _Literal["float"]_ - A float value. - `BOOL` _Literal["bool"]_ - A boolean value. - `DATE` _Literal["date"]_ - A date value. - `TIME` _Literal["time"]_ - A time value. DATETIME (Literal["date-time: - A datetime value. - `PERCENTAGE` _Literal["percentage"]_ - A percentage value represented as a string. Example "20.5%". - `ENUM` _Literal["enum"]_ - An enum value. - `LIST` _Literal["list"]_ - A list/array value. - `OBJECT` _Literal["object"]_ - An object/dictionary value. - `CHOICE` _Literal["choice"]_ - The options for a discrimated union. - `CASE` _Literal["case"]_ - A dictionary that contains a discrimated union. ## MessageHistory ```python MessageHistory = List[Dict[str, Union[Prompt, str]]] ``` ## ModelOrListOfModels ```python ModelOrListOfModels = Union[Type[BaseModel], Type[List[Type[BaseModel]]]] ``` ## ModelOrListOrDict ```python ModelOrListOrDict = Union[Type[BaseModel], Type[List[Type[BaseModel]]], Type[Dict[str, Type[BaseModel]]]] ``` ## ModelOrModelUnion ```python ModelOrModelUnion = Union[Type[BaseModel], Union[Type[BaseModel], Any]] ``` ## PydanticValidatorTuple ```python PydanticValidatorTuple = Tuple[Union[Validator, str, Callable], str] ``` ## PydanticValidatorSpec ```python PydanticValidatorSpec = Union[Validator, PydanticValidatorTuple] ``` ## UseValidatorSpec ```python UseValidatorSpec = Union[Validator, Type[Validator]] ``` ## UseManyValidatorTuple ```python UseManyValidatorTuple = Tuple[ Type[Validator], Optional[Union[List[Any], Dict[str, Any]]], Optional[Dict[str, Any]], ] ``` ## UseManyValidatorSpec ```python UseManyValidatorSpec = Union[Validator, UseManyValidatorTuple] ``` ## ValidatorMap ```python ValidatorMap = Dict[str, List[Validator]] ``` --- ### Api Reference/Validator # Validation ## Validator ```python @dataclass class Validator() ``` Base class for validators. #### \_\_init\_\_ ```python def __init__(on_fail: Optional[Union[Callable[[Any, FailResult], Any], OnFailAction]] = None, **kwargs) ``` #### validate ```python def validate(value: Any, metadata: Dict[str, Any]) -> ValidationResult ``` Do not override this function, instead implement _validate(). External facing validate function. This function acts as a wrapper for _validate() and is intended to apply any meta- validation requirements, logic, or pre/post processing. #### validate\_stream ```python def validate_stream(chunk: Any, metadata: Dict[str, Any], *, property_path: Optional[str] = "$", context_vars: Optional[ContextVar[Dict[ str, ContextVar[List[str]]]]] = None, context: Optional[Context] = None, **kwargs) -> Optional[ValidationResult] ``` Validates a chunk emitted by an LLM. If the LLM chunk is smaller than the validator's chunking strategy, it will be accumulated until it reaches the desired size. In the meantime, the validator will return None. If the LLM chunk is larger than the validator's chunking strategy, it will split it into validator-sized chunks and validate each one, returning an array of validation results. Otherwise, the validator will validate the chunk and return the result. #### with\_metadata ```python def with_metadata(metadata: Dict[str, Any]) ``` Assigns metadata to this validator to use during validation. #### to\_runnable ```python def to_runnable() -> Runnable ``` #### register\_validator ```python def register_validator( name: str, data_type: Union[str, List[str]], has_guardrails_endpoint: bool = False ) -> Callable[[Union[Type[V], Callable]], Union[Type[V], Type[Validator]]] ``` Register a validator for a data type. ## ValidationResult ```python class ValidationResult(IValidationResult, ArbitraryModel) ``` ValidationResult is the output type of Validator.validate and the abstract base class for all validation results. **Attributes**: - `outcome` _str_ - The outcome of the validation. Must be one of "pass" or "fail". - `metadata` _Optional[Dict[str, Any]]_ - The metadata associated with this validation result. - `validated_chunk` _Optional[Any]_ - The value argument passed to validator.validate or validator.validate_stream. ## PassResult ```python class PassResult(ValidationResult, IPassResult) ``` PassResult is the output type of Validator.validate when validation succeeds. **Attributes**: - `outcome` _Literal["pass"]_ - The outcome of the validation. Must be "pass". - `value_override` _Optional[Any]_ - The value to use as an override if validation passes. ## FailResult ```python class FailResult(ValidationResult, IFailResult) ``` FailResult is the output type of Validator.validate when validation fails. **Attributes**: - `outcome` _Literal["fail"]_ - The outcome of the validation. Must be "fail". - `error_message` _str_ - The error message indicating why validation failed. - `fix_value` _Optional[Any]_ - The auto-fix value that would be applied if the Validator's on_fail method is "fix". - `error_spans` _Optional[List[ErrorSpan]]_ - Segments that caused validation to fail. ## ErrorSpan ```python class ErrorSpan(IErrorSpan, ArbitraryModel) ``` ErrorSpan provide additional context for why a validation failed. They specify the start and end index of the segment that caused the failure, which can be useful when validating large chunks of text or validating while streaming with different chunking methods. **Attributes**: - `start` _int_ - Starting index relative to the validated chunk. - `end` _int_ - Ending index relative to the validated chunk. - `reason` _str_ - Reason validation failed for this chunk. ## ValidatorLogs ```python class ValidatorLogs(IValidatorLog, ArbitraryModel) ``` Logs for a single validator execution. **Attributes**: - `validator_name` _str_ - The class name of the validator - `registered_name` _str_ - The snake_cased id of the validator - `property_path` _str_ - The JSON path to the property being validated - `value_before_validation` _Any_ - The value before validation - `value_after_validation` _Optional[Any]_ - The value after validation; could be different if `value_override`s or `fix`es are applied - `validation_result` _Optional[ValidationResult]_ - The result of the validation - `start_time` _Optional[datetime]_ - The time the validation started - `end_time` _Optional[datetime]_ - The time the validation ended - `instance_id` _Optional[int]_ - The unique id of this instance of the validator ## ValidatorReference ```python class ValidatorReference(IValidatorReference) ``` ValidatorReference is a serialized reference for constructing a Validator. **Attributes**: - `id` _Optional[str]_ - The unique identifier for this Validator. Often the hub id; e.g. guardrails/regex_match. Default None. - `on` _Optional[str]_ - A reference to the property this validator should be applied against. Can be a valid JSON path or a meta-property such as `prompt` or `output`. Default None. - `on_fail` _Optional[str]_ - The OnFailAction to apply during validation. Default None. - `args` _Optional[List[Any]]_ - Positional arguments. Default None. - `kwargs` _Optional[Dict[str, Any]]_ - Keyword arguments. Default None. --- ### CONTRIBUTING # Contributing to Guardrails Welcome and thank you for your interest in contributing to Guardrails! We appreciate all contributions, big or small, from bug fixes to new features. Before diving in, let's go through some guidelines to make the process smoother for everyone. ## Getting Started 1. If you're fixing a bug or typo, feel free to submit a Pull Request directly. 2. For new features or bug fix discussions, open an issue or join our [Discord server](https://discord.gg/Jsey3mX98B) to chat with the community. ## Setting Up Your Environment 1. Clone the repository: `git clone https://github.com/guardrails-ai/guardrails.git` 2. Enter the project directory: `cd guardrails` 3. Install the project in developer mode (use a virtual environment if preferred): `make dev` 4. Install [pre-commit](https://pre-commit.com/): `pre-commit install` ## Development Workflow Follow these steps before committing your changes: 1. Ensure tests pass: `make test` 2. Format your code: `make autoformat` 3. Run static analysis: `make type` 4. Update documentation: `make docs-gen` ### Optional: Pre-Commit Hooks For convenience, consider [installing the pre-commit hooks](https://pre-commit.com/#installation) provided in the repository. These hooks automatically run tests and formatting checks each time you commit, reducing development overhead. ## Submitting a Pull Request 1. Ensure all tests pass and code is formatted. 2. Create a pull request with a clear description of your changes. Link to relevant issues or discussions. Follow [this guide](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request-from-a-fork) if needed. 3. Address any failing checks before requesting a review. 4. Engage in the code review process and make any necessary changes. 5. Celebrate when your pull request is merged! Your changes will be available in the next Guardrails release. Thank you for your contribution and happy coding! --- ### README
Guardrails AI Logo Guardrails AI Logo
[](https://opensource.org/licenses/Apache-2.0) [](https://pepy.tech/project/guardrails-ai) [](https://github.com/guardrails-ai/guardrails/actions/workflows/ci.yml) [](https://codecov.io/gh/guardrails-ai/guardrails) [](https://microsoft.github.io/pyright/) [](https://x.com/guardrails_ai) [](https://discord.gg/U9RKkZSBgx) [](https://guardrailsai.com/guardrails/docs) [](https://www.guardrailsai.com/blog) [](https://gurubase.io/g/guardrails)
## News and Updates - **[July 6, 2026]** Guardrails validators are moving to standard PyPI packages you install directly with `pip`, and Guardrails is discontinuing its hosted remote inferencing. See [How to Migrate](https://github.com/guardrails-ai/guardrails/issues/1560) for what to do. **Planned cutoff: August 6, 2026.** - **[Feb 12, 2025]** We just launched Guardrails Index -- the first of its kind benchmark comparing the performance and latency of 24 guardrails across 6 most common categories! Check out the index at index.guardrailsai.com ## What is Guardrails? Guardrails is a Python framework that helps build reliable AI applications by performing two key functions: 1. Guardrails runs Input/Output Guards in your application that detect, quantify and mitigate the presence of specific types of risks. To look at the full suite of risks, check out [Guardrails Hub](https://guardrailsai.com/hub/). 2. Guardrails help you generate structured data from LLMs.
Guardrails in your application
### Guardrails Hub Guardrails Hub is a collection of pre-built measures of specific types of risks (called 'validators'). Multiple validators can be combined together into Input and Output Guards that intercept the inputs and outputs of LLMs. Visit [Guardrails Hub](https://guardrailsai.com/hub/) to see the full list of validators and their documentation.
Guardrails Hub gif
## Installation ```python pip install guardrails-ai ``` ## Getting Started ### Create Input and Output Guards for LLM Validation 1. Download and configure the Guardrails Hub CLI. ```bash pip install guardrails-ai guardrails configure ``` 2. Install a guardrail from Guardrails Hub. ```bash pip install guardrails-ai-regex-match ``` 3. Create a Guard from the installed guardrail. ```python from guardrails import Guard, OnFailAction from guardrails_ai.regex_match import RegexMatch guard = Guard().use( RegexMatch, regex="\(?\d{3}\)?-? *\d{3}-? *-?\d{4}", on_fail=OnFailAction.EXCEPTION ) guard.validate("123-456-7890") # Guardrail passes try: guard.validate("1234-789-0000") # Guardrail fails except Exception as e: print(e) ``` Output: ```console Validation failed for field with errors: Result must match \(?\d{3}\)?-? *\d{3}-? *-?\d{4} ``` 4. Run multiple guardrails within a Guard. First, install the necessary guardrails from Guardrails Hub. ```bash pip install guardrails-ai-competitor-check guardrails-ai-toxic-language ``` Then, create a Guard from the installed guardrails. ```python from guardrails import Guard, OnFailAction from guardrails_ai.competitor_check import CompetitorCheck from guardrails_ai.toxic_language import ToxicLanguage guard = Guard().use( CompetitorCheck(["Apple", "Microsoft", "Google"], on_fail=OnFailAction.EXCEPTION), ToxicLanguage(threshold=0.5, validation_method="sentence", on_fail=OnFailAction.EXCEPTION) ) guard.validate( """An apple a day keeps a doctor away. This is good advice for keeping your health.""" ) # Both the guardrails pass try: guard.validate( """Shut the hell up! Apple just released a new iPhone.""" ) # Both the guardrails fail except Exception as e: print(e) ``` Output: ```console Validation failed for field with errors: Found the following competitors: [['Apple']]. Please avoid naming those competitors next time, The following sentences in your response were found to be toxic: - Shut the hell up! ``` ### Use Guardrails to generate structured data from LLMs Let's go through an example where we ask an LLM to generate fake pet names. To do this, we'll create a Pydantic [BaseModel](https://docs.pydantic.dev/latest/api/base_model/) that represents the structure of the output we want. ```py from pydantic import BaseModel, Field class Pet(BaseModel): pet_type: str = Field(description="Species of pet") name: str = Field(description="a unique pet name") ``` Now, create a Guard from the `Pet` class. The Guard can be used to call the LLM in a manner so that the output is formatted to the `Pet` class. Under the hood, this is done by either of two methods: 1. Function calling: For LLMs that support function calling, we generate structured data using the function call syntax. 2. Prompt optimization: For LLMs that don't support function calling, we add the schema of the expected output to the prompt so that the LLM can generate structured data. ```py from guardrails import Guard import openai prompt = """ What kind of pet should I get and what should I name it? ${gr.complete_json_suffix_v2} """ guard = Guard.for_pydantic(output_class=Pet, prompt=prompt) raw_output, validated_output, *rest = guard( llm_api=openai.completions.create, engine="gpt-3.5-turbo-instruct" ) print(validated_output) ``` This prints: ``` { "pet_type": "dog", "name": "Buddy } ``` ### Guardrails Server Guardrails can be set up as a standalone service served by Flask with `guardrails start`, allowing you to interact with it via a REST API. This approach simplifies development and deployment of Guardrails-powered applications. 1. Install: `pip install "guardrails-ai"` 2. Configure: `guardrails configure` 3. Create a config: `guardrails create --validators=hub://guardrails/two_words --guard-name=two-word-guard` 4. Start the dev server: `guardrails start --config=./config.py` 5. Interact with the dev server via the snippets below ``` # with the guardrails client import guardrails as gr gr.settings.use_server = True guard = gr.Guard(name='two-word-guard') guard.validate('this is more than two words') # or with the openai sdk import openai openai.base_url = "http://localhost:8000/guards/two-word-guard/openai/v1/" os.environ["OPENAI_API_KEY"] = "youropenaikey" messages = [ { "role": "user", "content": "tell me about an apple with 3 words exactly", }, ] completion = openai.chat.completions.create( model="gpt-4o-mini", messages=messages, ) ``` For production deployments, we recommend using Docker with Gunicorn as the WSGI server for improved performance and scalability. ## FAQ #### I'm running into issues with Guardrails. Where can I get help? You can reach out to us on [Discord](https://discord.gg/gw4cR9QvYE) or [Twitter](https://twitter.com/guardrails_ai). #### Can I use Guardrails with any LLM? Yes, Guardrails can be used with proprietary and open-source LLMs. Check out this guide on [how to use Guardrails with any LLM](https://guardrailsai.com/guardrails/docs/how-to-guides/using_llms). #### Can I create my own validators? Yes, you can create your own validators and contribute them to Guardrails Hub. Check out this guide on [how to create your own validators](https://guardrailsai.com/guardrails/docs/how-to-guides/custom_validators). #### Does Guardrails support other languages? Guardrails can be used with Python and JavaScript. Check out the docs on how to use Guardrails from JavaScript. We are working on adding support for other languages. If you would like to contribute to Guardrails, please reach out to us on [Discord](https://discord.gg/gw4cR9QvYE) or [Twitter](https://twitter.com/guardrails_ai). ## Contributing We welcome contributions to Guardrails! Get started by checking out Github issues and check out the [Contributing Guide](CONTRIBUTING.md). Feel free to open an issue, or reach out if you would like to add to the project! ---