Documentation/Website/Docs/Warning Codes/5001 Copy Section

---
id: code-5001-public
title: 5001 - Code Injection
sidebar_label: 5001 - Code Injection
---

TL;DR Copy Chapter

This category indicates that user-controlled input flows into a sink that allows code or shell command execution. This directly leads to Remote Code Execution which can be assumed to mean complete compromise of the server.

RCE via Code Injection (eval/exec) Copy Chapter

ISSUE Copy Section

The simplest kind of RCE involves user input flowing into a function such as eval or exec which are intended to interpret or run python code.

EXAMPLE Copy Section

def update_search_account_filtering(request: HttpRequest) -> HttpResponse:
    ...
    if action in ["delete", "add", "update"]:
        ...
        filter_by_username = eval(request.POST.get("filter_by_username", "True"))

RECOMMENDED SOLUTION Copy Section

RCE via Command Injection (os.system) Copy Chapter

ISSUE Copy Section

This kind of RCE involves user input flowing into a command executed in a system shell. If a user can control a portion of the command being executed in a shell, they can potentially add additional arbitrary commands to be executed.

EXAMPLE Copy Section

The following code is intended to run the spellcheck binary on a user provided text:

def spellcheck(request: HttpRequest):
    command = f"/usr/bin/spellcheck -l {request.GET['text']}"
    return os.system(command)

RECOMMENDED SOLUTION Copy Section

In general, we recommend avoiding creation of a subprocess and prefer using the API provided by the language.
However, if you need to create a subprocess, we recommend using an API such as subprocess.run, which allows you to separate arguments from the executable being invoked. DO NOT add the shell=True argument otherwise the code would still be vulnerable like the previous example

def spellcheck(request: HttpRequest):
    command = ["/usr/bin/spellcheck", "-l", request.GET['text']]
    subprocess.run(command)

---

Documentation/Website/Docs/Warning Codes/6065 Copy Section

---
id: code-6065-public
title: 6065 - Commandline arguments injection
sidebar_label: 6065 - Commandline arguments injection
---

TL;DR Copy Chapter

This category indicates that user-controlled input flows into a command-line argument used to execute an external process. Unlike category 5001, this leads to a Remote Code Execution issue only in specific cases (e.g., shell=True parameter or when executing particular binaries).

ISSUE Copy Section

subprocess.Popen, subprocess.run, subprocess.call, and other functions do a good job in preventing by default the command injection issues we described in category 5001. The values supplied in the args parameter (excluding the first which represents the executable) are considered only as arguments and not as commands to be interpreted in a system shell (more details in the python documentation). However, this safe behaviour can be manually bypassed by specifying the shell=True parameter, which reintroduces the command injection issue.

EXAMPLE Copy Section

The following code is intended to run the spellcheck binary on a user provided text:

def spellcheck(request: HttpRequest):
    command = "/usr/bin/spellcheck -l {}".format(request.GET['text'])
    return subprocess.run(command, shell=True)

RECOMMENDED SOLUTION Copy Section

In general, we recommend avoiding creation of a subprocess and prefer using the API provided by the language.
However, if you need to create a subprocess, we recommend using a safe API such as subprocess.run and avoiding use of the shell=True argument. If this is not possible, we recommend ensuring that the user-controlled values are shell-escaped with shlex.quote.

def spellcheck(request: HttpRequest):
    command = ["/usr/bin/spellcheck", "-l", request.GET['text']]
    subprocess.run(command)

---

Documentation/Website/Docs/Warning Codes/Overview Copy Section

---
id: overview-public
title: Overview
sidebar_label: Overview
---

Pysa tracks data flows from a set of sources to sinks. Each set of sources we track to sinks is assigned a unique code. When Pysa finds a flow, it emits issues associated with the unique code. This section of the website contains documentation for the warning codes we emit.

The source of truth for these warning codes is the taint_config.json files used in a given run.

---

Documentation/Website/Docs/Configuration Copy Section

---
id: configuration
title: Pyre Configuration
sidebar_label: Configuration
---

Pyre can be run without a configuration (see Command Line Arguments) but we do recommend that you create a configuration (see Getting Started) and commit that to your version control system to make sure everyone working on your project is using the same settings.

The Pyre Configuration Copy Chapter

$ pyre init

The configuration is a JSON file. For example,

{
  "source_directories": [
    "."
  ],
  "search_path": [
    "/external/library",
    {"site-package": "foo"}
  ]
}

You specify additional information to configure Pyre. The following fields are supported:

- source_directories: List of paths to type check. Paths can be a glob, for example, "./foo*".

Note: Pyre assumes that all imports are relative to the given source directory. For example, if your source directory is root/directory, then an import statement import module will be looking to import root.directory.module. If you wish to set a different import root for your source directory, you can provide an object {"import_root": "root", "source": "directory"} instead of "root/directory". In this case, import module will be looking to import root.module.

- search_path: List of paths to Python modules to include in the typing
environment. Relative paths are the best way to reference locations above the configuration directory. search_path elements take precendence over source_directories, and the order within the search path indicates precedence.
Individual items in the list can take one of the following forms:
+ A plain string, representing a path to the directories from which Pyre will search for modules. The paths can be globs, for example, "./foo*".
+ An object {"import_root": "root", "source": "directory"}, which can be used to control import root of the search path. See explaination for source_directories.
+ An object {"site-package": "package_name"}. It is equivalent to {"import_root": "site_root", "source": "package_name"}, where site_root is the first element in site_roots that has the site package named package_name installed. This can be useful when you want to manually specify which pip package you want the type checker to see as a dependency to your project (in which case it is recommended to set site_package_search_strategy to "none" to disable site package auto discovery).
+ An object {"site-package": "package_name", "is_toplevel_module": true}, to specify the name as a single-file module found in the site-root rather than as a package.

- binary: Location of Pyre's native binary.

- exclude: List of regular expressions such as ".\/node_modules\/." which
specify files and directories that should be completely ignored by Pyre. The
regular expression will be matched against the full path of files as opposed
to their relative path.

- extensions: Consider extensions in this list equivalent to .py for type checking.
Empty string indicates extensionless files.

- ignore_all_errors: A list of paths to omit from type-checking. This may be
useful for generated files, virtualenv directories, etc. These should be paths
relative to the location of the configuration file and support globs. Note: Files
matching these paths will still be processed (i.e. type and module names in those files are still visible to Pyre). Please refer to the exclude
configuration item if you have files that are intended to be hidden from Pyre.

- logger: Pyre will invoke this exectuable on every run, passing it statistics in JSON format.

- site_package_search_strategy: Configure how Pyre looks for type checking dependencies installed (e.g. by pip) on the local Python environment. Dependent libraries will not be type-checked, but they are consulted to determine the existence of globals/functions/classes. The value of this option can take one of the following forms:
+ "none". This indicates that Pyre should not attempt to search for any additional dependencies. Use this option if you know exactly what packages you depend on, and want to manually specify them with the search_path option.
+ "all". Pyre will pull in the entire site package roots (as specified in the site_roots option) as dependencies. Any libraries installed as site packages, regardless of whether they are typed or not, will be examined. Use this option if you do not know exactly which packages your code depend on, but want to make sure that no dependencies are missing.
+ "pep561". Similar to "all" but instead of pull in everything, Pyre will only pull in typed packages as dependencies according to rules specified in PEP 561. This is usually the recommended option, as the behavior is closer to what other type checkers would do by default.

Note: If incremental check is used, and the search strategy is set to "pep561", then a pyre restart is needed when new dependencies are installed -- Pyre will not automatically discover the new package by default. This is a limitation of the current implementation of Pyre and it may be lifted in the future.

- site_roots: List of path to where packages are installed.

If not specified, Pyre will consult the current Python interpreter using site.getusersitepackages() and site.getsitepackages(), which should work in most cases. But if your codebase uses a different Python interpreter, you may want to specify this option manually so Pyre knows the correct location to look for site packages.

- typeshed: Path to the Typeshed standard library, which
provides typed stubs for library functions.

- workers: Number of workers to spawn for multiprocessing.

- strict: Setting this to true will make strict mode the default in your project.

- python_version: This is expected to be a string of the form "X.Y.Z" which specifies what version of Python the checked code is expected to be executed with. By default, the version is set to the same as the Python interpreter that runs pyre itself.

- system_platform: This is expected to be a string that specifies the platform the checked code is expected to be executed on, and should be a legal value for sys.platform. By default, the platform is "linux".

Note: Since Pyre does not run your code, the setting of Python version and platform tends to have relatively small impact on the type checker's behavior. Currently, Pyre only uses Python version and platform to syntactically resolve version conditions of certain form.

Command Line Arguments Copy Chapter

Commands Copy Section

The first command you might come in contact with is
- initialize, init: Initial setup of a configuration for a project.

If Watchman is
installed, running Pyre with no positional arguments defaults to incremental,
otherwise defaults to check.
- check: Run Pyre end-to-end, i.e. not incrementally.
- incremental: Run Pyre incrementally. When invoked for the first time, the command will automatically start a server listening to changes to the filesystem. Subsequent invocations will be faster.

When Pyre is run incrementally, you can control the Pyre's server working in the background with the following commands.
- start: Start the Pyre server.
- stop: Stop the Pyre server.
- restart: Restart the Pyre server.
- servers: List all currently running Pyre servers.
- kill: In case something goes wrong and the server becomes unresponsive, kill will attempt to terminate any processes.
- rage: Print server logs for debugging or for context when reporting server errors.

Commonly Used Flags Copy Section

$ pyre --source-directory "." --noninteractive check
$ pyre --source-directory "." restart

- --noninteractive: Disable interactive logging, which by default overwrites intermediate
logging output and adds colors for a more streamlined user experience.
Non-interactive mode ensures all terminal output remains visible.

- --output {text, json, sarif}: Formatting for error return values. Defaults to text.

- --search-path SEARCH_PATH: Provide additional stubs or modules external to the project
being type-checked. Can also be set in .pyre_configuration.

- --source-directory SOURCE_DIRECTORY: Provide a path to the source root to check. This can also
be specified in .pyre_configuration.

- --typeshed TYPESHED: Path to the Typeshed standard library,
which provides typed stubs for library functions. This can also be set in .pyre_configuration.

- --version: Print the current version of Pyre.

Helper Commands Copy Section

Working with Multi-Project Repositories Copy Chapter

If you have a single repository with multiple independent Python projects, we recommend you
use a separate .pyre_configuration for each one. This allows each project to be
type checked independently.

If you use virtual environments to manage separate dependencies for each project, you can install
pyre as a development dependency in each one; by default Pyre will detect system packages
from the environment it is installed in, so this will cause each project to detect the
correct dependencies (assuming you activate the virtual environment before running Pyre).

---

Documentation/Website/Docs/Editor Integration Copy Section

---
id: editor-integration
title: Editor Integration
sidebar_label: Editor Integration
---
import {OssOnly, FbInternalOnly} from 'docusaurus-plugin-internaldocs-fb/internal';

---

Documentation/Website/Docs/Errors Copy Section

---
id: errors
title: Type Errors
sidebar_label: Type Errors
---

import Internal from './fb/errors.md';

Common Issues Copy Chapter

Covariance and Contravariance Copy Section

Pyre will error when, for instance, a List[int] is passed in when a List[float] is expected, as in the following example:

def to_seconds(milliseconds: List[float]) -> List[int]:
  return [int(x/1000.0) for x in milliseconds]
my_list: List[int] = [1]
my_list = to_seconds(my_list) # Pyre errors here!

$ pyre
Unbound name [10]: Name List is used but not defined in the current scope.

def halve_first_element(list: List[float]) -> None:
  list[0] /= 2
def function_taking_int(int: int) -> None:
    return None
my_list: List[int] = [1]
halve_first_element(my_list)
function_taking_int(my_list[0]) # Oh no, my_list[0] is 0.5!

$ pyre
Incompatible parameter type [6]: In call list.__setitem__, for 2nd positional argument, expected int but got float.

If we allowed passing in my_list to the halve_first_element function here, the above code would type check. It's perfectly valid from the perspective of the callee to modify the list's element to be a float, as it was annotated as taking a list of floats, but because this list escapes the scope of the callee, we can't allow this in the type checker.

To work around this, we can signal to the type checker that the parameter can't be modified. Here's how you can tell the type checker that you won't change the container in your function:

from typing import *
I can't modify milliseconds here, so it's safe to pass a Iterable[int].

def to_seconds(milliseconds: Iterable[float]) -> List[int]:
  return [int(x/1000.0) for x in milliseconds]
my_list: List[int] = [1]
my_list = to_seconds(my_list) # Type checks!

Most commonly used generic containers have immutable variants, and I would encourage you to use them for function parameters whenever you don't need to modify a container in your function.
Here are some immutable variants for commonly used containers:

typing.List → typing.Sequence (if you need random access via my_list[id])
typing.List → typing.Iterable (if you're just iterating over the list in a loop and want to support sets as well)
typing.Dict → typing.Mapping
typing.Set → typing.AbstractSet

Invariance, combined with type inference, comes with a few gotchas. When you write an expression, Pyre infers the most precise type possible. For instance, Pyre infers the List[int] type for [1, 2], even though List[float] would be a perfectly valid type here. This can cause issues, as in the following example:

def zeroes(number_of_elements: int) -> List[float]:
  a = [0] * number_of_elements
  return a # Pyre errors here!

$ pyre
Incompatible return type [7]: Expected List[float] but got List[int].

What happened above is that Pyre inferred a type of List[int] for a, and invariance kicked in. You can work around this by adding an explicit annotation when declaring a:

def zeroes(number_of_elements: int) -> List[float]:
  a: List[float] = [0.0] * number_of_elements
  return a # Type checks!

#### Contravariance

Callable, on the other hand, is contravariant#Contravariant_method_parameter_type) in its parameter types. This means that, when checking if Callable[[A], None] is compatible with Callable[[B], None], we check if B is compatible with A, not the other way around. This is because the former should be capable of accepting any arguments accepted by the latter.

For example, a function of type Callable[[Base], int] may be given an argument of type Child2. But if we passed in a function of type Callable[[Child1], int], this could fail at runtime:

class Base: pass
class Child1(Base):
    size: int = 42
No size field.

class Child2(Base): pass
def print_child2_size(get_size: Callable[[Base], int]) -> None:
    child2 = Child2()
    size = get_size(child2)
    print(size)
def size_of_child1(child1: Child1) -> int:
    return child1.size
print_child2_size(size_of_child1) # BAD!
At runtime:

AttributeError: 'Child2' object has no attribute 'size'

To prevent such errors, Pyre raises a type error when violating contravariance:

$ pyre
Incompatible parameter type [6]: Expected typing.Callable[[Base], int] for 1st positional only parameter to call print_child2_size but got typing.Callable(size_of_child1)[[Named(child1, Child1)], int].

Optional Attributes Copy Section

from typing import Optional
class Data:
  field: Optional[int]
def process_field(input: int) -> None:
  ...
def process_data(data: Data) -> None:
  if data.field is not None:
    # ... interleaving logic
    process_field(data.field)

$ pyre
Incompatible parameter type [6]: expected int but got Optional[int]

The above fails to type-check because Pyre cannot guarantee that data.field remains not None if the interleaving logic between the explicit check and the later reference contains anything that may have side effects, like function calls.

An interleaving call could set field back to None, since it's a non local variable and is mutable. Therefore any calls between the None check and the access will invalidate the "not None" refinement. If data.field is defined as a class property or if the parent class has overridden __getattr__, then all bets are off even if there are no interleaving calls.

The preferred way to make this code type-check is to either move the check closer to the access, or to mark the attribute Final if it is not meant to be reassigned to, and you can guarantee to the type checker that no interleaving side effects can modify this attribute.

from typing import Final, Optional
class Data:
  # Needs to be assigned in the constructor and cannot be changed afterwards.
  field: Final[Optional[int]] = 1

It is always safe to refine attributes when their types are Final.

Alternatively, it is also safe to assign the attribute to a local variable before accessing its value:

def process_data(data: Data) -> None:
  field = data.field
  if field is not None:
    # ... interleaving logic
    process_field(field)

or using Python 3.8's assignment expressions:

def process_data(data: Data) -> None:
    if (field := data.field) is not None:
        # ... interleaving logic
        process_field(field)

Third-Party Libraries Copy Section

Undefined attribute [16]: Module <library> has no attribute <some attribute>.

Since it is not always possible to annotate code, PEP 484 specifies a format for stub files with a .pyi extension. Pyre will look for stub files in typeshed, or next to your source code. You can also provide additional paths to Pyre to look for stubs (see Configuration).

<Internal />

Error Codes Copy Chapter

(venv) $ pyre
 ƛ Found 1 type error!
test.py:1:0 Incompatible variable type [9]: a is declared to have type int but is used as type str.

The 9 in the brackets indicates that we raised an error with code 9.

0: Unused Ignore Copy Section

pyre-fixme[7] # unused ignore

def foo() -> int:
  return 1

$ pyre
Unused ignore [0]: The pyre-ignore[7] or pyre-fixme[7] comment is not suppressing type errors, please remove it.

2: Missing Parameter Annotation Copy Section

If strict mode is turned on, Pyre will error when a function parameter is either annotated with a type that contains typing.Any or not annotated with any type at all (in which case Pyre will treat it as typing.Any by default). It will also error when a method parameter is not annotated, unless that parameter is the first parameter of a bound or static method (i.e. self, whose type pyre can infer).

We enforce typed argument because typing.Any can hide type errors that will happen at runtime:

from typing import Any
def say_hello(name) -> None:
    print("Hello " + name)

This line will raise at runtime, but no type error since say_hellos name has type Any.

say_hello(42)

You can silence this by adding a non-Any annotation to all parameters of functions and methods (other than self and cls for bound and class methods, which you may omit).

3: Missing Return Annotation Copy Section

If strict mode is turned on, Pyre will error when a function is either annotated with a return type that contains typing.Any, or is not annotated with any return type at all (in which case Pyre will treat it as returning typing.Any by default).

This is bad because a return type of typing.Any may potentially hiding legitimate type errors that may happen at runtime:

from typing import Any
def f():
  return 42
print("a" + f())

$ pyre
Missing return annotation [3]: Returning int but no return type is specified.

The best way to silence this error is to add non-Any return annotation to every function.

4: Missing Attribute Annotation Copy Section

In strict mode, Pyre will error when an attribute does not have an annotation.

def foo() -> str:
    return "Hello, World!"
class A:
    b = foo() # Missing attribute annotation

$ pyre
Missing attribute annotation [4]: Attribute b of class A has no type specified.

Adding a type annotation will resolve this error:

def foo() -> str:
    return "Hello, World!"
class A:
    b: str = foo()

This error can also occur when pyre is inferring attribute types from constructors.
For example, here we know that b is an int based on the parameter annotation in __init__:

class A:
    def __init__(self, b: int) -> None:
        self.b = b

But here we need a annotations because we can't just propagate an argument annotation:

class A:
    def __init__(self, arg: int) -> None:
        self.a = arg + 5
        self.b = arg + 5

$ pyre
Missing attribute annotation [4]: Attribute a of class A has type int but no type is specified.
Missing attribute annotation [4]: Attribute b of class A has type int but no type is specified.

We can fix this by making the annotation explicit, either in the class body or
in __init__:

class A:
    a: int
    def __init__(self, arg: int) -> None:
        self.a = arg + 5
        self.b: int = arg + 5

5: Missing Global Annotation Copy Section

If strict mode is turned on, Pyre will error when a globally accessible variable is not annotated. If pyre was able to infer a type for the
variable, it will emit this type in the error message. The fix is usually to add an annotation to the variable.

Note: This error has also arisen when there is some ambiguity of whether a declaration is a global expression or a type alias, in these cases pyre assumes it is an expression. Adding a : TypeAlias annotation lets pyre know that it is a type alias and solves the problem.

from typing_extensions import TypeAlias
This declaration would result in an error

MyTypeAlias = Dict[str, "AnotherTypeAlias"]

$ pyre
Missing global annotation [5]: Globally accessible variable MyTypeAlias has no type specified.

from typing_extensions import TypeAlias
This declaration ensures that pyre knows MyTypeAlias is a type alias

MyTypeAlias: TypeAlias = Dict[str, "AnotherTypeAlias"]

6: Incompatible Parameter Type Copy Section

def takes_int(x: int) -> None:
  pass
def f(x: Optional[int]) -> None:
  takes_int(x) # Incompatible parameter type error

$ pyre
Incompatible parameter type [6]: In call takes_int, for 1st positional argument, expected int but got Optional[int].

If you are seeing errors with invariant containers where some Container[T] is expected but you are passing Container[S] where S < T, please see Covariance and Contravariance.

7: Incompatible Return Type Copy Section

def foo() -> int:
  return "" # incompatible return type

$ pyre
Incompatible return type [7]: Expected int but got str.

def foo() -> str:
  return "" # compatible: No error

8: Incompatible Attribute Type Copy Section

class Foo:
  x: int = 0
def f(foo: Foo) -> None:
  foo.x = "abc" # Incompatible attribute type error

$ pyre
Incompatible attribute type [8]: Attribute x declared in class Foo has type int but is used as type str.

If you are seeing errors with invariant containers where some Container[T] is expected but you are passing Container[S] where S < T, please see Covariance and Contravariance.

9: Incompatible Variable Type Copy Section

That is, the following will error:

def f(x: int) -> None:
  x = "" # Incompatible variable type error
  y: int = 1
  y = "" # Incompatible variable type error

$ pyre
Incompatible variable type [9]: x is declared to have type int but is used as type str.
Incompatible variable type [9]: y is declared to have type int but is used as type str.

The rationale here is that it's surprising for an explicitly annotated variable to have an
incompatible type later on in the same function.

If you are constructing an object that is generic over an invariant type, you may run into an error:

from typing import TypeVar
_T = TypeVar('_T')
class Foo(Generic[_T]):
    def __init__(self, x: _T) -> None: ...
def f() -> None:
    foo: Foo[Optional[int]] = Foo(x=1) # Incompatible variable type error

$ pyre
Incompatible variable type [9]: foo is declared to have type Foo[Optional[int]] but is used as type Foo[int].

This is due to the fact that Foo[X] is not less than Foo[Y] even if X < Y when the type variable is invariant.
You can declare your intention to initialize the object with a wider type than is given to fix this error:

from typing import TypeVar
_T = TypeVar('_T')
class Foo(Generic[_T]):
    def __init__(self, x: _T) -> None: ...
def f() -> None:
    foo: Foo[Optional[int]] = Foo[Optional[int]](x=1)

10: Unbound Name Copy Section

Pyre produces an Unbound Name error when your code access a variable (local or global) that pyre believes is not defined.

In most cases code that does this is invalid and will always fail. For example this code will always fail at runtime:

def f() -> int:
    return x  # use of an unbound name x

$ pyre
Unbound name [10]: Name x is used but not defined in the current scope.

There are some cases where python code that works fine at runtime could produce this error, for example if a function implicitly sets a module-level global variable that is not declared in the toplevel. Pyre will not accept this because module-level globals require type annotations, and if they have no declaration there is nowhere to put the annotation:

def set_x() -> None:
    global x
    x = 42
def use_x() -> None:
    print(x)

this code will run fine, but pyre cannot analyze the type or use of the

implicitly-defined global x and will complain about an unbound name.

set_x()
use_x()

$ pyre
Unbound name [10]: Name x is used but not defined in the current scope.

You can fix this by explicitly adding a declaration of the top-level variable x, for example:

x : Optional[int] = None
def set_x() -> None:
    global x
    x = 42
def use_x() -> None:
    print(x)

this code will run fine

set_x()
use_x()

11, 31: Undefined or Invalid Type Copy Section

from typing_extensions import TypeAlias
INT_OR_STR: TypeAlias = Union[int, str]

If you use a name as an annotation that is not a valid type or valid alias, you will see this error:

from typing import Callable, List
from typing_extensions import Final, Literal
GLOBAL_VALUE = "string"
def f0() -> GLOBAL_VALUE: ... # Error! GLOBAL_VALUE is a value, not a type.
def f1() -> type(GLOBAL_VALUE): ...   # Error! Static type annotations cannot be dynamically computed.
def f2() -> [int]: ...  # Error! [int] is not a valid type. If you mean a list of int, use typing.List[int].
def f3() -> (int, str): ...  # Error! (int, str) is not a valid type. If you mean a pair of int and str, use typing.Tuple[int, str].
def f4() -> Callable[[int]]: ...  # Error! Callable[[int]] is not a valid type because the return type of the callable is missing. Good example: Callable[[int], int].
def f5() -> Callable[int, int]: ...  # Error! Callable[int, int] is not a valid type. The parameter types of the callable must be enclosed in square brackets. Good example: Callable[[int], int].
def f6() -> List[Final[int]]: ...  # Error! Final may only be used as the outermost type in annotations. See PEP 591.
def f7() -> Literal[GLOBAL_VALUE]: ...  # Error! Only literals are allowed as parameters for Literal. See PEP586. Good example: Literal[42] or Literal["string"].

$ pyre
Undefined or invalid type [11]: Annotation GLOBAL_VALUE is not defined as a type.
Invalid type [31]: Expression type(GLOBAL_VALUE) is not a valid type.
Invalid type [31]: Expression [int] is not a valid type.
Invalid type [31]: Expression (int, str) is not a valid type.
Invalid type [31]: Expression typing.Callable[[int]] is not a valid type.
Invalid type [31]: Expression typing.Callable[(int, int)] is not a valid type.
Invalid type [31]: Expression GLOBAL_VALUE is not a literal value.

You can fix this error by verifying that your annotation is
1. statically determined.
2. properly imported from typing if applicable.
3. properly defined in the module you are importing from. If the module you are importing from has a stub file, you should check the definition there.
4. properly adhere to the additional rules of special types (e.g. Callable, Final, and Literal).
#### Type Aliases
For type aliases, check that your type alias is defined
1. with a valid type on the RHS. If you provide an annotation for the TypeAlias assignment, it must be typing_extensions.TypeAlias.
2. on the global scope, not nested inside a function or class.
#### ParamSpec
For ParamSpec, check that you have used both args: P.args and *kwargs: P.kwargs in your function's parameters:

from typing import Callable
from pyre_extensions import ParameterSpecification
P = ParameterSpecification("P")
Error because kwargs: P.kwargs is missing.

def bad1(f: Callable[P, int], *args: P.args) -> int:
    return f(*args)
Error because *args: P.args is missing.

def bad2(f: Callable[P, int], kwargs: P.kwargs) -> int:
    return f(kwargs)

$ pyre
Undefined or invalid type [11]: Annotation P.args is not defined as a type.
Call error [29]: typing.Callable[P, int] cannot be safely called because the types and kinds of its parameters depend on a type variable.
Undefined or invalid type [11]: Annotation P.kwargs is not defined as a type.

from typing import Callable
from pyre_extensions import ParameterSpecification
P = ParameterSpecification("P")
OK

def good(f: Callable[P, int], args: P.args, *kwargs: P.kwargs) -> int:
    return f(args, *kwargs)

12: Incompatible Awaitable Type Copy Section

In strict mode, pyre will verify that all calls to await are on awaitable values, to ensure that you cannot get a runtime error awaiting an object that is not a coroutine.

A common situation where working code will produce this error is when pyre cannot statically verify that an awaitable is non-Null, for example:

import asyncio
async def f(flag: bool) -> None:
    if flag:
        task = asyncio.create_task(asyncio.sleep(1))
    else:
        task = None
    await task  # would throw a ValueError if flag were false
asyncio.run(f(True))

In this example, task will always be a valid awaitable unless some other module overwrites the global flag, but pyre cannot prove that this does not happen. The error we get has the message

$ pyre
Expected an awaitable but got typing.Optional[asyncio.tasks.Task[None]]

You can fix this error by ensuring that the awaited object has an awaitable type. In the case of optional values, you can use refinement to rule out None. The example above can be fixed by tweaking the definition of main:

async def f(flag: bool) -> None:
    if flag:
        task = asyncio.create_task(asyncio.sleep(1))
    else:
        task = None
    if task is not None:
      await task

13: Uninitialized Attribute Copy Section

In strict mode, pyre will throw an error for class attributes which are declared without default values if they are not initialized in a constructor, for example:

class A:
    x : int
    def __init__(self) -> None:
        pass

$ pyre
Uninitialized attribute [13]: Attribute x is declared in class A to have type int but is never initialized.

class A:
    x: int = 0
    def __init__(self) -> None:
        pass

class A:
    x: int
    def __init__(self, x: int = 0) -> None:
        self.x = x

#### Dataclass-like classes

One case where this can occur is when using a library providing a "dataclass-like" decorator that, for example, autogenerates a constructor setting attributes.

from dataclasses import dataclass
@dataclass
class A:
    x: int

Pyre currently knows that that uninitialized attributes of classes wrapped in dataclass and attrs decorators will generate constructors that set the attributes. But it does not understand many custom libraries that do similar things, for example test frameworks, or new decorators that wrap the dataclass decorator and add more logic.

There is not currently a way to fix this other than via pyre-ignore or pyre-fixme directives. The python typing community is aware of this problem but has not yet settled on a solution, you can see discussion here.

14,15: Behavioral Subtyping Copy Section

Method overrides should follow
Liskov's substitution principle.
In short, parameter types can't be more restrictive and return types
can't be more permissive in overridden methods. To see why, consider the following example:

def width(image: Image) -> float:
    return image.width()

Say we now have different implementations of our Image class, one of which
violates the substitution principle:

class Image:
    def width() -> float: pass
class JpegImage(Image):
    def width() -> int: return 10  # this is fine
class ComplexImage(Image):
    def width() -> complex: return 1j
def foo() -> None:
    image: Image = ComplexImage()
    print(int(image.width()))

The above code fails at runtime with TypeError: can't convert complex to int. The case for parameters follows analogously.

#### Common Reasons

+ Could not find parameter y in overriding signature.: Check if the overriding function can accept all arguments that the overridden function can.

class Base:
      def foo(self, x: int, y: str) -> None:
          pass
  class Child(Base):
      def foo(self, x: int) -> None:
          pass

+ Type Foo is not a subtype of the overridden attribute type Bar:

class Base:
      a: int = 0
  class Child(Base):
      a: str = ""
  def foo() -> None:
      base: Base = Child()
      base.a + 1

This would fail at runtime with TypeError: can only concatenate str (not "int") to str.

+ Returned type Foo is not a subtype of the overridden return Bar.: Check for reasons like invariance.

16: Missing Attributes Copy Section

Your code is most likely trying to access an attribute that Pyre does not know about.
Pyre has various ways of inferring what is an attribute of an object:

#### Explicitly Declare the Attribute

class Derp:
    my_attribute: int = 1
    @property
    def my_property(self) -> int: ...

#### Implicitly Declare the Attribute

class Derp:
    def __init__(self, foo: str) -> None:
        self.my_attribute: int = 1
        # The foo attribute is inferred to have type str because the
        # parameter foo has type str.
        self.foo = foo

#### Common Reasons

+ Optional type has no attribute foo.: See Optional attributes.

+ Foo has no attribute bar.: Check if you have explicitly provided the type for bar either in the constructor or as a class attribute.

+ Module foo has no attribute bar: Check if the library has stubs. If so, you may need to add the function, class, or global variable to the stub.

+ A library class has an attribute but it is not recognized by Pyre: Check if the library has stubs. If so, you may need to add the attribute to the class in the stub.

+ Your class has dynamic attributes: Consider using __getattr__ in a stub so that Pyre doesn't complain about those attributes.

17: Incompatible Constructor Annotation Copy Section

PEP 484 specifies that __init__ method of any class must be annotated to return None. Pyre will emit an error if the user's annotation does not conform to the specification.

pyre-strict
class A:
    def __init__(self) -> "A":  # Error 17: Invalid return annotation of __init__.
        ...
class B:
    def __init__(self) -> None:  # OK
        ...

$ pyre
Incompatible constructor annotation [17]: __init__ is annotated as returning A, but it should return None.

19: Too Many Arguments Copy Section

Pyre verifies that you pass a legal number of arguments to functions.

The most obvious way to encounter this error is to just pass too many arguments to a function:

def f(x: int) -> int:
    return x
f(5, 6)  # this would throw a TypeError at runtime, and pyre complains

$ pyre
Too many arguments [19]: Call f expects 1 positional argument, 2 were provided.

Pyre will also throw this error if you pass too many positional arguments to
a function that uses python's ability restrict arguments to be keyword-only
specified by PEP 3102:

def f(*, x: int) -> int:
    return x
f(5)  # As before, this throws a TypeError because x is positional-only
f(x=5)  # this line will typecheck and run without error

$ pyre
Too many arguments [19]: Call f expects 0 positional arguments, 1 was provided.

20: Missing Argument Copy Section

Pyre verifies that function calls provide all the expected arguments, so it will complain about code like this:

def f(x: int) -> ing:
    return x
f()

$ pyre
Missing argument [20]: Call f expects argument x.

To fix this, make sure all required arguments are provided.

21: Undefined Name, Undefined Import Copy Section

from my_module import my_function
my_function()

$ pyre
Undefined import [21]: Could not find a module corresponding to import my_module.

Pyre will raise error 21 instead ("Undefined import") when the import statement is present, but the module to be imported could not be found in the search path.
If the module provides stub files, please provide their location via the --search-path commandline parameter.

#### Namespace Package Modules

One case where you may run into undefined imports on code that works at runtime is when importing namespace modules.
The CPython runtime allows you to import a directory that is on your PYTHONPATH, even if it contains no __init__.py; this behavior is defined in PEP 420 and the module is called a namespace package.
In order to make Pyre both fast and consistent on incremental updates, in Pyre we only allow importing namespace packages that have at least one python file as a direct child.

So, for example, if I have a directory tree with just a/b/c.py then Pyre will allow import a.b.c and import a.b but not import a.
A namespace package module can never contain useful types or code so it is rare to directly import it, but in special cases it might be useful (for example to access the __name__ attribute).
In these cases, you'll need to suppress Pyre errors.

22: Redundant Cast Copy Section

from typing import cast
def foo(x: int) -> None:
    y = cast(int, x)

$ pyre
Redundant cast [22]: The value being cast is already of type int.

23: Unable to Unpack Copy Section

Pyre will warn you when trying to assign a value to a tuple with the wrong number of items.

def foo() -> None:
    a, b = (1, 2, 3)
    x, y = 42

$ pyre
Unable to unpack [23]: Unable to unpack 3 values, 2 were expected.
Unable to unpack [23]: Unable to unpack int into 2 values.

Common reasons:

+ Trying to assign an Optional value to a tuple:

def bar() -> None:
      x = None
      if 2 + 2 == 4:
          x = ("a", "b")
      a, b = x

$ pyre
  Unable to unpack [23]: Unable to unpack typing.Optional[typing.Tuple[str, str]] into 2 values.

+ Unpacking an incorrect number of elements when looping over a list:

for a, b in [1, 2, 3]:
      print(a, b)
  $ pyre
  Unable to unpack [23]: Unable to unpack int into 2 values.

24: Invalid Type Parameters Copy Section

#### "Generic type expects X type parameters ..."
Either too few or too many type parameters were provided for the container type. For example,

x: List[int, str] = [] # Invalid type parameters error

$ pyre
Invalid type parameters [24]: Generic type list expects 1 type parameter, received 2, use typing.List[<element type>] to avoid runtime subscripting errors.

x: List[Union[int, str]] = [] # No error

If you do not know or do not want to specify the type parameters, use typing.Any but still ensure the arity is correct.

x: List = [] # Invalid type parameters error
x: List[Any] = [] # No error

Note: You may see a suggestion to use typing.List instead of builtins list as the type annotation when providing type parameters. This is to avoid runtime errors, because the builtin list does not support subscripting and list[int] is therefore not runtime-friendly.

#### "Non-generic type cannot take type parameters ..."
Type parameters are only meaningful if the container type is generic. Passing in the type parameter binds the provided parameter type to the generic in the container class. For example,

class Container:
    def add(self,element: int) -> None: ...
    def get_element(self) -> int: ...
x: Container[int] = Container() # Invalid type parameter error

$ pyre
Invalid type parameters [24]: Non-generic type Container cannot take parameters.

from typing import TypeVar, Generic
T = TypeVar('T')
class Container(Generic[T]):
    def add(self,element: T) -> None: ...
    def get_element(self) -> T: ...
x: Container[int] = Container()
x.get_element() # returns int
y: Container[str] = Container()
y.get_element() # returns str

#### "Type parameter violates constraints ..."
If a container class is generic over a type variable with given type bounds, any type parameter used must comply with those type bounds. For example,

from typing import TypeVar, Union, Generic
T = TypeVar('T', bound=Union[int, bool])
class Container(Generic[T]):
    def add(self, element: T) -> None: ...
    def get_element(self) -> T: ...
x: Container[int] = Container() # No error
y: Container[str] = Container() # Invalid type parameter error

$ pyre
Invalid type parameters [24]: Type parameter str violates constraints on Variable[T (bound to typing.Union[bool, int])] in generic type Container.

26: Typed Dictionary Access With Non-Literal Copy Section

In python, typed dictionaries can only be accessed using literal strings that can be statically verified as valid. As a result, code like this will not typecheck even though it works at runtime, because we cannot statically verify that key in print_value is a valid Shape key:

from typing import TypedDict
class Shape(TypedDict):
    sides: int
    color: str
shape: Shape = {"sides": 4, "color": "blue"}
print(shape["sides"])  # this is fine because "sides" is a literal
for key in ["sides", "color"]:
    print(key, shape[key])  # pyre will complain here because it can't prove key is valid

$ pyre
TypedDict accessed with a non-literal [26]: TypedDict key must be a string literal. Expected one of ('color', 'sides').

The example above shows a situation where you might hit this error: when you want to iterate over the fields of a typed dict. A suggested fix is to use type-safe operations like dictionary.items instead. For example the following code produces the same results but type checks:

class Shape(TypedDict):
    sides: int
    color: str
shape: Shape = {"sides": 4, "color": "blue"}
print(shape["sides"])  # this is fine because "sides" is a literal
for key, value  in shape.items():
    print(key, value) # no error

In other cases where you need to access a TypedDict using a variable as a key, you can use dictionary.get(key).

from typing import TypedDict
class Shape(TypedDict):
    sides: int
    color: str
shape: Shape = {"sides": 4, "color": "blue"}
print(shape["sides"])  # this is fine because "sides" is a literal
for key in ["sides", "color"]:
    print(key, shape.get(key)) # no error

27: Typed Dictionary Key Not Found Copy Section

If you try to access a typed dictionary with a string literal that pyre knows is not a valid key, pyre will emit an error:

from typing import TypedDict
class Shape(TypedDict):
    sides: int
    color: str
def f(shape: Shape) -> None:
    print(shape["location"])  # error here

$ pyre
TypedDict accessed with a missing key [27]: TypedDict Shape has no key location.

A possible fix: pyre considers instances of any TypedDict with additional fields to be a subtype of Shape, so in many cases you could handle the need for a location field in some Shape dicts by creating a new type as follows:

from typing import TypedDict
class Shape(TypedDict):
    sides: int
    color: str
class ShapeWithLocation(TypedDict):
    sides: int
    color: str
    location: str
def f(shape: ShapeWithLocation) -> None:
    print(shape["location"])  # okay
    g(shape)  # also okay: ShapeWithLocation is a subtype of Shape
def g(shape: Shape) -> None:
    print(shape)

28: Unexpected Keyword Copy Section

Pyre will error if attempting to pass an argument by name and there are no parameters with a matching name. For example,

def foo(integer: int, string: str) -> None: ...
foo(1, "one")  # no error
foo(string="one", integer=1)  # no error
foo(integer=1, undefined="one")  # type error

$ pyre
Unexpected keyword [28]: Unexpected keyword argument undefined to call foo.

29: Call Error Copy Section

Pyre will emit an error on seeing a call of one of the following types:

1. The called object is not a function. This means that its inferred type is not Callable and it is not an instance of a class which implements a __call__ method. This could happen due to user error (the object is indeed not a function) or due to an incorrect or incomplete type stub for the object's class causing pyre to infer the wrong type.

2. The call cannot be safely typed since the types and kinds of its parameters depend on a type variable. This is seen when the callable is typed using a ParameterSpecification type variable and the args and *kwargs are not passed into the call correctly, i.e. together and in order. (For more details see PEP 612)

from pyre_extensions import ParameterSpecification
from typing import Callable
P = ParameterSpecification("P")
def decorator(f: Callable[P, int]) -> Callable[P, None]:
    def foo(args: P.args, *kwargs: P.kwargs) -> None:
        f(args, *kwargs)    # Accepted, should resolve to int
        f(*args)              # Rejected : error here
        f(kwargs, *args)    # Rejected : error here
        f(1, args, *kwargs) # Accepted
    return foo

$ pyre
Call error [29]: typing.Callable[sandbox.P, int] cannot be safely called because the types and kinds of its parameters depend on a type variable.
Call error [29]: typing.Callable[sandbox.P, int] cannot be safely called because the types and kinds of its parameters depend on a type variable.

30, 36: Terminating Analysis, Mutually Recursive Type Variables Copy Section

#### Overly-complex Functions

In very rare cases where a function has a lot of if branches or for-loops, Pyre may raise an error saying that is unable to analyze the function fully. Analyzing extremely complex functions in depth can be costly, so Pyre only does so up to a limit. This means that it won't infer precise types for some variables and won't catch errors related to their usage. For example:

def my_function() -> None:
    u = 42
    if foo():
        x1 = bar()
        if x1:
            x2 = baz()
            if x2:
                # <more branches of code>
        else:
            # <more branches>
        if foo2():
            # <code>
        if foo3():
            # <even more branches>
        # <and even more branches>
$ pyre
Analysis failure [30]: Pyre gave up inferring types for some variables because function foo was too complex.
Please simplify the function by factoring out some if-statements or for-loops.

To remedy this, factor out some of the branching code into separate functions out that each function has a limited amount of branching logic:

def do_stuff() -> None:
    if foo():
        # <code>
    if foo():
        # <even more branches>
    # <and even more branches>
def bar() -> None:
    u = 42
    if foo():
        x1 = bar()
        if x1:
            x2 = baz()
            if x2:
                # <more branches of code>
        else:
            # <more branches>
        do_stuff()

These errors usually indicates a bug in Pyre. Please open an issue on Github.

31: Invalid Type Copy Section

This indicates that you are using some expression that pyre does not understand as a type.

Some situations where you might run into this:

+ Using a list of types rather than List[type]:

x: [str] = ["a string"]

x: List[str] = ["a string"]

+ Using a constructor call rather than a bare class name:

class A:
      ...
  a: A() = A()

a: A = A()

32: Invalid Argument Copy Section

This error usually means you are using a variable in a way that is incompatible with its structure, either as an argument to a function call or as part of a data structure.

This could be from using an invalid variadic parameter (informally a "splat"):

x: int = 5
print(*x)   # invalid use of x, which is not iterable

$ pyre
Invalid argument [32]: Unpacked argument x must have an unpackable type but has type typing_extensions.Literal[5].

from typing import Dict
x: int = 5
d: Dict[int, int] = {x}  # invalid use of x, which is not a mapping
dict(d)  # invalid use of d; function kwargs must be a mapping with string keys

$ pyre
Invalid argument [32]: Keyword argument x has type typing_extensions.Literal[5] but must be a mapping.
Invalid argument [32]: Keyword argument d has type Dict[int, int] but must be a mapping with string keys.

It's also possible to hit this error code on constraint mismatches when using tuple variadic variables as specified in PEP 646, which are an advanced feature of pyre.

33: Prohibited Any Copy Section

import typing
from typing import Any, Dict
def foo() -> None:
    x: typing.Any = 1

$ pyre
Prohibited any [33]: Expression x has type int; given explicit type cannot be Any.

34: Invalid Type Variable Copy Section

Type variables can only be used as types when they have already been placed "in scope".
A type variable can be placed into scope via:

* Generic class declarations
* for example, class C(Generic[T]): puts T into scope for the body of the class
* The parameter types of a generic function
* for example, def foo(x: T) puts T into scope for the body and return type annotation of the function

For example:

from typing import List
class Base:
    foo: List[T] = []
$ pyre
Invalid type variable [34]: The current class isn't generic with respect to the type variable Variable[T].

def foo(x: int) -> List[T]:
    return [x, x]
$ pyre
Invalid type variable [34]: The type variable Variable[T] isn't present in the function's parameters.

from typing import Generic, List
class Base(Generic[T]):
    foo: List[T] = []
base: Base[int]
def foo(x: T) -> List[T]:
    return [x, x]

#### Decorator Factories

One common error is when defining a generic decorator factory. The Python type system doesn't currently place T into scope within a Callable type. So, it considers T to be a type variable from the outer scope. This can lead to errors for apparently valid code:

from typing import Callable, TypeVar
T = TypeVar("T")
R = TypeVar("R")
def my_decorator_factory(message: str) -> Callable[[Callable[[T], R]], Callable[[T], R]]:
    def _decorator(f: Callable[[T], R]) -> Callable[[T], R]:
        def _inner(x: T) -> R:
            print(message)
            return f(x)
        return _inner
    return _decorator
$ pyre
Invalid type variable [34]: The type variable Variable[R] isn't present in the function's parameters.
Invalid type variable [34]: The type variable Variable[T] isn't present in the function's parameters.

Suggested fix: Use a callback protocol to define the return type.

from typing import Callable, Protocol, TypeVar
T = TypeVar("T")
R = TypeVar("R")
class MyCallableProtocol(Protocol):
    def __call__(self, f: Callable[[T], R]) -> Callable[[T], R]: ...
def my_decorator_factory(message: str) -> MyCallableProtocol:
    def _decorator(f: Callable[[T], R]) -> Callable[[T], R]:
        def _inner(x: T) -> R:
            print(message)
            return f(x)
        return _inner
    return _decorator

If you are using a ParamSpec in your decorator, use the following:

from typing import Any, Callable, Coroutine, Protocol, TypeVar
from pyre_extensions import ParameterSpecification
import asyncio
R = TypeVar("R")
P = ParameterSpecification("P")
class MyCallableProtocol(Protocol):
    def __call__(self, f: Callable[P, Coroutine[object, object, R]]) -> Callable[P, Coroutine[object, object, R]]: ...
def my_decorator_factory(message: str) -> MyCallableProtocol:
    def _decorator(f: Callable[P, Coroutine[object, object, R]]) -> Callable[P, Coroutine[object, object, R]]:
        async def _inner(args: P.args, *kwargs: P.kwargs) -> R:
            print(message)
            return await f(args, *kwargs)
        return _inner
    return _decorator
@my_decorator_factory("hello!")
async def foo() -> int:
    return 1
asyncio.run(foo())

Note: Support for such callables is currently experimental and varies from one typechecker to another. This behavior may change in the future.

35: Illegal Annotation Target Copy Section

1. A variable is re-annotated after first declaration or an explicity annotated function parameter is re-annotated within the function body. This is not allowed as re-annotating variables reduces readability and causes the annotation of a variable to depend on the position in control flow.

def transformation(p: int) -> str:
      return str(p + 1)
  def foo(x: int) -> None:
      y: int = x + 2
      z = x + 3
      # Each of the following will produce an error
      x: str = transformation(x)
      y: str = transformation(y)
      z: int = 4

$ pyre
  Illegal annotation target [35]: Target x cannot be annotated after it is first declared.
  Illegal annotation target [35]: Target y cannot be annotated after it is first declared.
  Illegal annotation target [35]: Target z cannot be annotated after it is first declared.

2. Trying to annotate non-self attributes, i.e annotating the attributes of a different class than the one whose scope you are in:

class Foo:
      attribute: int = 1
  class Bar:
      def __init__(self) -> None:
          Foo.attribute: str = "hello"
  def some_method() -> None:
      Foo.attribute: int = 5

$ pyre
  Illegal annotation target [35]: Target sandbox.Foo.attribute cannot be annotated.
  Illegal annotation target [35]: Target sandbox.Foo.attribute cannot be annotated.

This is not allowed as Pyre needs to be able to statically determine the type of globally accessible values, including class attributes. Even if Pyre followed control flow across functions to determine class attribute annotations, such re-annotations imply very dynamic behavior that makes the code difficult to work with.

The fix for this situation, similar to the case above, is to annotate the class attribute at its definition in the class that owns it and remove any annotations elsewhere. If this attribute is from a third party library, then you can add a stub for the class and annotate the attribute there.

39: Invalid Inheritance Copy Section

1. The parent class is marked as final which means it explicitly is annotated as not supporting child classes.

@final
  class Base:
      ...
  class Derived(Base): # Invalid inheritance error
      ...

$ pyre
  Invalid inheritance [39]: Cannot inherit from final class Base.

2. The expression given in the base class field is not a class at all.

MY_GLOBAL: str = "string"
  class Foo(MY_GLOBAL): # Invalid inheritance error
      ...

Pyre does not support dynamic expressions as base classes, even if they may evaluate to a valid class at runtime. This is because the type checker relies on building up a valid class hierarchy before it can resolve types in the Python it is analyzing. On the other hand, type aliases are equivalent to types and are acceptable as base classes.

3. You are defining a typed dictionary that does not inherit from another typed dictionary.

from typing import TypedDict
  class NonTypedDict:
      ...
  class Movie(TypedDict):
      name: str
      year: int
  class BookBasedMovie(Movie): # No error
      based_on: str
  class BookBasedMovie(NonTypedDict): # Invalid inheritance error
      based_on: str

If inheriting from another typed dictionary, fields need to have a consistent type between child and parent, in order for subclassing to be sound. Similarly, a required field in the child must also be required for the parent.

40: Invalid Override Copy Section

class A:
    @staticmethod
    def foo() -> int:
        pass
class B(A):
    @classmethod # Non-static method B.foo cannot override a static method defined in A.
    def foo(cls) -> int:
        pass

from typing import final
class Foo:
    @final
    def bar(self) -> None:
        pass
class Bar(Foo):
    def bar(self) -> None: # Invalid override, because Foo.bar is final
        pass

41: Invalid Assignment Copy Section

from typing import Final, Optional
class Foo:
  field: Final[Optional[int]] = 1
  def foo() -> None:
    self.field = 2 # Invalid assignment
class Bar:
    _x = 1
    @property
    def x(self) -> int:
        return self._x
def bar(b: Bar) -> None:
    b.x = 1 # Invalid assignment

To fix this error, change the definition of this attribute to something that is mutable, if it is not intended to be read-only.

42: Missing Overload Implementation Copy Section

Pyre will throw this error if a source module specifies one or more overloads via typing.overload but fails to provide an implementation, for example:

from typing import overload
@overload
def f(x: int) -> float:
    ...
@overload
def f(x: str) -> str:
    ...

To fix it, provide exactly one implementation (a function of the same name without the typing.overload decorator). For example above we could implement f as follows:

from typing import overload, Union
@overload
def f(x: int) -> float:
    ...
@overload
def f(x: str) -> str:
    ...
@overload
def f(x: str) -> str:
    ...
def f(x: Union[int, str]) -> Union[float, str]:
    if isinstance(x, int):
        return float(x)
    else:
        return x

43: Incompatible Overload Implementation Copy Section

Pyre will error if you define one or more overloads using typing.overload, and your concrete implementation has an incompatible type signature.

For example, this code will produce an incompatible overload implementation error

pyre-strict
from typing import overload, Union
@overload
def f(x: int) -> float:
    ...
@overload
def f(x: float) -> int:
    ...
@overload
def f(x: str) -> str:
    ...
def f(x: Union[int, float, str]) -> Union[int, str]:
    if isinstance(x, float):
        return int(x)
    elif isinstance(x, int):
        return float(x)
    else:
        return x

You can fix this by either removing incorrect overload delcarations or making
sure all parameters and return type annotations on the concrete
implementation are general enough to be consistent with the overloads:

def f(x: Union[int, float, str]) -> Union[int, float, str]:
    <same implementation>

45: Invalid Class Instantiation Copy Section

In typed Python, some classes that represent abstract interfaces may not be directly instantiated. Pyre considers a class C abstract, and will error on invalid instantiation if you try to construct an instance directly in either of the following cases:

1. C contains one or more abstract methods that are left not overridden. Abstract methods are defined as methods that are decorated with @abc.abstractmethod.

For example, here Derived0 is abstract because it does not override bar, but Derived1 may be instantiated:

import abc
  from typing import Protocol
  class Base(abc.ABC):
      @abc.abstractmethod
      def foo(self) -> None:
          raise NotImplementedError
      @abc.abstractmethod
      def bar(self) -> str:
          raise NotImplementedError
  class Derived0(Base):
      def foo(self) -> None:
          print(self.bar())
  class Derived1(Derived0):
      def bar(self) -> str:
          return "bar"
  def test0() -> None:
      base = Base()  # Error! Class Base contains 2 abstract methods and therefore cannot be instantiated.
      derived0 = Derived0()  # Error! Class Derived0 contains 1 abstract method bar and therefore cannot be instantiated.
      derived1 = Derived1()  # OK

2. C directly inherits from typing.Protocol.

For example, here MyProtocol is abstract because it inherits directly from typing.Protocol, but MyClass (which implements the protocol interface) may be instantiated:

class MyProtocol(Protocol):
      def baz(self, x: int) -> int:
          ...
  class MyClass:
      def baz(self, x: int) -> int:
          return x
  def test1() -> None:
      object0 = MyProtocol()  # Error! Class MyProtocol cannot be instantiated.
      object1 = MyClass()  # OK

46: Invalid Type Variance Copy Section

In brief, read-only data types can be covariant, write-only data types can be contravariant, and data types that support both reads and writes must be invariant.
If a data type implements any functions accepting parameters of that type, we cannot guarantee that writes are not happening. If a data type implements any functions returning values of that type, we cannot guarantee that reads are not happening.
For example (note: int is a subclass of float in the type system and in these examples):
Writes taking covariants:

from typing import TypeVar, Generic
_T_co = TypeVar("_T_co", covariant=True)
class MyList(Generic[_T_co]):
    def write(self, element: _T_co) -> None:
        ... # adds element to list
def takes_float_list(float_list: MyList[float]) -> None:
    float_list.write(1.0)
int_list: MyList[int] = ...
takes_float_list(int_list)  # this call is OK because MyList is covariant: MyList[int] < MyList[float]
int_list contains floats

Reads returning contravariants:

from typing import TypeVar, Generic
_T_cont = TypeVar("_T_cont", contravariant=True)
class MyList(Generic[_T_cont]):
    def read(self) -> _T_cont:
        ... # returns first element from list
def takes_int_list(int_list: MyList[int]) -> int:
    return int_list.read()
float_list: MyList[float] = ...
takes_int_list(float_list)  # this call is OK because MyList is contravariant: MyList[float] < MyList[int]
problem with return above is clear

47: Invalid Method Signature Copy Section

Pyre will error if a non-static method fails to specify an expected implicit parameter like self for an instance method or cls for a class method, as this argument is always implicitly passed in a call and will cause a runtime crash if not specified. Additionally, Pyre will warn if this parameter is specified but typed as something incompatible with the type of the parent class.

Often times, the method may not need a self or cls and should be decorated with @staticmethod to resolve this error. For example,

class Foo:
    def foo() -> None: ...  # type error
class Foo:
    @staticmethod
    def foo() -> None: ... # no type error
class Foo:
    def foo(self) -> None: ... # no type error

Only type variables with compatible bounds can be used to annotate the self or cls parameter. For example,

from typing import TypeVar
P = TypeVar("T", bound="Parent")
A = TypeVar("S", bound="ChildA")
B = TypeVar("S", bound="ChildB")
class Parent: ...
class ChildA(Parent):
    @classmethod
    def foo(cls: Type[A]) -> A: ...  # no type error
class ChildB(Parent):
    def foo(self: A) -> A: ...  # type error
    def bar(self: B) -> B: ...  # no type error
    def baz(self: P) -> P: ...  # no type error

48: Invalid Exception Copy Section

In python, you can only raise objects that derive from BaseException (it's more common to subtype Exception or one of the standard library-defined errors like ValueError). Attempting to raise another object such as a bare string will result in a TypeError. As a result, pyre will flag code like this:

def f(x: int) -> None:
    if x > 1:
        raise "x is too big"

To fix this, wrap the information you are trying to raise (usually an error message) in some exception type, for example:

def f(x: int) -> None:
    if x > 1:
        raise ValueError("x is too big")

49: Unsafe Cast Copy Section

To allow "safe" casts that preserve type soundness, you can use pyre_extensions.safe_cast. This will verify that the type you are casting to is broader than the type of the expression. In cases where this is not the case, pyre will produce an Unsafe Cast error. For example:

from pyre_extensions import safe_cast
def foo(x: int) -> str:
    y = safe_cast(str, x) # Unsafe cast error
    z = safe_cast(Union[int, str], x) # No error
    return z # Invalid return type error

Some context on this: pyre_extensions.safe_cast is a type-safe alternative to typing.cast. The typing.cast function forces type checkers to accept a type for an expression that otherwise would not be valid, which is sometimes useful but also can hide clear type errors, for example:

from typing import cast
def foo(x: int) -> str:
    y = cast(str, x)
    return y # No type error, even though this is unsound.

51: Unused Local Mode Copy Section

This error will be thrown if you specify more than one local mode, by having multiple line comments of the form # pyre-strict or # pyre-unsafe in the header. Pyre will ask you to remove all but one local mode declaration if you have more than one because the mode needs to be unambiguous.

Context: Pyre supports two modes of type checking, unsafe and strict.
- By default, every file runs in unsafe mode, but you can change this default to strict in your configuration file.
- In addition, you can set the type checking mode of a module to differ from the default for the project by adding a comment in the form # pyre-strict or # pyre-unsafe comment on its own line to the file header.

52: Private Protocol Property Copy Section

Python Protocols provide a way to statically check "duck typing", what many languages would refer to as interfaces.

Because protocols specify only an interface, they should not include private fields and methods, which cannot not be accessed outside of the class where they are defined (including in subclasses). Pyre will complain about the following:

from typing import Protocol
class Duck(Protocol):
    def __quack(self) -> str:
        ...
class SomeDuck:
    def __quack(self) -> str:
        return "quack"

To signal a non-public part of an interface, use a protected field or method (single leading underscore), which is accessible by classes implementing the interface:

from typing import Protocol
class Duck(Protocol):
    def _quack(self) -> str:
        ...
class SomeDuck:
    def _quack(self) -> str:
        return "quack"

53: Missing Annotation For Captured Variables Copy Section

Pyre makes no attempt at trying to infer the types across function boundaries. The statement holds for nested functions as well.
From a nested function's perspective, a variable defined in an nesting function behaves similarly to a global variable. As with global variables, an explicit annotation is required if strict mode is turned on:

def outer_function0() -> int:
    x = foo()
    def inner_function() -> int:
        return x  # Due to the lack of explicit annotation, Pyre will treat this variable as having type Any.
    return inner_function()
def outer_function1() -> int:
    x: int = foo()
    def inner_function() -> int:
        return x  # This is ok: the type of x is known to be int.
    return inner_function()
def outer_function2() -> int:
    x = foo()
    def inner_function(x: int) -> int:
        return x  # This is also ok: even though the outer x is not annotated, the x parameter of the inner function is.
    return inner_function(x)

54: Invalid TypedDict Operation Copy Section

In accordance with PEP 598, code that tries to assign a value of the wrong type to a field of a TypedDict will not typecheck:

from typing import TypedDict
class MyDict(TypedDict):
    value: str
d: MyDict = {"value": "hello"}
d["value"] = 5  # Invalid TypedDict operation

To fix this you may need to change your field type to a Union, if variable types are actually needed for a field.

55: TypedDict Initialization Error Copy Section

Pyre will warn you when initializing a TypedDict with:

+ Missing required fields

from typing import TypedDict
  class Movie(TypedDict):
      name: str
      year: int
  movie: Movie = {"name": "The Matrix"}
  $ pyre
  TypedDict initialization error [55]: Missing required field year for TypedDict Movie.

+ Incorrect field type

movie: Movie = {"name": "The Matrix", "year": "1999"}
  $ pyre
  TypedDict initialization error [55]: Expected type int for Movie field year but got str.

+ Undefined fields

movie: Movie = {"name": "The Matrix", "year": 1999, "extra_field": "hello"}
  $ pyre
  TypedDict initialization error [55]: TypedDict Movie has no field extra_field.

56: Invalid Decoration Copy Section

This error code is a catch-all for a variety of problems that can arise in the course of resolving the type of a decorated function.
In all of these cases, these decoration failures will lead to the function being registered with type Any to avoid any spurious downstream errors.

#### "Pyre was not able to infer the type of the decorator ..."

This should only happen when the decorator access itself is invalid, e.g. when you use a decorator which isn't declared in the stubs for a third-party library.

#### "Pyre was not able to infer the type of argument ..."

When using the "decorator factory" pattern, we need to resolve the type of both the decorator factory itself as well as the arguments passed to the decorator factory.
This is because the types of these arguments can alter the behavior of the returned decorator via overloads or type variables.
However, this resolution has to happen early in the environment-building pipeline, when we don't yet have all of the context we need in order to resolve the types of arbitrary expressions.
We support resolving literals and simple globals as arguments, but using anything else will result in this error.

To work around this, you can statically type your arguments to the decorator factory as separate globals, which can be validated later in the type-checking pipeline.

from typing import TypeVar
T = TypeVar("T")
def decorator_factory(x: T) -> Callable[[Callable[[int], str]], Callable[[str], T]]:
    ...
pyre-fixme[56]: Pyre was not able to infer the type of argument

 complex_expression() to decorator factory decorator_factory.

@decorator_factory(complex_expression())
def foo(x: int) -> str:
    ...
argument: float = complex_expression()
@decorator_factory(argument) # Accepted!  bar resolves to Callable[[str], float]
def bar(x: int) -> str:
    ...

#### "Decorator factory \X\ could not be called"

This corresponds to when the decorator factory access resolves to a type that is not callable (i.e. has no __call__ method).

not_a_factory: int = 5
pyre-fixme[56]: Decorator factory not_a_factory could not be called, because its

type int is not callable

@not_a_factory(1)
def bar() -> None:
    pass

#### "Decorator \X\ could not be called"

Similarly, these errors correspond to when the entire decorator expression (potentially including arguments to a decorator factory), resolves to a non-callable type.

def foo() -> int:
    return 42
pyre-fixme[56]: Decorator foo() could not be called, because its

type int is not callable

@foo()
def bar() -> None:
    pass

#### "While applying decorator factory ..."

These errors are emitted from attempting to pass the resolved factory arguments to the factory, as with any other function call.

from typing import Callable
def factory(x: str) -> Callable[[object], object]:
    ...
pyre-fixme[56]: While applying decorator factory factory:

Expected str for 1st param but got int.

@factory(1)
def foo() -> None:
    pass

#### "While applying decorator ..."

Correspondingly, these errors are emitted from trying to pass the decorated function as an argument to the resolved decorator type.

from typing import Callable
def decorator(f: Callable[[int], str]) -> int:
    ...
pyre-fixme[56]: While applying decorator decorator:

Expected Callable[[int], str] for 1st param but got Callable[[str], int].

@decorator
def foo(x: str) -> int:
    return 5

57: Incompatible Async Generator Return Type Copy Section

An async generator function is an async function that contains at least one yield statement. The Python runtime ensures that all async generator would return an async generator object. Therefore, the return type of async generator functions should always be typing.AsyncGenerator or one of its supertypes.

from typing import AsyncGenerator
async def f() -> int:  # Error
    yield 0
async def g() -> AsyncGenerator[int, None]:  # OK
    if False:
        yield 1

58: Unsupported Operand Copy Section

Pyre will warn if an infix operator is not supported for the right or left operands provided.

In Python, an infix operator is converted to a method call on either of the operands - for example, a < b is equivalent to a.__lt__(b). Therefore, this type error can also be considered sugar for an error that method a.__lt__ does not accept the type of b as an argument.

For example,

from typing import Optional
def foo(x: Optional[int]) -> bool:
    return x < 0  # type error: Optional[int] is not a supported operand
def bar(x: Optional[int]) -> bool:
    if x:
        return x < 0  # no type error
    return False

59: Duplicate Type Variables Copy Section

This occurs when the same type variable is provided more than once to a Generic or Protocol. A type variable needs to be bound to a single value. Thus, if one wants two independent type variables with perhaps the same bounds or same properties, they have to be different variables.

from typing import TypeVar, Generic
T0 = TypeVar("T0")
T1 = TypeVar("T1")
T2 = TypeVar("T2")
class A(Generic[T0, T1, T0]):  # Error
    pass
class B(Generic[T0, T1, T2]):  # OK
    pass

60: Unable to Concatenate Tuple Copy Section

#### "Expected to unpack an iterable ..."

This can occur if during concatenation of a tuple one tries to unpack a non-iterable since non-iterables can't be unpacked. Either try to unpack an iterable, or concatenate without unpacking.

def foo(x: int, not_iterable: int, iterable: list[int]) -> None:
    y = (x, *not_iterable)  # Error
    z = (x, not_iterable) # OK
    w = (x, *iterable)  # OK

#### "Concatenation not yet supported for multiple variadic tuples ..."

This can occur if during concatenation one tries to use multiple variadic tuples. This is due the limitations of the current type system and there is no workaround currently. One may use # pyre-ignore[60] to suppress.

from typing import Tuple
from pyre_extensions import TypeVarTuple
Ts = TypeVarTuple("Ts")
def foo(xs: Tuple[*Ts]) -> None:
    y = (xs, xs)  # Error

61: Uninitialized Local Copy Section

This indicates that there are code paths along which a local variable may not be initialized. Below are some common code patterns that may cause this error:

#### Not initialized in all branches of condition

def f(x: int) -> None:
    z = None
    if x > 5:
        y = 2
        z = 2
    print(y)  # Error
    print(z)  # OK

- initialize y to a default value, outside the conditional or in the else branch
- refactor so that initialization and access are in the same conditional

Pyre static analysis does not reason about runtime values or potential side effects of interleaving calls, so for instance, in the example below we cannot guarantee that the two if statements will always be consistent and, hence, throw the same error:

def f(x: int) -> None:
    if x > 5:
        y = 2
    # ...some operations...
    if x > 5:
        print(y)    # Error

#### Initialized only inside a for loop

def f(xs: List[int]) -> None
    for x in xs:
        y = "yes"
    print("Last element is: ", x)  # Error
    print("Did we enter the loop?", y)  # Error

One way to remediate is to initialize outside the loop. For instance,

def f(xs: List[int]) -> None:
    x = None
    y = "no"
    for x in xs:
        y = "yes"
    print("Last element is: ", x)  # OK
    print("Did we enter the loop?", y)  # OK

#### Initialized in try block

def f(divisor: int) -> None:
    answer_good = None
    try:
        answer_bad = 5 / divisor
        answer_good = 5 / divisor
        answer_also_good = 5 / divisor
        print(f"5 divided by {divisor} is {answer_also_good}")   # OK
    except ZeroDivisionError:
        pass
    print(f"5 divided by {divisor} is {answer_bad}")  # Error
    print(f"5 divided by {divisor} is {answer_good}")  # OK

- Initialize any variables needed after the try block to a default value before entering the try block.
- Keep the access to variables initialized inside the try block within the try block.
- Consider if pulling the initialization as-is before the try block is possible. It is generally considered a good practice to minimize the code inside a try block, and keep it to exception throwing code. This also helps with Pyre, as it does not reason about which operations might throw exceptions.

def bad(divisor: int) -> Optional[int]:
    try:
        dividend = 5
        return dividend // divisor
    except ZeroDivisionError:
        print(f"Cannot divide {dividend} by 0")  # Error (according to Pyre)
def good(divisor: int) -> Optional[int]:
    dividend = 5
    try:
        return dividend // divisor
    except ZeroDivisionError:
        print(f"Cannot divide {dividend} by 0")  # OK

62: Non-literal string Copy Section

Pyre will error if you passs in a non-literal string into a function call that expects a LiteralString.

def query_user(conn: Connection, user_id: str) -> User:
    query = f"SELECT * FROM data WHERE user_id = {user_id}"
    conn.execute(query)  # Error: Expected LiteralString, got str.
    ...

def query_user(conn: Connection, user_id: str) -> User:
    query = "SELECT * FROM data WHERE user_id = ?"
    conn.execute(query, (user_id,))  # OK.

LiteralStrings are created either from an explicit string literal like "foo" or from combining multiple string literals or LiteralStrings. This is a security focused typing feature for safely calling powerful APIs. See PEP 675 for more details.

63: Suppression Comment Without Error Code Copy Section

When running Pyre in strict mode, suppression comments like pyre-ignore and pyre-fixme need to be associated with a specific error code or set of error codes.

OK:

def foo() -> int:
    # pyre-ignore[7]
    return ""

Not OK:

def foo() -> int:
    # pyre-ignore
    return ""

For more details on how to write suppression comments, refer to the section on Suppression later in this doc.

64: Inconsistent Method Resolution Order Copy Section

Python supports multiple inheritance via multiple base classes. In order to decide the order that methods from base classes are overridden, there needs to be a consistent order among the base classes of a particular class. When this order cannot be created, for example because there is a cycle, then Pyre raises this error.

class A(B):  # Error
    pass
class B(A):  # Error
    pass

For more details and examples for Python's Method Resolution Order (MRO), please refer to the official guide.

65: Duplicate Parameter Copy Section

Functions may not have multiple named parameters that share the same name.

def add(a: int, a: int) -> None:  # Error
    pass

66: Invalid Exception Handler Copy Section

Exception types listed in an except clause must extend BaseException.

try:
    pass
except (int, bool):  # Error
    pass

67: Invalid Exception Group Handler Copy Section

Exception types listed in an except* clause (Python 3.12 exception group handler) must extend BaseException and may not extend BaseExceptionGroup.

try:
    pass
except* ExceptionGroup as e:  # Error
    pass

try:
    pass
except* int as e:  # Error
    pass

68: Invalid Type Guard Copy Section

User defined type-guards are functions that return TypeIs[X] or TypeGuard[X]; at runtime these return a bool, but the static type indicates to the type checker that they narrow the type of the first positional argument, for example:

from typing import TypeIs, reveal_type
def is_int_or_str(val: object) -> TypeIs[int | str]: ...
def f(x: object):
    if is_int_or_str(x):
        reveal_type(x)  # int | str

(extending the previous example)
def is_int_or_str_typeguard(val: object) -> TypeGuard[int | str]: ...
def g(x: int | str | bytes):
    if is_int_or_str(x):
        reveal_type(x)  # int | str
    else:
        reveal_type(x)  # bytes
    if is_int_or_str_typeguard(x):
        reveal_type(x)  # int | str
    else:
        reveal_type(x)  # int | str | bytes (no narrowing here)

It is a type error if such a type guard function does not accept at least one positional argument; the type guard cannot actually be invoked correctly in this case. For example:

from typing import TypeIs, TypeGuard
Error: the argument is keyword-only, this is not a valid type guard.

def bad_type_guard(/, val: objec) -> TypeGuard[float]: ...
class CustomTypeGuard:
    # Error: this is a non-static method and self is bound, so it does not
    #  accept a positional argument.
    def guard(self) -> TypeIs[str]: ...

In addition, for TypeIs it is an error if the narrowed type (the type inside the TypeIs) is not assignable to the type of the first positional parameter.

from typing import TypeIs
Error: int is not a subtype of str

def inconsistent_type_is(x: str) -> TypeIs[int]: ...

You can read more about type guards and narrowing in the specification: https://typing.readthedocs.io/en/latest/spec/narrowing.html.

69: Invalid Positional-Only Parameter Copy Section

Python has positional-only parameters which may not be passed by name. Positional-only parameters cannot appear after parameters that may be passed by name.

Typically, all the parameters that proceed / in the parameter list are considered positional-only, but in functions without this syntax we treat parameters that are prefixed but not suffixed with __ as positional-only for backwards-compatibility purposes. Refer to https://typing.readthedocs.io/en/latest/spec/historical.html#positional-only-parameters for more details.

In the following example, __y is a positional-only that follows the regular parameter x, which is an error.

def foo(x: int, __y: int) -> None:  # Error
    pass

70: Assert Type Copy Section

assert_type is a static assertion that has no effect at runtime. Pyre will emit an error if the inferred type for the first argument does not match the type provided as the second argument.

x: int = 1
assert_type(x, str) # Error

71: Typed Dictionary Isinstance Copy Section

Typed dictionaries are structurally typed. This means that it is invalid to use isinstance to check whether something is an instance of a typed dictionary.

class Coord(TypedDict):
    x: int
    y: int
isinstance({}, Coord) # Error

72: Tuple Delete Copy Section

Tuples are immutable, so deleting elements is not allowed.

x = (1, 2, 3)
del x[0] # Error

73: Tuple Out of Bounds Copy Section

When a tuple with a concrete/known length is indexed with a literal, Pyre will emit an error if the index is out of bounds.

x = (1, 2, 3)
y: int = ...
x[0] # OK
x[-1] # OK
x[5] # Error
x[y] # OK

74: Named Tuple Default Fields Copy Section

Since named tuple constructors are generated based on the order fields are declared in, fields without defaults may not follow fields with defaults in a named tuple declaration.

This is OK

class MyTuple(NamedTuple):
    x: int
    y: int = 1
This is not allowed

class MyTuple2(NamedTuple):
    x: int = 1
    y: int

Suppression Copy Chapter

Suppressing Individual Errors Copy Section

- # pyre-fixme indicates there is an issue in the code that will be revisited later.
- # pyre-ignore indicates there's an issue with the type checker or the code is too dynamic and we have decided to not fix this. If this is a Pyre bug, make sure you open an issue on our tracker.

Both comment styles allow you to suppress individual error codes as well as adding additional context.

def foo() -> int:
    # pyre-fixme[7]: only suppresses return mismatches
    return ""

Pyre also supports # type: ignore comments for backwards-compatibility with MyPy.

Suppressing Errors within Format Strings Copy Section

If you want to suppress an error within an f-string, you can add a fixme comment on the line before the string. This will suppress all errors within the f-string matching that fixme:

def print_profile(name: str, age: Optional[int]) -> None:
    # pyre-fixme[58]: - is not supported for operand types Optional[int] and int.
    s = f"""
    Your personal details!!
    Name: {name}
    Age: {age - 3}
    """
    print(s)

Suppressing All Errors Copy Section

Suppressing Errors Across Files Copy Section

Furthermore Pyre supports suppressing all errors in an individual file if you add a # pyre-ignore-all-errors to your file. Like the other suppression comments, you can use square brackets to chose to only ignore one or more particular error types. For example, you can suppress all incompatible return type errors by adding:

pyre-ignore-all-errors[7]
def foo(x: int) -> str:
    return x  # pyre will not error here

Debugging Copy Chapter

---

Documentation/Website/Docs/Features Copy Section

---
id: features
title: Pyre-Exclusive Features
sidebar_label: Pyre-Exclusive Features
---

Pyre has custom support for Python idioms that would otherwise not be supported by the usual type annotations.

Registering attributes using PyTorch's register_buffer Copy Chapter

PyTorch allows subclasses of nn.Module to register a buffer in an object using self.register_buffer("foo", initial_value). Pyre supports this pattern when used within the constructor. It simply treats the buffer as a Tensor attribute of the class:

import torch
import torch.nn as nn
class Foo(nn.Module):
    def __init__(self) -> None:
        super(Foo, self).__init__()
        self.register_buffer("foo", torch.zeros(10, 20))
        self.register_buffer("foo_persistent", torch.zeros(10, 20), persistent=False)
    def bar(self) -> None:
        reveal_type(self.foo) # => torch.Tensor
        reveal_type(self.foo_persistent) # => torch.Tensor
def baz() -> None:
    y = Foo().foo
    reveal_type(y) # => torch.Tensor

Note that Pyre will not recognize buffers registered in methods other than the constructor (just like it doesn't recognize attributes defined in methods other than the constructor).

It will also not recognize buffers that are initialized with None since it cannot infer the exact type of the buffer. In such a case, you can tell Pyre about the attribute's type by explicitly defining it in the class:

import torch
import torch.nn as nn
from typing import Optional
class Foo(nn.Module):
    my_none_buffer: Optional[torch.Tensor]
    def __init__(self) -> None:
        super(Foo, self).__init__()
        self.register_buffer("my_none_buffer", None)
    def bar(self) -> None:
        reveal_type(self.my_none_buffer) # => Optional[torch.Tensor]

---

Documentation/Website/Docs/Getting Started Copy Section

---
id: getting-started
title: Getting Started with Pyre
sidebar_label: Getting Started
---
import {OssOnly, FbInternalOnly} from 'docusaurus-plugin-internaldocs-fb/internal';
import FbGettingStarted from './fb/getting_started.md';
import YouTube from 'react-youtube';

<FbInternalOnly>

<FbGettingStarted />

</FbInternalOnly>

<OssOnly>

Welcome to the wonderful world of static typing! This guide will get you from zero to a simple project that is type checked with Pyre.

Requirements Copy Chapter

$ brew install python3 watchman

$ sudo apt-get install python3 python3-pip python3-venv watchman

Setting up a Project Copy Chapter

$ mkdir my_project && cd my_project
$ python3 -m venv ~/.venvs/venv
$ source ~/.venvs/venv/bin/activate
(venv) $ pip install pyre-check

Next, we teach Pyre about our new project:

(venv) $ pyre init

Running Pyre Copy Chapter

(venv) $ echo "i: int = 'string'" > test.py
(venv) $ pyre
 ƛ Found 1 type error!
test.py:1:0 Incompatible variable type [9]: i is declared to have type int but is used as type str.

Introductory Video Copy Chapter

<YouTube videoId="k_xElpxw9aY" containerClassName="youtubeContainer" />

</OssOnly>

Further Reading Copy Chapter

This page should contain all of the basic information you need to get started with type checking your own project.

If you are new to the type system, the introduction to types in Python is recommended reading to familiarize with the type system, gradual typing, and common type errors.

<FbInternalOnly>

If you are looking for more options to configure your type checking experience, the configuration page explores command line and configuration file settings.

</FbInternalOnly>

<OssOnly>

If you are looking for more options to configure your type checking experience, the configuration page explores command line and configuration file settings.

</OssOnly>

<FbInternalOnly>

For questions and support, please use our Pyre Q&A workplace group. For more insight into our roadmap, new features, and progress, please read our Pyre FYI workplace group.

</FbInternalOnly>

---

Documentation/Website/Docs/Gradual Typing Copy Section

---
id: types-in-python
title: Types in Python
sidebar_label: Introduction
---

Python's type system was specified in PEP 484. If you are new to Python's type system and want to learn the basics, we highly recommend you take a look at mypy's cheatsheet as well as their type system reference. The following discussion focuses on Pyre's approach to "gradual typing" and how you can get from an untyped codebase to a fully typed codebase.

Why Types? Copy Chapter

Gradual Typing Copy Chapter

- only reporting errors on functions that have an explicit return or parameter type annotation,
- introducing an escape hatch: a special type Any that has all possible attributes and is both sub- and super-type of any other type,
- and assuming that all untyped fuctions implicitly return Any.

For example,

from typing import List
def unannotated():        # implictly returns Any
    return b"" + ""       # function body is not checked
def annotated() -> List:  # explicit return annotation means we type check annotated
    any = unannotated()
    any.attribute         # Any has all possible attributes
    return 1              # Error: returning int but expecting List

In combination, these rules allow you to slowly annotate code without getting overwhelmed by type errors in one sitting. Incrementally adding more annotations will give you stronger safety and consistency guarantees in your codebase.

In the example above, if you changed unannotated to return str, you would get a type error when accessing the attribute any.attribute in annotated.

Strict Mode Copy Chapter

- run on all functions, whether they are annotated or not,
- error on functions, globals, or attributes that are missing annotations,
- and error on annotations containing Any (with some exceptions to accommodate for common patterns).

In our previous example,

pyre-strict

from typing import List
def unannotated():        # Error: missing return annotation
    return b"" + ""       # Error: function body is checked
def annotated() -> List:  # Error: implicit Any for generic parameter to List
    any = unannotated()
    any.attribute         # Note: the type of any is still any.
    return 1              # Error: returning int but expecting List

As you can see in the example, Any can still sneak into modules that are strict, but increasing strict coverage and fixing the surfaced errors will gradually eliminate them.

Strict-By-Default Copy Section

How to move away from Any Any')" title="Copy chapter prompt for LLMs"> Copy Chapter

What if I want to use Any? Any?')" title="Copy section prompt for LLMs"> Copy Section

What then? Copy Section

About object object')" title="Copy section prompt for LLMs"> Copy Section

About TypeVar TypeVar')" title="Copy section prompt for LLMs"> Copy Section

When Source Code is not Available Copy Chapter

To address these cases, Pyre will give precedence to type stub files with a *.pyi extension over source files when these are specified in the search path in the project configuration or if they are located next to the implementation file.
Stub files have the same structure as implementation files but only contain class and function signatures:

my_dynamic_module.pyi

def dynamic_function() -> int: ...   # Function body is omitted

If a __getattr__ function is defined in the stub file as follows, Pyre will take it as a signal that the stub file is partially complete: accessing attributes whose name is not defined in the stub file will result in Any instead of a type error.

my_stub.pyi

from typing import Any
foo: int = 42
Parameter needs to be typed as str and return type needs to be Any

def __getattr__(name: str) -> Any: ...
my_source.py

import my_stub
reveal_type(my_stub.foo)        # Reveals int
reveal_type(my_stub.undefined)  # Reveals Any

Typeshed Copy Section

Strategies for Increasing Coverage Copy Chapter

Upgrade Copy Section

You can run pyre-upgrade with

(venv) $ pyre --output=json | pyre-upgrade fixme

(venv) $ pyre --output=json -l <project> | pyre-upgrade fixme

Automatic Type Inference Copy Section

Pyre can do static type inference. You can run

(venv) $ cd <path to project>; pyre infer -i

For dynamic inference we recommend you give MonkeyType a try.

---

Documentation/Website/Docs/Installation Copy Section

---
id: installation
title: Installation
sidebar_label: Installation
---

We recommend that you use our binary distribution through pypi inside of a virtual environment, which supports both MacOs and Linux. On Windows we have successfully gotten pyre to work through WSL, but do not officially support it.

Binary Distribution Copy Chapter

$ (venv) $ pip install pyre-check

IDE Integration Copy Chapter

Building from Source Copy Chapter

Requirements Copy Section

Building the OCaml binary Copy Section

$ git clone https://github.com/facebook/pyre-check

You can complete the setup of your development environment with:

$ cd pyre-check
$ ./scripts/setup.sh --local

This will compile Pyre and run all the unit tests. This is likely going to take some time on your system.

You can now make changes to the code. Run the following commands to compile and test your changes:

$ cd source
$ make
$ make test

Testing changes to the Python Client Copy Section

$ cd /path/to/pyre-check
$ pip install -r requirements.txt
$ ./scripts/run-python-tests.sh

When installing and running pyre from PyPi, the entry point to the executable is actually client/pyre.py. To be able to run this file from anywhere, add the directory containing the pyre-check directory to the PYTHONPATH environment variable and subsequently assign pyre as an alias for pyre-check.client.pyre. For the pyre command to correctly point to the compiled binary, also set the environment variable PYRE_BINARY to source/build/default/main.exe.

$ echo "alias pyre='PYTHONPATH=\"/path/to/pyre-check/..:\$PYTHONPATH\" python -m pyre-check.client.pyre'" >> ~/.bashrc
$ echo "export PYRE_BINARY=/path/to/pyre-check/source/_build/default/main.exe" >> ~/.bashrc
$ source ~/.bashrc

#### Testing changes for Plugin Development
VSCode will not pick up your shell aliases, so the alias step in the previous section will not work if you're doing VSCode Plugin development. To get around this, instead of creating an alias, we can create an executable script called pyre and place it in a directory in our PATH:

#!/bin/bash
PYTHONPATH="/path/to/pyre-check/..:$PYTHONPATH" python -m pyre-check.client.pyre "$@"

$ echo 'PATH="/path/to/pyre-check/scripts:$PATH"' >> ~/.bashrc
$ source ~/.bashrc

Building from Docker Copy Chapter

If you're having issues setting up or your OS is not yet supported, you can also use a Docker image. It runs Debian GNU/Linux 10 (buster) and builds pyre-check from source.

Before starting, ensure that Docker is installed on your computer.

1. Clone the pyre-check repository and navigate to the root directory.

git clone https://github.com/facebook/pyre-check.git
   cd pyre-check

2. Build the Docker image with the tag pyre-check (or another tag if you wish)

docker build -t pyre-docker .

3. Run the new image in a new container pyre-container (or another name if you wish)

docker run \
   --name pyre-container \
   -v /path/to/your/directory:/src \
   -t -i \
   pyre-check

4. Inside the container, run any Pyre command now with pyre!

Note: When initializing Pyre with pyre init, you may have to enter the following paths for the binary and typeshed:

ƛ No pyre.bin found, enter the path manually: /home/opam/pyre-check/source/_build/default/main.exe
   ƛ Unable to locate typeshed, please enter its root: /home/opam/pyre-check/stubs/typeshed/typeshed-master

For contributors: Inside the Docker container, the added pyre-check directory is only editable by the root user. To contribute to Pyre, make edits in your local filesystem and rebuild the Docker by running Step 2, then running a new Docker container in Steps 3-4.

Windows Subsystem for Linux (WSL) Install Copy Chapter

On x86_64 Windows pyre can run via Linux using WSL.
A brief summary to get this running on Ubuntu please follow:
- Install WSL (external Microsoft Documentation)
- Once you have a login to your Linux of choice:
- Optionally: Install build environment (some dependencies of pyre might need to be built)
- Use pip as described above or via a Python Virtual Environment
- Here is an example using Ubuntu with a venv:

$ sudo apt install python3-venv build-essential python3-dev libpython3-dev
$ python3 -m venv /tmp/tp
$ /tmp/tp/bin/pip install --upgrade pip setuptools wheel
$ /tmp/tp/bin/pip install pyre-check
$ source /tmp/tp/bin/activate
$ cd /mnt/c/path/to/repo
$ pyre --source-directory . check
$ (tp) cooper@TESTFAC-1FMHLI2:/mnt/c/path/to/repo$ pyre --source-directory . check
 ƛ Setting up a .pyre_configuration with pyre init may reduce overhead.
 ƛ No type errors found

---

Documentation/Website/Docs/Overview Copy Section

---
id: overview
title: Overview
sidebar_label: Overview
---

import {Redirect} from '@docusaurus/router';

const Home = () => {
return <Redirect to="/docs/getting-started" />;
};

---

Documentation/Website/Docs/Pyre Faq Copy Section

---
id: pyre-faq
title: FAQ & Troubleshooting
sidebar_label: FAQ
---
import {OssOnly, FbInternalOnly} from 'docusaurus-plugin-internaldocs-fb/internal';

<FbInternalOnly>

WIP

Thrift Stubs Copy Chapter

</FbInternalOnly>

<OssOnly>

WIP

</OssOnly>

Pyre Rage Copy Chapter

Typeshed & Stubs Copy Chapter

---

Documentation/Website/Docs/Pysa Advanced Copy Section

---
id: pysa-advanced
title: Advanced Topics
sidebar_label: Advanced Topics
---

This page documents more advanced bits of Pysa.

Conditional models Copy Chapter

Pysa models support if conditions for version and platform checks. This allows
for conditional parsing of models based on the Python version or the target
platform.

Conditional models based on Python version Copy Section

if sys.version == (3,9,0):
    def module.foo(): ...
else:
    def module.bar(): ...

sys.version is the only allowed left hand expression and the right hand expression
has to be a tuple of integers of the form (major, minor, micro). Only the major
version number is required and the other two are optional.

The comparison operators supported include == (equal to), != (not equal to),
< (less than), > greater than, <= (less than or equal to), and
>= (greater than or equal to).

If conditions can also be nested inside one another and follow the same behavior
as python if conditions.

Conditional models based on platform Copy Section

Models can also be conditionally enabled based on the target platform using
sys.platform:

if sys.platform == "linux":
    def module.foo(): ...
else:
    def module.bar(): ...

In this example, the first model will only be parsed if the target platform is
"linux". Otherwise, the second model will be used.

The supported platform strings include "linux", "win32", and "darwin",
matching Python's sys.platform values.

Only == (equal to) and != (not equal to) comparison operators are supported
for platform checks.

Platform and version conditions can be nested:

if sys.version >= (3, 10):
    if sys.platform == "linux":
        def module.foo(): ...

You can also combine conditions using and, or, and not:

if sys.version >= (3, 10) and sys.platform == "linux":
    def module.foo(): ...
if sys.version < (3, 9) or sys.platform == "win32":
    def module.bar(): ...
if not sys.platform == "darwin":
    def module.baz(): ...

Obscure models Copy Chapter

When Pysa does not have enough information about a function or method, it will
make basic assumptions about its behavior. This is referred to as an obscure
model. Most notably, it assumes that the function or method propagates the
taint from its arguments to its return value.

This usually happens when Pysa doesn't know about the callee of a function call:

def foo(f: Any):
    x = input()
    y = f(x) # no information about f, y will be considered tainted.
    eval(y)

Functions and methods defined in type stubs or in a different language (for
instance, in C or C++ bindings) will also be treated as obscure models.

To prevent a function or method from being marked as obscure, one can use the
@SkipObscure taint annotation in a .pysa file:

@SkipObscure
def module.foo(): ...

Parameter and return path Copy Chapter

When writing a model for a source, the ReturnPath annotation allows to specify
which index or attribute of the returned value is tainted. For instance:

def only_attribute_foo_tainted() -> TaintSource[Test, ReturnPath[_.foo]]: ...

Similarly, the ParameterPath annotation allows to specify which index or attribute
of an argument leads to a sink. For instance:

def only_arg_dot_bar_is_sink(arg: TaintSink[Test, ParameterPath[_.bar]]): ...

Access path definition Copy Section

The ParameterPath and ReturnPath annotation takes an access path as an argument.
An access path starts with an underscore _ which represents the whole argument or
return value (depending on the context). The underscore can be followed by attribute
accesses (e.g, _.foo.bar) and index accesses (e.g, _["foo"][0]["bar"]), or a
combination of both (e.g, _.foo[0]).

In addition to these, four special calls can be used: .all(), .keys(),
.parameter_name() and .all_static_fields().

#### all()

.all() is used to represent that any index might be tainted. This is usually when
the index cannot be known statically. For instance:

def foo(i: int):
  i = random.randint(0, 100)
  return {i: source()}

This can be represented by the model:

def foo(): TaintSource[Test, ReturnPath[_.all()]]: ...

#### keys()

.keys() is used to represent that any key of the dictionary might be tainted.
For instance:

def foo():
  return {source(): 0}

This can be represented by the model:

def foo(): TaintSource[Test, ReturnPath[_.keys()]]: ...

#### all_static_fields()

.all_static_fields() is used to mark all statically-known attributes of the
given parameter or return value as a source or sink. The set of attributes is
determined using the type annotation of the parameter or return value. If it is
not annotated or that we could not find any attributes, the whole parameter or
return value will be marked as a source or sink instead.

For instance:

class A:
  x: str
  y: str
def foo(a: A) -> B: ...

Using the following model:

def foo(a: TaintSink[Test, ParameterPath[_.all_static_fields()]]): ...

This will add a sink on a.x and a.y.

This can also be used on the return value:

def foo() -> TaintSource[Test, ReturnPath[_.all_static_fields()]]: ...

In general, we recommend to mark the whole parameter or return value as a
source or sink. This feature is only useful for power users that post
process the result of the analysis and extract leaf ports. It is also very
computationally expensive.

#### parameter_name()

.parameter_name() will be replaced by the name of the parameter that is being
modelled. This can only be used for TaintInTaintOut on parameters of functions
or methods. This is usually useful to model constructors of dataclass-like
classes.

For instance:

class A:
  def __init__(self, x, y, z):
    # method too complicated
    pass

Using the following model query:

ModelQuery(
  name="constructors",
  find="method",
  where=[fully_qualified_name.equals("A.__init__")],
  model=[
    Parameters(TaintInTaintOut[LocalReturn, NoCollapse, ReturnPath[_.parameter_name()]]),
    Modes([SkipAnalysis])
  ]
)

This will automatically propagate taint from parameters x, y, z to
self.x, self.y and self.z. Note that if the code is available and the
constructor is not too complex, Pysa will do that automatically without the need
for a model.

Taint In Taint Out Copy Section

ParameterPath and ReturnPath can also be used to give more information about
a propagation. For instance:

def foo(arg):
  return {"a": arg["b"][42]}

This can be represented by the model:

def foo(arg: TaintInTaintOut[ParameterPath[_["b"][42]], ReturnPath[_["a"]]]): ...

Note that Pysa will automatically infer propagations if it has access to the body
of the function. Writing taint-in-taint-out models should rarely be required.

When using the Updates annotation, the annotation UpdatePath is used instead
of ReturnPath. For instance:

def MyClass.updates_foo(self, x: TaintInTaintOut[Updates[self], UpdatePath[_.foo]]): ...

Taint propagation from arguments to self Copy Chapter

By default, Pysa infers taint propagation from arguments to self for all methods,
including constructors, property setters and the special __setitem__ method.

For instance:

class Foo:
  def __init__(self, x):
    self.x = x
  def set_x(self, x):
    self.x = x
def issue():
  foo = Foo(source())
  sink(foo)  # Issue (1) found.
  foo = Foo("")
  foo.set_x(source())
  sink(foo)  # Issue (2) found.

To disable the inference of propagations from arguments to self for all methods - except
constructors, property setters and __setitem__ - one can provide the command
line argument --no-infer-self-tito. This should reduce the analysis time, as well
as reduce false positives. The counterpart is that it can also lead to false negatives.
For instance, Pysa wouldn't find issue (2) anymore.

When using --no-infer-self-tito, we can use the taint annotation @InferSelfTito
in a .pysa file to enable inference for specific methods:

@InferSelfTito
def my_module.Foo.set_x(): ...

Taint propagation between arguments Copy Chapter

By default, Pysa does NOT infer taint propagation between arguments. For short,
it assumes that functions do not mutate their arguments.

For instance, this flow will NOT be found:

def append_wrapper(l: List[str], v: str) -> None:
  l.append(v)
def issue():
  l = []
  append_wrapper(l, source())
  sink(l[0])  # Issue NOT found.

To enable the inference of propagations between arguments for all functions and
methods, one can provide the command line argument --infer-argument-tito or
use the taint annotation @InferArgumentTito in a .pysa file:

@InferSelfTito
def my_module.append_wrapper(): ...

Taint broadening Copy Chapter

Taint broadening is an over-approximation performed by the taint analysis
for correctness or performance reasons. After applying broadening, Pysa
considers that a whole object or variable is tainted when only some attributes
or keys were initially tainted.

This is also called taint collapsing or tree collapsing because the taint
is internally represented as a tree structure where edges are attributes or keys.
Collapsing means merging the taint on all children into the root of the tree.

For instance, this happens when Pysa does not have access to the body of a
function:

def obscure_function(arg): ...

def foo():
  # Only x['a'] is tainted.
  x = {"a": source()}
  # Taint broadening happens, y and all its attributes are considered tainted.
  y = obscure_function(x)
  # This is considered an issue, even if only x['a'] was initially tainted.
  sink(y['b'])
  # Also an issue, y is entirely tainted.
  sink(y)

Note that whenever broadening happens, Pysa will automatically add a broadening
feature on the taint flow, which can help discard false
positives in post processing. Fine grained features are used for each different
scenario leading to broadening.

The most common causes for taint broadening are the following:

Broadening on obscure models Copy Section

Taint that flows through an obscure model - for instance,
when Pysa does not have access to the body of the callee - is collapsed, since
we must assume anything could get tainted, for correctness.

In this scenario, the tito-broadening and via:obscure:model features are
added to the flow.

Broadening on taint-in-taint-out (TITO) Copy Section

When specifying a taint propagation (also
called Taint In Taint Out or TITO) in a .pysa file, the propagation
will collapse the taint by default.

For instance:

models.pysa

def tito(arg: TaintInTaintOut): ...

def foo():
  x = {"a": source()}
  y = tito(x)
  sink(y['b']) # Considered an issue because of taint broadening.

In this scenario, the tito-broadening feature is added to the flow.

If the function is known to preserve the structure of the argument, the
NoCollapse annotation can be used to disable collapsing. For instance:

def tito(arg: TaintInTaintOut[NoCollapse]): ...

This would remove the issue from the previous example.

Note that this can be used in combination with
ParameterPath and ReturnPath.

Model broadening Copy Section

When the number of tainted attributes or keys hits a certain threshold, taint
broadening is applied to prevent the analysis from blowing up by tracking too
many values.

This is referred as Model broadening since this happens when the model (or
summary) of a function is computed.

For instance, this can happen when the number of tainted key-value pairs of a
dictionary hit a certain threshold. For scalability reasons, Pysa cannot track
an infinite amount of indices, and thus makes the approximation that the whole
object is tainted.

def foo(condition):
    d = {}
    if condition:
        d["a"] = source()
        d["b"] = source()
        # c, d, e, etc.
    else:
        d["1"] = source()
        d["2"] = source()
        # etc.
    return d # too many indexes, the whole return value is considered tainted.

In this scenario, the model-broadening feature is added to the flow.

See analysis thresholds for documentation about the
different scenarios of model broadening.

Note that model broadening can be disabled for a given function or method using
the @SkipModelBroadening annotation in a .pysa file:

@SkipModelBroadening
def foo(): ...

This can also be used in a ModelQuery using the
Modes clause.

Note that this should be used sparingly since this can potentially lead to an
increase in analysis time.

Model shaping Copy Section

When a specific attribute or key is tainted when the whole object is tainted
with the same taint kind (e.g, UserControlled), taint collapsing is applied
as an optimization to save analysis time. This is called model shaping and
is applied right before model broadening.

For instance:

def my_sink(x):
  sink(x)
  sink(x.foo)

The sink on x.foo (represented as formal(x)[foo]) is merged into the sink
on x.

Note that this is sound since attributes of a tainted object are also considered
tainted. Thus if my_sink is actually called with x.foo tainted, the flow
will be found as expected.

The downside is that this can lead to false positives in cases where my_sink
is called with another attribute (say x.bar) tainted. We would find a flow
from x.bar to x.foo in my_sink.

In this scenario, the model-shaping feature is added to the flow. The features
model-source-shaping, model-sink-shaping and model-tito-shaping are also
added to differentiate whether the shaping was on sources, sinks or tito.

Widen broadening Copy Section

When the number of tainted attributes or keys is potentially infinite because of
a loop or recursion, taint broadening is applied to allow the termination of the
analysis.

The term "widen" or "widening" refers to an operator that is applied to ensure
convergence. It commonly happens within loops.

For instance:

def foo(n):
  d = {}
  for _ in range(n):
    d = {
      "a": source(),
      "b": d,
    }
  return d

Technically, d['b']...['b']['a'] (with an infinite number of access to b)
could be tainted. To allow the analysis to terminate, Pysa stops at a certain
depth. See analysis thresholds for documentation about
the different scenarios of widen broadening.

Another example:

def foo(person):
    while person.parent is not None:
        person = person.parent
        # Infer sinks on person.name, person.parent.name, person.parent.parent.name, etc.
        sink(person.name)

In these scenarios, the widen-broadening feature is added to the flow.

Issue broadening Copy Section

When an object with a tainted attribute or key reaches a sink, Pysa considers
the flow as valid even if the whole object is not tainted.

For instance:

d = {"a": source(), "b": "foo"}
sink(d) # d itself is not tainted, but d["a"] is, thus we emit an issue.

In this scenario, the issue-broadening feature is added to the issue.

Tainting Specific kwargs kwargs')" title="Copy chapter prompt for LLMs"> Copy Chapter

Sometimes, a function can have potential sinks mixed together with benign
parameters in the keyword arguments (kwargs) that it accepts. In these cases,
tainting the whole kwargs variable will result in false positives when tainted
data flows into a benign kwarg. Instead, for a function like this:

def eval_and_log(kwargs):
    eval(kwargs["eval"])
    logging.debug(kwargs["log"])

We can lie a bit in our .pysa file, and break out the dangerous argument for
tainting:

def eval_and_log(, eval: TaintSink[RemoteCodeExecution], *kwargs): ...

This allows us to catch flows only into the eval keyword argument.

Instance attributes versus class attributes Copy Chapter

Models can specify sources and sinks on attributes, following the type annotation
syntax:

django.http.request.HttpRequest.GET: TaintSource[UserControlled]

Any access to request.GET will be tainted when request is an instance of
HttpRequest or any of its children. However, note that the access to the class
attribute (i.e, HttpRequest.GET) won't be considered tainted.

To specify sources and sinks on class attributes, use the __class__ prefix:

django.http.request.HttpRequest.__class__.GET: TaintSource[UserControlled]

To specify a source on both the class attribute and instance attribute, simply
use both lines.

Literal String Sources And Sinks Copy Chapter

Some security vulnerabilities are best captured by modeling strings of a given
form flowing to dangerous functions, or format strings that match a pattern getting
tainted data passed in.

To mark all literal strings matching a pattern as sources, you first need to add a
regular expression corresponding to the pattern to your taint.config:

{
  "sources": [
    {
      "name": "IPAddress"
    }
  ],
  "implicit_sources": {
     "literal_strings": [
       {
         "regexp": "\\d{1,3}(\\.\\d{1,3})+",
         "kind": "IPAddress",
         "description": "String that looks like an IP address."
       }
     ]
  }
}

With this regex in place, whenever Pysa sees a string such as 123.456.789.123, it will flag it
as a taint source with the kind IPAddress.

def test() -> None:
    ip_address = "123.456.789.123"
    dont_pass_an_ip_address(ip_address) # Pysa will now flag this.

The converse of supporting literal strings as sinks is also supported, for data flowing into a tainted string. The
syntax allows you to model data being used to format strings, like f-strings, manual string formatting, the string format() method, and printf-style string formatting with %.

Template strings and manual string formatting with more than two subexpressions are not yet supported.

To add a literal sink, first add the literal_sink to your configuration

{
  "sinks": [
    { "name": "MayBeRendered" },
    { "name": "MayBeSQL" }
  ],
  "implicit_sinks": {
     "literal_strings": [
       {
         "regexp": "^<.*>$",
         "kind": "MayBeRendered",
         "description": "Indicates a string whose contents may be rendered."
       },
       {
         "regexp": "^SELECT *.",
         "kind": "MayBeSQL",
         "description": "Indicates a string whose contents may be a SQL query."
       }
     ]
  }

Now, Pysa will treat any values flowing into a each of the following as a regular sink:

def may_render(parameter: str) -> None:
    result = f"<content={parameter}>"
    result = "<content={}>".format(parameter)
    result = "<content%s>" % (parameter,)

def build_sql_query(columns: str) -> None:
    result = f"SELECT {columns} FROM users;"
    result = "SELECT {} FROM users;".format(columns)
    result = "SELECT %s FROM users" % (columns,)
    result = "SELECT " + columns + " FROM users;"

Note that string literal sinks have some limitations. For instance, they cannot catch issues if the string literal is stored in a temporary variable (e.g., x = "SELECT {}"; x.format(input)). We recommend using string combine rules instead (see below).

Combined Source Rules Copy Chapter

Some security vulnerabilities are better modeled as two sources reaching
sinks at the same call site. For example, leaking credentials via requests.get could be modeled as
user controlled data flowing into the url parameter and credentials flowing
into the params parameter. These flows can be modeled by *combined source
rules*.

Sources for combined source rules are declared as normal in taint.config.
Sinks, however, are declared inside the rule definition, and are referred to as partial sinks.
The rule itself is declared in the combined_source_rules top level entry.
The actual flows are defined under section rule, which must contain two flows, one for each source. Each flow specifies a pair of (potentially multiple) sources and a single partial sink:

{
  "sources": [
    { "name": "UserControlled" },
    { "name": "Credentials" }
  ],
  "combined_source_rules": [
    {
       "name": "Credentials leaked through requests",
       "rule": [
        {
           "sources": [ "UserControlled" ],
           "partial_sink": "UserControlledRequestSink"
         },
         {
           "sources": [ "Credentials" ],
           "partial_sink": "CredentialsSink"
         }
       ],
       "code": 1,
       "message_format": "Credentials leaked through requests",
       "main_trace_source": "url",
    }
  ]
}

Sources are declared as normal in .pysa files. Instead of specifying sinks
with a TaintSink annotation, however, PartialSink annotations are used to
specify where each source needs to flow for the combined source rule. These
PartialSink must reference the ones that were declared by
the rule above:

def requests.api.get(
  url: PartialSink[UserControlledRequestSink],
  params: PartialSink[CredentialsSink],
  kwargs
): ...

With the above configuration, Pysa can detect cases where UserControlled flows
into url and Credentials flow into params at the same time (or at the same call site).

Note that the same partial sink can be used in different rules, which avoids duplicating a given model for each rule (in some cases).

String Combine Rules Copy Chapter

It is sometimes useful to detect data tainted with a source (e.g., UserControlled data) that is incorporated into a suspicious looking string (e.g., a string that looks like a SQL query). Detecting such a pattern is useful, because it allows Pysa to detect dynamic creation of SQL queries which could lead to SQL injection, even if the code under analysis is using a SQL framework that Pysa does not have models for.

To detect such flows, one can specify a variant of combined source rules, called string combine rules, to detect when the suspicious string (identified via regex match) and the other configured source both flow into string formatting call sites (such as calling str.__add__, str.__mod__, str.format or constructing f-strings).

For example, to detect flows from source UserControlled to sink StringMayBeSQL, one should specify the following contents in the taint configuration file, where UserControlled and StringMayBeSQL are both declared as sources:

{
  "sources": [
    { "name": "UserControlled" },
    { "name": "StringMayBeSQL" }
  ],
  "implicit_sources": {
    "literal_strings": [
      {
        "regexp": "SELECT.*",
        "kind": "StringMayBeSQL",
        "comment": "matches a SQL statement"
      }
    ]
  },
  "string_combine_rules": [
    {
       "name": "User controlled data flows into potential SQL strings",
       "rule": [
        {
           "sources": [ "UserControlled" ],
           "partial_sink": "UserControlledDataSink"
         },
         {
           "sources": [ "StringMayBeSQL" ],
           "partial_sink": "StringMayBeSQLSink"
         }
       ],
       "code": 4324,
       "message_format": "User controlled data flows into potential SQL strings"
    }
  ]
}

The above rule enables catching the following flows:

def issue():
  uc = user_controlled()
  f"SELECT {uc} FROM async_query"
  "SELECT " + uc + " FROM async_query"
  "SELECT %s FROM async_query" % uc
  "SELECT {} FROM async_query".format(uc)

Note that the string combine rules are strictly more powerful than the feature of implicit literal string sinks.
That is, any flow that can be detected via implicit literal string sinks can also be detected via string combine rules, but not vice versa.
Hence, we recommend using the string combine rules.

Prevent Inferring Models with SkipAnalysis SkipAnalysis')" title="Copy chapter prompt for LLMs"> Copy Chapter

In addition to the models defined in .pysa files, Pysa will infer models for
functions based what sources, sinks, etc. they call in their body. The
SkipAnalysis annotation can be used to prevent Pysa from inferring models, and
instead force it to use only the user defined models for determining taint flow:

@SkipAnalysis
def qualifier.dont_generate_models(argument): ...

SkipAnalysis can be applied at the class level as a shorthand to prevent pysa
from infering models for all functions in a class:

class skip_analysis.SkipMe(SkipAnalysis): ...

Ignoring overrides Copy Chapter

When a method is called on a base class, Pysa has to assume that that call could
actually invoke any subclass methods that override the base class's method. For
heavily overriden methods, this can lead to both performance impacts and false
positives. When running Pysa, you may see messages such as this in the output:

2020-09-02 09:25:50,677 WARNING object.__init__ has 106 overrides, this might slow down the analysis considerably.

The above message indicates that 106 subclasses of object have overridden
__init__. If Pysa sees taint flowing into object.__init__, then it will
treat all 106 overrides of object.__init__ as also receiving that taint.

The @SkipOverrides decorator can be applied to deal with false positives or
performance issues from having too many overrides on a given function:

@SkipOverrides
def object.__init__(self): ...

This annotation will cause Pysa not to propagate taint into to and from
overridden methods on subclasses, when analyzing functions that call the
overriden method on the base class.

maximum_overrides_to_analyze can be added the the options block of
taint.config to limit the number of overrides that Pysa will analyze:

{
  "sources": [],
  "sinks": [],
  "features": [],
  "rules": [],
  "options": {
    "maximum_overrides_to_analyze": 60
  }
}

This option can also be provided in the command line, using
--maximum-overrides-to-analyze.

This can speed up the analysis, but it will lead to false negatives, because
Pysa will only propagate taint to or from 60 (in the case of the above example)
overriden methods on subclasses. The remaining overriding methods will be
ignored and treated as if they weren't actually overriding the base class
method.

By default, Pysa skips overrides on some functions that are typically
problematic. You can find the full list of default-skipped functions in
stubs/taint/common/skipped_overrides.pysa

Force to analyze all overrides Copy Chapter

We also allow the use of decorator @AnalyzeAllOverrides to force analyzing all overriding methods of a given method, regardless of the configured maximum number of overrides to analyze (e.g., via command line option --maximum-overrides-to-analyze), or if there simultaneously exists an @SkipOverrides on the given method. An example is:

@AnalyzeAllOverrides
def BaseClass.method(self): ...

Decorator @AnalyzeAllOverrides is often used to reduce false negatives, by analyzing all overrides of some selected methods. This offers a more fine-grained option than tweaking --maximum-overrides-to-analyze for all methods. Adding decorator @AnalyzeAllOverrides to some selected methods is faster than using a large threshold of maximum overrides for all methods, but achieves better precision than using a small threshold.

Force to treat a callable as being called when passed as parameters Copy Chapter

We allow the usage of decorator @CalledWhenParameter to force treating the callable as being called, when the callable is passed as a parameter. An example is:

@CalledWhenParameter
def foo(): ...
def bar(f):
  f()
def main():
  bar(foo) # We would treat foo as being called here.

Skip inlining higher order functions with @SkipInliningHigherOrderFunctions @SkipInliningHigherOrderFunctions')" title="Copy chapter prompt for LLMs"> Copy Chapter

By default, when Pysa detects a higher order function (i.e., a function that
takes another function as a parameter and calls it), it creates parameterized
targets — specialized versions of the function for each concrete callable
passed as an argument. This allows Pysa to inline the callee and track taint
through the higher order call precisely.

For example, given:

def apply(f, x):
    return f(x)
def goes_to_sink(arg):
    sink(arg)
apply(goes_to_sink, source())  # Pysa creates apply[f=goes_to_sink] and finds the issue.

In some cases, this specialization is undesirable — for instance, if a function
is called with many different callables, creating parameterized targets for each
one can increase analysis time without providing useful results.

The @SkipInliningHigherOrderFunctions annotation prevents Pysa from creating
parameterized targets for a given function:

@SkipInliningHigherOrderFunctions
def module.apply(f, x): ...

With this annotation, Pysa will not inline the higher order parameter f at
call sites of module.apply. Instead, at a call site such as
apply(goes_to_sink, source()), Pysa will treat goes_to_sink as being called
directly at that call site (i.e., as if goes_to_sink(source()) was called),
rather than creating a specialized version apply[f=goes_to_sink]. See
Force to treat a callable as being called when passed as parameters
for more details on this behavior.

This is useful for reducing analysis cost on functions that are called with many
different callables, when the resulting precision is not needed.

Skip calling higher order parameters with @SkipCallHigherOrderFunctions @SkipCallHigherOrderFunctions')" title="Copy chapter prompt for LLMs"> Copy Chapter

The @SkipCallHigherOrderFunctions annotation prevents Pysa from recording the
implicit call to higher order parameters at the call site. Unlike
@SkipInliningHigherOrderFunctions, it does not prevent the creation of
parameterized targets — the higher order call graph fixpoint still inlines the
function normally when it can.

For example, given:

@SkipCallHigherOrderFunctions
def module.apply(f, x): ...

At a call site such as apply(goes_to_sink, source()), Pysa will:
- Still create a specialized version apply[f=goes_to_sink] and analyze it,
if the function has an available body (i.e, not a stub).
- Not treat goes_to_sink as being called directly at the call site
(i.e., the implicit call from the higher_order_parameters is removed).

This is useful when the implicit call recording causes false positives — for
instance, when the implicit call is redundant because the parameterized target
analysis already provides the correct result.

In most cases, you will want to use both annotations together:

@SkipInliningHigherOrderFunctions
@SkipCallHigherOrderFunctions
def module.apply(f, x): ...

This both prevents parameterized target creation and removes the implicit call,
effectively disabling all higher order function handling for this function.

Limit the trace length for better signal and performance Copy Chapter

By default, Pysa will find all flows from sources to sinks matching a rule.
This can lead to very long traces which are hard to understand and tend to be
false positives. This also brings down the performance a lot.

Pysa provides a --maximum-trace-length <integer> command line argument which
limits the length of traces that it finds. In general, this will also make Pysa
faster.

This option can also be added in the taint.config as follows:

{
  "sources": [],
  "sinks": [],
  "features": [],
  "rules": [],
  "options": {
    "maximum_trace_length": 20
  }
}

Note that this is not a silver bullet and that this might hide security
vulnerabilities. Use it with caution.

Limit the trace length for a given rule Copy Chapter

Similarly to the option described above, one can limit the trace length
for a given rule, using the filters option:

"rules": [
  {
    "name": "SQL injection.",
    "code": 1,
    "sources": [ "UserControlled" ],
    "sinks": [ "SQL" ],
    "message_format": "Data from [{$sources}] source(s) may reach [{$sinks}] sink(s)",
    "filters": {
      "maximum_source_distance": 10,
      "maximum_sink_distance": 5
    }
  }
]

This will limit the trace length from the root to the source by 10, and the
trace length from the root to the sink by 5, only for that specific rule.

Note: This is meant to be used to limit the number of issues written to the
database. Prefer using SAPP to
filter out false positives.

Limit the tito depth for better signal and performance Copy Chapter

Pysa automatically infers when a function propagate the taint from one argument
to its return value. This is called tito, for "Taint In Taint Out". In practice,
infering it can be very expensive since the taint can go through an arbitrary
number of hops (i.e, depth).

For instance:

def foo(x):
  return x
def bar(x):
  return foo(x)
def baz(x):
  return bar(x)

In this example, baz propagates the taint on its argument to the return value
using 3 hops.

Pysa provides a --maximum-tito-depth <integer> command line argument which
limints the depth of inferred propagations. In combination with the trace length
limit, this usually makes Pysa faster.

This option can also be added in the taint.config as follows:

{
  "sources": [],
  "sinks": [],
  "features": [],
  "rules": [],
  "options": {
    "maximum_tito_depth": 20
  }
}

Decorators Copy Chapter

By default, Pysa does not generally understand decorators, and will treat a call to a decorated function as an obscure call.

This will usually lead to false negatives if the decorated functions has sources or sinks. For instance:

def identity(f: Callable[[str], None]) -> Callable[[str], None]:
  return f
@identity
def decorated_sink(x: str) -> None:
  sink(x)
decorated_sink(source()) # False negative, issue will NOT be found!

This also leads to false negatives if the decorator adds a flow to a sink. For instance:

def with_sink(f: Callable[[str], None]) -> Callable[[str], None]:
  def inner(x: str) -> None:
    sink(x)
    f(x)
  return inner
@with_sink
def foo(x: str) -> None:
  print(x)
foo(source()) # False negative, issue will NOT be found!

Since the call to a decorated function is treated as an obscure call, it will conservatively propagate taint through decorated function:

def identity(f: Callable[[str], str]) -> Callable[[str], str]:
  return f
@identity
def decorated(x: str) -> str:
  # Whatever happens here will not be considered at the call site.
  return 'hello %s' % x
sink(decorated(source())) # Issue is properly found.

Pysa provides a few ways to deal with these limitations.

Ignoring decorators Copy Chapter

Pysa can entirely ignore a decorator, as if it was not present in the source code. This can be done safely when the decorator does not change the signature of the decorated function (i.e, it does not add or remove parameters).

To ignore a decorator, use the @IgnoreDecorator annotation in a .pysa file:

@IgnoreDecorator
def module.decorator(): ...

Single trace sanitizers with @SanitizeSingleTrace @SanitizeSingleTrace')" title="Copy chapter prompt for LLMs"> Copy Chapter

Sanitizers, as described in the Overview, are applied in both
the forward (i.e source) trace and backward (i.e sink) trace.

For instance, with the given .pysa file:

@Sanitize(TaintInTaintOut[TaintSink[RemoteCodeExecution]])
def shlex.quote(x): ...

And the following Python code:

import subprocess
from shlex import quote
def quoted_input():
  x = input() # source 'UserControlled'
  y = quote(x)
  return y
def echo(argument):
  subprocess.run(f'/bin/echo {argument}', shell=True) # sink 'RemoteCodeExecution'
def issue():
  x = quoted_input() # source trace: input -> quoted_input -> issue
  echo(x) # sink trace: issue -> echo -> subprocess.run

Pysa will NOT find an issue here, as expected.
This is because during the propagation of the 'UserControlled' source in the
forward trace, pysa remembers that it was sanitized for the sink 'RemoteCodeExecution'.

However, Pysa provides a simpler version of sanitizers, which only sanitizes in the
forward trace or the backward trace:

@SanitizeSingleTrace(TaintSource)
def f(): ...
@SanitizeSingleTrace(TaintSource[UserControlled])
def g(): ...
@SanitizeSingleTrace(TaintSink)
def h(): ...
@SanitizeSingleTrace(TaintSink[RemoteCodeExecution])
def i(): ...

These sanitizers are a lot cheaper and could save analysis time. However, these
might introduce false positives, so we recommend to use the default sanitizers.

Filtering the call graph with @Entrypoint @Entrypoint')" title="Copy chapter prompt for LLMs"> Copy Chapter

By default, Pysa will analyze the entire call graph of your program. This can lead to longer analysis times for larger programs, especially when you'd only like to perform analysis on specific parts of the program. This decorator will mark a specified function and the functions it calls as the only functions to be analyzed.

Note: the flag --limit-entrypoints must be passed to pyre analyze for call graph filtering to occur, even if the @Entrypoint decorator is present. This allows for call graph filtering to be easily enabled or disabled without editing your .pysa files.

If you have the following Python file:

class MyClass:
  def class_entrypoint():
    taint_sink(taint_source())
def my_bad_func_1():
  taint_sink(taint_source())
def my_bad_func_2():
  taint_sink(taint_source())
def func_entrypoint():
  my_bad_func_1()
def main():
  func_entrypoint()
  my_bad_func_2()
  MyClass().class_entrypoint()
main()

And the following .pysa file:

@Entrypoint
def my_file.MyClass.class_entrypoint(): ...
@Entrypoint
def func_entrypoint(): ...

Then issues will be found for taint in calls to class_entrypoint and my_bad_func_1, but not my_bad_func_2, since it isn't called by a function marked by an @Entrypoint.

Add a feature to all local variables on function call Copy Chapter

In some situations, one might want to filter out issues where a specific function was called,
but that call is not directly involved in the data flow.

For instance:

x = some_source()
if security_check():
    some_sink(x)

The problem here is that security_check() does not transfer taint.
Sanitizing this function would have no effect, Pysa will still find the issue.

A solution is to use @AddFeatureToState:

@AddFeatureToState(Via[security_check])
def my_module.security_check(): ...

This will add the feature via:security_check to all local variables when calling security_check().
In the example above, the issue from some_source() to some_sink() will therefore have that feature,
and it can be filtered out in post processing (see features).

Note that AddFeatureToState is not contagious (i.e it is not propagated up):

def security_check_wrapper():
  return security_check()
x = some_source()
if security_check_wrapper():
    some_sink(x)

Note also that AddBreadcrumbToState is an alias for AddFeatureToState.

Taint In Taint Out Transforms Copy Chapter

Taint in taint out transforms can be used to capture more precise flows.

As an example:

def read_file(path):
  with open(path, "r") as f:
    content = f.read()
  return content

To contrast with feature annotations, there are two differences:
* The filtering is done during analysis itself, and limits the issues generated (as opposed to a post-processing step by the user)
* Taint in taint out transforms can be used to reason about the order of events

Syntax Copy Section

{
  ...
  "transforms": [
    {
      "name": "MyTransform",
      "comment": "This is my transform"
    },
    ...
  ],
  ...
}

Then, one may use these transforms in rules as follows:

{
  ...
  "rules": [
    {
      "name": ...,
      "code": ...,
      "sources": ["SourceA"],
      "transforms": ["MyTransform1", "MyTransform2"],
      "sinks": ["SinkB"],
      "message_format": "[{$sources}] transformed by [${transforms}] may reach [${sinks}]"
    },
    ...
  ],
  ...
}

Finally, in .pysa model files a taint transform can be specified using a TaintInTaintOut[Transform[...]] annotation, where the parameter is the name of the transform.

def my_function(arg: TaintInTaintOut[Transform[MyTransform]]): ...

Semantics Copy Section

y = my_function(x)

Note that a transform modifies the taint itself. Hence, if a flow passes through a transform, it will no longer match rules which do not contain the transform.

RuleX: SourceA -> SinkB
RuleY: SourceA -> MyTransform -> SinkB
Flow1: SourceA -> SinkB
Flow2: SourceA -> MyTransform -> SinkB

Consider the scenario where we have an additional rule:

RuleZ: SourceC -> SinkD

Also note that the existing TaintInTaintOut annotation semantics of TITO being assumed (instead of inferred) on the argument are unchanged.

Analysis thresholds Copy Chapter

Pysa provides many options to fine tune the taint analysis. The following
options can be provided either via the command line or in the taint.config file,
under the options section.

For instance:

{
  "sources": [],
  "sinks": [],
  "features": [],
  "rules": [],
  "options": {
    "maximum_model_source_tree_width": 10,
    "maximum_model_sink_tree_width": 10,
    "maximum_model_tito_tree_width": 10
  }
}

When not provided, these are set to the following defaults:
``ocaml file=source/interprocedural_analyses/taint/taintConfiguration.ml start=DOCUMENTATION_CONFIGURATION_START end=DOCUMENTATION_CONFIGURATION_END

Maximum model source tree width  Copy Section
* Command line option: --maximum-model-source-tree-width
* taint.config option: maximum_model_source_tree_width
See taint broadening and model broadening.
This limits the width of the source tree in the model for a callable, i.e
the number of output paths in the return value.
For instance:

The source tree for foo has a width of 3. Above the provided threshold, pysa
will collapse the taint and consider the whole dictionary tainted. When that
happens, the features model-broadening and model-source-broadening will
be added to the flow.
Maximum model sink tree width  Copy Section
* Command line option: --maximum-model-sink-tree-width
* taint.config option: maximum_model_sink_tree_width
See taint broadening and model broadening.
This limits the width of the sink tree in the model for a callable, i.e
the number of input paths leading to a sink for a given parameter.
For instance:

The sink tree for foo and parameter arg has a width of 3.
Above the provided threshold, pysa will collapse the taint and consider that the
whole argument leads to a sink. When that happens, the features
model-broadening and model-sink-broadening will be added to the flow.
Maximum model tito tree width  Copy Section
* Command line option: --maximum-model-tito-tree-width
* taint.config option: maximum_model_tito_tree_width
See taint broadening and model broadening.
This limits the width of the taint-in-taint-out tree in the model for a callable,
i.e the number of input paths propagated to the return value, for a given parameter.
For instance:

The taint-in-taint-out tree for foo and parameter arg has a width of 3.
Above the provided threshold, pysa will collapse the taint and consider that the
taint on the whole argument is propagated to the return value. When that happens,
the features model-broadening and model-tito-broadening will be added to
the flow.
Maximum tree depth after widening  Copy Section
* Command line option: --maximum-tree-depth-after-widening
* taint.config option: maximum_tree_depth_after_widening
See taint broadening and widen broadening.
This limits the depth of the source, sink and tito trees within loops, i.e the
length of source, sink and tito paths for each variables.
For instance:

The source tree for variable has a depth of 3 (i.e, a -> b -> c).
Within a loop, pysa limits the depth to the provided threshold. For instance,
if that threshold is 1, we would consider that variable.a is entirely tainted.
When that happens, the feature widen-broadening will be added to the flow.
Maximum return access path width  Copy Section
* Command line option: --maximum-return-access-path-width
* taint.config option: maximum_return_access_path_width
See taint broadening and model broadening.
This limits the width of the return access path tree in the model for a callable,
i.e the number of output paths propagated to the return value, for a given parameter.
For instance:

The return access path tree for foo and parameter arg has a width of 3.
Above the provided threshold, pysa will collapse the taint and consider that the
whole return value is tainted whenever arg is tainted. When that happens,
the features model-broadening and model-tito-broadering will be added to
the flow.
Maximum return access path depth after widening  Copy Section
* Command line option: --maximum-return-access-path-depth-after-widening
* taint.config option: maximum_return_access_path_depth_after_widening
See taint broadening and widen broadening.
This limits the depth of the return access path tree within loops, i.e the
length of output paths propagated to the return value, for a given parameter.
For instance:

The return access path tree for foo and parameter arg has a depth  of 3
(i.e, a -> b -> c). Within a loop, pysa limits the depth to the provided
threshold. For instance, if that threshold is 2, we would cut the output path
to just a.b. When that happens, the feature model-broadening and
model-tito-broadening will be added to the flow.
Maximum tito collapse depth  Copy Section
* Command line option: --maximum-tito-collapse-depth
* taint.config option: maximum_tito_collapse_depth
This limits the depth of the taint tree after applying taint-in-taint-out,
i.e the length of paths for taint propagated from a parameter to the return
value.
For instance:

The taint tree for input has a depth of 3 (i.e, a -> b -> c).
When the taint is propagated to the return value of identity, we limit
the resulting taint tree to the given depth. For instance, if that threshold
is 1, we would consider that output['a'] is tainted.
This is also applied for sinks in the backward analysis:

With a threshold of 1, we would consider that output['a'] leads to a sink.
Maximum tito positions  Copy Section
* Command line option: --maximum-tito-positions
* taint.config option: maximum_tito_positions
This limits the number of positions to keep track of when propagating taint.
When taint is propagated through a function and returned (i.e, taint-in-taint-out),
pysa will keep track of the position of the argument, and display it in the trace.
For instance:

In this example, we have 2 tito positions. Above the provided threshold,
pysa simply discards all positions. Note that the taint is still propagated.
Partial Flows  Copy Chapter
SAPP supports the concept of partial flows, which helps with triaging issues from different rules with a common flow. Details are described here.
To leverage this capability in an end-to-end run (i.e., first running Pysa and then SAPP), you can add the following section to your taint.config file, so that they can be passed into the Pysa output files and then be picked up by SAPP:

In other words, the above configuration is not used by Pysa at all. Pysa simply passes them into the output files so that SAPP can read them.
---
Documentation/Website/Docs/Pysa Basics  Copy Section
---
id: pysa-basics
title: Pysa (Static Analysis for Security)
sidebar_label: Overview
---
import Internal from './fb/pysa_basics_internal.md';
import YouTube from 'react-youtube';
import useBaseUrl from '@docusaurus/useBaseUrl';
<img alt="Pysa Logo" style={{width: '400px'}} src={useBaseUrl('img/pysa.png')} />
Pyre has applications beyond type checking python code: it can also run static
analysis, more specifically called Taint Analysis, to identify potential security issues.
The Python Static Analyzer feature of Pyre is usually abbreviated to Pysa
(pronounced like the Leaning Tower of Pisa).
<YouTube videoId="LDxAczqkBiY" containerClassName="youtubeContainer" />
<Internal />
Taint Analysis  Copy Chapter
Tainted data is data that must be treated carefully. Pysa works by tracking
flows of data from where they originate (sources) to where they terminate in a
dangerous location (sinks). For example, we might use it to track flows where
user-controllable request data flows into an eval call, leading to a remote
code execution vulnerability. This analysis is made possible by user-created
models which provide annotations on source code, as well as rules that define
which sources are dangerous for which sinks. Pysa comes with many pre-written
models and rules for builtin and common python libraries.
Pysa propagates taint as operations are performed on tainted data. For example,
if we start with a tainted integer and perform a number of operations on it, the
end results will still be tainted:

Pysa will only analyze the code in the repo that it runs on, as well as code in
directories listed in the search_path of your
.pyre_configuration file. It does not see the source of
your dependencies. Just because you can see code in your editor
does not mean Pysa has access to that code during analysis. Because of this
limitation, Pysa makes some simplifying assumptions. If taint flows into a
function Pysa doesn't have the source for, it will assume that the return type
of that function has the same taint. This helps prevents false negatives, but can
also lead to false positives.
When an object is tainted, that means that all attributes of that object are
also tainted. Note that this is another source of potential false positives,
such as taint flows that include some_obj.__class__. This means that Pysa
will detect all of the following flows:

Pysa Configuration  Copy Chapter
Pysa uses two types of files for configuration: a single taint.config file,
and an unlimited number of files with a .pysa extension. The taint.config
file is a JSON document which stores definitions for sources, sinks, features,
and rules (discussed below). The .pysa files are model files (also discussed
below) which annotate your code with the sources, sinks, and features defined in
your taint.config file. Examples of these files can be found in the Pyre
repository.
These files live in the directory configured by taint_models_path in your
.pyre_configuration file. Any .pysa file found in this folder will be parsed
by Pysa and the models will be used during the analysis.

Sources  Copy Chapter
Sources are where tainted data originates. They are declared in your
taint.config file like this:

Models that indicate what is a source are then defined in .pysa
files. Sources are declared with the same syntax as type annotations in Python
3. Function return types,
class/model attributes, and even entire classes can be declared as sources by
adding TaintSource[SOURCE_NAME] wherever you would add a python type: