### Events Domain Events {#events} ===== ## Introduction Domain Events are a way for MediaWiki core components and extensions to be informed about events that have occurred in another component or extension by registering listeners. Domain Events in MediaWiki are similar to hooks, but offer more flexibility in data modeling and stronger guarantees around the invocation of listener code. They are introduced with MediaWiki 1.44 to improve the developer experience by making use of an established design pattern that is easy to use and helps reduce coupling, making the codebase more sustainable. The idea of domain event in MediaWiki is inspired by domain driven design where the term is used to describe something that happened in the past that users care about, typically something that changes the visible state of the system. Martin Fowler writes: "the essence of a Domain Event is that you use it to capture things that can trigger a change to the state of the application you are developing". For more information see the documentation on mediawiki.org: https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:Domain_events ## Motivation The Domain Event System is intended to enable event oriented processing within and eventually around MediaWiki. Events become first-class concepts for connecting MediaWiki core components and extensions as well as integrating MediaWiki with other services in a distributed system. Domain Events are designed to replace a certain type of hook as an extension interface, and to provide a mechanism for decoupling core components using the observer pattern. Eventually, MediaWiki should become able to broadcast events to and receive events from other wikis and other services. The overall motivation for introducing domain events is to make MediaWiki and extension development more sustainable: * Improve component boundaries between core components by applying the observer pattern (aka listener pattern). Listeners remove the need for code that affects a change to know about all code that needs to be informed about it. * Clarify the semantics of extension callbacks invoked as a result of a change, particularly with respect to transactional context. Standardize deferred update behavior, which will reduce boilerplate code and risk of misimplementation. * Make the extension interface more future proof by avoiding the rigidity imposed by using PHP interfaces to define hook parameters. Due to limitations of PHP, method signatures defined by extensions can’t be modified in a backwards-compatible way. * Prepare for the creation of a generic relay mechanism for broadcasting events over an event bus. Broadcasting itself is not in scope for the initial phase, but accommodating that use case is a design goal. The design follows the idea of domain events as defined in domain driven design: events represent changes maintained by a given component (or bounded context). --- ### Hooks Hooks ===== ## Introduction Hooks allow MediaWiki Core to call extensions or allow one extension to call another extension. For more information and a list of hooks, see https://www.mediawiki.org/wiki/Manual:Hooks Starting in MediaWiki 1.35, each hook called by MediaWiki Core has an associated interface with a single method. To call the hook, obtain a "hook runner" object, which implements the relevant interface, and call the relevant method. To handle a hook event in an extension, create a handler object which implements the interface. The name of the interface is the name of the hook with "Hook" added to the end. Interfaces are typically placed in the namespace of their primary caller. The method name for the hook is the name of the hook, prefixed with "on". Several hooks had colons in their name, which are invalid in an interface or method name. These hooks have interfaces and method names in which the colons are replaced with underscores. For example, if the hook is called `Mash`, we might have the interface: interface MashHook { public function onMash( $banana ); } Hooks can be defined and called by extensions. The extension should define a hook interface for each hook, as described above. ## HookContainer HookContainer is a service which is responsible for maintaining a list of hook handlers and calling those handlers when requested. HookContainer is not aware of hook interfaces or parameter types. HookContainer provides hook metadata. For example, `isRegistered()` tells us whether there are any handlers for a given hook event. A HookContainer instance can be obtained from the global service locator with MediaWikiServices::getHookContainer(). When implementing a service that needs to call a hook, a HookContainer object should be passed to the constructor of the service. ## Hook runner classes A hook runner is a class which implements hook interfaces, proxying the calls to `HookContainer::run()`. MediaWiki has two hook runner classes: HookRunner and ApiHookRunner. ApiHookRunner has proxy methods for all hooks which are called by the Action API. HookRunner has proxy methods for all hooks which are called by other parts of Core. Some hooks are implemented in both classes. Extensions which call hooks should create their own hook runner class, by analogy with the ones in Core. Hook runner classes are effectively internal to the module which calls the relevant hooks. Reorganisation of the hook calling code may lead to methods being removed from hook runner classes. Thus, it is safer for extensions to define their own hook runner classes even if they are calling Core hooks. New code should typically be written in a service which takes a HookContainer as a constructor parameter. However, for the convenience of existing static functions in MediaWiki Core, `Hooks::runner()` may be used to obtain a HookRunner instance. This is equivalent to new HookRunner( MediaWikiServices::getInstance()->getHookContainer() ) For example, to call the hook `Mash`, as defined above, in static code: Hooks::runner()->onMash( $banana ); ## How to handle a hook event in an extension In extension.json, there is a new attribute called `HookHandlers`. This is an object mapping the handler name to an ObjectFactory specification describing how to create the handler object. The specification will typically have a `class` member with the name of the handler class. For example, in an extension called `FoodProcessor`, we may have: "HookHandlers": { "main": { "class": "MediaWiki\\Extension\\FoodProcessor\\HookHandler" } } Then in the Hooks attribute, instead of a function name, the value will be the handler name: "Hooks": { "Mash": "main" } Or more explicitly, by using an object instead of a string for the handler: "Hooks": { "Mash": { "handler": "main" } } Note that while your HookHandler class will implement an interface that ends with the word "Hook", in `extension.json` you should omit the word "Hook" from the key in the `Hooks` definition. For example, in the definitions above, the key must be "Mash", not "MashHook". Then the extension will define a handler class: namespace MediaWiki\Extension\FoodProcessor; class HookHandler implements MashHook { public function onMash( $banana ) { // Implementation goes here } } ## Service dependencies The ObjectFactory specification in HookHandlers can contain a list of services which should be instantiated and provided to the constructor or factory function for the handler. For example: "HookHandlers": { "main": { "class": "MediaWiki\\Extension\\FoodProcessor\\HookHandler", "services": [ "ReadOnlyMode" ] } } However, care should be taken with this feature. Some services have expensive constructors, so requesting them when handling commonly-called hooks may damage performance. Also, some services may not be safe to construct from within a hook call. The safest pattern for service injection is to use a separate handler for each hook, and to inject only the services needed by that hook. Calling a hook with the `noServices` option disables service injection. If a handler for such a hook specifies services, an exception will be thrown when the hook is called. ## Returning and aborting If a hook handler returns false, HookContainer will stop iterating through the list of handlers and will immediately return false. If a hook handler returns true, or if there is no return value (causing it to effectively return null), then HookContainer will continue to call any other remaining handlers. Eventually HookContainer::run() will return true. If there were no registered handlers, HookContainer::run() will return true. Some hooks are declared to be "not abortable". If a handler for a non-abortable hook returns false, an exception will be thrown. A hook is declared to be not abortable by passing `[ "abortable" => false ]` in the $options parameter to HookContainer::run(). Aborting is properly used to enforce a convention that only one extension may handle a given hook call. Aborting is sometimes used as a generic return value, to indicate that the caller should stop performing some action. Most hook callers do not check the return value from HookContainer::run() and there is no real concept of aborting. The only effect of returning `false` from a handler of these hooks is to break other extensions. Theoretically, extensions which are registered first in LocalSettings.php will be called first, and thus will have the first opportunity to abort a hook call. This behaviour should not be relied upon. In the new hook system, handlers registered in the legacy way are called first, before handlers registered in the new way. ## Parameters passed by reference The typical way for a handler to return data to the caller is by modifying a parameter which was passed by reference. This is sometimes called "replacement". Reference parameters were somewhat overused in early versions of MediaWiki. You may find that some parameters passed by reference cannot reasonably be modified. Replacement either has no effect on the caller or would cause unexpected or inconsistent effects. Handlers should generally only replace a parameter when it is clear from the documentation that replacement is expected. ## How to define a new hook * Create a hook interface, typically in a subnamespace called `Hook` relative to the caller namespace. For example, if the caller is in a namespace called `MediaWiki\Foo`, the hook interface might be placed in `MediaWiki\Foo\Hook`. * Add an implementation to the relevant HookRunner class. ## Hook deprecation Core hooks are deprecated by adding them to an array in the DeprecatedHooks class. Hooks declared in extensions may be deprecated by listing them in the `DeprecatedHooks` attribute: "DeprecatedHooks": { "Mash": { "deprecatedVersion": "2.0", "component": "FoodProcessor" } } If the `component` is not specified, it defaults to the name of the extension. The hook interface should be marked as deprecated by adding @deprecated to the interface doc comment. The interface doc comment is a better place for @deprecated than the method doc comment, because this causes the interface to be deprecated for implementation. Deprecating the method only causes calling to be deprecated, not handling. Deprecating a hook in this way activates a migration system called **call filtering**. Extensions opt in to call filtering of deprecated hooks by **acknowledging** deprecation. An extension acknowledges deprecation with the `deprecated` parameter in the `Hooks` attribute: "Hooks": { "Mash": { "handler": "main", "deprecated": true } } If deprecation is acknowledged by the extension: * If MediaWiki knows that the hook is deprecated, the handler will not be called. The call to the handler is filtered. * If MediaWiki does not have the hook in its list of deprecated hooks, the handler will be called anyway. Deprecation acknowledgement is a way for the extension to say that it has made some other arrangement for implementing the relevant functionality and does not need the handler for the deprecated hook to be called. ### Call filtering example Suppose the hook `Mash` is deprecated in MediaWiki 2.0, and is replaced by a new one called `Slice`. In our example extension FoodProcessor 1.0, the `Mash` hook is handled. In FoodProcessor 2.0, both `Mash` and `Slice` have handlers, but deprecation of `Mash` is acknowledged. Thus: * With MediaWiki 2.0 and FoodProcessor 1.0, `onMash` is called but raises a deprecation warning. * With MediaWiki 2.0 and FoodProcessor 2.0, `onMash` is filtered, and `onSlice` is called. * With MediaWiki 1.0 and FoodProcessor 2.0, `onMash` is called, since it is not yet deprecated in Core. `onSlice` is not called since it does not yet exist in Core. So the call filtering system provides both forwards and backwards compatibility. ### Silent deprecation Developers sometimes use two stages of deprecation: "soft" deprecation in which the deprecated entity is merely discouraged in documentation, and "hard" deprecation in which a warning is raised. When you soft-deprecate a hook, it is important to register it as deprecated so that call filtering is activated. Activating call filtering simplifies the task of migrating extensions to the new hook. To deprecate a hook without raising deprecation warnings, use the "silent" flag: "DeprecatedHooks": { "Mash": { "deprecatedVersion": "2.0", "component": "FoodProcessor", "silent": true } } As with hard deprecation, @deprecated should be added to the interface. --- ### Injection Dependency Injection {#dependencyinjection} ======= This is an overview of how MediaWiki uses of dependency injection. The design originates from [RFC T384](https://phabricator.wikimedia.org/T384). The term "dependency injection" (DI) refers to a pattern in object oriented programming. DI tries to improve modularity by reducing strong coupling between classes. In practical terms, this means that anything an object needs to operate should be injected from the outside. The object itself should only know narrow interfaces, no concrete implementation of the logic it relies on. The requirement to inject everything typically results in an architecture based on two main kinds of objects: simple "value" objects with no business logic (and often immutable), and essentially stateless "service" objects that use other service objects to operate on value objects. As of 2022 (MediaWiki 1.39), MediaWiki has adopted dependency injection in much of its code. However, some operations still require the use of singletons or otherwise involve global state. ## Overview The heart of the DI in MediaWiki is the central service locator, MediaWikiServices, which acts as the top-level factory (or registry) for services. MediaWikiServices represents the tree (or network) of service objects that define MediaWiki's application logic. It acts as an entry point to all dependency injection for MediaWiki core. When `MediaWikiServices::getInstance()` is first called, it will create an instance of MediaWikiServices and populate it with the services defined by MediaWiki core in `includes/ServiceWiring.php`, as well as any additional bootstrapping files specified in `$wgServiceWiringFiles`. The service wiring files define the (default) service implementations to use, and specifies how they depend on each other ("wiring"). Extensions can add their own wiring files to `$wgServiceWiringFiles`, in order to define their own service. Extensions may also use the `MediaWikiServices` hook to replace ("redefine") a core service, by calling methods on the MediaWikiServices instance. It should be noted that the term "service locator" is often used to refer to a top-level factory that is accessed directly, throughout the code, to avoid explicit dependency injection. In contrast, the term "DI container" is often used to describe a top-level factory that is only accessed only inside service wiring code when instantiating service classes. We use the term "service locator" because it is more descriptive than "DI container", even though application logic is strongly discouraged from accessing MediaWikiServices directly. `MediaWikiServices::getInstance()` should ideally be accessed only in "static entry points" such as hook handler functions. See "Migration" below. ## Principles {#di-principles} Service classes generally only vary on site configuration and are deterministic and agnostic of global state. It is the responsibility of callers to a service object to obtain and derive information from a web request (such as title, user, language, WebRequest, RequestContext), and pass this to specific methods of a service class as-needed. See [T218555](https://phabricator.wikimedia.org/T218555) for related discussion. Consider using the factory pattern if your service would otherwise be unergonomic or slow, e.g. due to passing many parameters and/or recomputing the same derived information. This keeps the global state out of the service class, by having the service be a factory from which the caller can obtain a (re-usable) object for its specific context. This design ensures service classes are safe to use in both user-facing contexts on the web (e.g. index.php page views and special pages), as well as in an API, job, or maintenance script. It also ensures that within a web-facing context the same service can be safely used multiple times to perform different operations, without incorrectly implying certain commonalities between these calls. Lastly, this restriction allows services to be instantiated across wikis in the future. If a feature is not ready to meet these requirements, keep it outside the service container. This avoids false confidence in the safety of an injected service, and its ripple effect on other services. ### Principle exemption There is a limited exemption to the above principles for "inconsequential state". That is, global state may be used directly if and only if used for diagnostics or to optimise performance, so long as they do not change the observed functional outcome of a called method. Examples of safe and inconsequential state: * Use `$_SERVER['REQUEST_TIME_FLOAT']` or `ConvertibleTimestamp::now` to help compute a time measure that is sent to a metric service. * Use `wfHostname()`, `PHP_SAPI`, or `WikiMap::getCurrentWikiId()` to describe where, how, or for which wiki the overall process was created and send it as message context to a logging service. * Use `WebRequest::getRequestId()` to automatically inject a header into HTTP requests to other services. These are for tracking purposes only. * Use `function_exists('apcu_fetch')` to automatically enable use of caching. Examples of unsafe state in a service class: * Do not use `WikiMap::getCurrentWikiId()` as the default value to obtain a database connection. * Do not use `$_SERVER['SERVER_NAME']` to inject a header into HTTP requests to other services to control which wiki to operate on. ## Create a new service To create a new service in MediaWiki core, write a function that will return the appropriate class instantiation for that service in ServiceWiring.php. This makes the service available through the generic `getService()` method on the `MediaWikiServices` class. We then also add a wrapper method to MediaWikiServices.php with a discoverable method named and strictly typed return value to reduce mistakes and improve static analysis. ## Service Reset Services get their configuration injected, and changes to global configuration variables will not have any effect on services that were already instantiated. This would typically be the case for low level services like the ConfigFactory or the ObjectCacheManager, which are used during extension registration. To address this issue, Setup.php resets the global service locator instance by calling `MediaWikiServices::resetGlobalInstance()` once configuration and extension registration is complete. Note that "unmanaged" legacy services services that manage their own singleton must not keep references to services managed by MediaWikiServices, to allow a clean reset. After the global MediaWikiServices instance got reset, any such references would be stale, and using a stale service will result in an error. Services should either have all dependencies injected and be themselves managed by MediaWikiServices, or they should use the Service Locator pattern, accessing service instances via the global MediaWikiServices instance state when needed. This ensures that no stale service references remain after a reset. ## Configuration When the default MediaWikiServices instance is created, a Config object is provided to the constructor. This Config object represents the "bootstrap" configuration which will become available as the 'BootstrapConfig' service. As of MW 1.27, the bootstrap config is a GlobalVarConfig object providing access to the $wgXxx configuration variables. The bootstrap config is then used to construct a 'ConfigFactory' service, which in turn is used to construct the 'MainConfig' service. Application logic should use the 'MainConfig' service (or a more specific configuration object). 'BootstrapConfig' should only be used for bootstrapping basic services that are needed to load the 'MainConfig'. Note: Several well known services in MediaWiki core act as factories themselves, e.g. ApiModuleManager, ObjectCache, SpecialPageFactory, etc. The registries these factories are based on are currently managed as part of the configuration. This may however change in the future. ## Migration This section provides some recipes for improving code modularity by reducing strong coupling. The dependency injection mechanism described above is an essential tool in this effort. ### Migrate access to global service instances and config variables Assume `Foo` is a class that uses the `$wgScriptPath` global and calls `wfGetDB()` to get a database connection, in non-static methods. * Add `$scriptPath` as a constructor parameter and use `$this->scriptPath` instead of `$wgScriptPath`. * Add IConnectionProvider `$dbProvider` as a constructor parameter. Use `$this->dbProvider->getReplicaDatabase()` instead of `wfGetDB( DB_REPLICA )`, `$this->dbProvider->->getPrimaryDatabase()` instead of `wfGetDB( DB_PRIMARY )`. * Any code that calls `Foo`'s constructor would now need to provide the `$scriptPath` and `$dbProvider`. To avoid this, avoid direct instantiation of services all together - see below. ### Migrate services with multiple configuration variables When a service needs multiple configuration globals injected, a ServiceOptions object is commonly used with the service class defining a public constant (usually `CONSTRUCTOR_OPTIONS`) with an array of settings that the class needs access to. ```php assertRequiredOptions( self::CONSTRUCTOR_OPTIONS ); // $wgFoo is now available with $this->options->get( 'Foo' ) // $wgBar is now available with $this->options->get( 'Bar' ) } } ``` ServiceOptions objects are constructed within ServiceWiring.php and can also be created in tests. ```php 'DemoService' => static function ( MediaWikiServices $services ): DemoService { return new DemoService( new ServiceOptions( DemoService::CONSTRUCTOR_OPTIONS, $services->getMainConfig() ), ); }, ``` ### Migrate class-level singleton getters Assume class `Foo` has mostly non-static methods, and provides a static `getInstance()` method that returns a singleton (or default instance). * Add an instantiator function for `Foo` into ServiceWiring.php. The instantiator would do exactly what `Foo::getInstance()` did. However, it should replace any access to global state with calls to `$services->getXxx()` to get a service, or `$services->getMainConfig()->get()` to get a configuration setting. * Add a `getFoo()` method to MediaWikiServices. Don't forget to add the appropriate test cases in MediaWikiServicesTest. * Turn `Foo::getInstance()` into a deprecated alias for `MediaWikiServices::getInstance()->getFoo()`. Change all calls to `Foo::getInstance()` to use injection (see above). ### Migrate direct service instantiation Assume class `Bar` calls `new Foo()`. * Add an instantiator function for `Foo` into ServiceWiring.php and add a `getFoo()` method to MediaWikiServices. Don't forget to add the appropriate test cases in MediaWikiServicesTest. * In the instantiator, replace any access to global state with calls to `$services->getXxx()` to get a service, or `$services->getMainConfig()->get()` to get a configuration setting. * The code in `Bar` that calls `Foo`'s constructor should be changed to have a `Foo` instance injected; Eventually, the only code that instantiates `Foo` is the instantiator in ServiceWiring.php. * As an intermediate step, `Bar`'s constructor could initialize the `$foo` member variable by calling `MediaWikiServices::getInstance()->getFoo()`. This is acceptable as a stepping stone, but should be replaced by proper injection via a constructor argument. Do not however inject the MediaWikiServices object! ### Migrate parameterized helper instantiation Assume class `Bar` creates some helper object by calling `new Foo( $x )`, and `Foo` uses a global singleton of the `Xyzzy` service. * Define a `FooFactory` class (or a `FooFactory` interface along with a `MyFooFactory` implementation). `FooFactory` defines the method `newFoo( $x )` or `getFoo( $x )`, depending on the desired semantics (`newFoo` would guarantee a fresh instance). When Foo gets refactored to have `Xyzzy` injected, `FooFactory` will need a `Xyzzy` instance, so `newFoo()` can pass it to `new Foo()`. * Add an instantiator function for FooFactory into ServiceWiring.php and add a getFooFactory() method to MediaWikiServices. Don't forget to add the appropriate test cases in MediaWikiServicesTest. * The code in Bar that calls Foo's constructor should be changed to have a FooFactory instance injected; Eventually, the only code that instantiates Foo are implementations of FooFactory, and the only code that instantiates FooFactory is the instantiator in ServiceWiring.php. * As an intermediate step, Bar's constructor could initialize the $fooFactory member variable by calling `MediaWikiServices::getInstance()->getFooFactory()`. This is acceptable as a stepping stone, but should be replaced by proper injection via a constructor argument. Do not however inject the MediaWikiServices object! ### Migrate a handler registry Assume class `Bar` calls `FooRegistry::getFoo( $x )` to get a specialized `Foo` instance for handling `$x`. * Turn `getFoo` into a non-static method. * Add an instantiator function for `FooRegistry` into ServiceWiring.php and add a `getFooRegistry()` method to MediaWikiServices. Don't forget to add the appropriate test cases in MediaWikiServicesTest. * Change all code that calls `FooRegistry::getFoo()` statically to call this method on a `FooRegistry` instance. That is, `Bar` would have a `$fooRegistry` member, initialized from a constructor parameter. * As an intermediate step, Bar's constructor could initialize the `$fooRegistry` member variable by calling `MediaWikiServices::getInstance()->getFooRegistry()`. This is acceptable as a stepping stone, but should be replaced by proper injection via a constructor argument. Do not however inject the MediaWikiServices object! ### Migrate deferred service instantiation Assume class `Bar` calls `new Foo()`, but only when needed, to avoid the cost of instantiating Foo(). * Define a `FooFactory` interface and a `MyFooFactory` implementation of that interface. `FooFactory` defines the method `getFoo()` with no parameters. * Precede as for the "parameterized helper instantiation" case described above. ### Migrate a class with only static methods Assume `Foo` is a class with only static methods, such as `frob()`, which interacts with global state or system resources. * Introduce a `FooService` interface and a `DefaultFoo` implementation of that interface. `FooService` contains the public methods defined by Foo. * Add an instantiator function for `FooService` into ServiceWiring.php and add a `getFooService()` method to MediaWikiServices. Don't forget to add the appropriate test cases in MediaWikiServicesTest. * Add a private static `getFooService()` method to `Foo`. That method just calls `MediaWikiServices::getInstance()->getFooService()`. * Make all methods in `Foo` delegate to the `FooService` returned by `getFooService()`. That is, `Foo::frob()` would do `self::getFooService()->frob()`. * Deprecate `Foo`. Inject a `FooService` into all code that calls methods on `Foo`, and change any calls to static methods in foo to the methods provided by the `FooService` interface. ### Migrate static hook handler functions (to allow unit testing) Assume `MyExtHooks::onFoo` is a static hook handler function that is called with the parameter `$x`; Further assume `MyExt::onFoo` needs service `Bar`, which is already known to MediaWikiServices (if not, see above). * Create a non-static `doFoo( $x )` method in `MyExtHooks` that has the same signature as `onFoo( $x )`. Move the code from `onFoo()` into `doFoo()`, replacing any access to global or static variables with access to instance member variables. * Add a constructor to `MyExtHooks` that takes a Bar service as a parameter. * Add a static method called `newFromGlobalState()` with no parameters. It should just return `new MyExtHooks( MediaWikiServices::getInstance()->getBar() )`. * The original static handler method `onFoo( $x )` is then implemented as `self::newFromGlobalState()->doFoo( $x )`. ### Migrate a "smart record" Assume `Thingy` is a "smart record" that "knows" how to load and store itself. For this purpose, `Thingy` uses wfGetDB(). * Create a "dumb" value class `ThingyRecord` that contains all the information that `Thingy` represents (e.g. the information from a database row). The value object should not know about any service. * Create a DAO-style service for loading and storing `ThingyRecord`s, called `ThingyStore`. It may be useful to split the interfaces for reading and writing, with a single class implementing both interfaces, so we in the end have the `ThingyLookup` and `ThingyStore` interfaces, and a SqlThingyStore implementation. * Add instantiator functions for `ThingyLookup` and `ThingyStore` in ServiceWiring.php. Since we want to use the same instance for both service interfaces, the instantiator for `ThingyLookup` would return `$services->getThingyStore()`. * Add `getThingyLookup()` and `getThingyStore()` methods to MediaWikiServices. Don't forget to add the appropriate test cases in MediaWikiServicesTest. * In the old `Thingy` class, replace all member variables that represent the record's data with a single `ThingyRecord` object. * In the old Thingy class, replace all calls to static methods or functions, such as wfGetDB(), with calls to the appropriate services, such as `IConnectionProvider::getReplicaDatabase()`. * In Thingy's constructor, pull in any services needed, such as the IConnectionProvider, by using `MediaWikiServices::getInstance()`. These services cannot be injected without changing the constructor signature, which is often impractical for "smart records" that get instantiated directly in many places in the code base. * Deprecate the old `Thingy` class. Replace all usages of it with one of the three new classes: loading needs a `ThingyLookup`, storing needs a `ThingyStore`, and reading data needs a `ThingyRecord`. ### Migrate lazy loading Assume `Thingy` is a "smart record" as described above, but requires lazy loading of some or all the data it represents. * Instead of a plain object, define `ThingyRecord` to be an interface. Provide a "simple" and "lazy" implementations, called `SimpleThingyRecord` and `LazyThingyRecord`. `LazyThingyRecord` knows about some lower level storage interface, like a LoadBalancer, and uses it to load information on demand. * Any direct instantiation of a `ThingyRecord` would use the `SimpleThingyRecord` implementation. * `SqlThingyStore` however creates instances of `LazyThingyRecord`, and injects whatever storage layer service `LazyThingyRecord` needs to perform lazy loading. --- ### Introduction Introduction {#mainpage} =============================== Welcome to the MediaWiki autogenerated documentation system. If you are looking to use, install or configure your wiki, see the main site: . Helpful resources ------------------------------- - [General information](https://www.mediawiki.org/) - [Installation guide](https://www.mediawiki.org/wiki/Manual:Installation_guide) - [Configuration](https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:System_administration) - [New Developers](https://www.mediawiki.org/wiki/New_Developers) --- ### Language Language ======= The Language object handles all readable text produced by the software. See [MediaWiki.org](https://www.mediawiki.org/wiki/Localisation#General_use_.28for_developers.29) for documentation relating to using localized messages. --- ### LinkCache The LinkCache class maintains a list of article titles and the information about whether or not the article exists in the database. This is used to mark up links when displaying a page. If the same link appears more than once on any page, then it only has to be looked up once. In most cases, link lookups are done in batches with the LinkBatch class, or the equivalent in Parser::replaceLinkHolders(), so the link cache is mostly useful for short snippets of parsed text (such as the site notice), and for links in the navigation areas of the skin. The link cache was formerly used to track links used in a document for the purposes of updating the link tables. This application is now deprecated. To create a batch, you can use the following code: ~~~{.php} $pages = [ 'Main Page', 'Project:Help', /* ... */ ]; $titles = []; foreach( $pages as $page ){ $titles[] = Title::newFromText( $page ); } $linkBatchFactory = MediaWikiServices::getInstance()->getLinkBatchFactory(); $batch = $linkBatchFactory->newLinkBatch( $titles ); $batch->execute(); ~~~ --- ### Logger MediaWiki.Logger.LoggerFactory implements a [PSR-3] compatible message logging system. Named Psr.Log.LoggerInterface instances can be obtained from the MediaWiki.Logger.LoggerFactory::getInstance() static method. MediaWiki.Logger.LoggerFactory expects a class implementing the MediaWiki.Logger.Spi interface to act as a factory for new Psr.Log.LoggerInterface instances. The "Spi" in MediaWiki.Logger.Spi stands for "service provider interface". An SPI is an API intended to be implemented or extended by a third party. This software design pattern is intended to enable framework extension and replaceable components. It is specifically used in the MediaWiki.Logger.LoggerFactory service to allow alternate PSR-3 logging implementations to be easily integrated with MediaWiki. The service provider interface allows the backend logging library to be implemented in multiple ways. The $wgMWLoggerDefaultSpi global provides the classname of the default MediaWiki.Logger.Spi implementation to be loaded at runtime. This can either be the name of a class implementing the MediaWiki.Logger.Spi with a zero argument constructor or a callable that will return an MediaWiki.Logger.Spi instance. Alternately the MediaWiki.Logger.LoggerFactory::registerProvider() static method can be called to inject an MediaWiki.Logger.Spi instance into the LoggerFactory and bypass the use of the default configuration variable. The MediaWiki.Logger.LegacySpi class implements a service provider to generate MediaWiki.Logger.LegacyLogger instances. The MediaWiki.Logger.LegacyLogger class implements the PSR-3 logger interface and provides output and configuration equivalent to the historic logging output of wfDebug, wfDebugLog, wfLogDBError and wfErrorLog. The MediaWiki.Logger.LegacySpi class is the default service provider configured in DefaultSettings.php. It's usage should be transparent for users who are not ready or do not wish to switch to a alternate logging platform. The MediaWiki.Logger.MonologSpi class implements a service provider to generate Psr.Log.LoggerInterface instances that use the [Monolog] logging library. See the PHP docs (or source) for MediaWiki.Logger.MonologSpi for details on the configuration of this provider. The default configuration installs a null handler that will silently discard all logging events. The documentation provided by the class describes a more feature rich logging configuration. # Classes * MediaWiki.Logger.LoggerFactory: Factory for Psr.Log.LoggerInterface loggers * MediaWiki.Logger.Spi: Service provider interface for MediaWiki.Logger.LoggerFactory * MediaWiki.Logger.NullSpi: MediaWiki.Logger.Spi for creating instances that discard all log events * MediaWiki.Logger.LegacySpi: Service provider for creating MediaWiki.Logger.LegacyLogger instances * MediaWiki.Logger.LegacyLogger: PSR-3 logger that mimics the historical output and configuration of wfDebug, wfErrorLog and other global logging functions. * MediaWiki.Logger.MonologSpi: MediaWiki.Logger.Spi for creating instances backed by the monolog logging library * MediaWiki.Logger.Monolog.LegacyHandler: Monolog handler that replicates the udp2log and file logging functionality of wfErrorLog() * MediaWiki.Logger.Monolog.WikiProcessor: Monolog log processer that adds host: wfHostname() and wiki: wfWikiID() to all records # Globals * $wgMWLoggerDefaultSpi: Specification for creating the default service provider interface to use with LoggerFactory [PSR-3]: https://github.com/php-fig/fig-standards/blob/master/accepted/PSR-3-logger-interface.md [Monolog]: https://github.com/Seldaek/monolog --- ### Skin Skins ======= ## Core Skins MediaWiki includes four core skins: * Vector: The default skin. Introduced in the 1.16 release (2010), it has been set as the default in MediaWiki since the 1.17 release (2011), replacing Monobook. * Monobook: Named after the black-and-white photo of a book in the page background. Introduced in the 2004 release of 1.3, it had been the default skin since then, before being replaced by Vector. * Modern: An attractive blue/grey theme with sidebar and top bar. Derived from Monobook. * Cologne Blue: A lightweight skin with minimal formatting. The oldest of the currently bundled skins, largely rewritten in 2012 while keeping its appearance. ### Legacy core skins Several legacy skins were removed in the 1.22 release, as the burden of supporting them became too heavy to bear. Those were: * Standard (a.k.a. Classic): The old default skin written by Lee Crocker during the phase 3 rewrite, in 2002. * Nostalgia: A skin which looks like Wikipedia did in its first year (2001). This skin is now used for the old Wikipedia snapshot at https://nostalgia.wikipedia.org/ * Chick: A lightweight Monobook skin with no sidebar. The sidebar links were given at the bottom of the page instead. * Simple: A lightweight skin with a simple white-background sidebar and no top bar. * MySkin: Essentially Monobook without the CSS. The idea was that it could be customised using user-specific or site-wide CSS (see below). ## Custom CSS/JS It is possible to customise the site CSS and JavaScript without editing any server-side source files. This is done by editing some pages on the wiki: * `MediaWiki:Common.css` for skin-independent CSS * `MediaWiki:Common.js` for skin-independent JavaScript * `MediaWiki:Vector.css`, `MediaWiki:Monobook.css`, etc. for skin-dependent CSS * `MediaWiki:Vector.js`, `MediaWiki:Monobook.js`, etc. for skin-dependent JavaScript These can also be customised on a per-user basis, by editing `User:/vector.css`, `User:/vector.js`, etc. ## Custom skins Several custom skins are available as of 2019. List of all skins is available at [MediaWiki.org](https://www.mediawiki.org/wiki/Special:MyLanguage/Category:All_skins). Installing a skin requires adding its files in a subdirectory under `skins/` and adding an appropriate `wfLoadSkin` line to `LocalSettings.php`, similarly to how extensions are installed. You can then make that skin the default by adding: ```php $wgDefaultSkin = ''; ``` Or disable it entirely by removing the `wfLoadSkin` line. (User settings will not be lost if it's reenabled later.) See https://www.mediawiki.org/wiki/Manual:Skinning for more information on writing new skins. ### Legacy custom skins Until MediaWiki 1.25 it used to be possible to just put a `.php` file in MediaWiki's `skins/` directory, which would be loaded and expected to contain the `Skin` class. This way has always been discouraged because of its limitations (inability to add localisation messages, ResourceLoader modules, etc.) and awkwardness in managing such skins. For information on migrating skins using this old method, see https://www.mediawiki.org/wiki/Manual:Skin_autodiscovery. --- ### Title The MediaWiki software's "Title" class represents article titles, which are used for many purposes: as the human-readable text title of the article, in the URL used to access the article, the wikitext link to the article, the key into the article database, and so on. The class in instantiated from one of these forms and can be queried for the others, and for other attributes of the title. This is intended to be an immutable "value" class, so there are no mutator functions. To get a new instance, call Title::newFromText(). Once instantiated, the non-static accessor methods can be used, such as getText(), getDBkey(), getNamespace(), etc. Note that Title::newFromText() may return false if the text is illegal according to the rules below. The prefix rules: a title consists of an optional interwiki prefix (such as "m:" for meta or "de:" for German), followed by an optional namespace, followed by the remainder of the title. Both interwiki prefixes and namespace prefixes have the same rules: they contain only letters, digits, space, and underscore, must start with a letter, are case insensitive, and spaces and underscores are interchangeable. Prefixes end with a ":". A prefix is only recognized if it is one of those specifically allowed by the software. For example, "de:name" is a link to the article "name" in the German Wikipedia, because "de" is recognized as one of the allowable interwikis. The title "talk:name" is a link to the article "name" in the "talk" namespace of the current wiki, because "talk" is a recognized namespace. Both may be present, and if so, the interwiki must come first, for example, "m:talk:name". If a title begins with a colon as its first character, no prefixes are scanned for, and the colon is just removed. Note that because of these rules, it is possible to have articles with colons in their names. "E. Coli 0157:H7" is a valid title, as is "2001: A Space Odyssey", because "E. Coli 0157" and "2001" are not valid interwikis or namespaces. It is not possible to have an article whose bare name includes a namespace or interwiki prefix. An initial colon in a title listed in wiki text may however suppress special handling for interlanguage links, image links, and category links. It is also used to indicate the main namespace in template inclusions. Once prefixes have been stripped, the rest of the title processed this way: * Spaces and underscores are treated as equivalent and each is converted to the other in the appropriate context (underscore in URL and database keys, spaces in plain text). * Multiple consecutive spaces are converted to a single space. * Leading or trailing space is removed. * If $wgCapitalLinks is enabled (the default), the first letter is capitalised, using the capitalisation function of the content language object. * The unicode characters LRM (U+200E) and RLM (U+200F) are silently stripped. * Invalid UTF-8 sequences or instances of the replacement character (U+FFFD) are considered illegal. * A percent sign followed by two hexadecimal characters is illegal * Anything that looks like an XML/HTML character reference is illegal * Any character not matched by the $wgLegalTitleChars regex is illegal * Zero-length titles (after whitespace stripping) are illegal All titles except special pages must be less than 255 bytes when encoded with UTF-8, because that is the size of the database field. Special page titles may be up to 512 bytes. Note that Unicode Normal Form C (NFC) is enforced by MediaWiki's user interface input functions, and so titles will typically be in this form. getArticleID() needs some explanation: for "internal" articles, it should return the "page_id" field if the article exists, else it returns 0. For all external articles it returns 0. All of the IDs for all instances of Title created during a request are cached, so they can be looked up quickly while rendering wiki text with lots of internal links. See LinkCache.md. --- ### Contenthandler # ContentHandler The *ContentHandler* facility adds support for arbitrary content types on wiki pages, instead of relying on wikitext for everything. It was introduced in MediaWiki 1.21. Each kind of content ("content model") supported by MediaWiki is identified by unique name. The content model determines how a page's content is rendered, compared, stored, edited, and so on. Built-in content types are: * wikitext - wikitext, as usual * javascript - user provided javascript code * json - simple implementation for use by extensions, etc. * css - user provided css code * text - plain text In PHP, use the corresponding `CONTENT_MODEL_XXX` constant. A page's content model is available using the `Title::getContentModel()` method. A page's default model is determined by `ContentHandler::getDefaultModelFor($title)` as follows: * The global setting `$wgNamespaceContentModels` specifies a content model for the given namespace. * The hook `ContentHandlerDefaultModelFor` may be used to override the page's default model. * Pages in `NS_MEDIAWIKI` and `NS_USER` default to the CSS or JavaScript model if they end in .css or .js, respectively. Pages in `NS_MEDIAWIKI` default to the wikitext model otherwise. * Otherwise, the wikitext model is used. Note that there is no guarantee that revisions of a page will all have the same content model. To find the content model of the slot of a revision, use `SlotRecord::getModel()` - the content model of the main slot can for now be assumed to be the content model for the overall revision. ## Architecture Two class hierarchies are used to provide the functionality associated with the different content models: * Content interface (and `AbstractContent` base class) define functionality that acts on the concrete content of a page, and * `ContentHandler` base class provides functionality specific to a content model, but not acting on concrete content. The most important function of ContentHandler is to act as a factory for the appropriate implementation of Content. These `Content` objects are to be used by MediaWiki everywhere, instead of passing page content around as text. All manipulation and analysis of page content must be done via the appropriate methods of the Content object. For each content model, a subclass of ContentHandler has to be registered with `$wgContentHandlers`. The ContentHandler object for a given content model can be obtained using `ContentHandler::getForModelID( $id )`. Also `Title` and `WikiPage` now have `getContentHandler()` methods for convenience. `ContentHandler` objects are singletons that provide functionality specific to the content type, but not directly acting on the content of some page. `ContentHandler::makeEmptyContent()` and `ContentHandler::unserializeContent()` can be used to create a Content object of the appropriate type. However, it is recommended to instead use `WikiPage::getContent()` resp. `RevisionRecord::getContent()` to get a page's content as a Content object. These two methods should be the ONLY way in which page content is accessed. For `WikiPage::getContent()` the content of the main slot is returned, other slots can be retrieved by using `RevisionRecord::getContent()` and specifying the slot. Another important function of ContentHandler objects is to define custom action handlers for a content model, see `ContentHandler::getActionOverrides()`. This is similar to what `WikiPage::getActionOverrides()` was already doing. ## Serialization With the ContentHandler facility, page content no longer has to be text based. Objects implementing the Content interface are used to represent and handle the content internally. For storage and data exchange, each content model supports at least one serialization format via `ContentHandler::serializeContent( $content )`. The list of supported formats for a given content model can be accessed using `ContentHandler::getSupportedFormats()`. Content serialization formats are identified using MIME type like strings. The following formats are built in: * text/x-wiki - wikitext * text/javascript - for js pages * text/css - for css pages * text/plain - for future use, e.g. with plain text messages. * text/html - for future use, e.g. with plain html messages. * application/vnd.php.serialized - for future use with the api and for extensions * application/json - for future use with the api, and for use by extensions * application/xml - for future use with the api, and for use by extensions In PHP, use the corresponding `CONTENT_FORMAT_XXX` constant. Note that when using the API to access page content, especially `action=edit`, `action=parse` and `action=query&prop=revisions`, the model and format of the content should always be handled explicitly. Without that information, interpretation of the provided content is not reliable. The same applies to XML dumps generated via `maintenance/dumpBackup.php` or `Special:Export`. Also note that the API will provide encapsulated, serialized content - so if the API was called with `format=json`, and contentformat is also json (or rather, application/json), the page content is represented as a string containing an escaped json structure. Extensions that use JSON to serialize some types of page content may provide specialized API modules that allow access to that content in a more natural form. ## Compatibility The ContentHandler facility is introduced in a way that should allow all existing code to keep functioning at least for pages that contain wikitext or other text based content. However, a number of functions and hooks have been deprecated in favor of new versions that are aware of the page's content model, and will now generate warnings when used. Most importantly, the following functions have been deprecated: * `Revision::getText()` was deprecated in favor of `Revision::getContent()` (though the Revision class was later fully removed as part of the migration to Multi-Content Revisions (MCR), see [documentation of mediawiki.org][mediawiki.org/wiki/Multi-Content_Revisions]. * `WikiPage::getText()` is deprecated in favor of `WikiPage::getContent()` Also, the old `Article::getContent()` (which returns text) is superceded by `Article::getContentObject()`. However, both methods should be avoided since they do not provide clean access to the page's actual content. For instance, they may return a system message for non-existing pages. Use `WikiPage::getContent()` instead. Code that relies on a textual representation of the page content should eventually be rewritten. However, `ContentHandler::getContentText()` provides a stop-gap that can be used to get text for a page. Its behavior is controlled by `$wgContentHandlerTextFallback`; per default it will return the text for text based content, and null for any other content. For rendering page content, `Content::getParserOutput()` should be used instead of accessing the parser directly. `WikiPage::makeParserOptions()` can be used to construct appropriate options. Besides some functions, some hooks have also been replaced by new versions (see hooks.txt for details). These hooks will now trigger a warning when used: * `ArticleAfterFetchContent` was replaced by `ArticleAfterFetchContentObject`, later replaced by `ArticleRevisionViewCustom` * `ArticleInsertComplete` was replaced by `PageContentInsertComplete`, later replaced by `PageSaveComplete` * `ArticleSave` was replaced by `PageContentSave` * `ArticleSaveComplete` was replaced by `PageContentSaveComplete`, later replaced by `PageSaveComplete` * `ArticleViewCustom` was replaced by `ArticleContentViewCustom`, which was later removed entirely * `EditFilterMerged` was replaced by `EditFilterMergedContent` * `EditPageGetDiffText` was replaced by `EditPageGetDiffContent` * `EditPageGetPreviewText` was replaced by `EditPageGetPreviewContent` * `ShowRawCssJs` was deprecated in favor of custom rendering implemented in the respective ContentHandler object. ## Database Storage Page content is stored in the database using the same mechanism as before. Non-text content is serialized first. Each revision's content model and serialization format is stored in the revision table (resp. in the archive table, if the revision was deleted). The page's (current) content model (that is, the content model of the latest revision) is also stored in the page table. Note however that the content model and format is only stored if it differs from the page's default, as determined by `ContentHandler::getDefaultModelFor( $title )`. The default values are represented as `NULL` in the database, to preserve space. ## Globals There are some new globals that can be used to control the behavior of the ContentHandler facility: * `$wgContentHandlers` associates content model IDs with the names of the appropriate ContentHandler subclasses or callbacks that create an instance of the appropriate ContentHandler subclass. * `$wgNamespaceContentModels` maps namespace IDs to a content model that should be the default for that namespace. * `$wgContentHandlerTextFallback` determines how the compatibility method `ContentHandler::getContentText()` will behave for non-text content: * `'ignore'` causes null to be returned for non-text content (default). * `'serialize'` causes the serialized form of any non-text content to be returned (scary). * `'fail'` causes an exception to be thrown for non-text content (strict). ## Caveats There are some changes in behavior that might be surprising to users: * Javascript and CSS pages are no longer parsed as wikitext (though pre-save transform is still applied). Most importantly, this means that links, including categorization links, contained in the code will not work. * `action=edit` will fail for pages with non-text content, unless the respective ContentHandler implementation has provided a specialized handler for the edit action. This is true for the API as well. * `action=raw` will fail for all non-text content. This seems better than serving content in other formats to an unsuspecting recipient. This will also cause client-side diffs to fail. * File pages provide their own action overrides that do not combine gracefully with any custom handlers defined by a ContentHandler. If for example a File page used a content model with a custom revert action, this would be overridden by WikiFilePage's handler for the revert action. --- ### Database # Database Access *Some information about database access in MediaWiki. By Tim Starling, January 2006.* ## Database Layout For information about the MediaWiki database layout, such as a description of the tables and their contents, please see: * [The manual](https://www.mediawiki.org/wiki/Manual:Database_layout) * [https://phabricator.wikimedia.org/diffusion/MW/browse/master/maintenance/tables.sql](https://phabricator.wikimedia.org/diffusion/MW/browse/master/maintenance/tables.sql) ## API To make a read query, something like this usually suffices: ```php $dbr = wfGetDB( DB_REPLICA ); $res = $dbr->select( /* ...see docs... */ ); foreach ( $res as $row ) { ... } ``` For a write query, use something like: ```php $dbw = wfGetDB( DB_PRIMARY ); $dbw->insert( /* ...see docs... */ ); ``` We use the convention `$dbr` for read and `$dbw` for write to help you keep track of whether the database object is a replica (read-only) or a primary (read/write). If you write to a replica, the world will explode. Or to be precise, a subsequent write query which succeeded on the primary may fail when propagated to the replica due to a unique key collision. Replication will then stop and it may take hours to repair the database and get it back online. Setting `read_only` in `my.cnf` on the replica will avoid this scenario, but given the dire consequences, we prefer to have as many checks as possible. We provide a `query()` function for raw SQL, but the wrapper functions like `select()` and `insert()` are usually more convenient. They take care of things like table prefixes and escaping for you. If you really need to make your own SQL, please read the documentation for `tableName()` and `addQuotes()`. You will need both of them. ## Basic query optimisation MediaWiki developers who need to write DB queries should have some understanding of databases and the performance issues associated with them. Patches containing unacceptably slow features will not be accepted. Unindexed queries are generally not welcome in MediaWiki, except in special pages derived from `QueryPage`. It's a common pitfall for new developers to submit code containing SQL queries which examine huge numbers of rows. Remember that `COUNT(*)` is **O(N)**, counting rows in a table is like counting beans in a bucket. ## Replication The largest installation of MediaWiki, Wikimedia, uses a large set of replica MySQL servers replicating writes made to a primary MySQL server. It is important to understand the issues associated with this setup if you want to write code destined for Wikipedia. It's often the case that the best algorithm to use for a given task depends on whether or not replication is in use. Due to our unabashed Wikipedia-centrism, we often just use the replication-friendly version, but if you like, you can use `LoadBalancer::getServerCount() > 1` to check to see if replication is in use. ## Lag Lag primarily occurs when large write queries are sent to the primary. Writes on the primary are executed in parallel, but they are executed in serial when they are replicated to the replicas. The primary writes the query to the binlog when the transaction is committed. The replicas poll the binlog and start executing the query as soon as it appears. They can service reads while they are performing a write query, but will not read anything more from the binlog and thus will perform no more writes. This means that if the write query runs for a long time, the replicas will lag behind the primary for the time it takes for the write query to complete. Lag can be exacerbated by high read load. MediaWiki's load balancer will stop sending reads to a replica when it is lagged by more than 30 seconds. If the load ratios are set incorrectly, or if there is too much load generally, this may lead to a replica permanently hovering around 30 seconds lag. If all replicas are lagged by more than 30 seconds, MediaWiki will stop writing to the database. All edits and other write operations will be refused, with an error returned to the user. This gives the replicas a chance to catch up. Before we had this mechanism, the replicas would regularly lag by several minutes, making review of recent edits difficult. In addition to this, MediaWiki attempts to ensure that the user sees events occurring on the wiki in chronological order. A few seconds of lag can be tolerated, as long as the user sees a consistent picture from subsequent requests. This is done by saving the primary binlog position in the session, and then at the start of each request, waiting for the replica to catch up to that position before doing any reads from it. If this wait times out, reads are allowed anyway, but the request is considered to be in "lagged replica mode". Lagged replica mode can be checked by calling `LoadBalancer::getLaggedReplicaMode()`. The only practical consequence at present is a warning displayed in the page footer. ## Lag avoidance To avoid excessive lag, queries which write large numbers of rows should be split up, generally to write one row at a time. Multi-row `INSERT ... SELECT` queries are the worst offenders should be avoided altogether. Instead do the select first and then the insert. ## Working with lag Despite our best efforts, it's not practical to guarantee a low-lag environment. Lag will usually be less than one second, but may occasionally be up to 30 seconds. For scalability, it's very important to keep load on the primary low, so simply sending all your queries to the masprimaryter is not the answer. So when you have a genuine need for up-to-date data, the following approach is advised: 1) Do a quick query to the primary for a sequence number or timestamp 2) Run the full query on the replica and check if it matches the data you got from the primary 3) If it doesn't, run the full query on the primary To avoid swamping the primary every time the replicas lag, use of this approach should be kept to a minimum. In most cases you should just read from the replica and let the user deal with the delay. ## Lock contention Due to the high write rate on Wikipedia (and some other wikis), MediaWiki developers need to be very careful to structure their writes to avoid long-lasting locks. By default, MediaWiki opens a transaction at the first query, and commits it before the output is sent. Locks will be held from the time when the query is done until the commit. So you can reduce lock time by doing as much processing as possible before you do your write queries. Often this approach is not good enough, and it becomes necessary to enclose small groups of queries in their own transaction. Use the following syntax: ```php $dbw = wfGetDB( DB_PRIMARY ); $dbw->begin( __METHOD__ ); /* Do queries */ $dbw->commit( __METHOD__ ); ``` Use of locking reads (e.g. the `FOR UPDATE` clause) is not advised. They are poorly implemented in InnoDB and will cause regular deadlock errors. It's also surprisingly easy to cripple the wiki with lock contention. Instead of locking reads, combine your existence checks into your write queries, by using an appropriate condition in the `WHERE` clause of an `UPDATE`, or by using unique indexes in combination with `INSERT IGNORE`. Then use the affected row count to see if the query succeeded. ## Query groups MediaWiki supports database query groups, a way to indicate a preferred group of database hosts to use for a given query. Query groups are only supported for connections to child (non-primary) databases, making them only viable for read operations. It should be noted that using query groups does not _guarantee_ a given group of hosts will be used, but rather that the query prefers such group. Making use of query groups can be benficial in many cases. One benefit is a reduction of cache misses. Directing reads for a category of queries (e.g. all logging queries) to a given host can result in more deterministic and faster performing queries. Another benefit is that it allows high-traffic wikis to configure some of their database hosts to handle some types of queries more optimally than others. For example, optimizing with different table indices for faster performance. Query groups are especially beneficial for queries expected to have a long execution time. Such queries can exhaust a database of its resources (e.g. cache space and I/O time), so targeting a specific group of hosts prevents more urgent queries from suffering a performance decrease. Additionally, expensive queries can delay database maintenance operations which may increase latency for other queries. For example, while a database read is executing, if other queries have performed updates to any tables those tables must retain all stale versions of its rows until the read is complete. Now, other potentially unrelated queries must now spend additional time scanning over obsolete rows that are waiting to be purged. Directing these long running queries to dedicated hosts helps prevent other queries in suffering a performance hit. MediaWiki currently supports the following query groups: * api * Only use for queries specific to api.php requests; the method ApiBase::getDB() is provided for this purpose. * dump * Only use in MediaWiki dump maintenance scripts. In such scripts, all queries, even fast ones, should use this group. * vslow * Only use for queries that are expected to have a long execution time. For example, when calculating per-wiki site statistics. Use the below example syntax to connect to a database when your query falls into one of the above 3 categories: ```php $lb = MediaWikiServices::getInstance()->getDBLoadBalancer(); $lb->getConnectionRef( DB_REPLICA, 'vslow' ); ``` ## Supported DBMSs MediaWiki is written primarily for use with MySQL. Queries are optimized for it and its schema is considered the canonical version. However, MediaWiki does support the following other DBMSs to varying degrees: * PostgreSQL * SQLite More information can be found about each of these databases (known issues, level of support, extra configuration) in the `databases` subdirectory in this folder. ## Use of `GROUP BY` MySQL supports `GROUP BY` without checking anything in the `SELECT` clause. Other DBMSs (especially Postgres) are stricter and require that all the non-aggregate items in the `SELECT` clause appear in the `GROUP BY`. For this reason, it is highly discouraged to use `SELECT *` with `GROUP BY` queries. --- ### Magicword Magic Words ==================================== Magic words are localizable keywords used in wikitext. They are used for many small fragments of text, including: * The names of parser functions e.g. `{{urlencode:...}}` * The names of variables, e.g. `{{CURRENTDAY}}` * Double-underscore behavior switches, e.g. `__NOTOC__` * Image link parameter names Magic words have a synonym list, with the canonical English word always present, and a case sensitivity flag. The MagicWord class provides facilities for matching a magic word by converting it to a regex. A magic word has a unique ID. Often, the ID is the canonical English synonym in lowercase. To add a magic word in an extension, add a file to the **ExtensionMessagesFiles** attribute in extension.json, and in that file, set a variable called **$magicWords**. This array is associative with the language code in the first dimension key and an ID in the second key. The third level array is numerically indexed: the element with key 0 contains the case sensitivity flag, with 0 for case-insensitive and 1 for case-sensitive. The subsequent elements of the array are the synonyms in the relevant language. To add a magic word in core, add it to $magicWords in MessagesEn.php, following the comment there. For example, to add a new parser function in an extension: create a file called **ExtensionName.i18n.magic.php** with the following contents: ```php [ 0, 'custom' ], ]; $magicWords['es'] = [ 'mag_custom' => [ 0, 'aduanero' ], ]; ``` Then in extension.json: ```json { "ExtensionMessagesFiles": { "ExtensionNameMagic": "ExtensionName.i18n.magic.php" }, "Hooks": { "ParserFirstCallInit": "MyExtensionHooks::onParserFirstCallInit" } } ``` It is important that the key "ExtensionNameMagic" is unique. It must not be used by another extension. And in the class file: ```php setFunctionHook( 'mag_custom', [ self::class, 'expandCustom' ] ); return true; } public static function expandCustom( $parser, $var1, $var2 ) { return "custom: var1 is $var1, var2 is $var2"; } } ``` - Online documentation (contains more informations): - Magic words: - Variables: - Parser functions: --- ### Memcached Memcached ==================================== MediaWiki has optional support for memcached, a "high-performance, distributed memory object caching system". For general information on it, see: Memcached is likely more trouble than a small site will need, but for a larger site with heavy load, like Wikipedia, it should help lighten the load on the database servers by caching data and objects in memory. Installation -------------------------------- Packages are available for Fedora, Debian, Ubuntu and probably other Linux distributions. If there's no package available for your distribution, you can compile it from source. Compilation -------------------------------- * PHP must be compiled with --enable-sockets * libevent: (as of 2003-08-11, 0.7a is current) * optionally, epoll-rt patch for Linux kernel: * memcached: (as of this writing, 1.1.9 is current) Memcached and libevent are under BSD-style licenses. The server should run on Linux and other Unix-like systems... you can run multiple servers on one machine or on multiple machines on a network; storage can be distributed across multiple servers, and multiple web servers can use the same cache cluster. **W A R N I N G ! ! ! ! !** Memcached has no security or authentication. Please ensure that your server is appropriately firewalled, and that the port(s) used for memcached servers are not publicly accessible. Otherwise, anyone on the internet can put data into and read data from your cache. An attacker familiar with MediaWiki internals could use this to steal passwords and email addresses, or to make themselves a sysop and install malicious javascript on the site. There may be other types of vulnerability, no audit has been done -- so be safe and keep it behind a firewall. **W A R N I N G ! ! ! ! !** Setup -------------------------------- If you installed memcached using a distro, the daemon should be started automatically using /etc/init.d/memcached To start the daemon manually, use something like: memcached -d -l 127.0.0.1 -p 11211 -m 64 (to run in daemon mode, accessible only via loopback interface, on port 11211, using up to 64 MiB of memory) In your LocalSettings.php file, set: ```php $wgMainCacheType = CACHE_MEMCACHED; $wgMemCachedServers = [ "127.0.0.1:11211" ]; ``` The wiki should then use memcached to cache various data. To use multiple servers (physically separate boxes or multiple caches on one machine on a large-memory x86 box), just add more items to the array. To increase the weight of a server (say, because it has twice the memory of the others and you want to spread usage evenly), make its entry a subarray: ```php $wgMemCachedServers = [ "127.0.0.1:11211", # one gig on this box [ "192.168.0.1:11211", 2 ] # two gigs on the other box ]; ``` PHP client for memcached -------------------------------- MediaWiki uses a fork of Ryan T. Dean's pure-PHP memcached client. It also supports the PECL PHP extension for memcached. MediaWiki uses the ObjectCache class to retrieve instances of BagOStuff by purpose, controlled by the following variables: * $wgMainCacheType * $wgParserCacheType * $wgMessageCacheType If you set one of these to CACHE_NONE, MediaWiki still creates a BagOStuff object, but calls it to it are no-ops. If the cache daemon can't be contacted, it should also disable itself fairly smoothly. Keys used -------------------------------- (incomplete, out of date) Date Formatter: key: $wgDBname:dateformatter ex: wikidb:dateformatter stores: a single instance of the DateFormatter class cleared by: nothing expiry: one hour Difference Engine: key: $wgDBname:diff:version:{MW_DIFF_VERSION}:oldid:$old:newid:$new ex: wikidb:diff:version:1.11a:oldid:1:newid:2 stores: body of a difference cleared by: nothing expiry: one week Interwiki: key: $wgDBname:interwiki:$prefix ex: wikidb:interwiki:w stores: object from the interwiki table of the database expiry: $wgInterwikiExpiry cleared by: nothing Lag time of the databases: key: $wgDBname:lag_times ex: wikidb:lag_times stores: array mapping the database id to its lag time expiry: 5 secondes cleared by: nothing Localisation: key: $wgDBname:localisation:$lang ex: wikidb:localisation:de stores: array of localisation settings set in: Language::loadLocalisation() expiry: none cleared by: Language::loadLocalisation() Message Cache: See MessageCache.php. Newtalk: key: $wgDBname:newtalk:ip:$ip ex: wikidb:newtalk:ip:123.45.67.89 stores: integer, 0 or 1 set in: User::loadFromDatabase() cleared by: User::saveSettings() # ? expiry: 30 minutes Parser Cache: access: ParserCache backend: $wgParserCacheType key: $wgDBname:pcache:idhash:$pageid-$renderkey!$hash $pageid: id of the page $renderkey: 1 if action=render, 0 otherwise $hash: hash of user options applied to the page, see ParserOptions::optionsHash() ex: wikidb:pcache:idhash:1-0!1!0!!en!2 stores: ParserOutput object modified by: WikiPage::doEditUpdates() or PoolWorkArticleView::doWork() expiry: $wgParserCacheExpireTime or less if it contains short lived functions key: $wgDBname:pcache:idoptions:$pageid stores: CacheTime object with an additional list of used options for the hash, serves as ParserCache pointer. modified by: ParserCache::save() expiry: The same as the ParserCache entry it points to. Ping limiter: controlled by: $wgRateLimits key: $wgDBname:limiter:action:$action:ip:$ip, $wgDBname:limiter:action:$action:user:$id, mediawiki:limiter:action:$action:ip:$ip and mediawiki:limiter:action:$action:subnet:$sub ex: wikidb:limiter:action:edit:ip:123.45.67.89, wikidb:limiter:action:edit:user:1012 mediawiki:limiter:action:edit:ip:123.45.67.89 and mediawiki:limiter:action:$action:subnet:123.45.67 stores: number of action made by user/ip/subnet cleared by: nothing expiry: expiry set for the action and group in $wgRateLimits Proxy Check: (deprecated) key: $wgDBname:proxy:ip:$ip ex: wikidb:proxy:ip:123.45.67.89 stores: 1 if the ip is a proxy cleared by: nothing expiry: $wgProxyMemcExpiry Revision text: key: $wgDBname:revisiontext:textid:$id ex: wikidb:revisiontext:textid:1012 stores: text of a revision cleared by: nothing expiry: $wgRevisionCacheExpiry Sessions: controlled by: $wgSessionsInObjectCache key: $wgBDname:session:$id ex: wikidb:session:38d7c5b8d3bfc51egf40c69bc40f8be3 stores: $SESSION, useful when using a multi-sever wiki expiry: one hour cleared by: session_destroy() Sidebar: access: WANObjectCache controlled by: $wgEnableSidebarCache key: $wgDBname:sidebar ex: wikidb:sidebar stores: the html output of the sidebar expiry: $wgSidebarCacheExpiry cleared by: MessageCache::replace() Special:Allpages: key: $wgDBname:allpages:ns:$ns ex: wikidb:allpages:ns:0 stores: array of pages in a namespace expiry: one hour cleared by: nothing Special:Recentchanges (feed): backend: $wgMessageCacheType key: $wgDBname:rcfeed:$format:$limit:$hideminor:$target and rcfeed:$format:timestamp ex: wikidb:rcfeed:rss:50:: and rcfeed:rss:timestamp stores: xml output of feed expiry: one day clear by: maintenance/rebuildrecentchanges.php script, or calling Special:Recentchanges?action=purge&feed=rss, Special:Recentchanges?action=purge&feed=atom, but note need $wgGroupPermissions[...]['purge'] permission. ... more to come ... --- ### Pageupdater PageUpdater =========== This document provides an overview of the usage of PageUpdater and DerivedPageDataUpdater. ## `PageUpdater` `PageUpdater` is the canonical way to create page revisions, that is, to perform edits. `PageUpdater` is a stateful, handle-like object that allows new revisions to be created on a given wiki page using the `saveRevision()` method. `PageUpdater` provides setters for defining the new revision's content as well as meta-data such as change tags. `saveRevision()` stores the new revision's primary content and metadata, and triggers the necessary updates to derived secondary data and cached artifacts e.g. in the `ParserCache` and the CDN layer, using a `DerivedPageDataUpdater`. `PageUpdater` instances follow the below life cycle, defined by a number of methods: +----------------------------+ | | | new | | | +------|--------------|------+ | | grabParentRevision()-| | or hasEditConflict()-| | | | +--------v-------+ | | | | | parent known | | | | | Enables---------------+--------|-------+ | safe operations based on | |-saveRevision() the parent revision, e.g. | | section replacement or | | edit conflict resolution. | | | | saveRevision()-| | | | +------v--------------v------+ | | | creation committed | | | Enables-----------------+----------------------------+ wasSuccess() isUnchanged() isNew() getState() getNewRevision() etc. The stateful nature of `PageUpdater` allows it to be used to safely perform transformations that depend on the new revision's parent revision, such as replacing sections or applying 3-way conflict resolution, while protecting against race conditions using a compare-and-swap (CAS) mechanism: after calling code used the `grabParentRevision()` method to access the edit's logical parent, `PageUpdater` remembers that revision, and ensure that that revision is still the page's current revision when performing the atomic database update for the revision's primary meta-data when `saveRevision()` is called. If another revision was created concurrently, `saveRevision()` will fail, indicating the problem with the "edit-conflict" code in the status object. Typical usage for programmatic revision creation (with `$page` being a WikiPage as of 1.32, to be replaced by a repository service later): ```php $updater = $page->newPageUpdater( $user ); $updater->setContent( SlotRecord::MAIN, $content ); $updater->setRcPatrolStatus( RecentChange::PRC_PATROLLED ); $newRev = $updater->saveRevision( $comment ); ``` Usage with content depending on the parent revision ```php $updater = $page->newPageUpdater( $user ); $parent = $updater->grabParentRevision(); $content = $parent->getContent( SlotRecord::MAIN )->replaceSection( $section, $sectionContent ); $updater->setContent( SlotRecord::MAIN, $content ); $newRev = $updater->saveRevision( $comment, EDIT_UPDATE ); ``` In both cases, all secondary updates will be triggered automatically. # `DerivedPageDataUpdater` `DerivedPageDataUpdater` is a stateful, handle-like object that caches derived data representing a revision, and can trigger updates of cached copies of that data, e.g. in the links tables, `page_props`, the `ParserCache`, and the CDN layer. `DerivedPageDataUpdater` is used by `PageUpdater` when creating new revisions, but can also be used independently when performing meta data updates during undeletion, import, or when puring a page. It's a stepping stone on the way to a more complete refactoring of WikiPage. **NOTE**: Avoid direct usage of `DerivedPageDataUpdater`. In the future, we want to define interfaces for the different use cases of `DerivedPageDataUpdater`, particularly providing access to post-PST content and `ParserOutput` to callbacks during revision creation, which currently use `WikiPage::prepareContentForEdit`, and allowing updates to be triggered on purge, import, and undeletion, which currently use `WikiPage::doEditUpdates()` and `Content::getSecondaryDataUpdates()`. The primary reason for `DerivedPageDataUpdater` to be stateful is internal caching of state that avoids the re-generation of `ParserOutput` and re-application of pre-save-transformations (PST). `DerivedPageDataUpdater` instances follow the below life cycle, defined by a number of methods: +---------------------------------------------------------------------+ | | | new | | | +---------------|------------------|------------------|---------------+ | | | grabCurrentRevision()-| | | | | | +-----------v----------+ | | | | |-prepareContent() | | knows current | | | | | | | Enables------------------+-----|-----|----------+ | | pageExisted() | | | | wasRedirect() | |-prepareContent() | |-prepareUpdate() | | | | | | +-------------v------------+ | | | | | | | +----> has content | | | | | | Enables------------------------|----------+--------------------------+ | isChange() | | | isCreation() |-prepareUpdate() | | getSlots() | prepareUpdate()-| | getTouchedSlotRoles() | | | getCanonicalParserOutput() | +-----------v------------v-----------------+ | | | +------------------> has revision | | | Enables-------------------------------------------+------------------------|-----------------+ updateParserCache() | runSecondaryDataUpdates() |-doUpdates() | +-----------v---------+ | | | updates done | | | +---------------------+ - `grabCurrentRevision()` returns the logical parent revision of the target revision. It is guaranteed to always return the same revision for a given `DerivedPageDataUpdater` instance. If called before `prepareUpdate()`, this fixates the logical parent to be the page's current revision. If called for the first time after `prepareUpdate()`, it returns the revision passed as the 'oldrevision' option to `prepareUpdate()`, or, if that wasn't given, the parent of $revision parameter passed to `prepareUpdate()`. - `prepareContent()` is called before the new revision is created, to apply pre-save-transformation (PST) and allow subsequent access to the canonical `ParserOutput` of the revision. `getSlots()` and `getCanonicalParserOutput()` as well as `getSecondaryDataUpdates()` may be used after `prepareContent()` was called. Calling `prepareContent()` with the same parameters again has no effect. Calling it again with mismatching parameters, or calling it after `prepareUpdate()` was called, triggers a `LogicException`. - `prepareUpdate()` is called after the new revision has been created. This may happen right after the revision was created, on the same instance on which `prepareContent()` was called, or later (possibly much later), on a fresh instance in a different process, due to deferred or asynchronous updates, or during import, undeletion, purging, etc. `prepareUpdate()` is required before a call to `doUpdates()`, and it also enables calls to `getSlots()` and `getCanonicalParserOutput()` as well as `getSecondaryDataUpdates()`. Calling `prepareUpdate()` with the same parameters again has no effect. Calling it again with mismatching parameters, or calling it with parameters mismatching the ones `prepareContent()` was called with, triggers a `LogicException`. - `getSecondaryDataUpdates()` returns `DataUpdates` that represent derived data for the revision. These may be used to update such data, e.g. in `ApiPurge`, `RefreshLinksJob`, and the `refreshLinks` script. - `doUpdates()` triggers the updates defined by `getSecondaryDataUpdates()`, and also causes updates to cached artifacts in the `ParserCache`, the CDN layer, etc. This is primarily used by PageUpdater, but also by `PageArchive` during undeletion, and when importing revisions from XML. `doUpdates()` can only be called after `prepareUpdate()` was used to initialize the `DerivedPageDataUpdater` instance for a specific revision. Calling it before `prepareUpdate()` is called raises a `LogicException`. A `DerivedPageDataUpdater` instance is intended to be re-used during different stages of complex update operations that often involve callbacks to extension code via MediaWiki's hook mechanism, or deferred or even asynchronous execution of Jobs and `DeferredUpdates`. Since these mechanisms typically do not provide a way to pass a `DerivedPageDataUpdater` directly, `WikiPage::getDerivedPageDataUpdater()` has to be used to obtain a `DerivedPageDataUpdater` for the update currently in progress - re-using the same `DerivedPageDataUpdater` if possible avoids re-generation of `ParserOutput` objects and other expensively derived artifacts. This mechanism for re-using a `DerivedPageDataUpdater` instance without passing it directly requires a way to ensure that a given `DerivedPageDataUpdater` instance can actually be used in the calling code's context. For this purpose, `WikiPage::getDerivedPageDataUpdater()` calls the `isReusableFor()` method on `DerivedPageDataUpdater`, which ensures that the given instance is applicable to the given parameters. In other words, `isReusableFor()` predicts whether calling `prepareContent()` or `prepareUpdate()` with a given set of parameters will trigger a `LogicException.` In that case, `WikiPage::getDerivedPageDataUpdater()` creates a fresh `DerivedPageDataUpdater` instance. --- ### Schema Schema ====== The most up-to-date schema for the tables in the database will always be `tables.sql` in the maintenance directory, which is called from the installation script. That file has been commented with details of the usage for each table and field. Historical information and some other notes are available at . --- ### Sitelist Sitelist ======== This document describes the XML format used to represent information about external sites known to a MediaWiki installation. This information about external sites is used to allow "inter-wiki" links, cross-language navigation, as well as close integration via direct access to the other site's web API or even directly to their database. Lists of external sites can be imported and exported using the *importSites.php* and *exportSites.php* scripts. In the database, external sites are described by the `sites` and `site_ids` tables. The formal specification of the format used by *importSites.php* and *exportSites.php* can be found in the *sitelist-1.0.xsd* file. Below is an example and a brief description of what the individual XML elements and attributes mean: ```xml acme.com acme Vendor http://acme.com/ meta.wikimedia.org de.wikidik.example de Dictionary http://acme.com/ ``` The XML elements are used as follows: - `sites`: The root element, containing a set of site tags. May have a `version` attribute with the value `1.0`. - `site`: A site entry, representing an external website. May have a `type` attribute with one of the following values: + `unknown`: (default) any website + `mediawiki`: A MediaWiki site - `globalid`: A unique identifier for the site. For a given site, the same unique global ID must be used across all wikis in a wiki farm (aka wiki family). - `localid`: An identifier for the site, for use on the local wiki. Multiple local IDs may be assigned to a given site. The same local ID can be used to refer to different sites by different wikis on the same farm/family. The `localid` element may have a type attribute with one of the following values: + `interwiki`: Used as an "interwiki" link prefix, for creating cross-wiki links. + `equivalent`: Used as a "language" link prefix, for cross-linking equivalent content in different languages. - `group`: The site group (e.g. wiki family) the site belongs to. - `path`: A URL template for accessing resources on the site. Several paths may be defined for a given site, for accessing different kinds of resources, identified by the `type` attribute, using one of the following values: + `link`: Generic URL template, often the document root. + `page_path`: (for `mediawiki` sites) URL template for wiki pages (corresponds to the target wiki's `$wgArticlePath` setting) + `file_path`: (for `mediawiki` sites) URL pattern for application entry points and resources (corresponds to the target wiki's `$wgScriptPath` setting). - `forward`: Whether using a prefix defined by a `localid` tag in the URL will cause the request to be redirected to the corresponding page on the target wiki (currently unused). E.g. whether should be forwarded to . (CAVEAT: not yet implement, can be specified but has no effect) ---