# Technical Documentation: typelevel/cats > ℹ️ **Provenance:** Multi-Tier Ingestion · [CodeWiki Reference](https://codewiki.google/github.com/typelevel/cats) ### Overview (`README.md`) ## Cats [](https://index.scala-lang.org/typelevel/cats/cats-core) [](https://opencollective.com/typelevel) [](https://discord.gg/XF3CXcMzqD) ### Overview Cats is a library which provides abstractions for functional programming in the [Scala programming language](https://scala-lang.org). Scala supports both object-oriented and functional programming, and this is reflected in the hybrid approach of the standard library. Cats strives to provide functional programming abstractions that are core, [binary compatible](#binary-compatibility-and-versioning), [modular](https://typelevel.org/cats/motivations.html#modularity), [approachable](https://typelevel.org/cats/motivations.html#approachability) and [efficient](https://typelevel.org/cats/motivations.html#efficiency). A broader goal of Cats is to provide a foundation for an [ecosystem of pure, typeful libraries](https://typelevel.org/cats/typelevelEcosystem.html) to support functional programming in Scala applications. For more detail about Cats' motivations, go [here](https://typelevel.org/cats/motivations.html). ### Why "cats"? The name is a playful shortening of the word *category*, from "[category theory](https://en.wikipedia.org/wiki/Category_theory)". Regardless, you do not need to know anything about category theory to use Cats. ## Contributors ### Code Contributors This project exists thanks to [all the people who contribute](https://github.com/typelevel/cats/graphs/contributors). We welcome contributions to Cats and would love for you to help build Cats. See our [contributor guide](CONTRIBUTING.md) for more information about how you can get involved as a developer. If you are looking for something to start with, [here is a beginner friendly list](https://github.com/typelevel/cats/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22). ### Financial Contributors [Become a financial contributor](https://opencollective.com/typelevel) and help us sustain our community. Donations directly support office hours for maintainers, better documentation and strategic initiatives.

Platinum Sponsors

Platinum sponsorship starts at $950 USD/month.

Gold Sponsors

Gold Sponsorship starts at $420 USD/month.

Silver Sponsors

Silver Sponsorship starts at $180 USD/month.

Backers

Become a Backer with a recurring donation of just $5 USD/month.

Other contributors

We thankfully accept one-time and recurring contributions as well.

Platinum Sponsors

Platinum sponsorship starts at $950 USD/month.

Gold Sponsors

Gold Sponsorship starts at $420 USD/month.

Silver Sponsors

Silver Sponsorship starts at $180 USD/month.

Backers

Become a Backer with a recurring donation of just $5 USD/month.

Other contributors

We thankfully accept one-time and recurring contributions as well.

### Getting Started Cats is available for [Scala.js](http://www.scala-js.org/) and [Scala Native](https://www.scala-native.org/), as well as the standard JVM runtime. Cats relies on improved type inference via the fix for [SI-2712](https://github.com/scala/bug/issues/2712), which is not enabled by default. For **Scala 2.12** you should add the following to your `build.sbt`: ```scala scalacOptions += "-Ypartial-unification" ``` (Partial unification is on by default since Scala 2.13, the compiler no longer accepts `-Ypartial-unification`) And then create the Cats dependency, by adding the following to your `build.sbt`: ```scala libraryDependencies += "org.typelevel" %% "cats-core" % "@VERSION@" ``` This will pull in the cats-core module. If you require some other functionality, you can pick-and-choose from amongst these modules (used in place of `"cats-core"`): * `cats-kernel`: Small set of basic type classes (*required*). * `cats-core`: Most core type classes and functionality (*required*). * `cats-laws`: Laws for testing type class instances. * `cats-free`: Free structures such as the free monad, and supporting type classes. * `cats-testkit`: lib for writing tests for type class instances using laws. * `algebra`: Type classes to represent algebraic structures. * `alleycats-core`: Cats instances and classes which are not lawful. There are several other Cats modules that are in separate repos so that they can maintain independent release cycles. * [`cats-effect`](https://github.com/typelevel/cats-effect): standard `IO` type together with `Sync`, `Async` and `Effect` type classes * [`cats-mtl`](https://github.com/typelevel/cats-mtl): transformer typeclasses for Cats' Monads, Applicatives and Functors. * [`mouse`](https://github.com/typelevel/mouse): a small companion to Cats that provides convenient syntax (aka extension methods) * [`kittens`](https://github.com/typelevel/kittens): automatic type class instance derivation for Cats and generic utility functions * [`cats-tagless`](https://github.com/typelevel/cats-tagless): Utilities for tagless final encoded algebras * [`cats-collections`](https://github.com/typelevel/cats-collections): Data structures which facilitate pure functional programming * [`cats-testkit-scalatest`](https://github.com/typelevel/cats-testkit-scalatest): Cats testkit integration with Scalatest Past release notes for Cats are available in [CHANGES.md](https://github.com/typelevel/cats/blob/main/CHANGES.md). ### Community Discussion around Cats is currently happening on GitHub issues, PR pages, and Discord: The [Typelevel Discord](https://discord.gg/XF3CXcMzqD) has \#cats and \#cats-dev channels, as well as community channels such as \#beginners. Please join us! People are expected to follow the [Typelevel Code of Conduct](https://typelevel.org/code-of-conduct.html) when discussing Cats on GitHub, Discord, or other venues. We hope that our community will be respectful, helpful, and kind. If you find yourself embroiled in a situation that becomes heated, or that fails to live up to our expectations, you should disengage and contact one of the [project maintainers](#maintainers) in private. We hope to avoid letting minor aggressions and misunderstandings escalate into larger problems. If you are being harassed, please contact one of [us](#maintainers) immediately so that we can support you. ### Binary compatibility and versioning After `1.0.0` release, we [decided](https://github.com/typelevel/cats/issues/1233) to use *MAJOR.MINOR.PATCH* [Semantic Versioning 2.0.0](http://semver.org/) going forward, which is different from the *EPOCH.MAJOR.MINOR* scheme common among Java and Scala libraries (including the Scala lang). Cats strives to provide a solid and stable foundation for an ecosystem of FP libraries. Thus, we treat backward binary compatibility maintenance with a high priority. In semantic versioning, backward breaking change is **ONLY** allowed between *MAJOR* versions. We will maintain backward binary compatibility between *PATCH* AND *MINOR* versions. For example, when we release Cats `1.1.0`, it will be backward binary compatible with the previous `1.0.x` versions. I.E. the new JAR will be a drop-in replacement for the old one. This is critical when your application has a diamond dependency on Cats - depending on two or more libraries that all depend on Cats. If one library upgrades to the new `1.1.0` Cats before the other one does, your application still runs thanks to this backward binary compatibility. Also worth noting is that according to semantic versioning, *MINOR* version Y (x.Y.z | x > 0) MUST be incremented if new, backwards compatible functionality is introduced to the public API. It MUST be incremented if any public API functionality is marked as deprecated. Any binary breaking changes will require a *MAJOR* version bump, which we will be very cautious about. We will also consider using `organization` and package name for major versioning in the future. But that decision is yet to be made. ### Adopters A (non-exhaustive) list of companies that use Cats in production is featured on the [Adopters page]. Don't see yours? [You can add it in a PR!](https://github.com/typelevel/cats/edit/main/ADOPTERS.md) And if you can, consider [supporting us](https://opencollective.com/typelevel). ### Maintainers The current maintainers (people who can merge pull requests) are: * [rossabaker](https://github.com/rossabaker) Ross Baker * [armanbilge](https://github.com/armanbilge) Arman Bilge * [johnynek](https://github.com/johnynek) P. Oscar Boykin * [adelbertc](https://github.com/adelbertc) Adelbert Chang * [danicheg](https://github.com/danicheg) Daniel Esik * [LukaJCB](https://github.com/LukaJCB) Luka Jacobowitz * [peterneyens](https://github.com/peterneyens) Peter Neyens * [tpolecat](https://github.com/tpolecat) Rob Norris * [non](https://github.com/non) Erik Osheim * [barambani](https://github.com/barambani) Filippo Mariotti * [mpilquist](https://github.com/mpilquist) Michael Pilquist * [milessabin](https://github.com/milessabin) Miles Sabin * [djspiewak](https://github.com/djspiewak) Daniel Spiewak * [fthomas](https://github.com/fthomas) Frank Thomas * [satorg](https://github.com/satorg) Sergey Torgashov * [julien-truffaut](https://github.com/julien-truffaut) Julien Truffaut * [kailuowang](https://github.com/kailuowang) Kailuo Wang Retired committers include: * [ceedubs](https://github.com/ceedubs) Cody Allen * [travisbrown](https://github.com/travisbrown) Travis Brown We are currently following a practice of requiring at least two sign-offs to merge code PRs (and for large or contentious issues we may wait for more). For typos, documentation improvements or minor build fix we relax this to a single sign-off. More detail in the [process document](https://github.com/typelevel/cats/blob/main/PROCESS.md). ### Copyright and License All code is available to you under the MIT license, available at http://opensource.org/licenses/mit-license.php and also in the [COPYING](https://github.com/typelevel/cats/blob/main/COPYING) file. The design is informed by many other projects, in particular [Scalaz](https://github.com/scalaz/scalaz). Copyright the maintainers, 2015-2024. [Adopters page]: ADOPTERS.md --- ### Algebra (`docs/algebra.md`) {% laika.title = Algebra %} # Algebra Overview Algebra uses type classes to represent algebraic structures. You can use these type classes to represent the abstract capabilities (and requirements) you want generic parameters to possess. This section will explain the structures available. ## algebraic properties and terminology We will be talking about properties like *associativity* and *commutativity*. Here is a quick explanation of what those properties mean: |Name |Description | |-------------|--------------------------------------------------------------------------------| |Associative | If `⊕` is associative, then `a ⊕ (b ⊕ c)` = `(a ⊕ b) ⊕ c`. | |Commutative | If `⊕` is commutative, then `a ⊕ b` = `b ⊕ a`. | |Identity | If `id` is an identity for `⊕`, then `a ⊕ id` = `id ⊕ a` = `a`. | |Inverse | If `¬` is an inverse for `⊕` and `id`, then `a ⊕ ¬a` = `¬a ⊕ a` = `id`. | |Distributive | If `⊕` and `⊙` distribute, then `a ⊙ (b ⊕ c)` = `(a ⊙ b) ⊕ (a ⊙ c)` and `(a ⊕ b) ⊙ c` = `(a ⊙ c) ⊕ (b ⊙ c)`. | |Idempotent | If `⊕` is idempotent, then `a ⊕ a` = `a`. If `f` is idempotent, then `f(f(a))` = `f(a)` | Though these properties are illustrated with symbolic operators, they work equally-well with functions. When you see `a ⊕ b` that is equivalent to `f(a, b)`: `⊕` is an infix representation of the binary function `f`, and `a` and `b` are values (of some type `A`). Similarly, when you see `¬a` that is equivalent to `g(a)`: `¬` is a prefix representation of the unary function `g`, and `a` is a value (of some type `A`). ## basic algebraic structures The most basic structures can be found in the `algebra` package. They all implement a method called `combine`, which is associative. The identity element (if present) will be called `empty`, and the inverse method (if present) will be called `inverse`. |Name |Associative?|Commutative?|Identity?|Inverse?|Idempotent?| |--------------------|------------|------------|---------|--------|-----------| |Semigroup | ✓| | | | | |CommutativeSemigroup| ✓| ✓| | | | |Monoid | ✓| | ✓| | | |Band | ✓| | | | ✓| |Semilattice | ✓| ✓| | | ✓| |Group | ✓| | ✓| ✓| | |CommutativeMonoid | ✓| ✓| ✓| | | |CommutativeGroup | ✓| ✓| ✓| ✓| | |BoundedSemilattice | ✓| ✓| ✓| | ✓| (For a description of what each column means, see [§algebraic properties and terminology](#algebraic-properties-and-terminology).) ## ring-like structures The `algebra.ring` package contains more sophisticated structures which combine an *additive* operation (called `plus`) and a *multiplicative* operation (called `times`). Additive identity and inverses will be called `zero` and `negate` (respectively); multiplicative identity and inverses will be called `one` and `reciprocal` (respectively). All ring-like structures are associative for both `+` and `*`, have commutative `+`, and have a `zero` element (an identity for `+`). |Name |Has `negate`?|Has `1`?|Has `reciprocal`?|Commutative `*`?| |--------------------|-------------|--------|-----------------|----------------| |Semiring | | | | | |Rng | ✓| | | | |Rig | | ✓| | | |CommutativeRig | | ✓| | ✓| |Ring | ✓| ✓| | | |CommutativeRing | ✓| ✓| | ✓| |Semifield | | ✓| ✓| | |CommutativeSemifield| | ✓| ✓| ✓| |Field | ✓| ✓| ✓| ✓| (For a description of what the terminology in each column means, see [§algebraic properties and terminology](#algebraic-properties-and-terminology).) ## lattice-like structures The `algebra.lattice` package contains more structures that can be somewhat ring-like. Rather than `plus` and `times` we have `meet` and `join` both of which are always associative, commutative and idempotent, and as such each can be viewed as a semilattice. Meet can be thought of as the greatest lower bound of two items while join can be thought of as the least upper bound between two items. When `zero` is present, `join(a, zero)` = `a`. When `one` is present `meet(a, one)` = `a`. When `meet` and `join` are both present, they obey the absorption law: - `meet(a, join(a, b))` = `join(a, meet(a, b)) = a` Sometimes meet and join distribute, we say it is distributive in this case: - `meet(a, join(b, c))` = `join(meet(a, b), meet(a, c))` - `join(a, meet(b, c))` = `meet(join(a, b), join(a, c))` Sometimes an additional binary operation `imp` (for impliciation, also written as →, meet written as ∧) is present. Implication obeys the following laws: - `a → a` = `1` - `a ∧ (a → b)` = `a ∧ b` - `b ∧ (a → b)` = `b` - `a → (b ∧ c)` = `(a → b) ∧ (a → c)` The law of the excluded middle can be expressed as: - `(a ∨ (a → 0))` = `1` |Name |Has `join`?|Has `meet`?|Has `zero`?|Has `one`?|Distributive|Has `imp`?|Excludes middle?| |--------------------------|-----------|-----------|-----------|----------|------------|----------|----------------| |JoinSemilattice | ✓| | | | | | | |MeetSemilattice | | ✓| | | | | | |BoundedJoinSemilattice | ✓| | ✓| | | | | |BoundedMeetSemilattice | | ✓| | ✓| | | | |Lattice | ✓| ✓| | | | | | |DistributiveLattice | ✓| ✓| | | ✓| | | |BoundedLattice | ✓| ✓| ✓| ✓| | | | |BoundedDistributiveLattice| ✓| ✓| ✓| ✓| ✓| | | |Heyting | ✓| ✓| ✓| ✓| ✓| ✓| | |Bool | ✓| ✓| ✓| ✓| ✓| ✓| ✓| Note that a `BoundedDistributiveLattice` gives you a `CommutativeRig`, but not the other way around: rigs aren't distributive with `a + (b * c) = (a + b) * (a + c)`. Also, a `Bool` gives rise to a `BoolRing`, since each element can be defined as its own negation. Note, Bool's `.asBoolRing` is not an extension of the `.asCommutativeRig` method as the `plus` operations are defined differently. ### Documentation Help We'd love your help with this documentation! You can edit this page in your browser by clicking [this link](https://github.com/typelevel/cats/edit/main/docs/algebra.md). --- ### Alleycats (`docs/alleycats.md`) # Alleycats ## Overview Alleycats is a module of the [Cats](https://github.com/typelevel/cats) project that exists to support types which are not entirely savory or *law-abiding* but which may prove useful or necessary in some situations. In some cases, a type class instance can almost (but not quite) obey the required laws (e.g. a `Monad` instance for `scala.util.Try`). In other cases, type classes which lack laws or constraints may still be useful in some cases (e.g. `Empty[_]`, a type class which provides some notion of "emptiness"). Rather than argue about whether to permit these types in Cats proper, we provide a (slightly disreputable) home for them here. ## Type classes Alleycats introduces several new type classes. Here is an overview of the instances introduced. ### Empty\[A\], Zero\[A\], and One\[A\] A commonly-requested type class is one that encodes the idea of a set having an identity element. Normally this would be done by defining a `Monoid[A]` instance, but in some cases the idea of emptiness is independent of a particular associative operation. In this case, `Empty[A]` may be used in place of `Monoid[A]`. It provides access to the *identity* element (via the `.empty` method), and can also provide `.isEmpty` and `.nonEmpty` if an `Eq[A]` is available. The two other type classes, `Zero[A]` and `One[A]`, are similar, except they correspond to `AdditiveMonoid[A]` and `MultiplicativeMonoid[A]` (found in the `algebra.ring` package). Their methods are called `zero` and `one`, respectively. While none of these type classes have their own laws, they are required not to violate the monoid laws when applicable. This means that if `Empty[A]` and `Semigroup[A]` are both available, that `Empty[A].empty` **must** function as an identity element for `Semigroup[A].combine`. In fact, together these instances can be viewed as a `Monoid[A]` (and there is an implicit method to that effect). The same rules apply to `Zero[A]` and `One[A]` and their respective associative operations. ### Pure\[F\[\_\]\] and Extract\[F\[\_\]\] The `Pure[F]` type class represents the `pure` method of `Applicative[F]` separated from its `map` and `ap` methods. Like the previous type classes, if `Pure[F]` and `Apply[F]` are both available they are required to be consistent (and should provide a valid `Applicative[F]` instance). Similarly, `Extract[F]` represents the `extract` method of `Comonad[F]` without `coflatMap` and `map` methods. When `Extract[F]` and `CoflatMap[F]` are available, they should provide a valid `Comonad[F]` instance. ### EmptyK\[F\[\_\]\] Finally, `EmptyK[F]` generalizes the `empty[A]` method from `MonoidK[F]`. The pattern here is the same as before -- `SemigroupK[F]` and `EmptyK[F]` should provide a valid `MonoidK[F]` instance. ## Instances Alleycats also provides some "disreputable" type class instances. ### Set\[\_\] instances Scala's `Set[_]` takes advantage of the universal availability of `.hashCode` and `.equals`. This makes it difficult to use [parametricity](https://failex.blogspot.jp/2013/06/fake-theorems-for-free.html) to reason about sets, and casts some doubt on their use with functors and monads. Alleycats provides `Monad[Set]` and `Traverse[Set]`. You can import these instances via `import alleycats.std.set._`. ### Try\[\_\] instances Scala's `Try[_]` is intended to replace the need for `try { ... } catch { ... }` syntax in Scala programs, to ease error-handling, and to transport exceptions as data. Due to the way `Try` transparently catches exceptions in `.map` and `.flatMap`, some people are skeptical that `Try` fulfills the necessary functor/monad laws. Alleycats provides a `Monad[Try]`. You can import this instance via `import alleycats.std.try_._`. ### Iterable\[\_\] instances Scala's `collection.Iterable[_]` offers no guarantees that it's immutable, since it abstracts over the `mutable` and `immutable` variants. However it's the type used to represent a `Map`s `values`, and its often desirable to treat the values of a map as a `Foldable` collection. Alleycats provides a `Foldable[Iterable]`, eg: ``` import cats.syntax.all._ import alleycats.std.iterable._ // Result "AppleOrange" Map(1 -> "Apple", 2 -> "Orange").values.combineAll ``` ## Contributing This module's goal is to be very liberal about accepting type class instances, but to only provide instances which are absent from Cats proper. Law-abiding instances will end up in Cats, and everything else will end up here. --- ### Colophon (`docs/colophon.md`) # Colophon Cats has been made a much better project, and is a much more enjoyable project to work on because of many of the other projects on which Cats is built. Many of these projects have had enhancements made in order to benefit Cats, for which we are grateful. We would like to thank the maintainers of these supporting projects, and we'd encourage you to check out these projects and consider integrating them into your own projects. * [scalacheck](http://scalacheck.org) for property-based testing * [discipline](https://github.com/typelevel/discipline) for encoding and testing laws * [kind-projector](https://github.com/typelevel/kind-projector) for type lambda syntax * [mdoc](https://github.com/scalameta/mdoc) type-checked example code makes sure that our examples stay in sync with the rest of our source There are other libraries that aim to foster Functional Programming in the Scala programming language which Cats has a relationship to: * [scalaz](https://github.com/scalaz/scalaz) The project which directly inspires Cats. Currently Cats borrows some code directly from scalaz. * [Structures](https://github.com/mpilquist/Structures) A project very similar in nature to Cats, also derived from scalaz. The Structures and Cats projects have had a healthy relationship of sharing both ideas and code. For a full list of Cats and Typelevel related projects and libraries, take a look at [The Typelevel Ecosystem](typelevelEcosystem.md) --- ### Datatypes (`docs/datatypes.md`) {% laika.title = "Data Types" %} @:navigationTree { entries = [ { target = "datatypes", depth = 2 } ] } --- ### Faq (`docs/faq.md`) {% laika.title = FAQ %} # Frequently Asked Questions @:navigationTree { entries = [ { target = "faq.md", depth = 2 } ] } ## What imports do I need? The easiest approach to Cats imports is to import everything that's commonly needed: ```scala mdoc:silent import cats._ import cats.data._ import cats.syntax.all._ ``` This should be all that you need, but if you'd like to learn more about the details of imports than you can check out the [import guide](imports.md). ## I am new to pure functional programming, what quick wins can I get from Cats? Please refer to the [jump start guide](jump_start_guide.md). ## What is the difference between Cats and Scalaz? Cats and [Scalaz](https://github.com/scalaz/scalaz) have the same goal: to facilitate pure functional programming in Scala applications. However the underlying core strategy is different; Scalaz took the approach of trying to provide a single batteries-included *standard library* for FP that powers the Scala applications. Cats, on the other hand, aims to help build an [ecosystem](typelevelEcosystem.md) of pure FP libraries by providing a solid and stable foundation; these libraries can have their own styles and personalities, competing with each other, while at the same time playing nice. It is through this ecosystem of FP libraries (cats included) that Scala applications can be powered with "FP awesome-ness" and beyond by picking whatever best fit their needs. Based on this core strategy, Cats takes a [modular](motivations.md#modularity) approach and focuses on providing core, [binary compatible](index.md#binary-compatibility-and-versioning), [approachable](motivations.md#approachability) and [efficient](motivations.md#efficiency) abstractions. It provides a welcoming and supportive environment for the [user community](https://discord.gg/XF3CXcMzqD) governed by the [Typelevel Code of Conduct](https://typelevel.org/code-of-conduct.html). It also takes great effort in supplying a comprehensive and beginner-friendly [documentation](motivations.md#documentation). ## Where is right-biased Either? Up through Cats 0.7.x we had `cats.data.Xor`, which was effectively `scala.util.Either`, but right-biased by default and with a bunch of useful combinators around it. In Scala 2.12.x `Either` [became right-biased](https://github.com/scala/scala/pull/5135) so we revisited the use of `Xor` and [decided](https://github.com/typelevel/cats/issues/1192) that in the interest of interoperability, we would remove `Xor` in the Cats 0.8.0 release and fill in the gaps in the `scala.util.Either` API via [syntax enrichment](https://github.com/typelevel/cats/blob/main/core/src/main/scala/cats/syntax/either.scala). This syntax and the type class instances for `Either` can be imported using `cats.syntax._`, which will also bring in syntactic enrichment and instances for other standard library types, or you can import only the `Either` enrichment with `cats.syntax.either._`. There are a few minor mismatches between `Xor` and `Either`. For example, in some cases you may need to specify a type parameter for an enrichment method on `Either` (such as `leftMap`) even though it was properly inferred for `Xor`, due to `Either` having covariant type parameters. Similarly, `cats.data.XorT` has been replaced with `cats.data.EitherT`, although since this is a type defined in Cats, you don't need to import syntax or instances for it (although you may need imports for the underlying monad). ## Why is the compiler having trouble with types with more than one type parameter? When you encounter a situation where the same code works fine with a type with one type parameter, e.g. `List[A]`, but doesn't work with types with more than one, e.g. `Either[A, B]`, you probably hit [SI-2712](https://issues.scala-lang.org/browse/SI-2712). Without going into the details, it's highly recommended to enable a partial SI-2712 fix in your project. The easiest way to achieve that is through this [sbt plugin](https://github.com/fiadliel/sbt-partial-unification). Cats used to provide mitigation to this issue semi-transparently, but given the fact that the fix is now mainstream, we decided to drop that mitigation machinery in favor of reducing the complexity. See this [issue](https://github.com/typelevel/cats/issues/1073) for details. ## Why is some example code not compiling for me? A portion of example code requires either the [Kind-projector](https://github.com/typelevel/kind-projector) compiler plugin or partial unification turned on in scalac. The easiest way to turn partial unification on is through this [sbt plugin](https://github.com/fiadliel/sbt-partial-unification). ## Why can't the compiler find implicit instances for Future? If you have already followed the [imports advice](#what-imports-do-i-need) but are still getting error messages like `could not find implicit value for parameter e: cats.Monad[scala.concurrent.Future]` or `value |+| is not a member of scala.concurrent.Future[Int]`, then make sure that you have an implicit `scala.concurrent.ExecutionContext` in scope. The easiest way to do this is to `import scala.concurrent.ExecutionContext.Implicits.global`, but note that you may want to use a different execution context for your production application. ## Where are implicit instances for `Seq`? As of `cats-2.3`, instances for `collection.immutable.Seq` are provided by cats. Mind that, up to `scala-2.12`, `Seq` was an alias for `collection.Seq` and lawful instances can't be provided for it due to its potential mutability. In `scala-2.13`, `Seq` was changed to `collection.immutable.Seq` which greatly improves `Seq`'s interoperability with cats. ## How can I turn my List of `` into a `` of a list? It's really common to have a `List` of values with types like `Option`, `Either`, or `Validated` that you would like to turn "inside out" into an `Option` (or `Either` or `Validated`) of a `List`. The `sequence` and `traverse` methods are _really_ handy for this. You can read more about them in the [Traverse documentation](typeclasses/traverse.md). ## Where is ListT? There are monad transformers for various types, such as [OptionT](datatypes/optiont.md), so people often wonder why there isn't a `ListT`. For example, in the following example, people might reach for `ListT` to simplify making nested `map` and `exists` calls: ```scala mdoc:reset:silent val l: Option[List[Int]] = Some(List(1, 2, 3, 4, 5)) def isEven(i: Int): Boolean = i % 2 == 0 ``` ```scala mdoc l.map(_.map(_ + 1)) l.exists(_.exists(isEven)) ``` A naive implementation of `ListT` suffers from associativity issues; see [this gist](https://gist.github.com/tpolecat/1227e22e3161b5816e014c00650f3b57) for an example. It's possible to create a `ListT` that doesn't have these issues, but it tends to be pretty inefficient. For many use-cases, [Nested](https://github.com/typelevel/cats/blob/main/core/src/main/scala/cats/data/Nested.scala) can be used to achieve the desired results. Here is how we could achieve the effect of the previous example using `Nested`: ```scala mdoc:silent import cats.data.Nested import cats.syntax.all._ ``` ```scala mdoc val nl = Nested(l) nl.map(_ + 1) nl.exists(isEven) ``` We can even perform more complicated operations, such as a `traverse` of the nested structure: ```scala mdoc:silent import cats.data.ValidatedNel type ErrorsOr[A] = ValidatedNel[String, A] def even(i: Int): ErrorsOr[Int] = if (i % 2 == 0) i.validNel else s"$i is odd".invalidNel ``` ```scala mdoc nl.traverse(even) ``` ## Where are `Applicative`s for monad transformers? An `Applicative` instance for `OptionT[F, *]`/`EitherT[F, E, *]`, built without a corresponding `Monad` instance for `F`, would be unlawful, so it's not included. See [the guidelines](guidelines.md#applicative-instances-for-monad-transformers) for a more detailed explanation. As an alternative, using `.toNested` on the monad transformer is recommended, although its `ap` will still be inconsistent with the Monad instance's.`. ## Where is IO/Task? In purely functional programming, a monadic `IO` or `Task` type is often used to handle side effects such as file/network IO. In some languages and frameworks, such a type also serves as the primary abstraction through which parallelism is achieved. Nearly every real-world purely functional application or service is going to require such a data type, and this gives rise to an obvious question: why doesn't Cats include such a type? The answer is that Cats *does* include an `IO`, it just isn't included in the core library. The decision was made to split `IO` away from cats-core and (indeed the whole Cats release cycle!) in order to make it easier to ensure modular versioning and compatibility across the ecosystem. The [cats-effect](https://github.com/typelevel/cats-effect) project defines a type, `cats.effect.IO`, which is intended to be a very minimal, very performant data type for managing synchronous and asynchronous side-effects, integrated into the Cats ecosystem. However, we acknowledge that this type may not meet everyone's needs. The cats-effect project characterizes the space of side-effect-capturing data types with a set of typeclasses (deriving from `cats.Monad`), and so all such data types are, broadly-speaking, mutually compatible and interchangeable in many generic contexts. For example, [Monix](https://monix.io) provides support for IO, concurrency, and streaming and integrates with the cats-effect type classes. It may be worth keeping in mind that `IO` and `Task` are pretty blunt instruments (they are essentially the `Any` of side effect management), and you may want to narrow the scope of your effects throughout most of your application. The [free monad](datatypes/freemonad.md) documentation describes a way to abstractly define controlled effects and interpret them into a type such as `IO` or `Task` as late as possible. As more of your code becomes pure through these controlled effects the less it matters which type you end up choosing to represent your side effects. ## What do types like `?` and `λ` mean? Cats defines a wealth of type classes and type class instances. For a number of the type class and instance combinations, there is a mismatch between the type parameter requirements of the type class and the type parameter requirements of the data type for which the instance is being defined. For example, the `Either` data type is a type constructor with two type parameters. We would like to be able to define a [Monad](typeclasses/monad.md) for `Either`, but the `Monad` type class operates on type constructors having only one type parameter. **Enter type lambdas!** Type lambdas provide a mechanism to allow one or more of the type parameters for a particular type constructor to be fixed. In the case of `Either` then, when defining a `Monad` for `Either`, we want to fix one of the type parameters at the point where a `Monad` instance is summoned, so that the type parameters line up. As `Either` is right biased, a type lambda can be used to fix the left type parameter and allow the right type parameter to continue to vary when `Either` is treated as a `Monad`. **Enter [kind-projector](https://github.com/typelevel/kind-projector)!** kind-projector is a compiler plugin which provides a convenient syntax for dealing with type lambdas. The symbols `?` and `λ` are treated specially by kind-projector, and expanded into the more verbose definitions that would be required were it not to be used. You can read more about kind-projector at the [project page](https://github.com/typelevel/kind-projector). ## What is `tailRecM`? The `FlatMap` type class has a `tailRecM` method with the following signature: ```scala def tailRecM[A, B](a: A)(f: A => F[Either[A, B]]): F[B] ``` When you are defining a `FlatMap` instance, its `tailRecM` implementation must have two properties in order for the instance to be considered lawful. The first property is that `tailRecM` must return the same result that you would get if you recursively called `flatMap` until you got a `Right` value (assuming you had unlimited stack space—we'll get to that in a moment). In other words, it must give the same result as this implementation: ```scala mdoc:silent trait Monad[F[_]] { def pure[A](x: A): F[A] = ??? def flatMap[A, B](fa: F[A])(f: A => F[B]): F[B] = ??? def tailRecM[A, B](a: A)(f: A => F[Either[A, B]]): F[B] = flatMap(f(a)) { case Right(b) => pure(b) case Left(nextA) => tailRecM(nextA)(f) } } ``` The reason we can't simply use this implementation for all type constructors (and the reason that `tailRecM` is useful at all) is that for many monadic types, recursively `flatMap`-ing in this way will quickly exhaust the stack. `Option` is one example of a monadic type whose `flatMap` consumes stack in such a way that nesting `flatMap` calls deeply enough (usually around a couple thousand levels) will result in a stack overflow. We can provide a stack-safe `tailRecM` implementation for `Option`, though: ```scala mdoc:silent import cats.FlatMap import scala.annotation.tailrec implicit val optionFlatMap: FlatMap[Option] = new FlatMap[Option] { def map[A, B](fa: Option[A])(f: A => B): Option[B] = fa.map(f) def flatMap[A, B](fa: Option[A])(f: A => Option[B]): Option[B] = fa.flatMap(f) @tailrec def tailRecM[A, B](a: A)(f: A => Option[Either[A, B]]): Option[B] = f(a) match { case None => None case Some(Left(a1)) => tailRecM(a1)(f) case Some(Right(b)) => Some(b) } } ``` Now we don't have to worry about overflowing the stack, no matter how many times we have to call `tailRecM` before we get a `Right`. This is useful because any operation that you would write using recursive `flatMap`s can be rewritten to use `tailRecM`, and if the `FlatMap` instance for your type constructor is lawful, you don't have to worry about stack safety. The downside is that how you write a lawful `tailRecM` for your type constructor may not always be obvious. For some type constructors, such as `Future`, recursively `flatMap`-ing is already safe, and the first simple implementation above will be lawful. For types like `Option` and `Try`, you'll need to arrange the recursion in such a way that the `tailRecM` calls are tail calls (which you can confirm with Scala's `tailrec` annotation). Collection types require yet another approach (see for example the [implementation for `List`](https://github.com/typelevel/cats/pull/1041/files#diff-e4d8b82ab5544972195d955591ffe18cR31)). If you're having trouble figuring out how to implement `tailRecM` lawfully, you can try to find an instance in Cats itself for a type that is semantically similar to yours (all of the `FlatMap` instances provided by Cats have lawful, stack-safe `tailRecM` implementations). In some cases you may decide that providing a lawful `tailRecM` may be impractical or even impossible (if so we'd like to hear about it). For these cases we provide a way of testing all of the monad laws _except_ for the stack safety of `tailRecM`: just replace `MonadTests[F].monad[A, B, C]` in your tests with `MonadTests[F].stackUnsafeMonad[A, B, C]`. ## What does this symbol mean? Below is a list of symbols used in Cats. The `~>`, `⊥`, `⊤`, `:<:` and `:≺:` symbols can be imported with `import cats._`. All other symbols can be imported with `import cats.syntax.all._` | Symbol | Name | Nickname | Type Class | Signature | | -------------------------------- | -------------------------| ---------------- | ----------------------- | --------------------------------------------------------------------| | `fa *> fb` | product right | | `Apply[F[_]]` | `productR(fa: F[A])(fb: F[B]): F[B]` | | `fa <* fb` | product left | | `Apply[F[_]]` | `productL(fa: F[A])(fb: F[B]): F[A]` | | `x === y` | equals | | `Eq[A]` | `eqv(x: A, y: A): Boolean` | | `x =!= y` | not equals | | `Eq[A]` | `neqv(x: A, y: A): Boolean` | | `fa >>= f` | flatMap | | `FlatMap[F[_]]` | `flatMap(fa: F[A])(f: A => F[B]): F[B]` | | `fa >> fb` | followed by | | `FlatMap[F[_]]` | `>>(fb: => F[B]): F[B]` | | x |-| y | remove | | `Group[A]` | `remove(x: A, y: A): A` | | `x > y` | greater than | | `PartialOrder[A]` | `gt(x: A, y: A): Boolean` | | `x >= y` | greater than or equal | | `PartialOrder[A]` | `gteq(x: A, y: A): Boolean` | | `x < y` | less than | | `PartialOrder[A]` | `lt(x: A, y: A): Boolean` | | `x <= y` | less than or equal | | `PartialOrder[A]` | `lteq(x: A, y: A): Boolean` | | x |+| y | Semigroup combine | | `Semigroup[A]` | `combine(x: A, y: A): A` | | `x <+> y` | SemigroupK combine | | `SemigroupK[F[_]]` | `combineK(x: F[A], y: F[A]): F[A]` | | `f <<< g` | Arrow compose | | `Compose[F[_, _]]` | `compose(f: F[B, C], g: F[A, B]): F[A, C]` | | `f >>> g` | Arrow andThen | | `Compose[F[_, _]]` | `andThen(f: F[A, B], g: F[B, C]): F[A, C]` | | `f &&& g` | Arrow merge | | `Arrow[F[_, _]]` | `merge[A, B, C](f: F[A, B], g: F[A, C]): F[A, (B, C)]` | | `f -< g` | Arrow combine and bypass | | `Arrow[F[_, _]]` | `combineAndByPass[A, B, C](f: F[A, B], g: F[B, C]): F[A, (B, C)]` | | `F ~> G` | natural transformation | | `FunctionK[F[_], G[_]]` | `FunctionK` alias | | `F :<: G` | injectK | | `InjectK[F[_], G[_]]` | `InjectK` alias | | `F :≺: G` | injectK | | `InjectK[F[_], G[_]]` | `InjectK` alias | | `fa &> fb` | parallel product right | | `Parallel[M[_]]` | `parProductR[A, B](ma: M[A])(mb: M[B]): M[B]` | | `fa <& fb` | parallel product left | | `Parallel[M[_]]` | `parProductL[A, B](ma: M[A])(mb: M[B]): M[A]` | | `⊥` | bottom | | N/A | `Nothing` | | `⊤` | top | | N/A | `Any` | | `fa << fb` (Deprecated) | product left | | `FlatMap[F[_]]` | `productL(fa: F[A])(fb: F[B]): F[A]` | ## How can I test instances against their type classes' laws? You can find more information [here](typeclasses/lawtesting.md). ## How can I help? The Сats community welcomes and encourages contributions, even if you are completely new to Сats and functional programming. Here are a few ways to help out: - Find an undocumented method and write a ScalaDoc entry for it. See [Arrow.scala](https://github.com/typelevel/cats/blob/main/core/src/main/scala/cats/arrow/Arrow.scala) for some examples of ScalaDoc entries that use [sbt-doctest](https://github.com/tkawachi/sbt-doctest). - Find an [open issue](https://github.com/typelevel/cats/issues?q=is%3Aopen+is%3Aissue+label%3Aready), leave a comment on it to let people know you are working on it, and submit a pull request. If you are new to Сats, you may want to look for items with the [low-hanging-fruit](https://github.com/typelevel/cats/issues?q=is%3Aopen+is%3Aissue+label%3A%22low-hanging+fruit%22) label. See the [contributing guide](CONTRIBUTING.md) for more information. ## How to try Cats in a REPL? The easiest way is probably using [Ammonite-REPL](http://ammonite.io/). Install it following the instructions there. Then in the amm console you can type in ```scala // interp.configureCompiler(_.settings.YpartialUnification.value = true) // If using scala 2.11 or 2.12 import $ivy.`org.typelevel::cats-core:2.1.1`, cats._, cats.data._, cats.implicits._ ``` Or if you want, you can add these lines to `~/.ammonite/predef.sc` so that they are enabled every ammonite session. ## Why aren't monad transformers like `OptionT` and `EitherT` covariant like `Option` and `Either`? Please see [Variance of Monad Transformers](https://typelevel.org/blog/2018/09/29/monad-transformer-variance.html) on the Typelevel blog. ## Why isn't `Set` a lawful `Functor` (and hence `Applicative`, `Monad`, etc.)? Although in many cases `Set` can behave as a `Functor` indeed, there are some edge cases where it doesn't. For example, `Set` can violate the "covariant composition" law of `Functor`, which is: ``` fa.map(f).map(g) <-> fa.map(f.andThen(g)) ``` Since `Set` is internally based on universal hashing and equality, it violates this law when used with a data type that overrides `equals` and `hashCode` in a such a way that its instances may appear equal but de-facto contain different values. One example of this type is `scala.concurrent.duration.Duration`: ```scala mdoc import scala.concurrent.duration.* val f: String => Duration = Duration.apply val g: Duration => String = _.toString val set = Set("1 minute", "60 seconds", "60000 milliseconds") set.map(f).map(g) // ==> Set("1 minute") set.map(f.andThen(g)) // ==> Set("1 minute", "60 seconds", "60000 milliseconds") ``` Another good real-world example to be aware of is [CIString](https://typelevel.org/case-insensitive). That said, there's a `Monad[Set]` instance in the [Alleycats](https://typelevel.org/cats/alleycats.html#set-_-instances) module, which you can use on your own risk if you are sure about data types you are working with. --- ### Guidelines (`docs/guidelines.md`) # Guidelines All guidelines in Сats should have clear justifications. There is no room for tribal wisdom in a simple library. ## Syntax ### Composing Implicit Conversions in Traits Implicit syntax conversions provided in publicly-exposed traits should be marked final so that any composition of the traits provides conversions that can all be inlined. ### Ops Classes Ops classes should be marked final and extend AnyVal, to take full advantage of inlining and prevent unnecessary allocations. The most notable exception is the case where all of the ops in the class are provided by zero-cost macros anyway, for example with Simulacrum. ### Partially-Applied Type In Scala, when there are multiple type parameters in a function, either scalac infers all type parameters or the user has to specify all of them. Often we have functions where there are one or more types that are inferable but not all of them. For example, there is helper function in `OptionT` that creates an `OptionT[F, A]` from an `A`. It could be written as: ```scala mdoc:silent import cats._ import cats.syntax.all._ import cats.data.OptionT ``` ```scala mdoc def pure[F[_], A](a: A)(implicit F: Applicative[F]): OptionT[F, A] = OptionT(F.pure(Some(a))) pure[List, Int](1) ``` Note that the type `A` should've been given by the `a: A` argument, but since scalac cannot infer `F[_]`, the user still has to specify all type params. In Сats, we use a technique described in Rob Norris’s [Kinda-Curried Type Parameters](https://tpolecat.github.io/2015/07/30/infer.html) to overcome this restriction of scala inference. Here is a version of the `pure` using this technique in Сats. ```scala package cats.data object OptionT { private[data] final class PurePartiallyApplied[F[_]](val dummy: Boolean = true ) extends AnyVal { def apply[A](value: A)(implicit F: Applicative[F]): OptionT[F, A] = OptionT(F.pure(Some(value))) } def pure[F[_]]: PurePartiallyApplied[F] = new PurePartiallyApplied[F] } ``` We introduced an intermediate or, as the name suggested, type parameter partially applied type `PurePartiallyApplied` to divide the function into two steps: the first step is a construction of the partially applied type, for which the type `F[_]` is given by the user; the second step is the `apply` method inside partially applied type, for which the `A` can be inferred from the argument. Now we can write: ```scala mdoc OptionT.pure[List](1) ``` The user doesn't need to specify the type `A` which is given by the parameter. You probably noticed that there is a `val dummy: Boolean` in the `PurePartiallyApplied` class. This is a trick we used to make this intermediate class a [Value Class](http://docs.scala-lang.org/overviews/core/value-classes.html) so that there is no cost of allocation, i.e. at runtime, it doesn't create an instance of `PurePartiallyApplied`. We also hide this partially applied class by making it package private and placing it inside an object. ### Implicit naming In a widely-used library it's important to minimize the chance that the names of implicits will be used by others and therefore name our implicits according to the following rules: - Implicits should start with "cats" followed by the package name (where the instance is defined). - If the package contains `instances` leave `instances` out. - The type and the type class should be mentioned in the name. - If the instance is for multiple type classes, use `InstancesFor` instead of a type class name. - If the instance is for a standard library type add `Std` after the package. i.e. `catsStdShowForVector` and `catsKernelStdGroupForTuple`. As an example, an implicit instance of `Monoid` for `List` defined in the package `Kernel` should be named `catsKernelStdMonoidForList`. This rule is relatively flexible. Use what you see appropriate. The goal is to maintain uniqueness and avoid conflicts. ### Implicit instance priority When there are multiple instances provided implicitly, if the type class of them are in the same inheritance hierarchy, the instances need to be separated out into different abstract class/traits so that they don't conflict with each other. The names of these abstract classes/traits should be numbered with a priority with 0 being the highest priority. The abstract classes/trait with higher priority inherits from the ones with lower priority. The most specific (whose type class is the lowest in the hierarchy) instance should be placed in the abstract class/ trait with the highest priority. Here is an example. ```scala trait Functor[F[_]] trait Monad[F[_]] extends Functor ... object Kleisli extends KleisliInstance0 abstract class KleisliInstance0 extends KleisliInstance1 { implicit def catsDataMonadForKleisli[F[_], A]: Monad[Kleisli[F, A, *]] = ... } abstract class KleisliInstance1 { implicit def catsDataFunctorForKleisli[F[_], A]: Functor[Kleisli[F, A, *]] = ... } ``` ### Type classes that ONLY define laws. We can introduce new type classes for the sake of adding laws that don't apply to the parent type class, e.g. `CommutativeSemigroup` and `CommutativeArrow`. ### Applicative instances for monad transformers We explicitly don't provide an instance of `Applicative` for e.g. `EitherT[F, String, *]` given an `Applicative[F]`. An attempt to construct one without a proper `Monad[F]` instance would be inconsistent in `ap` with the provided `Monad` instance for `EitherT[F, String, *]`. Such an instance will be derived if you use `Nested` instead: ```scala import cats._, cats.data._, cats.implicits._ val a = EitherT(List(Left("err"), Right(1))) val x = (a *> a).value > x: List[Either[String, Int]] = List(Left("err"), Left("err"), Right(1)) val y = (a.toNested *> a.toNested).value > y: List[Either[String, Int]] = List(Left("err"), Left("err"), Left("err"), Right(1)) x === y > false ``` ### Classes extending AnyVal AnyVal-extending class constructor parameters must be marked as private. --- ### Imports (`docs/imports.md`) # Imports The easiest approach to Сats imports is to import everything that's commonly needed: ```scala mdoc:reset:silent import cats._ import cats.data._ import cats.syntax.all._ ``` The `cats._` import brings in quite a few [type classes](typeclasses.md) (similar to interfaces) such as [Monad](typeclasses/monad.md), [Semigroup](typeclasses/semigroup.md), and [Foldable](typeclasses/foldable.md). Instead of the entire `cats` package, you can import only the types that you need, for example: ```scala mdoc:reset:silent import cats.Monad import cats.Semigroup import cats.Foldable ``` The `cats.data._`, import brings in data structures such as [Validated](datatypes/validated.md) and [State](datatypes/state.md). Instead of the entire `cats.data` package, you can import only the types that you need, for example: ```scala mdoc:reset:silent import cats.data.Validated import cats.data.State ``` The `cats.syntax.all._` import adds syntax enrichment onto certain types to provide some handy methods such as right-biased `Either` combinators: ```scala mdoc:reset import cats.syntax.all._ // Сats adds right-biased combinators to the standard library's Either val e: Either[String, Int] = Right(3) e.map(_ + 1) // cats adds an orEmpty method to the standard library's Option val o: Option[String] = None o.orEmpty ``` If you'd like to import à la carte, you can do so, by importing from `cats.syntax` for syntax enrichment. For example, if you'd like to import the `Semigroup` syntax: ```scala mdoc:reset import cats.syntax.semigroup._ "Hello, " |+| "World!" ``` The import adds the `|+|` syntax. **Note**: Beware that if you import a type class instance or its syntax twice, you will receive conflicting implicits with a less than helpful error message. This usually happens when importing different type classes in the same hierarchy or when importing syntax enrichment for all type classes using `cats.syntax.all._` or `cats.implicits._` together with a more specific import like `cats.syntax.option._` or `cats.instances.either._`. Below is an example of this phenomenon: ```scala mdoc:reset:silent:fail import cats.syntax.all._ 1 |+| 2 import cats.syntax.semigroup._ 3 |+| 5 // error: value |+| is not a member of Int ``` Compilation fails on the second invocation of `|+|` because we now have conflicting implicits from `Semigroup`. --- ### Jump Start Guide (`docs/jump_start_guide.md`) {% laika.title = Jump Start Guide %} # Introduction This jump start guide barely scratches the surface of what Cats can do, but instead provides a concise hands-on introduction to the patterns you're most likely to take advantage of in your Scala project. If you're using the constructs like [`Future`](http://www.scala-lang.org/api/current/scala/concurrent/Future.html), [`Option`](http://www.scala-lang.org/api/current/scala/Option.html) or [`Either`](http://www.scala-lang.org/api/current/scala/util/Either.html) on a daily basis, it's very likely that Cats can simplify and improve the readability of your code. Please refer to the [wiki on GitHub](https://github.com/typelevel/cats) for guidelines on how to add the library to your project dependencies. We are sticking to [version 2.13.0](https://github.com/typelevel/cats/releases/tag/v2.13.0) in the entire post. Let's go through the library package-wise, looking at the syntax available in each package. # Option Helpers ```scala mdoc:silent import cats.syntax.all._ ``` Importing this package enables `obj.some` syntax — equivalent to `Some(obj)`. The only real difference is that the value is already upcast to `Option[T]` from `Some[T]`. Using `obj.some` instead of `Some(obj)` can sometimes e.g. improve readability when you need to provide dummy implementation for service methods for the purpose of testing. For example, if you put the following implicit class into the scope: ```scala mdoc:silent import scala.concurrent.Future implicit class ToFutureSuccessful[T](obj: T) { def asFuture: Future[T] = Future.successful(obj) } ``` then you can use the chained syntax shown below: ```scala mdoc:silent class Account { /* ... */ } trait AccountService { def getAccountById(id: Int): Future[Option[Account]] } class DummyAccountServiceImpl extends AccountService { def dummyAccount: Account = ??? /* ... */ override def getAccountById(id: Int): Future[Option[Account]] = dummyAccount.some.asFuture /* ... */ } ``` That's more readable than `Future.successful(Some(dummyAccount))`, especially if this pattern repeats frequently. Chaining `.some.asFuture` at the end rather than putting it at the front also helps focus on what's actually being returned rather than on the expected wrapper type. `none[T]`, in turn, is shorthand for `Option.empty[T]` which is just `None`, but already upcast from `None.type` to `Option[T]`. Providing a more specialized type sometimes helps the Scala compiler properly infer the type of expressions containing `None`. # Either Helpers ```scala mdoc:silent import cats.syntax.all._ ``` `obj.asRight` is `Right(obj)`, `obj.asLeft` is `Left(obj)`. In both cases the type of returned value is widened from `Right` or `Left` to `Either`. Just as was the case with `.some`, these helpers are handy to combine with `.asFuture` to improve readability: ```scala mdoc:silent case class User(accountId: Long) { /* ... */ } trait UserService { def ensureUserExists(id: Int): Future[Either[Exception, User]] } class UserServiceSpec extends UserService { def dummyUser: User = ??? /* ... */ override def ensureUserExists(id: Int): Future[Either[Exception, User]] = dummyUser.asRight.asFuture // instead of Future.successful(Right(dummyUser)) /* ... */ } ``` `Either.fromOption(option: Option[A], ifNone: => E)`, in turn, is a useful helper for converting an `Option` to an `Either`. If the provided `option` is `Some(x)`, it becomes `Right(x)`. Otherwise it becomes `Left` with the provided `ifNone` value inside. # Tuples The `apply` package provides `(..., ..., ...).mapN` syntax, which allows for an intuitive construct for applying a function that takes more than one parameter to multiple effectful values (like futures). Let's say we have 3 futures, one of type `Int`, one of type `String`, one of type `User` and a method accepting three parameters — `Int`, `String` and `User`. ```scala mdoc:silent import scala.concurrent.ExecutionContext.Implicits.global class ProcessingResult { /* ... */ } def intFuture: Future[Int] = { /* ... */ ??? } def stringFuture: Future[String] = { /* ... */ ??? } def userFuture: Future[User] = { /* ... */ ??? } def process(value: Int, contents: String, user: User): ProcessingResult = { /* ... */ ??? } ``` Our goal is to apply the function to the values computed by those 3 futures. With `apply` syntax this becomes very easy and concise: ```scala mdoc:silent import cats.syntax.all._ def processAsync: Future[ProcessingResult] = { (intFuture, stringFuture, userFuture).mapN { (value, contents, user) => process(value, contents, user) } } ``` By default the implicit instances (namely, [`Functor[Future]`](@API_URL@/cats/Functor.html) and [`Semigroupal[Future]`](@API_URL@/cats/Semigroupal.html)) required for `mapN` to work properly are always visible. They are present in the respective companion objects of the instances and hence we do not need to import them explicitly. This above idea can be expressed even shorter, just: ```scala mdoc:silent def processAsync2: Future[ProcessingResult] = (intFuture, stringFuture, userFuture).mapN(process) ``` If any of the chained futures fails, the resulting future will also fail with the same exception as the first failing future in the chain (this is fail-fast behavior). What's important, all futures will run in parallel, as opposed to what would happen in a `for` comprehension: ```scala mdoc:silent def processAsync3: Future[ProcessingResult] = { for { value <- intFuture contents <- stringFuture user <- userFuture } yield process(value, contents, user) } ``` In the above snippet (which under the hood translates to `flatMap` and `map` calls), `stringFuture` will not run until `intFuture` is successfully completed, and in the same way `userFuture` will be run only after `stringFuture` completes. But since the computations are independent of one another, it's perfectly viable to run them in parallel with `mapN` instead. # Traversing ```scala mdoc:silent import cats.syntax.all._ ``` ## `traverse` If you have an instance `obj` of type `F[A]` that can be mapped over (like `Future`) and a function `fun` of type `A => G[B]`, then calling `obj.map(fun)` would give you `F[G[B]]`. In many common real-life cases, like when `F` is `Option` and `G` is `Future`, you would get `Option[Future[B]]`, which most likely isn't what you wanted. `traverse` comes as a solution here. If you call `traverse` instead of `map`, like `obj.traverse(fun)`, you'll get `G[F[A]]`, which will be `Future[Option[B]]` in our case; this is much more useful and easier to process than `Option[Future[B]]`. ```scala mdoc:silent def updateUser(user: User): Future[User] = { /* ... */ ??? } def updateUsers(users: List[User]): Future[List[User]] = { users.traverse(updateUser) } ``` As a side note, there is also a dedicated method `Future.traverse` in the [`Future`](http://www.scala-lang.org/api/current/scala/concurrent/Future$.html) companion object, but the Cats version is far more readable and can easily work on any structure for which certain type classes are available. ## `sequence` `sequence` represents an even simpler concept: it can be thought of as simply swapping the types from `F[G[A]]` to `G[F[A]]` without even mapping the enclosed value like `traverse` does. ```scala mdoc:silent val foo: List[Future[String]] = List(Future("hello"), Future("world")) val bar = foo.sequence // will have Future[List[String]] type ``` `obj.sequence` is in fact implemented in Cats as `obj.traverse(identity)`. On the other hand, `obj.traverse(fun)` is roughly equivalent to `obj.map(fun).sequence`. ## `flatTraverse` If you have an `obj` of type `F[A]` and a function `fun` of type `A => G[F[B]]`, then doing `obj.map(f)` yields result of type `F[G[F[B]]]` — very unlikely to be what you wanted. Traversing the `obj` instead of mapping helps a little — you'll get `G[F[F[B]]` instead. Since `G` is usually something like `Future` and `F` is `List` or `Option`, you would end up with `Future[Option[Option[A]]` or `Future[List[List[A]]]` — a bit awkward to process. ```scala mdoc:silent lazy val valueOpt: Option[Int] = { /* ... */ ??? } def compute(value: Int): Future[Option[Int]] = { /* ... */ ??? } def computeOverValue: Future[Option[Option[Int]]] = valueOpt.traverse(compute) // has type Future[Option[Option[Int]]], not good ``` The solution could be to map the result with a ```_.flatten``` call like: ```scala mdoc:silent def computeOverValue2: Future[Option[Int]] = valueOpt.traverse(compute).map(_.flatten) ``` and this way you'll get the desired type `G[F[B]]` at the end. However, there is a neat shortcut for this called `flatTraverse`: ```scala mdoc:silent def computeOverValue3: Future[Option[Int]] = valueOpt.flatTraverse(compute) ``` and that solves our problem for good. # Monad Transformers ## OptionT ```scala mdoc:silent import cats.data.OptionT ``` An instance of [`OptionT[F, A]`](datatypes/optiont.md) can be thought of as a wrapper over `F[Option[A]]` which adds a couple of useful methods specific to nested types that aren't available in `F` or `Option` itself. Most typically, your `F` will be `Future` (or sometimes slick's `DBIO`, but this requires having an implementation of Cats type classes like `Functor` or `Monad` for `DBIO`). Wrappers such as `OptionT` are generally known as _monad transformers_. A quite common pattern is mapping the inner value stored inside an instance of `F[Option[A]]` to an instance of `F[Option[B]]` with a function of type `A => B`. This can be done with rather verbose syntax like: ```scala mdoc:silent lazy val resultFuture: Future[Option[Int]] = ??? def mappedResultFuture: Future[Option[String]] = resultFuture.map { maybeValue => maybeValue.map { value => // Do something with the value and return String ??? } } ``` With the use of `OptionT`, this can be simplified as follows: ```scala mdoc:silent def mappedResultFuture2: OptionT[Future, String] = OptionT(resultFuture).map { value => // Do something with the value and return String ??? } ``` The above `map` will return a value of type `OptionT[Future, String]`. To get the underlying `Future[Option[String]]` value, simply call `.value` on the `OptionT` instance. It's also a viable solution to fully switch to `OptionT[Future, A]` in method parameter/return types and completely (or almost completely) ditch `Future[Option[A]]` in type declarations. There are several ways to construct an `OptionT` instance. The method headers in the table below are slightly simplified: the type parameters and type classes required by each method are skipped. | Method | Takes | Returns | | :---: | :---: | :---: | | `OptionT.apply` or `OptionT(...)` | `F[Option[A]]` | `OptionT[F, A]` | | `OptionT.fromOption` | `Option[A]` | `OptionT[F, A]` | | `OptionT.liftF` | `F[A]` | `OptionT[F, A]` | | `OptionT.pure` | `A` | `OptionT[F, A]` | In production code you'll most commonly use the `OptionT(...)` syntax in order to wrap an instance of `Future[Option[A]]` into `OptionT[F, A]`. The other methods, in turn, prove useful to set up `OptionT`-typed dummy values in unit tests. We have already come across one of `OptionT`'s methods, namely `map`. There are several other methods available and they mostly differ by the signature of the function they accept as the parameter. As was the case with the previous table, the expected type classes are skipped. | Method | Takes | Returns | :---: | :---: | :---: | | `map[B]` | `A => B` | `OptionT[F, B]` | | `subflatMap[B]` | `A => Option[B]` | `OptionT[F, B]` | | `semiflatMap[B]` | `A => F[B]` | `OptionT[F, B]` | | `flatMapF[B]` | `A => F[Option[B]]` | `OptionT[F, B]` | | `flatMap[B]` | `A => OptionT[F, B]` | `OptionT[F, B]` | In practice, you're most likely to use `map` and `semiflatMap`. As is always the case with `flatMap` and `map`, you can use it not only explicitly, but also under the hood in `for` comprehensions, as in the example below: ```scala mdoc:silent class Money { /* ... */ } def findUserById(userId: Long): OptionT[Future, User] = { /* ... */ ??? } def findAccountById(accountId: Long): OptionT[Future, Account] = { /* ... */ ??? } def getReservedFundsForAccount(account: Account): OptionT[Future, Money] = { /* ... */ ??? } def getReservedFundsForUser(userId: Long): OptionT[Future, Money] = for { user <- findUserById(userId) account <- findAccountById(user.accountId) funds <- getReservedFundsForAccount(account) } yield funds ``` The `OptionT[Future, Money]` instance returned by `getReservedFundsForUser` will enclose a `None` value if any of the three composed methods returns an `OptionT` corresponding to `None`. Otherwise, if the result of all three calls contains `Some`, the final outcome will also contain `Some`. ## EitherT ```scala mdoc:silent import cats.data.EitherT ``` [`EitherT[F, A, B]`](datatypes/eithert.md) is the monad transformer for `Either` — you can think of it as a wrapper over a `F[Either[A, B]]` value. Just as in the above section, I simplified the method headers, skipping type parameters or their context bounds and lower bounds. Let's have a quick look at how to create an `EitherT` instance: | Method | Takes | Returns | | :---: | :---: | :---: | | `EitherT.apply` or `EitherT(...)` | `F[Either[A, B]]` | `EitherT[F, A, B]` | | `EitherT.fromEither` | `Either[A, B]` | `EitherT[F, A, B]` (wraps the provided `Either` value into `F`) | | `EitherT.right` or `EitherT.liftF` | `F[B]` | `EitherT[F, A, B]` (wraps value inside `F[B]` into `Right`) | | `EitherT.left` | `F[A]` | `EitherT[F, A, B]` (wraps value inside `F[B]` into `Left`) | | `EitherT.pure` | `A` | `EitherT[F, A, B]` (wraps value into `Right` and then into `F`) | Another useful way to construct an `EitherT` instance is to use `OptionT`'s methods `toLeft` and `toRight`: ```scala mdoc:silent abstract class BaseException(message: String) extends Exception(message) case class UserNotFoundException(message: String) extends BaseException(message) def getUserById(userId: Int): Future[Option[User]] = { /* ... */ ??? } def ensureUserExists(userId: Int): EitherT[Future, BaseException, User] = { OptionT(getUserById(userId)) .toRight(left = UserNotFoundException(s"user not found, userId=$userId")) } ``` `toRight` is pretty analogous to the method `Either.fromOption` mentioned before: just as `fromOption` built an `Either` from an `Option`, `toRight` creates an `EitherT` from an `OptionT`. If the original `OptionT` stores `Some` value, it will be wrapped into `Right`; otherwise the value provided as the `left` parameter will be wrapped into a `Left`. To provide the `left` value within the monad, there is corresponding `toRightF` method. `toLeft` is `toRight`'s counterpart which wraps the `Some` value into `Left` and transforms `None` into `Right` enclosing the provided `right` value. This is less commonly used in practice, but can serve e.g. for enforcing uniqueness checks in code. We return `Left` if the value has been found, and `Right` if it doesn't yet exist in the system. The methods available in `EitherT` are pretty similar to those we've seen in `OptionT`, but there are some notable differences. You might get into some confusion at first when it comes to e.g. `map`. In the case of `OptionT`, it was pretty obvious what should be done: `map` should go over the `Option` enclosed within `Future`, and then map the enclosed `Option` itself. This is slightly less obvious in case of `EitherT`: should it map over both `Left` and `Right` values, or only the `Right` value? The answer is that `EitherT` is _right-biased_, therefore plain `map` actually deals with the `Right` value. This is unlike `Either` in the Scala standard library up to 2.11, which is in turn _unbiased_: there's no `map` available in `Either`, only for its left and right projections. Having said that, let's take a quick look at the right-biased methods that `EitherT` offers: | Method | Takes | Returns | | :---: | --- | --- | | `map[D]` | `B => D` | `EitherT[F, A, D]` | | `subflatMap[D]` | `B => Either[A, D]` | `EitherT[F, A, D]` | | `semiflatMap[D]` | `B => F[D]` | `EitherT[F, A, D]` | | `flatMapF[D]` | `B => F[Either[A, D]]` | `EitherT[F, A, D]` | | `flatMap[D]` | `B => EitherT[F, A, D]` | `EitherT[F, A, D]` | As a side note, there are also certain methods in `EitherT` (that you're likely to need at some point) which map over the `Left` value, like `leftMap`, or over both `Left` and `Right` values, like `fold` or `bimap`. `EitherT` is very useful for fail-fast chained verifications: ```scala mdoc:silent case class Item(state: String) class ItemOrder { /* ... */ } case class ItemNotFoundException(message: String) extends BaseException(message) case class InvalidItemStateException(message: String) extends BaseException(message) def getItemById(itemId: Int): Future[Option[Item]] = { /* .. */ ??? } def ensureItemExists(itemId: Int): EitherT[Future, BaseException, Item] = { OptionT(getItemById(itemId)) .toRight(ItemNotFoundException(s"item not found, itemId = $itemId")) } def ensureItemStateIs(actual: String, expected: String): EitherT[Future, BaseException, Unit] = { // Returns a Unit value wrapped into Right and then into Future if condition is true, // otherwise the provided exception wrapped into Left and then into Future. EitherT.cond(actual == expected, (), InvalidItemStateException(s"actual=$actual, expected=$expected")) } def placeOrderForItem(userId: Int, itemId: Int, count: Int): Future[ItemOrder] = { /* ... */ ??? } def buyItem(userId: Int, itemId: Int, count: Int): EitherT[Future, BaseException, ItemOrder] = { for { user <- ensureUserExists(userId) item <- ensureItemExists(itemId) _ <- ensureItemStateIs(item.state, "AVAILABLE_IN_STOCK") // EitherT.liftF is necessary to make EitherT[Future, BaseException, ItemOrder] out of Future[ItemOrder] placedOrder <- EitherT.liftF(placeOrderForItem(userId, itemId, count)) } yield placedOrder } ``` In the above example, we're running various checks against the item one by one. If any of the checks fails, the resulting `EitherT` will contain a `Left` value. Otherwise, if all of the checks yield a `Right` (of course we mean a `Right` wrapped into an `EitherT`), then the final outcome will also contain `Right`. This is a fail-fast behavior: we're effectively stopping the `for` comprehension flow at the first `Left`-ish result. If you're instead looking for validation that accumulates the errors (e.g. when dealing with user-provided form data), `cats.data.Validated` may be a good choice. # Common issues The Cats type class instances for standard library types are available in implicit scope and hence no longer have to be imported. If anything doesn't compile as expected, first make sure all the required Cats syntax implicits are in the scope — try importing ```cats.syntax.all._``` and see if the problem persists. The only exception is here is Cat's own ```Order``` and ```PartialOrder``` type classes which are available by importing ```cats.implicits._```. As mentioned before, though, it's better to use narrow imports, but if the code doesn't compile it's sometimes worth just importing the entire library to check if it solves the problem. If you're using `Futures`, make sure to provide an implicit `ExecutionContext` in the scope, otherwise Cats won't be able to infer implicit instances for `Future`'s type classes. IntelliJ sometimes reports errors in Cats-loaded code even though the source passes under scalac. One such example are invocations of the methods of `cats.data.Nested` class, which compile correctly under scalac, but don't type check under IntelliJ's presentation compiler. It should work without trouble under [Scala IDE](http://scala-ide.org/), though. As an advice for your future learning: the `Applicative` type class, despite it's key significance in functional programming, comes slightly difficult to understand. In my opinion it's much less intuitive than `Functor` or `Monad`, even though it actually stands right between `Functor` and `Monad` in the inheritance hierarchy. The best approach to grasp `Applicative` is to first understand how `product` (which transforms an `F[A]` and `F[B]` to an `F[(A, B)]`) works rather than focus on the somewhat exotic `ap` operation itself. --- ### Motivations (`docs/motivations.md`) {% laika.title = Motivations %} #### Approachability As this library evolves, we are placing a lot of emphasis on trying to make decisions which will help keep this library approachable for people new to the concepts in this library. We are collecting anecdotes from successes and failures we have experienced in the past in trying to teach others about these concepts, and trying to make decisions which will help ease the process of getting acquainted to the library for a newcomer. If you have any feedback for us in this regard, we would love to hear from you. See the [Contributing page](CONTRIBUTING.md) to find out ways to give us feedback. #### Modularity We are trying to make the library modular. It will have a tight core which will contain only the [type classes](typeclasses.md), the bare minimum of data structures that are needed to support them, and type class instances for those data structures and standard library types. #### Documentation We feel that having lots of documentation is a very important goal for our project. It will be a big win towards our goal of approachability. We will strive to have the code well documented, we will strive to have lots of documentation external to the code, and we will strive to have a large corpus of compiler verified examples of how the software can be used. Writing documentation is a huge part of developing software, and one that is often neglected. It is also a very easy way to get started with [contributing](CONTRIBUTING.md) to the project. #### Efficiency Although unfortunately there are times when programming only with pure functions and writing efficient code in Scala can be at odds, we are attempting to do our best at keeping our library as efficient as we can without making unnecessary sacrifices of purity and usability. Where sacrifices have to be made, we will strive to make these obvious, and will keep them well documented. --- ### Nomenclature (`docs/nomenclature.md`) # Glossary This is a catalogue of the major functions, type classes, and data types in `Cats`. It serves as a bird's-eye view of each class capabilities. It is also intended as a go-to reference for `Cats` users, who may not recall the answer to questions like these: - What is the difference between `unit` and `void`? - To discard the first value and keep only the first effect, is it `<*` or `*>`? - How do I make a computation `F[A]` fail by checking a condition on the value? The signatures and type-classes have been simplified, are described [below](#simplifications). If you want a printable version, you can also check out this [cats-cheatsheet](https://arosien.github.io/cats-cheatsheets/typeclasses.pdf). _WARNING_: this page is written manually, and not automatically generated, so many things may be missing. If you find a mistake, or addition, please submit a PR following the guidelines below. ## Type-Classes over an `F[_]` ### Functor | Type | Method Name | Notes | |--------------------------------|----------------|-------| | `F[A] => F[Unit]` | `void` | | `F[A] => B => F[B]` | `as` | | `F[A] => (A => B) => F[B]` | `map` | | `F[A] => (A => A1) => F[A1])` | `mapOrKeep` | A1 >: A, the (A => A1) is a PartialFunction | `F[A] => (A => B) => F[(A,B)]` | `fproduct` | | `F[A] => (A => B) => F[(B,A)]` | `fproductLeft` | | `F[A] => B => F[(B, A)]` | `tupleLeft` | | `F[A] => B => F[(A, B)]` | `tupleRight` | | `(A => B) => (F[A] => F[B])` | `lift` | ### Apply | Type | Method Name | Symbol | | ------------- |--------------|------------| | `F[A] => F[B] => F[A]` | `productL` | `<*` | `F[A] => F[B] => F[B]` | `productR` | `*>` | `F[A] => F[B] => F[(A,B)]` | `product` | | `F[A => B] => F[A] => F[B]` | `ap` | `<*>` | `F[A => B => C] => F[A] => F[B] => F[C]` | `ap2` | | `F[A] => F[B] => (A => B => C) => F[C]` | `map2` | ### Applicative | Type | Method Name | Notes | | ------------- |--------------|------------| | `A => F[A]` | `pure` | | `=> F[Unit]` | `unit` | | `Boolean => F[Unit] => F[Unit]` | `whenA` | Performs effect iff condition is true | | `unlessA` | Adds effect iff condition is false ### FlatMap | Type | Method Name | | ------------- |---------------| | `F[F[A]] => F[A]` | `flatten` | | `F[A] => (A => F[B]) => F[B]` | `flatMap` | `F[A] => (A => F[B]) => F[(A,B)]` | `mproduct` | `F[Boolean] => F[A] => F[A] => F[A]` | `ifM` | `F[A] => (A => F[B]) => F[A]` | `flatTap` ### FunctorFilter | Type | Method Name | Notes | |-------------|---------------|--------| | `F[A] => (A => Boolean) => F[A]` | `filter` | | `F[A] => (A => Option[B]) => F[B]` | `mapFilter` | | `F[A] => (A => B) => F[B]` | `collect` | The `A => B` is a PartialFunction | `F[Option[A]] => F[A]` | `flattenOption` | ### ApplicativeError The source code of `Cats` uses the `E` type variable for the error type. | Type | Method Name | Notes | |--------------|--------------|-------| | `E => F[A]` | `raiseError` | | `F[A] => F[Either[E,A]]` | `attempt` | | `F[A] => (E => A) => F[A]` | `handleError` | | `F[A] => (E => F[A]) => F[A]` | `handleErrorWith` | | `F[A] => (E => A) => F[A]` | `recover` | The `E => A` is a PartialFunction. | `F[A] => (E => F[A]) => F[A]` | `recoverWith` | The `E => F[A]` is a PartialFunction. | `F[A] => (E => F[Unit]) => F[A]` | `onError` | The `E => F[Unit]` is a PartialFunction. | `Either[E,A] => F[A]` | `fromEither` | | `Option[A] => E => F[A]` | `liftFromOption` | ### MonadError Like the previous section, we use the `E` for the error parameter type. | Type | Method Name | Notes | | ------------- |--------------|--------| | `F[A] => E => (A => Boolean) => F[A]` | `ensure` | `F[A] => (A => E) => (A => Boolean) => F[A]` | `ensureOr` | `F[A] => (E => E) => F[A]` | `adaptError` | The `E => E` is a PartialFunction. | `F[Either[E,A]] => F[A]` | `rethrow` ### UnorderedFoldable | Type | Method Name | Constraints | ------------- |--------------|---------------- | `F[A] => Boolean` | `isEmpty` | | `F[A] => Boolean` | `nonEmpty` | | `F[A] => Long` | `size` | | `F[A] => (A => Boolean) => Boolean`| `forall` | | `F[A] => (A => Boolean) => Boolean`| `exists` | | `F[A] => A` | `unorderedFold` | `A: CommutativeMonoid` | `F[A] => (A => B) => B`| `unorderedFoldMap` | `B: CommutativeMonoid` ### Foldable | Type | Method Name | Constraints | ------------- |--------------|----------- | `F[A] => A` | `fold` | `A: Monoid` | `F[A] => B => ((B,A) => B) => B` | `foldLeft` | `F[A] => (A => B) => B` | `foldMap` | `B: Monoid` | `F[A] => (A => G[B]) => G[B]` | `foldMapM` | `G: Monad` and `B: Monoid` | `F[A] => (A => B) => Option[B]` | `collectFirst` | The `A => B` is a `PartialFunction` | `F[A] => (A => Option[B]) => Option[B]` | `collectFirstSome` | | `F[A] => (A => G[B]) => G[Unit]` | `traverse_` | `G: Applicative` | `F[G[A]] => G[Unit]` | `sequence_` | `G: Applicative` | `F[A] => (A => Either[B, C]) => (F[B], F[C])` | `partitionEither` | `G: Applicative` ### Reducible | Type | Method Name | Constraints | ------------- |--------------|----------- | `F[A] => ((A,A) => A) => A` | `reduceLeft` | | `F[A] => A` | `reduce` | `A: Semigroup` | ### Traverse | Type | Method Name | Constraints | |------------|--------------|-----------| | `F[G[A]] => G[F[A]]` | `sequence` | `G: Applicative` | | `F[A] => (A => G[B]) => G[F[B]]` | `traverse` | `G: Applicative` | | `F[A] => (A => G[F[B]]) => G[F[B]]` | `flatTraverse` | `F: FlatMap` and `G: Applicative` | `F[G[F[A]]] => G[F[A]]` | `flatSequence` | `G: Applicative` and `F: FlatMap` | `F[A] => F[(A,Int)]` | `zipWithIndex` | | `F[A] => ((A,Int) => B) => F[B]` | `mapWithIndex` | | `F[A] => ((A,Int) => G[B]) => G[F[B]]` | `traverseWithIndexM` | `F: Monad` ### SemigroupK | Type | Method Name | Constraints | |------------|--------------|-----------| | `F[A] => F[A] => F[A]`| `combineK` | | `F[A] => Int => F[A]` | `combineNK` | `F[A] => F[B] => F[Either[A, B]]` | `sum` | `F: Functor` | `IterableOnce[F[A]] => Option[F[A]]` | `combineAllOptionK` ### MonoidK | Type | Method Name | Constraints | |------------|--------------|-----------| | `F[A]` | `empty` | `F[A] => Boolean` | `isEmpty` | `IterableOnce[F[A]] => F[A]` | `combineAllK` ### Alternative | Type | Method Name | Constraints | |------------|--------------|-----------| | `F[G[A]] => F[A]` | `unite` | `F: FlatMap` and `G: Foldable` | `F[G[A, B]] => (F[A], F[B])` | `separate` | `F: FlatMap` and `G: Bifoldable` | `F[G[A, B]] => (F[A], F[B])` | `separateFoldable` | `F: Foldable` and `G: Bifoldable` | `Boolean => F[Unit]` | `guard` | `IterableOnce[A] => F[A]` | `fromIterableOnce` | `G[A] => F[A]` | `fromFoldable` | `G: Foldable` ### NonEmptyAlternative | Type | Method Name | Constraints | |------------|--------------|-----------| | `A => F[A] => F[A]` | `prependK` | `F[A] => A => F[A]` | `appendK` ## Transformers ### Constructors and wrappers | Data Type | is an alias or wrapper of | |------------|--------------| | `OptionT[F[_], A]` | `F[Option[A]]` | `EitherT[F[_], A, B]` | `F[Either[A,B]` | `Kleisli[F[_], A, B]` | `A => F[B]` | `Reader[A, B]` | `A => B` | `ReaderT[F[_], A, B]` | `Kleisli[F, A, B]` | `Writer[A, B]` | `(A,B)` | `WriterT[F[_], A, B]` | `F[(A,B)]` | `Tuple2K[F[_], G[_], A]` | `(F[A], G[A])` | `EitherK[F[_], G[_], A]` | `Either[F[A], G[A]]` | `FunctionK[F[_], G[_]]` | `F[X] => G[X]` for every `X` | `F ~> G` | Alias of `FunctionK[F, G]` ### OptionT For convenience, in these types we use the symbol `OT` to abbreviate `OptionT`. | Type | Method Name | Constraints | |----------|--------------|-------------| | `=> OT[F, A]` | `none` | `F: Applicative` | | `A => OT[F, A]` | `some` or `pure` | `F: Applicative` | `F[A] => OT[F, A]` | `liftF` | `F: Functor` | `Boolean => F[A] => OT[F, A]` | `whenF` | `F: Applicative` | `F[Boolean] => F[A] => OT[F, A]` | `whenM` | `F: Monad` | `OT[F, A] => F[Option[A]]` | `value` | `OT[F, A] => A => Boolean => OT[F, A]` | `filter` | `F: Functor` | `OT[F, A] => A => F[Boolean] => OT[F, A]` | `filterF` | `F: Monad` | `OT[F, A] => (A => B) => OT[F, B]` | `map` | `F: Functor` | `OT[F, A] => (F ~> G) => OT[G, B]` | `mapK` | `OT[F, A] => (A => Option[B]) => OT[F, B]` | `mapFilter` | `F: Functor` | `OT[F, A] => B => (A => B) => F[B]` | `fold` or `cata` | `OT[F, A] => (A => OT[F, B]) => OT[F,B]` | `flatMap` | `OT[F, A] => (A => F[Option[B]]) => OT[F,B]` | `flatMapF` | `F: Monad` | | `OT[F, A] => A => F[A]` | `getOrElse` | `F: Functor` | | `OT[F, A] => F[A] => F[A]` | `getOrElseF` | `F: Monad` | | `OT[F, A] => OT[F, A] => OT[F, A]` | ### EitherT Here, we use `ET` to abbreviate `EitherT`; and we use `A` and `B` as type variables for the left and right sides of the `Either`. | Type | Method Name | Constraints | |----------|--------------|-------------| | `A => ET[F, A, B]` | `leftT` | `F: Applicative` | | `B => ET[F, A, B]` | `rightT` | `F: Applicative` | | | `pure` | `F: Applicative` | | `F[A] => ET[F, A, B]` | `left` | `F: Applicative` | | `F[B] => ET[F, A, B]` | `right` | `F: Applicative` | | | `liftF` | `F: Applicative` | | `Either[A, B] => ET[F, A, B]` | `fromEither` | `F: Applicative` | | `Option[B] => A => ET[F, A, B]` | `fromOption` | `F: Applicative` | | `F[Option[B]] => A => ET[F, A, B]` | `fromOptionF` | `F: Functor` | | `F[Option[B]] => F[A] => ET[F, A, B]` | `fromOptionM` | `F: Monad` | | `Boolean => B => A => ET[F, A, B]` | `cond` | `F: Applicative` | | `ET[F, A, B] => (A => C) => (B => C) => F[C]` | `fold` | `F: Functor` | | `ET[F, A, B] => ET[F, B, A]` | `swap` | `F: Functor` | | `ET[F, A, A] => F[A]` | `merge` | ### Kleisli (or ReaderT) Here, we use `Ki` as a short-hand for `Kleisli`. | Type | Method Name | Constraints | |----------|--------------|-------------| | `Ki[F, A, B] => (A => F[B])` | `run` | | `Ki[F, A, B] => A => F[B]` | `apply` | | `A => Ki[F, A, A]` | `ask` | `F: Applicative` | `B => Ki[F, A, B]` | `pure` | `F: Applicative` | `F[B] => Ki[F, A, B]` | `liftF` | | `Ki[F, A, B] => (C => A) => Ki[F, C, B]` | `local` | | `Ki[F, A, B] => Ki[F, A, A]` | `tap` | | `Ki[F, A, B] => (B => C) => Ki[F, A, C]` | `map` | | `Ki[F, A, B] => (F ~> G) => Ki[G, A, B]` | `mapK` | | `Ki[F, A, B] => (F[B] => G[C]) => Ki[F, A, C]` | `mapF` | | `Ki[F, A, B] => Ki[F, A, F[B]]` | `lower` | ## Type Classes for types `F[_, _]` ### Bifunctor | Type | Method Name | | ------------- |--------------| | `F[A,B] => (A => C) => F[C,B]` | `leftMap` | | `F[A,B] => (B => D) => F[A,D]` | `.rightFunctor` and `.map` | | `F[A,B] => (A => C) => (B => D) => F[C,D]` | `bimap` | #### Profunctor | Type | Method Name | --------|------------- | `F[A, B] => (B => C) => F[A, C]` | `rmap` | | `F[A, B] => (C => A) => F[C, B]` | `lmap` | | `F[A, B] => (C => A) => (B => D) => F[C,D]` | `dimap` | #### Strong Profunctor | Type | Method Name | --------|-------------| | `F[A, B] => F[(A,C), (B,C)]` | `first` | | `F[A, B] => F[(C,A), (C,B)]` | `second` | #### Compose, Category, Choice | Type | Method Name | Symbol | --------|-------------|--------------| | `F[A, B] => F[C, A] => F[C, B]` | `compose` | `<<<` | | `F[A, B] => F[B, C] => F[A, C]` | `andThen` | `>>>` | | `=> F[A,A]` | `id` | | `F[A, B] => F[C, B] => F[Either[A, C], B]` | `choice` | `|||` | | `=> F[ Either[A, A], A]` | `codiagonal` | #### Arrow | Type | Method Name | Symbol | |----------------|--------------|--------------| | `(A => B) => F[A, B]` | `lift` | | `F[A,B] => F[C,D] => F[(A,C), (B,D)]` | `split` | `***` | | `F[A,B] => F[A,C] => F[A, (B,C)]` | `merge` | `&&&` | #### ArrowChoice | Type | Method Name | Symbol | --------|-------------|--------------| | `F[A,B] => F[C,D] => F[Either[A, C], Either[B, D]]` | `choose` | `+++` | `F[A,B] => F[Either[A, C], Either[B, C]]` | `left` | | `F[A,B] => F[Either[C, A], Either[C, B]]` | `right` | ## Simplifications Because `Сats` is a Scala library and Scala has many knobs and switches, the actual definitions and the implementations of the functions and type-classes in `Сats` can be a bit obfuscated at first. To alleviate this, in this glossary we focus on the plain type signatures of the method, and ignore many of the details from Scala. In particular, in our type signatures: - We use `A,B,C` for type variables of kind `*`, and `F, G, H` for type variables of a higher kind. - We write type signatures in currified form: parameters are taken one at a time, and they are separated with the arrow `=>` operation. In Scala, a method's parameters may be split in several comma-separated lists. - We do not differentiate between methods from the type-class trait (e.g. `trait Functor`), or the companion object, or the syntax companion (`implicit class`). - For functions defined as method of the typeclass trait, we ignore the receiver object. - We ignore implicit parameters that represent type-class constraints; and write them on a side column instead. - We use `A => B` for both `Function1[A, B]` and `PartialFunction[A, B]` parameters, without distinction. We add a side note when one is a `PartialFunction`. - Some functions are defined through the [Partially Applied Type Params](guidelines.md#partially-applied-type) pattern. We ignore this. - We ignore the distinction between by-name and by-value input parameters. We use the notation `=> A`, without parameters, to indicate constant functions. - We ignore Scala variance annotations. We also ignore extra type parameters, which in some methods are added with a subtype-constraint, (e.g. `B >: A`). These are usually meant for flexibility, but we replace each one by its bound. --- ### Resources For Learners (`docs/resources_for_learners.md`) {% laika.title = Resources for Learners %} # Books * [Functional Programming in Scala](https://www.manning.com/books/functional-programming-in-scala-second-edition) by Michael Pilquist, Rúnar Bjarnason, and Paul Chiusano - While this book does not specifically cover Cats, it is a great introduction to functional programming in Scala in general and covers some of the abstractions that can be found in Cats. * [Scala with Cats](https://underscore.io/books/scala-with-cats/) by Noel Welsh and Dave Gurnell - The aims of this book are two-fold: to introduce monads, functors, and other functional programming patterns as a way to structure program design, and to explain how these concepts are implemented in Cats. # Tutorials * [herding cats](http://eed3si9n.com/herding-cats/) by Eugene Yokota # Example Projects * [Scala Pet Store](https://github.com/pauljamescleary/scala-pet-store) Demonstrates FP techniques while maintaining approachability for people with an OO background. Uses tagless final for the core domain, and popular Typelevel libraries including Cats, Http4s, Circe and Doobie. # Videos * [Functional Structures in Scala](https://www.youtube.com/playlist?list=PLFrwDVdSrYE6dy14XCmUtRAJuhCxuzJp0) by [Michael Pilquist](https://github.com/MPilquist) offers a series of lectures about basic functional structures. * **Category Theory for Programmers** by [Bartosz Milewski](https://github.com/BartoszMilewski) is a series of category theory lectures oriented towards programmers. The code examples are written in Haskell and C++, but the course is designed for programmers of any background, even the ones without a strong maths knowledge. * [part 1](https://www.youtube.com/playlist?list=PLbgaMIhjbmEnaH_LTkxLI7FMa2HsnawM_) * [part 2](https://www.youtube.com/playlist?list=PLbgaMIhjbmElia1eCEZNvsVscFef9m0dm) * [part 3](https://www.youtube.com/playlist?list=PLbgaMIhjbmEn64WVX4B08B4h2rOtueWIL) * [Easy and efficient data validation with Cats](https://www.youtube.com/watch?v=P8nGAo3Jp-Q) by [Daniela Sfregola](https://danielasfregola.com/) this talk introduces Cats, and some of its most interesting components for data validation. In particular, we’ll discuss some options to achieve efficient and expressive data validation. We will also argue that, compared to other options in the language, Cats is particularly suited for the task thanks to its easy-to-use data types and more approachable syntax. * [Cats by example](https://www.youtube.com/playlist?list=PLy4hTMY6erDBmvZiEDjtCTP8XCdhaXc2c) by [Thomas Höfer](https://twitter.com/TomTriple) provides a beginner series on various datatypes from Cats. # Social network * [@impurepics](https://twitter.com/impurepics) regularly tweet informative and kawaii educational pieces about Cats and FP in Scala. --- ### Typeclasses (`docs/typeclasses.md`) # Type Classes Type classes are a powerful tool used in functional programming to enable ad-hoc polymorphism, more commonly known as overloading. Where many object-oriented languages leverage subtyping for polymorphic code, functional programming tends towards a combination of parametric polymorphism (think type parameters, like Java generics) and ad-hoc polymorphism. ## Example: collapsing a list The following code snippets show code that sums a list of integers, concatenates a list of strings, and unions a list of sets. ```scala mdoc:silent def sumInts(list: List[Int]): Int = list.foldRight(0)(_ + _) def concatStrings(list: List[String]): String = list.foldRight("")(_ ++ _) def unionSets[A](list: List[Set[A]]): Set[A] = list.foldRight(Set.empty[A])(_ union _) ``` All of these follow the same pattern: an initial value (0, empty string, empty set) and a combining function (`+`, `++`, `union`). We'd like to abstract over this so we can write the function once instead of once for every type so we pull out the necessary pieces into an interface. ```scala mdoc:silent trait Monoid[A] { def empty: A def combine(x: A, y: A): A } // Implementation for Int val intAdditionMonoid: Monoid[Int] = new Monoid[Int] { def empty: Int = 0 def combine(x: Int, y: Int): Int = x + y } ``` The name `Monoid` is taken from abstract algebra which specifies precisely this kind of structure. We can now write the functions above against this interface. ```scala mdoc:silent def combineAll[A](list: List[A], m: Monoid[A]): A = list.foldRight(m.empty)(m.combine) ``` ## Type classes vs. subtyping The definition above takes an actual monoid argument instead of doing the usual object-oriented practice of using subtype constraints. ```scala mdoc:silent // Subtyping def combineAll[A <: Monoid[A]](list: List[A]): A = ??? ``` This has a subtle difference with the earlier explicit example. In order to seed the `foldRight` with the empty value, we need to get a hold of it given only the type `A`. Taking `Monoid[A]` as an argument gives us this by calling the appropriate `empty` method on it. With the subtype example, the `empty` method would be on a **value** of type `Monoid[A]` itself, which we are only getting from the `list` argument. If `list` is empty, we have no values to work with and therefore can't get the empty value. Not to mention the oddity of getting a constant value from a non-static object. --- For another motivating difference, consider the simple pair type. ```scala mdoc:silent final case class Pair[A, B](first: A, second: B) ``` Defining a `Monoid[Pair[A, B]]` depends on the ability to define a `Monoid[A]` and `Monoid[B]`, where the definition is point-wise, i.e. the first element of the first pair combines with the first element of the second pair and the second element of the first pair combines with the second element of the second pair. With subtyping such a constraint would be encoded as something like ```scala mdoc:silent:nest final case class Pair[A <: Monoid[A], B <: Monoid[B]](first: A, second: B) extends Monoid[Pair[A, B]] { def empty: Pair[A, B] = ??? def combine(x: Pair[A, B], y: Pair[A, B]): Pair[A, B] = ??? } ``` Not only is the type signature of `Pair` now messy but it also forces all instances of `Pair` to have a `Monoid` instance, whereas `Pair` should be able to carry any types it wants and if the types happens to have a `Monoid` instance then so would it. We could try bubbling down the constraint into the methods themselves. ```scala mdoc:fail:nest final case class Pair[A, B](first: A, second: B) extends Monoid[Pair[A, B]] { def empty(implicit eva: A <:< Monoid[A], evb: B <:< Monoid[B]): Pair[A, B] = ??? def combine(x: Pair[A, B], y: Pair[A, B])(implicit eva: A <:< Monoid[A], evb: B <:< Monoid[B]): Pair[A, B] = ??? } ``` But now these don't conform to the interface of `Monoid` due to the implicit constraints. ### Implicit derivation Note that a `Monoid[Pair[A, B]]` is derivable given `Monoid[A]` and `Monoid[B]`: ```scala mdoc:silent:nest final case class Pair[A, B](first: A, second: B) def deriveMonoidPair[A, B](A: Monoid[A], B: Monoid[B]): Monoid[Pair[A, B]] = new Monoid[Pair[A, B]] { def empty: Pair[A, B] = Pair(A.empty, B.empty) def combine(x: Pair[A, B], y: Pair[A, B]): Pair[A, B] = Pair(A.combine(x.first, y.first), B.combine(x.second, y.second)) } ``` One of the most powerful features of type classes is the ability to do this kind of derivation automatically. We can do this through Scala's implicit mechanism. ```scala mdoc:reset-object import cats.Monoid object Demo { final case class Pair[A, B](first: A, second: B) object Pair { implicit def tuple2Instance[A, B](implicit A: Monoid[A], B: Monoid[B]): Monoid[Pair[A, B]] = new Monoid[Pair[A, B]] { def empty: Pair[A, B] = Pair(A.empty, B.empty) def combine(x: Pair[A, B], y: Pair[A, B]): Pair[A, B] = Pair(A.combine(x.first, y.first), B.combine(x.second, y.second)) } } } ``` We also change any functions that have a `Monoid` constraint on the type parameter to take the argument implicitly, and any instances of the type class to be implicit. ```scala mdoc:silent implicit val intAdditionMonoid: Monoid[Int] = new Monoid[Int] { def empty: Int = 0 def combine(x: Int, y: Int): Int = x + y } def combineAll[A](list: List[A])(implicit A: Monoid[A]): A = list.foldRight(A.empty)(A.combine) ``` Now we can also `combineAll` a list of `Pair`s as long as `Pair`'s type parameters themselves have `Monoid` instances. ```scala mdoc:silent implicit val stringMonoid: Monoid[String] = new Monoid[String] { def empty: String = "" def combine(x: String, y: String): String = x ++ y } ``` ```scala mdoc import Demo.{Pair => Paired} combineAll(List(Paired(1, "hello"), Paired(2, " "), Paired(3, "world"))) ``` ## A note on syntax In many cases, including the `combineAll` function above, the implicit arguments can be written with syntactic sugar. ```scala mdoc:compile-only def combineAll[A : Monoid](list: List[A]): A = ??? ``` While nicer to read as a user, it comes at a cost for the implementer. ```scala mdoc:silent:reset import cats.Monoid // Defined in the standard library, shown for illustration purposes // Implicitly looks in implicit scope for a value of type `A` and just hands it back def implicitly[A](implicit ev: A): A = ev def combineAll[A : Monoid](list: List[A]): A = list.foldRight(implicitly[Monoid[A]].empty)(implicitly[Monoid[A]].combine) ``` For this reason, many libraries that provide type classes provide a utility method on the companion object of the type class, usually under the name `apply`, that skirts the need to call `implicitly` everywhere. ```scala mdoc:silent:nest object Monoid { def apply[A : Monoid]: Monoid[A] = implicitly[Monoid[A]] } def combineAll[A : Monoid](list: List[A]): A = list.foldRight(Monoid[A].empty)(Monoid[A].combine) ``` # Laws Conceptually, all type classes come with laws. These laws constrain implementations for a given type and can be exploited and used to reason about generic code. For instance, the `Monoid` type class requires that `combine` be associative and `empty` be an identity element for `combine`. That means the following equalities should hold for any choice of `x`, `y`, and `z`. ``` combine(x, combine(y, z)) = combine(combine(x, y), z) combine(x, id) = combine(id, x) = x ``` With these laws in place, functions parametrized over a `Monoid` can leverage them for say, performance reasons. A function that collapses a `List[A]` into a single `A` can do so with `foldLeft` or `foldRight` since `combine` is assumed to be associative, or it can break apart the list into smaller lists and collapse in parallel, such as ```scala mdoc:silent val list = List(1, 2, 3, 4, 5) val (left, right) = list.splitAt(2) ``` ```scala mdoc // Imagine the following two operations run in parallel val sumLeft = combineAll(left) val sumRight = combineAll(right) // Now gather the results val result = Monoid[Int].combine(sumLeft, sumRight) ``` Cats provides laws for type classes via the `kernel-laws` and `laws` modules which makes law checking type class instances easy. You can find out more about law testing [here](typeclasses/lawtesting.md). ## Type classes in Cats

From [cats-infographic by @tpolecat](https://github.com/tpolecat/cats-infographic). ## Incomplete type class instances in cats Originally from [@hobwekiva](https://gist.github.com/hobwekiva/d63508ddb6a728015ace53cb70a1fd5d) | Type | Functor | Apply | Applicative | Monad | MonoidK | ApplicativeError | MonadError | CoflatMap | Comonad | Bimonad | | --------------- |:-------:|:-----------------:|:-----------:|:-----:|:-------:|:-----------------:|:----------:|:---------:|:-------:|:-------:| | `Id[A]` | ✔ | ✔ | ✔ | ✔ | ✗ | ✗ | ✗ | ✔ | ✔ |✔ | | `Eval[A]` | ✔ | ✔ | ✔ | ✔ | ✗ | ✗ | ✗ | ✔ | ✔ |✔ | | `Option[A]` | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✗ |✗ | | `Const[K, A]` | ✔ | ✔ (`K:Monoid`) | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ |✗ | | `Either[E, A]` | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✗ | ✗ |✗ | | `List[A]` | ✔ | ✔ | ✔ | ✔ | ✔ | ✗ | ✗ | ✔ | ✗ |✗ | | `NonEmptyList[A]` | ✔ | ✔ | ✔ | ✔ | ✗ | ✗ | ✗ | ✔ | ✔ |✔ | | `Stream[A]` | ✔ | ✔ | ✔ | ✔ | ✔ | ✗ | ✗ | ✔ | ✗ |✗ | | `Map[K, A]` | ✔ | ✔ | ✗ | ✗ | ✔ | ✗ | ✗ | ✗ | ✗ |✗ | | `Validated[E, A]` | ✔ | ✔ (`E: Semigroup`)| ✔ | ✗ | ✗ | ✔ (`E: Semigroup`)| ✗ | ✗ | ✗ |✗ | | `Reader[E, A]` | ✔ | ✔ | ✔ | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ |✗ | | `Writer[E, A]` | ✔ | ✔ (`E:Monoid`) | ✔ | ✔ | ✗ | ✗ | ✗ | ✔ | ✗ |✗ | ## Further reading * [Returning the "Current" Type in Scala][fbounds] [fbounds]: http://tpolecat.github.io/2015/04/29/f-bounds.html "Returning the "Current" Type in Scala" --- ### TypelevelEcosystem (`docs/typelevelEcosystem.md`) # Typelevel Ecosystem # By sharing the same set of type classes, instances and data types provided by Cats, projects can speak the same “Cats language”, and integrate with each other with ease. ## Libraries ## Here you can find a comprehensive list of libraries and projects related to Typelevel: | Project | Description | |:--------------------------------------------------------------------------------------------------------------------------:|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:| | [Mules][Mules] | In memory caching. | | [agitation](https://github.com/davenverse/agitation) | Cooperative Cancelation for [Cats Effect][Cats Effect]. | | [ancient-concurrent](https://christopherdavenport.github.io/ancient-concurrent/) | [Cats Effect][Cats Effect] concurrent for cats-effect 0.10. | | [atto](https://tpolecat.github.io/atto/) | Friendly little text parsers. | | [bank](https://github.com/ChristopherDavenport/bank) | Concurrent Vault Management. | | [canoe](https://github.com/augustjune/canoe) | A purely functional library for building [Telegram](https://telegram.org/) chatbots. | | [Cats Effect][Cats Effect] | High-performance, asynchronous, composable framework for building real-world applications in a purely functional style. | | [cats-effect-testing](https://github.com/typelevel/cats-effect-testing) | Experimental integration between [Cats Effect][Cats Effect] and testing frameworks. | | [cats-effect-time](https://github.com/davenverse/cats-effect-time) | Java Time from [Cats Effect][Cats Effect]. | | [cats-parse](https://github.com/typelevel/cats-parse) | A parsing library for the Cats ecosystem. | | [cats-retry](https://cb372.github.io/cats-retry/) | Composable retry logic for [Cats][Cats] and [Cats Effect][Cats Effect]. | | [cats-scalacheck](https://github.com/davenverse/keysemaphore) | [Cats][Cats] typeclass instances for [ScalaCheck](https://www.scalacheck.org/). | | [cats-stm](https://timwspence.github.io/cats-stm/) | Software Transactional Memory for [Cats Effect][Cats Effect]. | | [cats-time](https://typelevel.org/cats-time/) | [Cats][Cats] typeclass instances for [java.time](https://docs.oracle.com/javase/8/docs/api/java/time/package-summary.html). | | [chromaprint](https://github.com/mgdigital/Chromaprint.scala) | A Scala implementation of the Chromaprint/AcoustID audio fingerprinting algorithm, built with [Fs2][Fs2] streams / [Cats Effect][Cats Effect]. | | [circe-fs2](https://github.com/circe/circe-fs2) | Streaming JSON manipulation with [Circe][Circe]. | | [Circe][Circe] | Pure functional JSON library. | | [circuit](https://davenverse.github.io/circuit/) | Functional Circuit Breaker implementation. | | [ciris](https://cir.is/) | Lightweight, extensible, and validated configuration loading in Scala. | | [console4cats](https://console4cats.profunktor.dev/) | Console I/O for [Cats Effect][Cats Effect]. | | [cormorant](https://davenverse.github.io/cormorant/) | CSV handling library for FP. | | [coulomb-cats](https://github.com/erikerlandson/coulomb) | [Cats][Cats] typeclass instances for coulomb Quantity. | | [decline](https://ben.kirw.in/decline/) | A composable command-line parser. | | [doobie-pool](https://christopherdavenport.github.io/doobie-pool/) | Doobie Pooling Mechanism in Pure FP. | | [doobie](https://tpolecat.github.io/doobie/) | Pure functional JDBC built on [Fs2][Fs2]. | | [droste](https://github.com/higherkindness/droste) | Recursion schemes for [Cats][Cats]. | | [dsl-domains-cats](https://github.com/ThoughtWorksInc/dsl-domains-cats) | The !-notation for creating [Cats][Cats] monadic expressions. | | [dynamapath](https://christopherdavenport.github.io/dynamapath/) | Dynamic Path Parsing. | | [eff](https://atnos-org.github.io/eff/) | Functional effects and effect handlers (alternative to monad transformers). | | [epimetheus-circuit](https://github.com/davenverse/epimetheus-circuit) | Prometheus Rejected Execution Metrics. | | [epimetheus-http4s](https://github.com/davenverse/epimetheus-http4s) | Prometheus Http4s Metrics Middleware and Push Gateway Support. | | [epimetheus-log4cats](https://github.com/davenverse/epimetheus-log4cats) | Prometheus Logging Metrics. | | [epimetheus-redis4cats](https://github.com/davenverse/epimetheus-redis4cats) | Prometheus Redis4cats Metrics. | | [epimetheus](https://davenverse.github.io/epimetheus/) | Prometheus Core Representations. | | [extruder](https://janstenpickle.github.io/extruder/) | Populate case classes from any data source. | | [fastparse-cats](https://github.com/johnynek/fastparse-cats) | [Cats][Cats] Monad and Alternative instances for [fastparse](https://com-lihaoyi.github.io/fastparse/). | | [fd4s/fs2-kafka](https://fd4s.github.io/fs2-kafka/) | Functional [Kafka][Kafka] Streams for Scala. | | [fetch](https://xebia-functional.github.io/fetch/) | Efficient data access to heterogeneous data sources. | | [finch](https://finagle.github.io/finch/) | Scala combinator api for building [finagle](https://twitter.github.io/finagle/) http services. | | [fly4s](https://geirolz.github.io/fly4s/) | A lightweight, simple and functional wrapper for Flyway using [Cats Effect][Cats Effect]. | | [frameless](https://github.com/typelevel/frameless) | Expressive types for [Spark](https://spark.apache.org/). | | [freestyle](http://frees.io/) | Pure functional framework for free and tagless final applications and libraries. | | [fs2-aws](https://github.com/laserdisc-io/fs2-aws) | [Fs2][Fs2] streams to interact with AWS utilities. | | [fs2-blobstore](https://fs2-blobstore.github.io/fs2-blobstore/) | Minimal, idiomatic, stream-based Scala interface for S3, GCS, SFTP and other key/value store implementations. | | [fs2-cassandra](https://spinoco.github.io/fs2-cassandra/) | [Cassandra](https://cassandra.apache.org/_/index.html) bindings for [Fs2][Fs2]. | | [fs2-columns](https://gitlab.com/lJoublanc/fs2-columns) | A Chunk that uses [Shapeless](https://github.com/milessabin/shapeless) to store case class data column-wise. | | [fs2-cron](https://gitlab.com/lJoublanc/fs2-columns) | [Fs2][Fs2] streams based on cron expressions. | | [fs2-crypto](https://github.com/Spinoco/fs2-crypto) | TLS support for [Fs2][Fs2]. | | [fs2-data](https://fs2-data.gnieh.org/) | A set of data parsers and manipulation libraries for various formats for [Fs2][Fs2]. | | [fs2-elastic](https://github.com/amarrella/fs2-elastic) | Simple client for Elasticsearch. | | [fs2-ftp](https://github.com/regis-leray/fs2-ftp) | Simple client for Ftp/Ftps/Sftp using [Fs2][Fs2] and [Cats Effect][Cats Effect]. | | [fs2-google-pubsub](https://github.com/permutive/fs2-google-pubsub) | A Google Cloud Pub/Sub implementation using [Fs2][Fs2] and [Cats Effect][Cats Effect]. | | [fs2-grpc](https://github.com/fiadliel/fs2-grpc) | GRPC implementation for [Fs2][Fs2] / [Cats Effect][Cats Effect]. | | [fs2-http](https://github.com/Spinoco/fs2-http) | Http server and client library implemented in [Fs2][Fs2]. | | [fs2-jms](https://github.com/kiambogo/fs2-jms) | Java Messaging Service (JMS) connectors for [Fs2][Fs2] streams. | | [fs2-log](https://github.com/Spinoco/fs2-log) | Simple logging facade for [Fs2][Fs2]. | | [fs2-mail](https://github.com/Spinoco/fs2-mail) | Fully asynchronous Java non-blocking email client using [Fs2][Fs2]. | | [fs2-mqtt](https://github.com/user-signal/fs2-mqtt/) | A purely functional MQTT client library based on [Fs2][Fs2] and [Cats Effect][Cats Effect]. | | [fs2-rabbit](https://fs2-rabbit.profunktor.dev/) | [RabbitMQ](https://www.rabbitmq.com/) stream-based client built on top of [Fs2][Fs2] and [Cats Effect][Cats Effect]. | | [fs2-reactive-streams](https://github.com/zainab-ali/fs2-reactive-streams) | A reactive streams implementation for [Fs2][Fs2]. | | [fs2-ssh](https://github.com/precog/fs2-ssh) | A wrapper around Apache SSHD targeting [Cats Effect][Cats Effect] and [Fs2][Fs2]. | | [fs2-throttler](https://github.com/kovstas/fs2-throttler) | Throttling for fs2 based on the token bucket algorithm. | | [fs2-zk](https://github.com/Spinoco/fs2-zk) | Simple Apache [Zookeeper](https://zookeeper.apache.org/) bindings for [Fs2][Fs2]. | | [Fs2][Fs2] | Functional streams for Scala (streaming I/O library). | | [fuuid-annotation](https://github.com/davenverse/fuuid-annotation) | Automatically create an inner Id tagged FUUID type with convenient methods for its creation. | | [fuuid](https://davenverse.github.io/fuuid/) | Functional UUID's. | | [gato-parsec](https://github.com/davenverse/gato-parsec/) | Generalized attoparsec with [Cats][Cats]. | | [github4s](http://47degrees.github.io/github4s/) | Wrapper around the GitHub API. | | [github](https://github.com/davenverse/github) | Github Integration for Scala. | | [grafter](https://opensource.zalando.com/grafter/) | Dependency-injection library using the Reader pattern. | | [graphite4s](https://github.com/YannMoisan/graphite4s) | Lightweight graphite client. | | [gsheets4s](https://github.com/BenFradet/gsheets4s) | Wrapper around the Google Sheets API. | | [hammock](https://hammock.pepegar.com/) | Purely functional HTTP client. | | [henkan](https://github.com/kailuowang/henkan) | Type safe conversion between case class instances with similar fields. | | [http4s](https://http4s.org/) | Minimal, idiomatic Scala interface for HTTP services using [Fs2][Fs2]. | | [iota](https://github.com/frees-io/iota) | Fast (co)product types with a clean syntax. | | [itto-csv](https://github.com/gekomad/itto-csv) | Pure functional library for working with CSV. | | [jms4s](https://fpinbo.dev/jms4s) | A functional wrapper for jms | | [kafka4s](https://banno.github.io/kafka4s/) | Functional programming with [Kafka][Kafka] and Scala. | | [kamon-monix](https://github.com/bogdanromanx/kamon-monix) | [Kamon](https://github.com/kamon-io/Kamon) Integration for [Monix][Monix]. | | [keysemaphore](https://github.com/davenverse/keysemaphore) | Keyed Semaphore implementation. | | [laserdisc](http://laserdisc.io/) | LaserDisc is a(nother) Scala driver for Redis. | | [library.g8](https://github.com/davenverse/library.g8) | Library Starter Template. | | [linebacker](https://christopherdavenport.github.io/linebacker/) | Thread pool management for Scala: Enabling functional blocking where needed. | | [log4cats-log4s](https://github.com/ChristopherDavenport/log4cats-log4s) | Log4s support for [Log4Cats][Log4Cats]. | | [log4cats-mtl](https://christopherdavenport.github.io/log4cats-mtl/) | Provide MTL support for [Log4Cats][Log4Cats]. | | [log4cats-scribe](https://www.javadoc.io/doc/io.chrisdavenport/log4cats-scribe_2.12/latest/index.html) | Scribe support for [Log4Cats][Log4Cats]. | | [Log4Cats][Log4Cats] | Functional logging. | | [mapref](https://davenverse.github.io/mapref/) | Refs optimized around maps. | | [mau](https://github.com/kailuowang/mau) | A tiny library for an auto polling ref. | | [monadic-html](https://olivierblanvillain.github.io/monadic-html/) | Tiny DOM binding library for Scala.js. | | [mongo4cats](https://github.com/Kirill5k/mongo4cats) | MongoDB Scala client wrapper for [Cats Effect][Cats Effect] & [Fs2][Fs2]. | | [mongosaur](https://gitlab.com/lJoublanc/mongosaur) | [Fs2][Fs2]-based MongoDB driver. | | [monix bio](https://bio.monix.io/) | Io alternative with typed errors. | | [monix-connect](https://connect.monix.io/) | stream integrations for [Monix][Monix]. | | [monix-kafka](https://github.com/monix/monix-kafka) | [Monix][Monix] integration with [Kafka][Kafka]. | | [monix-nio](https://github.com/monix/monix-nio) | Java NIO utilities for usage with [Monix][Monix]. | | [monix](https://monix.io/) | High-performance library for composing asynchronous and event-based programs. | | [monocle](https://www.optics.dev/Monocle/) | An optics library for Scala (and Scala.js) strongly inspired by [Haskell Lens](https://hackage.haskell.org/package/lens). | | [monoids](https://typelevel.org/monoids/) | Generic Monoids in Scala. | | [morphling](https://github.com/danslapman/morphling) | Free applicative schemas & instance derivation. | | [mules-http4s-redis](https://github.com/davenverse/mules-http4s-redis) | [Mules][Mules] Distributed Http Caching Support. | | [mules-http4s](https://davenverse.github.io/mules-http4s/) | [Mules][Mules] Http4s Caching Implementation for Caching Servers and Client Http Interactions According to the Http Spec. | | [mules-redis](https://github.com/davenverse/mules-redis) | [Mules][Mules] Redis Support. | | [mysql-binlog-stream](https://github.com/allwefantasy/spark-binlog) | Stream MySQL binlog events with [Fs2][Fs2]. | | [natchez](https://github.com/tpolecat/natchez) | An opentracing library, to use with Jaegger, Honeycomb, OpenCensus, and Lightstep. | | [neotypes](https://neotypes.github.io/neotypes/) | A Scala lightweight, type-safe & asynchronous driver for neo4j. Support streaming queries using [Fs2][Fs2]. | | [neutron](https://cr-org.github.io/neutron/) | Apache Pulsar bindings for [Fs2][Fs2]. | | [newts](https://julien-truffaut.github.io/newts/) | Defines newtypes compatible with [Cats][Cats] type classes. | | [odin](https://github.com/valskalla/odin) | Fast & functional logger with own logging backend. | | [origami](https://github.com/atnos-org/origami) | Monadic folds. | | [phony](https://github.com/alirezameskin/phony) | Fake data generator. | | [pulsar4s](https://github.com/sksamuel/pulsar4s) | concise, idiomatic, reactive, type safe Scala client for [Apache Pulsar](https://pulsar.apache.org/). | | [pureconfig](https://pureconfig.github.io/) | A boilerplate-free library for loading configuration files. | | [rainier](https://rainier.fit/) | Bayesian inference in Scala. | | [random](https://christopherdavenport.github.io/random/) | Pure randomness in Scala. | | [read](https://github.com/ChristopherDavenport/read) | Read Typeclass. | | [redis4cats](https://redis4cats.profunktor.dev/) | [Redis](https://redis.io/) client built on top of [Fs2][Fs2] and [Cats Effect][Cats Effect]. | | [refined](https://github.com/fthomas/refined) | Simple refinement types for Scala. | | [sangria-monix](https://github.com/sangria-graphql/sangria-monix) | Sangria integration for [Monix][Monix]. | | [sbt-mima-version-check](https://christopherdavenport.github.io/sbt-mima-version-check/) | Plugin to automate which mima versions to check. | | [scala cache](https://cb372.github.io/scalacache/) | A facade for the most popular cache implementations for Scala. | | [scala-forex](https://github.com/snowplow/scala-forex) | Exchange rate lookups. | | [scala-maxmind-ip-lookups](https://github.com/snowplow/scala-maxmind-iplookups) | IP geolocation through the Maxmind database. | | [scala-referer-parser](https://github.com/snowplow-referer-parser/scala-referer-parser) | Referer parsing. | | [scala-rl](https://www.scalarl.com/) | Functional, monadic reinforcement learning in Scala. | | [scala-weather](https://github.com/snowplow/scala-weather) | Weather lookups. | | [scanamo](https://www.scanamo.org/) | Simpler [DynamoDB](https://aws.amazon.com/dynamodb/) access for Scala. | | [scarctic](https://gitlab.com/lJoublanc/scarctic) | [Fs2][Fs2]-based driver for MAN/AHL’s Arctic data store. | | [scodec-protocols](https://github.com/scodec/scodec-protocols) | A library for working with libpcap files. Contains many interesting pipes (e.g., working with time series and playing back streams at various rates). | | [scodec-stream](https://github.com/scodec/scodec-stream) | A library for streaming binary decoding and encoding, built using [Fs2][Fs2] and scodec. | | [seals](https://github.com/durban/seals) | Tools for schema evolution and language-integrated schemata. | | [semigroups](https://github.com/typelevel/semigroups) | Set of generic semigroup types and accompanying instances very useful for abstract programming. | | [shims](https://github.com/djspiewak/shims) | Seamless interoperability for [Cats][Cats] and [Scalaz](https://scalaz.github.io/7/) typeclasses and datatypes. | | [skunk](https://tpolecat.github.io/skunk/) | A data access library for Scala + [Postgres](https://www.postgresql.org/). | | [snowflake4s](https://github.com/qwbarch/snowflake4s) | Functional snowflake id generator. | | [spata](https://github.com/fingo/spata) | CSV parser build on [Fs2][Fs2]. | | [spinoco/fs2-kafka](https://github.com/Spinoco/fs2-kafka) | Simple client for Apache [Kafka][Kafka]. | | [sqs4s](https://d2a4u.github.io/sqs4s/) | A pure Scala client for [AWS SQS](https://www.postgresql.org/) using [Fs2][Fs2]. | | [streamz](https://github.com/krasserm/streamz) | A library that supports the conversion of [Akka Stream](https://doc.akka.io/docs/akka/current/stream/index.html) Sources, Flows and Sinks to and from [Fs2][Fs2] Streams, Pipes and Sinks, respectively. It also supports the usage of [Apache Camel](https://camel.apache.org/) endpoints in [Fs2][Fs2] Streams and Akka Stream Sources, Flows and SubFlows. | | [sttp](https://github.com/softwaremill/sttp) | The Scala HTTP client you always wanted. | | [sup](https://sup.kubukoz.com/) | Composable, purely functional healthchecks in Scala. | | [synchronized](https://christopherdavenport.github.io/synchronized/) | Synchronous resource access. | | [system-effect](https://github.com/ChristopherDavenport/system-effect) | Console and Environment Simple Tools. | | [testcontainers-specs2](https://github.com/davenverse/testcontainers-specs2) | Test Containers Support for Specs2. | | [tsec](https://jmcardon.github.io/tsec/) | Typesafe, functional, general purpose cryptography and security library. | | [unique](https://typelevel.org/unique/) | Unique functional values for Scala. | | [upperbound](https://github.com/SystemFw/upperbound) | A purely functional, interval-based rate limiter with support for backpressure. | | [vault](https://typelevel.org/vault/) | Type-safe, persistent storage for values of arbitrary types. | | [vinyldns](https://www.vinyldns.io/) | A DNS governance system using [Fs2][Fs2] for throttling updates to DNS backends. | | [whale-tail](https://github.com/davenverse/whale-tail) | Docker Daemon Integration. | | | | Your project talks Cats too? [Submit a PR to add it here!](https://github.com/typelevel/cats/edit/main/docs/typelevelEcosystem.md) *The full-size [Cats logo](img/cats-logo.png) is available for use for Cats related projects, contents, souvenirs, etc.* *We offer a [Cats Friendly Badge](img/cats-badge.svg) to let others know your project works with Cats!* Below are quick html and markdown snippets to use the badge in your own project. ```html

``` ```markdown ``` [Cats]: index.md "Cats" [Fs2]: https://fs2.io "Fs2" [Kafka]: https://kafka.apache.org/ "Kafka" [Monix]: https://monix.io/ "Monix" [Mules]: https://github.com/davenverse/mules/ "Mules" [Cats Effect]: https://typelevel.org/cats-effect/ "Cats Effect" [Circe]: https://circe.github.io/circe/ "Circe" [Log4Cats]: https://typelevel.org/log4cats/ "Log4Cats" --- ### Datatypes Chain (`docs/datatypes/chain.md`) # Chain API Documentation: @:api(cats.data.Chain) `Chain` is an immutable sequence data structure that allows constant time prepending, appending and concatenation. This makes it especially efficient when used as a [Monoid], e.g. with [Validated] or [Writer]. As such it aims to be used where @:api(scala.collection.immutable.List) and @:api(scala.collection.immutable.Vector) incur a performance penalty. Cats also includes type class implementations to support using `Chain` as a general-purpose collection type, including [Traverse], [Monad], and [Alternative]. ## Motivation `List` is a great data type, it is very simple and easy to understand. It has very low overhead for the most important functions such as [fold][Foldable] and [map][Functor] and also supports prepending a single element in constant time. Traversing a data structure with something like [Writer\[List\[Log\], A\]][Writer] or [ValidatedNel\[Error, A\]][Validated] is powerful and allows us to precisely specify what kind of iteration we want to do while remaining succinct. However, in terms of efficiency it's a whole different story unfortunately. That is because both of these traversals make use of the `List` monoid (or the [NonEmptyList] semigroup), which by the nature of `List` is very inefficient. If you use [traverse][Traverse] with a data structure with `n` elements and [Writer] or [Validated] as the [Applicative] type, you will end up with a runtime of `O(n^2)`. This is because, with `List`, appending a single element requires iterating over the entire data structure and therefore takes linear time. So @:api(scala.collection.immutable.List) isn't all that great for this use case, so let's use @:api(scala.collection.immutable.Vector) or @:api(cats.data.NonEmptyVector)` instead, right? Well, `Vector` has its own problems and in this case it's unfortunately not that much faster than `List` at all. You can check [this blog post](http://www.lihaoyi.com/post/BenchmarkingScalaCollections.html#vectors-are-ok) by Li Haoyi for some deeper insight into `Vector`'s issues. `Chain` evolved from what used to be `fs2.Catenable` and Erik Osheim's [Chain](https://github.com/non/chain) library. Similar to `List`, it is also a very simple data structure, but unlike `List` it supports constant O(1) time `append`, `prepend` and `concat`. This makes its [Monoid] instance [super performant][Benchmarks] and a much better fit for usage with [Validated], [Writer], [Ior] or [Const]. ## NonEmptyChain [NonEmptyChain][nec] is the non-empty version of `Chain`. It does not have a [Monoid] instance since it cannot be empty, but it does have a [Semigroup] instance. Likewise, it defines a [NonEmptyTraverse] instance, but no @:api(cats.TraverseFilter) instance. To simplify the usage of `NonEmptyChain`, Cats includes type aliases like [ValidatedNec](validated.md#meeting-applicative) and [IorNec](ior.md#using-with-nonemptychain), as well as helper functions like `groupByNec` and `Validated.invalidNec`. There are numerous ways to construct a `NonEmptyChain`, e.g. you can create one from a single element, a `NonEmptyList` or a `NonEmptyVector`: ```scala mdoc import cats.data._ NonEmptyChain(1, 2, 3, 4) NonEmptyChain.fromNonEmptyList(NonEmptyList(1, List(2, 3))) NonEmptyChain.fromNonEmptyVector(NonEmptyVector(1, Vector(2, 3))) NonEmptyChain.one(1) ``` You can also create an @:api(scala.Option) of `NonEmptyChain` from a `Chain` or any other collection type: ```scala mdoc import cats.data._ NonEmptyChain.fromChain(Chain(1, 2, 3)) NonEmptyChain.fromSeq(List.empty[Int]) NonEmptyChain.fromSeq(Vector(1, 2, 3)) ``` Sometimes, you'll want to prepend or append a single element to a chain and return the result as a `NonEmptyChain`: ```scala mdoc import cats.data._ NonEmptyChain.fromChainAppend(Chain(1, 2, 3), 4) NonEmptyChain.fromChainAppend(Chain.empty[Int], 1) NonEmptyChain.fromChainPrepend(1, Chain(2, 3)) ``` ## How it works `Chain` is implemented as a simple unbalanced binary tree ADT with four cases: an empty `Chain` with no elements, a singleton `Chain` with exactly one element, a concatenation of two chains, or a wrapper for a @:api(scala.collection.immutable.Seq). In code it looks like this: ```scala mdoc sealed abstract class Chain[+A] case object Empty extends Chain[Nothing] case class Singleton[A](a: A) extends Chain[A] case class Append[A](left: Chain[A], right: Chain[A]) extends Chain[A] case class Wrap[A](seq: Seq[A]) extends Chain[A] ``` The `Append` constructor is what gives us the fast concatenation ability. Concatenating two existing `Chain`s is just a call to the `Append` constructor, which is always constant time `O(1)`. In case we want to append or prepend a single element, all we have to do is wrap the element with the `Singleton` constructor and then use the `Append` constructor to append or prepend the `Singleton` `Chain`. The `Wrap` constructor lifts any `Seq` into a `Chain`. This can be useful for concatenating already created collections that don't have great concatenation performance. `Append(Wrap(list1), Wrap(list2)).foldMap(f)` will in general be much faster than just concatenating `list1` and `list2` and folding the result. `Chain` doesn't directly expose the `Append` and `Wrap` constructors, because the arguments might refer to an empty `Chain` or `Seq`. Instead of calling `Append` directly you can simply use `Chain.concat` or `++`, which will check if one of the arguments is empty: ```scala def concat[A](c: Chain[A], c2: Chain[A]): Chain[A] = if (c.isEmpty) c2 else if (c2.isEmpty) c else Append(c, c2) ``` To construct a `Chain` from a `Seq` you should use `Chain.fromSeq` which will also check for emptiness: ```scala def fromSeq[A](s: Seq[A]): Chain[A] = if (s.isEmpty) nil else if (s.lengthCompare(1) == 0) one(s.head) else Wrap(s) ``` In conclusion `Chain` supports constant time concatenation, because it builds an unbalance tree of `Append`s. `append` and `prepend` are treated as concatenation with single element collection to keep the same performance characteristics. This unbalanced tree will always allow iteration in linear time. ## Benchmarks To get a good idea of performance of `Chain`, here are some benchmarks that test monoidal append (higher score is better): ``` Benchmark Mode Cnt Score Error Units CollectionMonoidBench.accumulateChain thrpt 25 81.973 ± 3.921 ops/s CollectionMonoidBench.accumulateList thrpt 25 21.150 ± 1.756 ops/s CollectionMonoidBench.accumulateVector thrpt 25 11.725 ± 0.306 ops/s ``` As you can see accumulating things with `Chain` is almost 4 times faster than `List` and nearly 8 times faster than `Vector`. So appending is a lot more performant than the standard library collections, but what about operations like `map` or `fold`? Fortunately we've also benchmarked these (again, higher score is better): ``` /* Detailed source-code truncated for AI context efficiency. */ ``` While not dominant, `Chain` performance is in the middle of the pack for most operations benchmarked. `Chain` does have poor random access performance, and should be avoided in favor of `Vector` for random access heavy use cases. Chain excels with concatenation heavy workloads and has comparable performance to `List` and `Vector` for most other operations. So next time you write any code that uses `List` or `Vector` as a [Monoid], be sure to use `Chain` instead! > Note: All benchmarks above were run using JMH 1.32 with Scala 2.13.8 on JDK 11. For full details, see [here](https://github.com/typelevel/cats/pull/4264). You can also check out the [benchmark source code](https://github.com/typelevel/cats/blob/v@VERSION@/bench/src/main/scala/cats/bench). [nec]: @API_LINK_BASE@/cats/data/index.html#NonEmptyChain:cats.data.NonEmptyChainImpl.type --- ### Datatypes Const (`docs/datatypes/const.md`) # Const API Documentation: @:api(cats.data.Const) At first glance `Const` seems like a strange data type - it has two type parameters, yet only stores a value of the first type. What possible use is it? As it turns out, it does have its uses, which serve as a nice example of the consistency and elegance of functional programming. ## Thinking about `Const` The `Const` data type can be thought of similarly to the `const` function, but as a data type. ```scala mdoc:silent def const[A, B](a: A)(b: => B): A = a ``` The `const` function takes two arguments and simply returns the first argument, ignoring the second. ```scala mdoc:silent case class Const[A, B](getConst: A) ``` The `Const` data type takes two type parameters, but only ever stores a value of the first type parameter. Because the second type parameter is not used in the data type, the type parameter is referred to as a "phantom type". ## Why do we care? It would seem `Const` gives us no benefit over a data type that would simply not have the second type parameter. However, while we don't directly use the second type parameter, its existence becomes useful in certain contexts. ### Example 1: Lens The following is heavily inspired by [Julien Truffaut](https://github.com/julien-truffaut)'s [blog post](http://functional-wizardry.blogspot.co.uk/2014/02/lens-implementation-part-1.html) on [Monocle](https://github.com/julien-truffaut/Monocle), a fully-fledged optics library in Scala. Types that contain other types are common across many programming paradigms. It is of course desirable in many cases to get out members of other types, or to set them. In traditional object-oriented programming this is handled by getter and setter methods on the outer object. In functional programming, a popular solution is to use a lens. A lens can be thought of as a first class getter/setter. A `Lens[S, A]` is a data type that knows how to get an `A` out of an `S`, or set an `A` in an `S`. ```scala mdoc:silent trait Lens[S, A] { def get(s: S): A def set(s: S, a: A): S def modify(s: S)(f: A => A): S = set(s, f(get(s))) } ``` It can be useful to have effectful modifications as well - perhaps our modification can fail (`Option`) or can return several values (`List`). ```scala mdoc:nest:silent trait Lens[S, A] { def get(s: S): A def set(s: S, a: A): S def modify(s: S)(f: A => A): S = set(s, f(get(s))) def modifyOption(s: S)(f: A => Option[A]): Option[S] = f(get(s)).map(a => set(s, a)) def modifyList(s: S)(f: A => List[A]): List[S] = f(get(s)).map(a => set(s, a)) } ``` Note that both `modifyOption` and `modifyList` share the *exact* same implementation. If we look closely, the only thing we need is a `map` operation on the data type. Being good functional programmers, we abstract. ```scala mdoc:nest:silent import cats.Functor import cats.syntax.all._ trait Lens[S, A] { def get(s: S): A def set(s: S, a: A): S def modify(s: S)(f: A => A): S = set(s, f(get(s))) def modifyF[F[_] : Functor](s: S)(f: A => F[A]): F[S] = f(get(s)).map(a => set(s, a)) } ``` We can redefine `modify` in terms of `modifyF` by using `cats.Id`. We can also treat `set` as a modification that simply ignores the current value. Due to these modifications however, we must leave `modifyF` abstract since having it defined in terms of `set` would lead to infinite circular calls. ```scala mdoc:nest:silent import cats.Id trait Lens[S, A] { def modifyF[F[_] : Functor](s: S)(f: A => F[A]): F[S] def set(s: S, a: A): S = modify(s)(_ => a) def modify(s: S)(f: A => A): S = modifyF[Id](s)(f) def get(s: S): A } ``` What about `get`? Certainly we can't define `get` in terms of the others.. the others are to modify an existing value, whereas `get` is to retrieve it. Let's give it a shot anyways. Looking at `modifyF`, we have an `S` we can pass in. The tricky part will be the `A => F[A]`, and then somehow getting an `A` out of `F[S]`. If we imagine `F` to be a type-level constant function however, we could imagine it would simply take any type and return some other constant type, an `A` perhaps. This suggests our `F` is a `Const`. We then take a look at the fact that `modifyF` takes an `F[_]`, a type constructor that takes a single type parameter. `Const` takes two, so we must fix one. The function returns an `F[S]`, but we want an `A`, which implies we have the first type parameter fixed to `A` and leave the second one free for the function to fill in as it wants. Substituting in `Const[A, _]` wherever we see `F[_]`, the function wants an `A => Const[A, A]` and will give us back a `Const[A, S]`. Looking at the definition of `Const`, we see that we only ever have a value of the first type parameter and completely ignore the second. Therefore, we can treat any `Const[X, Y]` value as equivalent to `X` (plus or minus some wrapping into `Const`). This leaves us with needing a function `A => A`. Given the type, the only thing we can do is to take an `A` and return it right back (lifted into `Const`). Before we plug and play however, note that `modifyF` has a `Functor` constraint on `F[_]`. This means we need to define a `Functor` instance for `Const`, where the first type parameter is fixed. *Note*: the example below assumes usage of the [kind-projector compiler plugin](https://github.com/typelevel/kind-projector) and will not compile if it is not being used in a project. ```scala mdoc:reset:silent import cats.Functor import cats.data.Const implicit def constFunctor[X]: Functor[Const[X, *]] = new Functor[Const[X, *]] { // Recall Const[X, A] ~= X, so the function is not of any use to us def map[A, B](fa: Const[X, A])(f: A => B): Const[X, B] = Const(fa.getConst) } ``` Now that that's taken care of, let's substitute and see what happens. ```scala mdoc:silent import cats.Id trait Lens[S, A] { def modifyF[F[_] : Functor](s: S)(f: A => F[A]): F[S] def set(s: S, a: A): S = modify(s)(_ => a) def modify(s: S)(f: A => A): S = modifyF[Id](s)(f) def get(s: S): A = { val storedValue = modifyF[Const[A, *]](s)(a => Const(a)) storedValue.getConst } } ``` It works! We get a `Const[A, S]` out on the other side, and we simply just retrieve the `A` value stored inside. What's going on here? We can treat the effectful "modification" we are doing as a store operation - we take an `A` and store it inside a `Const`. Knowing only `F[_]` has a `Functor` instance, it can only `map` over the `Const` which will do nothing to the stored value. After `modifyF` is done getting the new `S`, we retrieve the stored `A` value and we're done! ### Example 2: Traverse In the popular [The Essence of the Iterator Pattern](https://www.cs.ox.ac.uk/jeremy.gibbons/publications/iterator.pdf) paper, Jeremy Gibbons and Bruno C. d. S. Oliveria describe a functional approach to iterating over a collection of data. Among the abstractions presented are `Foldable` and `Traverse`, replicated below (also available in Cats). ```scala mdoc:silent import cats.{Applicative, Monoid} trait Foldable[F[_]] { // Given a collection of data F[A], and a function mapping each A to a B where B has a Monoid instance, // reduce the collection down to a single B value using the monoidal behavior of B def foldMap[A, B : Monoid](fa: F[A])(f: A => B): B } trait Traverse[F[_]] { // Given a collection of data F[A], for each value apply the function f which returns an effectful // value. The result of traverse is the composition of all these effectful values. def traverse[G[_] : Applicative, A, B](fa: F[A])(f: A => G[B]): G[F[B]] } ``` These two type classes seem unrelated - one reduces a collection down to a single value, the other traverses a collection with an effectful function, collecting results. It may be surprising to see that in fact `Traverse` subsumes `Foldable`. ```scala mdoc:nest:silent trait Traverse[F[_]] extends Foldable[F] { def traverse[G[_] : Applicative, A, X](fa: F[A])(f: A => G[X]): G[F[X]] def foldMap[A, B : Monoid](fa: F[A])(f: A => B): B } ``` To start, we observe that if we are to implement `foldMap` in terms of `traverse`, we will want a `B` out at some point. However, `traverse` returns a `G[F[X]]`. It would seem there is no way to unify these two. However, if we imagine `G[_]` to be a sort of type-level constant function, where the fact that it's taking a `F[X]` is irrelevant to the true underlying value, we can begin to see how we might be able to pull this off. `traverse` however wants `G[_]` to have an `Applicative` instance, so let's define one for `Const`. Since `F[X]` is the value we want to ignore, we treat it as the second type parameter and hence, leave it as the free one. ```scala mdoc:nest:silent import cats.data.Const implicit def constApplicative[Z]: Applicative[Const[Z, *]] = new Applicative[Const[Z, *]] { def pure[A](a: A): Const[Z, A] = ??? def ap[A, B](f: Const[Z, A => B])(fa: Const[Z, A]): Const[Z, B] = ??? } ``` Recall that `Const[Z, A]` means we have a `Z` value in hand, and don't really care about the `A` type parameter. Therefore we can more or less treat the type `Const[Z, A]` as just `Z`. In functions `pure` and `ap` we have a problem. In `pure`, we have an `A` value, but want to return a `Z` value. We have no function `A => Z`, so our only option is to completely ignore the `A` value. But we still don't have a `Z`! Let's put that aside for now, but still keep it in the back of our minds. In `ap` we have two `Z` values, and want to return a `Z` value. We could certainly return one or the other, but we should try to do something more useful. This suggests composition of `Z`s, which we don't know how to do. So now we need a constant `Z` value, and a binary function that takes two `Z`s and produces a `Z`. Sound familiar? We want `Z` to have a `Monoid` instance! ```scala mdoc:nest:silent implicit def constApplicative[Z : Monoid]: Applicative[Const[Z, *]] = new Applicative[Const[Z, *]] { def pure[A](a: A): Const[Z, A] = Const(Monoid[Z].empty) def ap[A, B](f: Const[Z, A => B])(fa: Const[Z, A]): Const[Z, B] = Const(Monoid[Z].combine(fa.getConst, f.getConst)) } ``` We have our `Applicative`! Going back to `Traverse`, we fill in the first parameter of `traverse` with `fa` since that's the only value that fits. Now we need a `A => G[B]`. We have an `A => B`, and we've decided to use `Const` for our `G[_]`. We need to fix the first parameter of `Const` since `Const` takes two type parameters and `traverse` wants a type constructor which only takes one. The first type parameter which will be the type of the actual values we store, and therefore will be the type of the value we get out at the end, so we leave the second one free, similar to the `Applicative` instance. We don't care about the second type parameter and there are no restrictions on it, so we can just use `Nothing`, the type that has no values. So to summarize, what we want is a function `A => Const[B, Nothing]`, and we have a function `A => B`. Recall that `Const[B, Z]` (for any `Z`) is the moral equivalent of just `B`, so `A => Const[B, Nothing]` is equivalent to `A => B`, which is exactly what we have, we just need to wrap it. ```scala mdoc:nest:silent trait Traverse[F[_]] extends Foldable[F] { def traverse[G[_] : Applicative, A, X](fa: F[A])(f: A => G[X]): G[F[X]] def foldMap[A, B : Monoid](fa: F[A])(f: A => B): B = { val const: Const[B, F[Nothing]] = traverse[Const[B, *], A, Nothing](fa)(a => Const(f(a))) const.getConst } } ``` Hurrah! What's happening here? We can see `traverse` is a function that goes over a collection, applying an effectful function to each value, and combining all of these effectful values. In our case, the effect is mapping each value to a value of type `B`, where we know how to combine `B`s via its `Monoid` instance. The `Monoid` instance is exactly what is used when `traverse` goes to collect the effectful values together. Should the `F[A]` be "empty", it can use `Monoid#empty` as a value to return back. Pretty nifty. `traverse`-ing over a collection with an effectful function is more general than traversing over a collection to reduce it down to a single value. --- ### Datatypes Contt (`docs/datatypes/contt.md`) # ContT API Documentation: @:api(cats.data.ContT) A pattern that appears sometimes in functional programming is that of a function first computing some kind of intermediate result and then passing that result to another function which was passed in as an argument, in order to delegate the computation of the final result. For example: ```scala mdoc:silent case class User(id: Int, name: String, age: Int) sealed abstract class UserUpdateResult case class Succeeded(updatedUserId: Int) extends UserUpdateResult case object Failed extends UserUpdateResult ``` ```scala mdoc import cats.Eval def updateUser(persistToDatabase: User => Eval[UserUpdateResult]) (existingUser: User, newName: String, newAge: Int): Eval[UserUpdateResult] = { val trimmedName = newName.trim val cappedAge = newAge max 150 val updatedUser = existingUser.copy(name = trimmedName, age = cappedAge) persistToDatabase(updatedUser) } ``` (Note: We will be using `Eval` throughout the examples on this page. If you are not familiar with `Eval`, it's worth reading [the Eval documentation](eval.md) first.) Our `updateUser` function takes in an existing user and some updates to perform. It sanitises the inputs and updates the user model, but it delegates the database update to another function which is passed in as an argument. This pattern is known as "continuation passing style" or CPS, and the function passed in (`persistToDatabase`) is known as a "continuation". Note the following characteristics: * The return type of our `updateUser` function (`Eval[UserUpdateResult]`) is the same as the return type of the continuation function that was passed in. * Our function does a bit of work to build an intermediate value, then passes that value to the continuation, which takes care of the remainder of the work. In Cats we can encode this pattern using the `ContT` data type: ```scala mdoc import cats.data.ContT def updateUserCont(existingUser: User, newName: String, newAge: Int): ContT[Eval, UserUpdateResult, User] = ContT.apply[Eval, UserUpdateResult, User] { next => val trimmedName = newName.trim val cappedAge = newAge min 150 val updatedUser = existingUser.copy(name = trimmedName, age = cappedAge) next(updatedUser) } ``` We can construct a computation as follows: ```scala mdoc val existingUser = User(100, "Alice", 42) val computation = updateUserCont(existingUser, "Bob", 200) ``` And then call `run` on it, passing in a function of type `User => Eval[UserUpdateResult]` as the continuation: ```scala mdoc val eval = computation.run { user => Eval.later { println(s"Persisting updated user to the DB: $user") Succeeded(user.id) } } ``` Finally we can run the resulting `Eval` to actually execute the computation: ```scala mdoc eval.value ``` ## Composition You might be wondering what the point of all this was, as the function that uses `ContT` seems to achieve the same thing as the original function, just encoded in a slightly different way. The point is that `ContT` is a monad, so by rewriting our function into a `ContT` we gain composibility for free. For example we can `map` over a `ContT`: ```scala mdoc val anotherComputation = computation.map { user => Map( "id" -> user.id.toString, "name" -> user.name, "age" -> user.age.toString ) } val anotherEval = anotherComputation.run { userFields => Eval.later { println(s"Persisting these fields to the DB: $userFields") Succeeded(userFields("id").toInt) } } anotherEval.value ``` And we can use `flatMap` to chain multiple `ContT`s together. The following example builds 3 computations: one to sanitise the inputs and update the user model, one to persist the updated user to the database, and one to publish a message saying the user was updated. It then chains them together in continuation-passing style using `flatMap` and runs the whole computation. ```scala mdoc:nest val updateUserModel: ContT[Eval, UserUpdateResult, User] = updateUserCont(existingUser, "Bob", 200).map { updatedUser => println("Updated user model") updatedUser } val persistToDb: User => ContT[Eval, UserUpdateResult, UserUpdateResult] = { user => ContT.apply[Eval, UserUpdateResult, UserUpdateResult] { next => println(s"Persisting updated user to the DB: $user") next(Succeeded(user.id)) } } val publishEvent: UserUpdateResult => ContT[Eval, UserUpdateResult, UserUpdateResult] = { userUpdateResult => ContT.apply[Eval, UserUpdateResult, UserUpdateResult] { next => userUpdateResult match { case Succeeded(userId) => println(s"Publishing 'user updated' event for user ID $userId") case Failed => println("Not publishing 'user updated' event because update failed") } next(userUpdateResult) } } val chainOfContinuations = updateUserModel flatMap persistToDb flatMap publishEvent val eval = chainOfContinuations.run { finalResult => Eval.later { println("Finished!") finalResult } } eval.value ``` ## Why Eval? If you're wondering why we used `Eval` in our examples above, it's because the `Monad` instance for `ContT[M[_], A, B]` requires an instance of `cats.Defer` for `M[_]`. This is an implementation detail - it's needed in order to preserve stack safety. In a real-world application, you're more likely to be using something like cats-effect `IO`, which has a `Defer` instance. --- ### Datatypes Either (`docs/datatypes/either.md`) {% laika.excludeFromNavigation = true %} # Either The `Either` type is provided by the Scala standard library, but Cats provides [syntax enrichment](../jump_start_guide.md#either-helpers) and typeclass instances to extend its functionality, including [Functor], [Applicative], [Monad] and [MonadError](../typeclasses/applicativemonaderror.md). If you would like to accumulate errors instead of the short-circuiting behavior of `Either`, see the [Parallel] typeclass, or the [Validated] data type. --- ### Datatypes Eithert (`docs/datatypes/eithert.md`) # EitherT API Documentation: @:api(cats.data.EitherT) `Either` can be used for error handling in most situations. However, when `Either` is placed into effectful types such as `Option` or`Future`, a large amount of boilerplate is required to handle errors. For example, consider the following program: ```scala mdoc import scala.util.Try import cats.syntax.all._ def parseDouble(s: String): Either[String, Double] = Try(s.toDouble).map(Right(_)).getOrElse(Left(s"$s is not a number")) def divide(a: Double, b: Double): Either[String, Double] = Either.cond(b != 0, a / b, "Cannot divide by zero") def divisionProgram(inputA: String, inputB: String): Either[String, Double] = for { a <- parseDouble(inputA) b <- parseDouble(inputB) result <- divide(a, b) } yield result divisionProgram("4", "2") // Right(2.0) divisionProgram("a", "b") // Left("a is not a number") ``` Suppose `parseDouble` and `divide` are rewritten to be asynchronous and return `Future[Either[String, Double]]` instead. The for-comprehension can no longer be used since `divisionProgram` must now compose `Future` and `Either` together, which means that the error handling must be performed explicitly to ensure that the proper types are returned: ```scala mdoc:silent import scala.concurrent.ExecutionContext.Implicits.global import scala.concurrent.Future def parseDoubleAsync(s: String): Future[Either[String, Double]] = Future.successful(parseDouble(s)) def divideAsync(a: Double, b: Double): Future[Either[String, Double]] = Future.successful(divide(a, b)) def divisionProgramAsync(inputA: String, inputB: String): Future[Either[String, Double]] = parseDoubleAsync(inputA) flatMap { eitherA => parseDoubleAsync(inputB) flatMap { eitherB => (eitherA, eitherB) match { case (Right(a), Right(b)) => divideAsync(a, b) case (Left(err), _) => Future.successful(Left(err)) case (_, Left(err)) => Future.successful(Left(err)) } } } ``` Clearly, the updated code is less readable and more verbose: the details of the program are now mixed with the error handling. In addition, as more `Either`s and `Futures` are included, the amount of boilerplate required to properly handle the errors will increase dramatically. ## EitherT `EitherT[F[_], A, B]` is a lightweight wrapper for `F[Either[A, B]]` that makes it easy to compose `Either`s and `F`s together. To use `EitherT`, values of `Either`, `F`, `A`, and `B` are first converted into `EitherT`, and the resulting `EitherT` values are then composed using combinators. For example, the asynchronous division program can be rewritten as follows: ```scala mdoc:nest import cats.data.EitherT import cats.syntax.all._ def divisionProgramAsync(inputA: String, inputB: String): EitherT[Future, String, Double] = for { a <- EitherT(parseDoubleAsync(inputA)) b <- EitherT(parseDoubleAsync(inputB)) result <- EitherT(divideAsync(a, b)) } yield result divisionProgramAsync("4", "2").value divisionProgramAsync("a", "b").value ``` Note that when `F` is a monad, then `EitherT` will also form a monad, allowing monadic combinators such as `flatMap` to be used in composing `EitherT` values. ## From `A` or `B` to `EitherT[F, A, B]` To obtain a left version or a right version of `EitherT` when given an `A` or a `B`, use `EitherT.leftT` and `EitherT.rightT` (which is an alias for `EitherT.pure`), respectively. ```scala mdoc:silent val number: EitherT[Option, String, Int] = EitherT.rightT(5) val error: EitherT[Option, String, Int] = EitherT.leftT("Not a number") ``` ## From `F[A]` or `F[B]` to `EitherT[F, A, B]` Similarly, use `EitherT.left` and `EitherT.right` to convert an `F[A]` or an `F[B]` into an `EitherT`. It is also possible to use `EitherT.liftF` as an alias for `EitherT.right`. ```scala mdoc:nest:silent val numberO: Option[Int] = Some(5) val errorO: Option[String] = Some("Not a number") val number: EitherT[Option, String, Int] = EitherT.right(numberO) val error: EitherT[Option, String, Int] = EitherT.left(errorO) ``` ## From `Either[A, B]` or `F[Either[A, B]]` to `EitherT[F, A, B]` Use `EitherT.fromEither` to lift a value of `Either[A, B]` into `EitherT[F, A, B]`. An `F[Either[A, B]]` can be converted into `EitherT` using the `EitherT` constructor. ```scala mdoc:silent val numberE: Either[String, Int] = Right(100) val errorE: Either[String, Int] = Left("Not a number") val numberFE: List[Either[String, Int]] = List(Right(250)) val numberET: EitherT[List, String, Int] = EitherT.fromEither(numberE) val errorET: EitherT[List, String, Int] = EitherT.fromEither(errorE) val numberFET: EitherT[List, String, Int] = EitherT(numberFE) ``` ## From `Option[B]` or `F[Option[B]]` to `EitherT[F, A, B]` An `Option[B]` or an `F[Option[B]]`, along with a default value, can be passed to `EitherT.fromOption` and `EitherT.fromOptionF`, respectively, to produce an `EitherT`. For `F[Option[B]]` and default `F[A]`, there is `EitherT.fromOptionM`. ```scala mdoc val myOption: Option[Int] = None val myOptionList: List[Option[Int]] = List(None, Some(2), Some(3), None, Some(5)) val myOptionET = EitherT.fromOption[Future](myOption, "option not defined") val myOptionListET = EitherT.fromOptionF(myOptionList, "option not defined") val myOptionListETM = EitherT.fromOptionM(myOptionList, List("option not defined")) ``` ## From `ApplicativeError[F, E]` to `EitherT[F, E, A]` An `ApplicativeError[F, E]` or a `MonadError[F, E]`, can be converted into `EitherT[F, E, A]` using the `attemptT` method on them. ```scala mdoc:silent val myTry: Try[Int] = Try(2) val myFuture: Future[String] = Future.failed(new Exception()) val myTryET: EitherT[Try, Throwable, Int] = myTry.attemptT val myFutureET: EitherT[Future, Throwable, String] = myFuture.attemptT ``` ## Extracting an `F[Either[A, B]]` from an `EitherT[F, A, B]` Use the `value` method defined on `EitherT` to retrieve the underlying `F[Either[A, B]]`: ```scala mdoc:nest val errorT: EitherT[Future, String, Int] = EitherT.leftT("foo") val error: Future[Either[String, Int]] = errorT.value ``` --- ### Datatypes Eval (`docs/datatypes/eval.md`) # Eval API Documentation: @:api(cats.Eval) Eval is a data type for controlling synchronous evaluation. Its implementation is designed to provide stack-safety at all times using a technique called trampolining. There are two different factors that play into evaluation: memoization and laziness. Memoized evaluation evaluates an expression only once and then remembers (memoizes) that value. Lazy evaluation refers to when the expression is evaluated. We talk about eager evaluation if the expression is immediately evaluated when defined and about lazy evaluation if the expression is evaluated when it's first used. For example, in Scala, a `lazy val` is both lazy and memoized, a method definition `def` is lazy, but not memoized, since the body will be evaluated on every call. A normal `val` evaluates eagerly and also memoizes the result. `Eval` is able to express all of these evaluation strategies and allows us to chain computations using its `Monad` instance. #### Eval.now First of the strategies is eager evaluation, we can construct an `Eval` eagerly using `Eval.now`: ```scala mdoc import cats.Eval import cats.syntax.all._ val eager = Eval.now { println("Running expensive calculation...") 1 + 2 * 3 } ``` We can run the computation using the given evaluation strategy anytime by using the `value` method. ```scala mdoc eager.value ``` #### Eval.later If we want lazy evaluation, we can use `Eval.later`: ```scala mdoc val lazyEval = Eval.later { println("Running expensive calculation...") 1 + 2 * 3 } lazyEval.value lazyEval.value ``` Notice that "Running expensive calculation" is printed only once, since the value was memoized internally. `Eval.later` is different to using a `lazy val` in a few different ways. First, it allows the runtime to perform garbage collection of the thunk after evaluation, leading to more memory being freed earlier. Secondly, when `lazy val`s are evaluated, in order to preserve thread-safety, the Scala compiler will lock the whole surrounding class, whereas `Eval` will only lock itself. #### Eval.always If we want lazy evaluation, but without memoization akin to `Function0`, we can use `Eval.always` ```scala mdoc val always = Eval.always { println("Running expensive calculation...") 1 + 2 * 3 } always.value always.value ``` Here we can see, that the expression is evaluated every time we call `.value`. ### Chaining lazy computations One of the most useful applications of `Eval` is its ability to chain together computations in a stack-safe way. You can see one such usage when looking at the `foldRight` method found in [`Foldable`](../typeclasses/foldable.md). Another great example are mutual tail-recursive calls: ```scala mdoc object MutualRecursion { def even(n: Int): Eval[Boolean] = Eval.always(n == 0).flatMap { case true => Eval.True case false => odd(n - 1) } def odd(n: Int): Eval[Boolean] = Eval.always(n == 0).flatMap { case true => Eval.False case false => even(n - 1) } } MutualRecursion.odd(199999).value ``` Because `Eval` guarantees stack-safety, we can chain a lot of computations together using `flatMap` without fear of blowing up the stack. You can also use `Eval.defer` to defer any computation that will return an `Eval[A]`. This is useful, because nesting a call to `.value` inside any of the `Eval` creation methods can be unsafe. --- ### Datatypes Freeapplicative (`docs/datatypes/freeapplicative.md`) # Free Applicative API Documentation: @:api(cats.free.FreeApplicative) `FreeApplicative`s are similar to `Free` (monads) in that they provide a nice way to represent computations as data and are useful for building embedded DSLs (EDSLs). However, they differ from `Free` in that the kinds of operations they support are limited, much like the distinction between `Applicative` and `Monad`. ## Dependency If you'd like to use cats' free applicative, you'll need to add a library dependency for the `cats-free` module. ## Example Consider building an EDSL for validating strings - to keep things simple we'll just have a way to check a string is at least a certain size and to ensure the string contains numbers. ```scala mdoc:silent sealed abstract class ValidationOp[A] case class Size(size: Int) extends ValidationOp[Boolean] case object HasNumber extends ValidationOp[Boolean] ``` Much like the `Free` monad tutorial, we use smart constructors to lift our algebra into the `FreeApplicative`. ```scala mdoc:silent import cats.free.FreeApplicative import cats.free.FreeApplicative.lift type Validation[A] = FreeApplicative[ValidationOp, A] def size(size: Int): Validation[Boolean] = lift(Size(size)) val hasNumber: Validation[Boolean] = lift(HasNumber) ``` Because a `FreeApplicative` only supports the operations of `Applicative`, we do not get the nicety of a for-comprehension. We can however still use `Applicative` syntax provided by Cats. ```scala mdoc:silent import cats.syntax.all._ val prog: Validation[Boolean] = (size(5), hasNumber).mapN { case (l, r) => l && r} ``` As it stands, our program is just an instance of a data structure - nothing has happened at this point. To make our program useful we need to interpret it. ```scala mdoc:silent import cats.Id import cats.arrow.FunctionK import cats.syntax.all._ // a function that takes a string as input type FromString[A] = String => A val compiler = new FunctionK[ValidationOp, FromString] { def apply[A](fa: ValidationOp[A]): FromString[A] = str => fa match { case Size(size) => str.size >= size case HasNumber => str.exists(c => "0123456789".contains(c)) } } ``` ```scala mdoc val validator = prog.foldMap[FromString](compiler) validator("1234") validator("12345") ``` ## Differences from `Free` So far everything we've been doing has been not much different from `Free` - we've built an algebra and interpreted it. However, there are some things `FreeApplicative` can do that `Free` cannot. Recall a key distinction between the type classes `Applicative` and `Monad` - `Applicative` captures the idea of independent computations, whereas `Monad` captures that of dependent computations. Put differently `Applicative`s cannot branch based on the value of an existing/prior computation. Therefore when using `Applicative`s, we must hand in all our data in one go. In the context of `FreeApplicative`s, we can leverage this static knowledge in our interpreter. ### Parallelism Because we have everything we need up front and know there can be no branching, we can easily write a validator that validates in parallel. ```scala mdoc:silent import cats.data.Kleisli import cats.syntax.all._ import scala.concurrent.Future import scala.concurrent.ExecutionContext.Implicits.global // recall Kleisli[Future, String, A] is the same as String => Future[A] type ParValidator[A] = Kleisli[Future, String, A] val parCompiler = new FunctionK[ValidationOp, ParValidator] { def apply[A](fa: ValidationOp[A]): ParValidator[A] = Kleisli { str => fa match { case Size(size) => Future { str.size >= size } case HasNumber => Future { str.exists(c => "0123456789".contains(c)) } } } } val parValidator = prog.foldMap[ParValidator](parCompiler) ``` ### Logging We can also write an interpreter that simply creates a list of strings indicating the filters that have been used - this could be useful for logging purposes. Note that we need not actually evaluate the rules against a string for this, we simply need to map each rule to some identifier. Therefore we can completely ignore the return type of the operation and return just a `List[String]` - the `Const` data type is useful for this. ```scala mdoc:silent import cats.data.Const import cats.syntax.all._ type Log[A] = Const[List[String], A] val logCompiler = new FunctionK[ValidationOp, Log] { def apply[A](fa: ValidationOp[A]): Log[A] = fa match { case Size(size) => Const(List(s"size >= $size")) case HasNumber => Const(List("has number")) } } def logValidation[A](validation: Validation[A]): List[String] = validation.foldMap[Log](logCompiler).getConst ``` ```scala mdoc logValidation(prog) logValidation(size(5) *> hasNumber *> size(10)) logValidation((hasNumber, size(3)).mapN(_ || _)) ``` ### Why not both? It is perhaps more plausible and useful to have both the actual validation function and the logging strings. While we could easily compile our program twice, once for each interpreter as we have above, we could also do it in one go - this would avoid multiple traversals of the same structure. Another useful property `Applicative`s have over `Monad`s is that given two `Applicative`s `F[_]` and `G[_]`, their product `type FG[A] = (F[A], G[A])` is also an `Applicative`. This is not true in the general case for monads. Therefore, we can write an interpreter that uses the product of the `ParValidator` and `Log` `Applicative`s to interpret our program in one go. We can create this interpreter easily by using `FunctionK#and`. ```scala mdoc:silent import cats.data.Tuple2K type ValidateAndLog[A] = Tuple2K[ParValidator, Log, A] val prodCompiler: FunctionK[ValidationOp, ValidateAndLog] = parCompiler and logCompiler val prodValidation = prog.foldMap[ValidateAndLog](prodCompiler) ``` ### The way FreeApplicative#foldMap works Despite being an imperative loop, there is a functional intuition behind `FreeApplicative#foldMap`. The new `FreeApplicative`'s `foldMap` is a sort of mutually-recursive function that operates on an argument stack and a function stack, where the argument stack has type `List[FreeApplicative[F, _]]` and the functions have type `List[Fn[G, _, _]]`. `Fn[G[_, _]]` contains a function to be `Ap`'d that has already been translated to the target `Applicative`, as well as the number of functions that were `Ap`'d immediately subsequently to it. #### Main re-association loop Pull an argument out of the stack, eagerly remove right-associated `Ap` nodes, by looping on the right and adding the `Ap` nodes' arguments on the left to the argument stack; at the end, pushes a single function to the function stack of the applied functions, the rest of which will be pushed in this loop in later iterations. Once all of the `Ap` nodes on the right are removed, the loop resets to deal with the ones on the left. Here's an example `FreeApplicative` value to demonstrate the loop's function, at the end of every iteration. Every node in the tree is annotated with an identifying number and the concrete type of the node (A -> `Ap`, L -> `Lift`, P -> `Pure`), and an apostrophe to denote where `argF` (the current argument) currently points; as well the argument and function branches off `Ap` nodes are explicitly denoted. ``` ==> begin. '1A / \ arg/ \fun / \ / \ 2A 3A arg/ \fun arg/ \fun / \ / \ 4L 5P 6L 7L args: Nil functions: Nil ==> loop. 1A / \ arg/ \fun / \ / \ 2A '3A arg/ \fun arg/ \fun / \ / \ 4L 5P 6L 7L args: 2A :: Nil functions: Nil ==> loop. 1A / \ arg/ \fun / \ / \ 2A 3A arg/ \fun arg/ \fun / \ / \ 4L 5P 6L '7L args: 6L :: 2A :: Nil functions: Fn(gab = foldArg(7L), argc = 2) :: Nil ==> finished. ``` At the end of the loop the entire right branch of `Ap`s under `argF` has been peeled off into a single curried function, all of the arguments to that function are on the argument stack and that function itself is on the function stack, annotated with the amount of arguments it takes. #### Function application loop Once `argF` isn't an `Ap` node, a loop runs which pulls functions from the stack until it reaches a curried function, in which case it applies the function to `argF` transformed into a `G[Any]` value, and pushes the resulting function back to the function stack, before returning to the main loop. I'll continue the example from before here: ``` ==> loop. 1A / \ arg/ \fun / \ / \ 2A 3A arg/ \fun arg/ \fun / \ / \ 4L 5P '6L 7L args: 2A :: Nil functions: Fn(gab = foldArg(7L) ap foldArg(6L), argc = 1) :: Nil ==> finished. ``` At the end of this loop every function on the top of the function stack with `length == 1` (not curried) has been applied to a single argument from the argument stack, and the first curried function (`length != 1`) on the stack has been applied to a single argument from the argument stack. The reason we can't keep applying the curried function to arguments is that the node on top of the argument stack *must* be an `Ap` node if the function is curried, so we can't translate it directly to `G[_]`. Once the last function has been applied to the last argument, the fold has finished and the result is returned. ## References Deeper explanations can be found in this paper [Free Applicative Functors by Paolo Capriotti](http://www.paolocapriotti.com/assets/applicative.pdf) --- ### Datatypes Freemonad (`docs/datatypes/freemonad.md`) # Free Monad API Documentation: @:api(cats.free.Free) ## What is it? A *free monad* is a construction which allows you to build a *monad* from any *Functor*. Like other *monads*, it is a pure way to represent and manipulate computations. In particular, *free monads* provide a practical way to: - represent stateful computations as data, and run them - run recursive computations in a stack-safe way - build an embedded DSL (domain-specific language) - retarget a computation to another interpreter using natural transformations > (In Cats, the type representing a *free monad* is abbreviated as `Free[_]`.) ## Using Free Monads If you'd like to use Cats' free monad, you'll need to add a library dependency for the `cats-free` module. A good way to get a sense for how *free monads* work is to see them in action. The next section uses `Free[_]` to create an embedded DSL (Domain Specific Language). If you're interested in the theory behind *free monads*, the [What is Free in theory?](#for-the-curious-ones-what-is-free-in-theory) section discusses free monads in terms of category theory. ### Study your topic Let's imagine that we want to create a DSL for a key-value store. We want to be able to do three things with keys: - *put* a `value` into the store, associated with its `key`. - *get* a `value` from the store given its `key`. - *delete* a `value` from the store given its `key`. The idea is to write a sequence of these operations in the embedded DSL as a "program", compile the "program", and finally execute the "program" to interact with the actual key-value store. For example: ```scala put("toto", 3) get("toto") // returns 3 delete("toto") ``` But we want: - the computation to be represented as a pure, immutable value - to separate the creation and execution of the program - to be able to support many different methods of execution ### Study your grammar We have 3 commands to interact with our KeyValue store: - `Put` a value associated with a key into the store - `Get` a value associated with a key out of the store - `Delete` a value associated with a key from the store ### Create an ADT representing your grammar ADT stands for *Algebraic Data Type*. In this context, it refers to a closed set of types which can be combined to build up complex, recursive values. We need to create an ADT to represent our key-value operations: ```scala mdoc:silent sealed trait KVStoreA[A] case class Put[T](key: String, value: T) extends KVStoreA[Unit] case class Get[T](key: String) extends KVStoreA[Option[T]] case class Delete(key: String) extends KVStoreA[Unit] ``` ### Free your ADT There are five basic steps to "freeing" the ADT: 1. Create a type based on `Free[_]` and `KVStoreA[_]`. 2. Create smart constructors for `KVStore[_]` using `liftF`. 3. Build a program out of key-value DSL operations. 4. Build a compiler for programs of DSL operations. 5. Execute our compiled program. #### 1. Create a `Free` type based on your ADT ```scala mdoc:silent import cats.free.Free type KVStore[A] = Free[KVStoreA, A] ``` #### 2. Create smart constructors using `liftF` These methods will make working with our DSL a lot nicer, and will lift `KVStoreA[_]` values into our `KVStore[_]` monad (note the missing "A" in the second type). ```scala mdoc:silent import cats.free.Free.liftF // Put returns nothing (i.e. Unit). def put[T](key: String, value: T): KVStore[Unit] = liftF[KVStoreA, Unit](Put[T](key, value)) // Get returns a T value. def get[T](key: String): KVStore[Option[T]] = liftF[KVStoreA, Option[T]](Get[T](key)) // Delete returns nothing (i.e. Unit). def delete(key: String): KVStore[Unit] = liftF(Delete(key)) // Update composes get and set, and returns nothing. def update[T](key: String, f: T => T): KVStore[Unit] = for { vMaybe <- get[T](key) _ <- vMaybe.map(v => put[T](key, f(v))).getOrElse(Free.pure(())) } yield () ``` #### 3. Build a program Now that we can construct `KVStore[_]` values we can use our DSL to write "programs" using a *for-comprehension*: ```scala mdoc:silent def program: KVStore[Option[Int]] = for { _ <- put("wild-cats", 2) _ <- update[Int]("wild-cats", (_ + 12)) _ <- put("tame-cats", 5) n <- get[Int]("wild-cats") _ <- delete("tame-cats") } yield n ``` This looks like a monadic flow. However, it just builds a recursive data structure representing the sequence of operations. #### 4. Write a compiler for your program As you may have understood now, `Free[_]` is used to create an embedded DSL. By itself, this DSL only represents a sequence of operations (defined by a recursive data structure); it doesn't produce anything. `Free[_]` is a programming language inside your programming language! **So, like any other programming language, we need to compile our abstract language into an _effective_ language and then run it.** To do this, we will use a *natural transformation* between type containers. Natural transformations go between types like `F[_]` and `G[_]` (this particular transformation would be written as `FunctionK[F,G]` or as done here using the symbolic alternative as `F ~> G`). In our case, we will use a simple mutable map to represent our key value store: ```scala mdoc:silent import cats.arrow.FunctionK import cats.{Id, ~>} import scala.collection.mutable // the program will crash if a type is incorrectly specified. def impureCompiler: KVStoreA ~> Id = new (KVStoreA ~> Id) { // a very simple (and imprecise) key-value store val kvs = mutable.Map.empty[String, Any] def apply[A](fa: KVStoreA[A]): Id[A] = fa match { case Put(key, value) => println(s"put($key, $value)") kvs(key) = value () case Get(key) => println(s"get($key)") kvs.get(key).asInstanceOf[A] case Delete(key) => println(s"delete($key)") kvs.remove(key) () } } ``` Please note this `impureCompiler` is impure -- it mutates `kvs` and also produces logging output using `println`. The whole purpose of functional programming isn't to prevent side-effects, it is just to push side-effects to the boundaries of your system in a well-known and controlled way. `Id[_]` represents the simplest type container: the type itself. Thus, `Id[Int]` is just `Int`. This means that our program will execute immediately, and block until the final value can be returned. However, we could easily use other type containers for different behavior, such as: - `Future[_]` for asynchronous computation - `List[_]` for gathering multiple results - `Option[_]` to support optional results - `Either[E, *]` to support failure - a pseudo-random monad to support non-determinism - and so on... #### 5. Run your program The final step is naturally running your program after compiling it. `Free[_]` is just a recursive structure that can be seen as sequence of operations producing other operations. In this way it is similar to `List[_]`. We often use folds (e.g. `foldRight`) to obtain a single value from a list; this recurses over the structure, combining its contents. The idea behind running a `Free[_]` is exactly the same. We fold the recursive structure by: - consuming each operation. - compiling the operation into our effective language using `impureCompiler` (applying its effects if any). - computing next operation. - continue recursively until reaching a `Pure` state, and returning it. This operation is called `Free.foldMap`: ```scala final def foldMap[M[_]](f: FunctionK[S,M])(M: Monad[M]): M[A] = ... ``` `M` must be a `Monad` to be flattenable (the famous monoid aspect under `Monad`). As `Id` is a `Monad`, we can use `foldMap`. To run your `Free` with previous `impureCompiler`: ```scala mdoc val result: Option[Int] = program.foldMap(impureCompiler) ``` An important aspect of `foldMap` is its **stack-safety**. It evaluates each step of computation on the stack then unstack and restart. This process is known as trampolining. As long as your natural transformation is stack-safe, `foldMap` will never overflow your stack. Trampolining is heap-intensive but stack-safety provides the reliability required to use `Free[_]` for data-intensive tasks, as well as infinite processes such as streams. #### 6. Use a pure compiler (optional) The previous examples used an effectful natural transformation. This works, but you might prefer folding your `Free` in a "purer" way. The [State](state.md) data structure can be used to keep track of the program state in an immutable map, avoiding mutation altogether. ```scala mdoc:silent import cats.data.State type KVStoreState[A] = State[Map[String, Any], A] val pureCompiler: KVStoreA ~> KVStoreState = new (KVStoreA ~> KVStoreState) { def apply[A](fa: KVStoreA[A]): KVStoreState[A] = fa match { case Put(key, value) => State.modify(_.updated(key, value)) case Get(key) => State.inspect(_.get(key).asInstanceOf[A]) case Delete(key) => State.modify(_ - key) } } ``` (You can see that we are again running into some places where Scala's support for pattern matching is limited by the JVM's type erasure, but it's not too hard to get around.) ```scala mdoc:nest val result: (Map[String, Any], Option[Int]) = program.foldMap(pureCompiler).run(Map.empty).value ``` ## Composing Free monads ADTs. Real world applications often time combine different algebras. The injection type class described by Swierstra in [Data types à la carte](http://www.staff.science.uu.nl/~swier004/publications/2008-jfp.pdf) lets us compose different algebras in the context of `Free`. Let's see a trivial example of unrelated ADT's getting composed as a `EitherK` that can form a more complex program. ```scala mdoc:silent import cats.data.EitherK import cats.free.Free import cats.{Id, InjectK, ~>} import scala.collection.mutable.ListBuffer ``` ```scala mdoc:silent /* Handles user interaction */ sealed trait Interact[A] case class Ask(prompt: String) extends Interact[String] case class Tell(msg: String) extends Interact[Unit] /* Represents persistence operations */ sealed trait DataOp[A] case class AddCat(a: String) extends DataOp[Unit] case class GetAllCats() extends DataOp[List[String]] ``` Once the ADTs are defined we can formally state that a `Free` program is the EitherK of its Algebras. ```scala mdoc:silent type CatsApp[A] = EitherK[DataOp, Interact, A] ``` In order to take advantage of monadic composition we use smart constructors to lift our Algebra to the `Free` context. ```scala mdoc:silent class Interacts[F[_]](implicit I: InjectK[Interact, F]) { def tell(msg: String): Free[F, Unit] = Free.liftInject[F](Tell(msg)) def ask(prompt: String): Free[F, String] = Free.liftInject[F](Ask(prompt)) } object Interacts { implicit def interacts[F[_]](implicit I: InjectK[Interact, F]): Interacts[F] = new Interacts[F] } class DataSource[F[_]](implicit I: InjectK[DataOp, F]) { def addCat(a: String): Free[F, Unit] = Free.liftInject[F](AddCat(a)) def getAllCats: Free[F, List[String]] = Free.liftInject[F](GetAllCats()) } object DataSource { implicit def dataSource[F[_]](implicit I: InjectK[DataOp, F]): DataSource[F] = new DataSource[F] } ``` ADTs are now easily composed and trivially intertwined inside monadic contexts. ```scala mdoc:nest:silent def program(implicit I : Interacts[CatsApp], D : DataSource[CatsApp]): Free[CatsApp, Unit] = { import I._, D._ for { cat <- ask("What's the kitty's name?") _ <- addCat(cat) cats <- getAllCats _ <- tell(cats.toString) } yield () } ``` Finally we write one interpreter per ADT and combine them with a `FunctionK` to `EitherK` so they can be compiled and applied to our `Free` program. ```scala mdoc:invisible def readLine(): String = "snuggles" ``` ```scala mdoc:silent object ConsoleCatsInterpreter extends (Interact ~> Id) { def apply[A](i: Interact[A]) = i match { case Ask(prompt) => println(prompt) readLine() case Tell(msg) => println(msg) } } object InMemoryDatasourceInterpreter extends (DataOp ~> Id) { private[this] val memDataSet = new ListBuffer[String] def apply[A](fa: DataOp[A]) = fa match { case AddCat(a) => memDataSet.append(a); () case GetAllCats() => memDataSet.toList } } val interpreter: CatsApp ~> Id = InMemoryDatasourceInterpreter or ConsoleCatsInterpreter ``` Now if we run our program and type in "snuggles" when prompted, we see something like this: ```scala mdoc:silent import DataSource._, Interacts._ ``` ```scala mdoc val evaled: Unit = program.foldMap(interpreter) ``` ## For the curious ones: what is Free in theory? Mathematically-speaking, a *free monad* (at least in the programming language context) is a construction that is left adjoint to a forgetful functor whose domain is the category of Monads and whose co-domain is the category of Endofunctors. Huh? Concretely, **it is just a clever construction that allows us to build a _very simple_ Monad from any _functor_**. The above forgetful functor takes a `Monad` and: - forgets its *monadic* part (e.g. the `flatMap` function) - forgets its *pointed* part (e.g. the `pure` function) - finally keeps the *functor* part (e.g. the `map` function) By reversing all arrows to build the left-adjoint, we deduce that the free monad is basically a construction that: - takes a *functor* - adds the *pointed* part (e.g. `pure`) - adds the *monadic* behavior (e.g. `flatMap`) In terms of implementation, to build a *monad* from a *functor* we use the following classic inductive definition: ```scala sealed abstract class Free[F[_], A] case class Pure[F[_], A](a: A) extends Free[F, A] case class Suspend[F[_], A](a: F[Free[F, A]]) extends Free[F, A] ``` (_This generalizes the concept of fixed point functor_.) In this representation: - `Pure` builds a `Free` instance from an `A` value (it _reifies_ the `pure` function) - `Suspend` builds a new `Free` by applying `F` to a previous `Free` (it _reifies_ the `flatMap` function) So a typical `Free` structure might look like: ```scala Suspend(F(Suspend(F(Suspend(F(....(Pure(a)))))))) ``` `Free` is a recursive structure. It uses `A` in `F[A]` as the recursion "carrier" with a terminal element `Pure`. From a computational point of view, `Free` recursive structure can be seen as a sequence of operations. - `Pure` returns an `A` value and ends the entire computation. - `Suspend` is a continuation; it suspends the current computation with the suspension functor `F` (which can represent a command for example) and hands control to the caller. `A` represents a value bound to this computation. Please note this `Free` construction has the interesting quality of _encoding_ the recursion on the heap instead of the stack as classic function calls would. This provides the stack-safety we heard about earlier, allowing very large `Free` structures to be evaluated safely. ### For the very curious ones If you look at implementation in cats, you will see another member of the `Free[_]` ADT: ```scala case class FlatMapped[S[_], B, C](c: Free[S, C], f: C => Free[S, B]) extends Free[S, B] ``` `FlatMapped` represents a call to a subroutine `c` and when `c` is finished, it continues the computation by calling the function `f` with the result of `c`. It is actually an optimization of `Free` structure allowing to solve a problem of quadratic complexity implied by very deep recursive `Free` computations. It is exactly the same problem as repeatedly appending to a `List[_]`. As the sequence of operations becomes longer, the slower a `flatMap` "through" the structure will be. With `FlatMapped`, `Free` becomes a right-associated structure not subject to quadratic complexity. ## FreeT Often times we want to interleave the syntax tree when building a Free monad with some other effect not declared as part of the ADT. FreeT solves this problem by allowing us to mix building steps of the AST with calling action in other base monad. In the following example a basic console application is shown. When the user inputs some text we use a separate `State` monad to track what the user typed. As we can observe in this case `FreeT` offers us the alternative to delegate denotations to `State` monad with stronger equational guarantees than if we were emulating the `State` ops in our own ADT. ``` /* Detailed source-code truncated for AI context efficiency. */ ``` Another example is more basic usage of `FreeT` with some context `Ctx` for which we provide `Try` interpreter, combined with `OptionT` for reducing boilerplate. ```scala mdoc:reset import cats.free._ import cats._ import cats.data._ import cats.syntax.all._ import scala.util.Try sealed trait Ctx[A] case class Action(value: Int) extends Ctx[Int] def op1: FreeT[Ctx, Option, Int] = FreeT.liftF[Ctx, Option, Int](Action(7)) def op2: FreeT[Ctx, Option, Int] = FreeT.liftT[Ctx, Option, Int](Some(4)) def op3: FreeT[Ctx, Option, Int] = FreeT.pure[Ctx, Option, Int](1) val opComplete: FreeT[Ctx, Option, Int] = for { a <- op1 b <- op2 c <- op3 } yield a + b + c /* Our interpreters */ type OptTry[A] = OptionT[Try, A] def tryInterpreter: Ctx ~> OptTry = new (Ctx ~> OptTry) { def apply[A](fa: Ctx[A]): OptTry[A] = { fa match { case Action(value) => OptionT.liftF(Try(value)) } } } def optTryLift: Option ~> OptTry = new (Option ~> OptTry) { def apply[A](fa: Option[A]): OptTry[A] = { fa match { case Some(value) => OptionT(Try(Option(value))) case None => OptionT.none } } } val hoisted = opComplete.hoist(optTryLift) val evaluated = hoisted.foldMap(tryInterpreter) val result = evaluated.value ``` ## Future Work (TODO) There are many remarkable uses of `Free[_]`. In the future, we will include some here, such as: - Trampoline - Option - Iteratee - Source - etc... We will also discuss the *Coyoneda Trick*. ## Credits This article was written by [Pascal Voitot](https://github.com/mandubian) and edited by other members of the Cats community. --- ### Datatypes Functionk (`docs/datatypes/functionk.md`) # FunctionK API Documentation: @:api(cats.arrow.FunctionK) A `FunctionK` transforms values from one first-order-kinded type (a type that takes a single type parameter, such as `List` or `Option`) into another first-order-kinded type. This transformation is universal, meaning that a `FunctionK[List, Option]` will translate all `List[A]` values into an `Option[A]` value for all possible types of `A`. This explanation may be easier to understand if we first step back and talk about ordinary functions. ## Ordinary Functions Consider the following scala method: ```scala mdoc:silent def first(l: List[Int]): Option[Int] = l.headOption ``` This isn't a particularly helpful method, but it will work as an example. Instead of writing this as a method, we could have written this as a function _value_: ```scala mdoc:silent val first: List[Int] => Option[Int] = l => l.headOption ``` And here, `=>` is really just some syntactic sugar for `Function1`, so we could also write that as: ```scala mdoc:silent:reset val first: Function1[List[Int], Option[Int]] = l => l.headOption ```` Let's cut through the syntactic sugar even a little bit further. `Function1` isn't really a special type. It's just a trait that looks something like this: ```scala mdoc:silent // we are calling this `MyFunction1` so we don't collide with the actual `Function1` trait MyFunction1[A, B] { def apply(a: A): B } ``` So if we didn't mind being a bit verbose, we could have written our function as: ```scala mdoc:silent:reset val first: Function1[List[Int], Option[Int]] = new Function1[List[Int], Option[Int]] { def apply(l: List[Int]): Option[Int] = l.headOption } ``` ## Abstracting via Generics Recall our `first` method: ```scala mdoc:silent:reset def first(l: List[Int]): Option[Int] = l.headOption ``` The astute reader may have noticed that there's really no reason that this method needs to be tied directly to `Int`. We could use generics to make this a bit more general: ```scala mdoc:silent:reset def first[A](l: List[A]): Option[A] = l.headOption ``` But how would we represent this new `first` method as a `=>`/`Function1` value? We are looking for something like a type of `List[A] => Option[A] forAll A`, but this isn't valid scala syntax. `Function1` isn't quite the right fit, because its `apply` method doesn't take a generic type parameter. ## Higher Kinds to the Rescue It turns out that we can represent our universal `List` to `Option` transformation with something that looks a bit like `Function1` but that adds a type parameter to the `apply` method and utilizes higher kinds: ```scala mdoc:silent:reset trait MyFunctionK[F[_], G[_]] { def apply[A](fa: F[A]): G[A] } ``` Cats provides this type as `FunctionK` (we used `MyFunctionK` for our example type to avoid confusion). So now we can write `first` as a `FunctionK[List, Option]` value: ```scala mdoc:silent import cats.arrow.FunctionK val first: FunctionK[List, Option] = new FunctionK[List, Option] { def apply[A](l: List[A]): Option[A] = l.headOption } ``` ## Syntactic Sugar If the example above looks a bit too verbose for you, the [kind-projector](https://github.com/typelevel/kind-projector) compiler plugin [provides](https://github.com/typelevel/kind-projector#polymorphic-lambda-values) a more concise syntax. After adding the plugin to your project, you could write the `first` example as: ```scala mdoc:silent:nest val first: FunctionK[List, Option] = λ[FunctionK[List, Option]](_.headOption) ``` Cats also provides a `~>` type alias for `FunctionK`, so an even more concise version would be: ```scala mdoc:silent:nest import cats.~> val first: List ~> Option = λ[List ~> Option](_.headOption) ``` Being able to use `~>` as an alias for `FunctionK` parallels being able to use `=>` as an alias for `Function1`. ## Use-cases `FunctionK` tends to show up when there is abstraction over higher-kinds. For example, interpreters for [free monads](freemonad.md) and [free applicatives](freeapplicative.md) are represented as `FunctionK` instances. ## Types with more than one type parameter Earlier it was mentioned that `FunctionK` operates on first-order-kinded types (types that take a single type parameter such as `List` or `Option`). It's still possible to use `FunctionK` with types that would normally take more than one type parameter (such as `Either`) if we fix all of the type parameters except for one. For example: ```scala mdoc:silent type ErrorOr[A] = Either[String, A] val errorOrFirst: FunctionK[List, ErrorOr] = λ[FunctionK[List, ErrorOr]](_.headOption.toRight("ERROR: the list was empty!")) ``` ## Natural Transformation In category theory, a [natural transformation](https://ncatlab.org/nlab/show/natural+transformation) provides a morphism between Functors while preserving the internal structure. It's one of the most fundamental notions of category theory. If we have two Functors `F` and `G`, `FunctionK[F, G]` is a natural transformation via parametricity. That is, given `fk: FunctionK[F, G]`, for all functions `A => B` and all `fa: F[A]` the following are equivalent: ```Scala fk(F.map(fa)(f)) <-> G.map(fk(fa))(f) ``` We don't need to write a law to test the implementation of the `fk` for the above to be true. It's automatically given by parametricity. Thus natural transformation can be implemented in terms of `FunctionK`. This is why a parametric polymorphic function `FunctionK[F, G]` is sometimes referred as a natural transformation. However, they are two different concepts that are not isomorphic. For more details, Bartosz Milewski has written a great blog post titled ["Parametricity: Money for Nothing and Theorems for Free"](https://bartoszmilewski.com/2014/09/22/parametricity-money-for-nothing-and-theorems-for-free/). --- ### Datatypes Id (`docs/datatypes/id.md`) # Id API Documentation: [Id](@API_LINK_BASE@cats/index.html#Id[A]=A) The identity monad can be seen as the ambient monad that encodes the effect of having no effect. It is ambient in the sense that plain pure values are values of `Id`. It is encoded as: ```scala type Id[A] = A ``` That is to say that the type `Id[A]` is just a synonym for `A`. We can freely treat values of type `A` as values of type `Id[A]`, and vice-versa. ```scala mdoc import cats._ val x: Id[Int] = 1 val y: Int = x ``` Using this type declaration, we can treat our Id type constructor as a [`Monad`](../typeclasses/monad.md) and as a [`Comonad`](../typeclasses/comonad.md). The `pure` method, which has type `A => Id[A]` just becomes the identity function. The `map` method from `Functor` just becomes function application: ```scala mdoc import cats.Functor val one: Int = 1 Functor[Id].map(one)(_ + 1) ``` Compare the signatures of `map` and `flatMap` and `coflatMap`: ```scala def map[A, B](fa: Id[A])(f: A => B): Id[B] def flatMap[A, B](fa: Id[A])(f: A => Id[B]): Id[B] def coflatMap[A, B](a: Id[A])(f: Id[A] => B): Id[B] ``` You'll notice that in the flatMap signature, since `Id[B]` is the same as `B` for all `B`, we can rewrite the type of the `f` parameter to be `A => B` instead of `A => Id[B]`, and this makes the signatures of the two functions the same, and, in fact, they can have the same implementation, meaning that for `Id`, `flatMap` is also just function application: ```scala mdoc:nest import cats.Monad val one: Int = 1 Monad[Id].map(one)(_ + 1) Monad[Id].flatMap(one)(_ + 1) ``` And that similarly, `coflatMap` is just function application: ```scala mdoc import cats.Comonad Comonad[Id].coflatMap(one)(_ + 1) ``` --- ### Datatypes Ior (`docs/datatypes/ior.md`) # Ior API Documentation: @:api(cats.data.Ior) `Ior` represents an inclusive-or relationship between two data types. This makes it very similar to the `Either` data type, which represents an "exclusive-or" relationship. What this means, is that an `Ior[A, B]` (also written as `A Ior B`) can contain either an `A`, a `B`, or both an `A` and `B`. Another similarity to `Either` is that `Ior` is right-biased, which means that the `map` and `flatMap` functions will work on the right side of the `Ior`, in our case the `B` value. You can see this in the function signature of `map`: ```scala def map[B, C](fa: A Ior B)(f: B => C): A Ior C ``` We can create `Ior` values using `Ior.left`, `Ior.right` and `Ior.both`: ```scala mdoc import cats.data._ val right = Ior.right[String, Int](3) val left = Ior.left[String, Int]("Error") val both = Ior.both("Warning", 3) ``` Cats also offers syntax enrichment for `Ior`. The `leftIor` and `rightIor` functions can be imported from `cats.syntax.ior._`: ```scala mdoc:nest import cats.syntax.all._ val right = 3.rightIor val left = "Error".leftIor ``` When we look at the `Monad` or `Applicative` instances of `Ior`, we can see that they actually require a `Semigroup` instance on the left side. This is because `Ior` will actually accumulate failures on the left side, very similar to how the [`Validated`](validated.md) data type does. This means we can accumulate data on the left side while also being able to short-circuit upon the first left-side-only value. For example, sometimes, we might want to accumulate warnings together with a valid result and only halt the computation on a "hard error" Here's an example of how we might be able to do that: ```scala mdoc:reset-object:silent import cats.syntax.all._ import cats.data.{ NonEmptyChain => Nec, Ior} type Failures = Nec[String] case class Username(value: String) extends AnyVal case class Password(value: String) extends AnyVal case class User(name: Username, pw: Password) def validateUsername(u: String): Failures Ior Username = { if (u.isEmpty) Nec.one("Can't be empty").leftIor else if (u.contains(".")) Ior.both(Nec.one("Dot in name is deprecated"), Username(u)) else Username(u).rightIor } def validatePassword(p: String): Failures Ior Password = { if (p.length < 8) Nec.one("Password too short").leftIor else if (p.length < 10) Ior.both(Nec.one("Password should be longer"), Password(p)) else Password(p).rightIor } def validateUser(name: String, password: String): Failures Ior User = (validateUsername(name), validatePassword(password)).mapN(User) ``` Now we're able to validate user data and also accumulate non-fatal warnings: ```scala mdoc validateUser("John", "password12") validateUser("john.doe", "password") validateUser("jane", "short") ``` To extract the values, we can use the `fold` method, which expects a function for each case the `Ior` can represent: ```scala mdoc validateUser("john.doe", "password").fold( errorNec => s"Error: ${errorNec.head}", user => s"Success: $user", (warnings, user) => s"Warning: ${user.name.value}; The following warnings occurred: ${warnings.show}" ) ``` ## Using with NonEmptyChain Similar to [Validated](validated.md), there is also a type alias for using a `NonEmptyChain` on the left side. ```scala mdoc:silent import cats.data.NonEmptyChain type IorNec[B, A] = Ior[NonEmptyChain[B], A] ``` ```scala mdoc:nest import cats.syntax.all._, cats.data.NonEmptyChain val left: IorNec[String, Int] = Ior.fromEither("Error".leftNec[Int]) ``` ## Conversions We can also convert our `Ior` to `Either`, `Validated` or `Option`. All of these conversions will discard the left side value if both are available: ```scala mdoc Ior.both("Warning", 42).toEither ``` --- ### Datatypes Iort (`docs/datatypes/iort.md`) # IorT API Documentation: @:api(cats.data.IorT) `IorT[F[_], A, B]` is a light wrapper on an `F[Ior[A, B]]`. Similar to `OptionT[F[_], A]` and `EitherT[F[_], A, B]`, it is a monad transformer for `Ior`, that can be more convenient to work with than using `F[Ior[A, B]]` directly. ## The boilerplate Consider the following program that uses `Ior` to propagate log messages when validating an address: ```scala mdoc:silent import cats.data.Ior import cats.data.{ NonEmptyChain => Nec } import cats.syntax.all._ import scala.util.{Success, Try} type Logs = Nec[String] def parseNumber(input: String): Ior[Logs, Option[Int]] = Try(input.trim.toInt) match { case Success(number) if number > 0 => Ior.Right(Some(number)) case Success(_) => Ior.Both(Nec.one(s"'$input' is non-positive number"), None) case _ => Ior.Both(Nec.one(s"'$input' is not a number"), None) } def parseStreet(input: String): Ior[Logs, String] = { if (input.trim.isEmpty) Ior.Left(Nec.one(s"'$input' is not a street")) else Ior.Right(input) } def numberToString(number: Option[Int]): Ior[Logs, String] = number match { case Some(n) => Ior.Right(n.toString) case None => Ior.Both(Nec.one("used default address number"), "n/a") } def addressProgram(numberInput: String, streetInput: String): Ior[Logs, String] = for { number <- parseNumber(numberInput) street <- parseStreet(streetInput) sNumber <- numberToString(number) } yield s"$sNumber, $street" ``` Due to the monadic nature of `Ior` combining the results of `parseNumber`, `parseStreet`, and `numberToString` can be as concise as a for-comprehension. As the following examples demonstrate, log messages of the different processing steps are combined when using `flatMap`. ```scala mdoc addressProgram("7", "Buckingham Palace Rd") addressProgram("SW1W", "Buckingham Palace Rd") addressProgram("SW1W", "") ``` Suppose `parseNumber`, `parseStreet`, and `numberToString` are rewritten to be asynchronous and return `Future[Ior[Logs, *]]` instead. The for-comprehension can no longer be used since `addressProgram` must now compose `Future` and `Ior` together, which means that the error handling must be performed explicitly to ensure that the proper types are returned: ```scala mdoc:silent import scala.concurrent.Future import scala.concurrent.ExecutionContext.Implicits.global def parseNumberAsync(input: String): Future[Ior[Logs, Option[Int]]] = Future.successful(parseNumber(input)) def parseStreetAsync(input: String): Future[Ior[Logs, String]] = Future.successful(parseStreet(input)) def numberToStringAsync(number: Option[Int]): Future[Ior[Logs, String]] = Future.successful(numberToString(number)) def programHelper(number: Option[Int], streetInput: String): Future[Ior[Logs, String]] = parseStreetAsync(streetInput).flatMap { streetIor => numberToStringAsync(number).map { sNumberIor => for { street <- streetIor sNumber <- sNumberIor } yield s"$sNumber, $street" } } def addressProgramAsync(numberInput: String, streetInput: String): Future[Ior[Logs, String]] = parseNumberAsync(numberInput).flatMap { case Ior.Left(logs) => Future.successful(Ior.Left(logs)) case Ior.Right(number) => programHelper(number, streetInput) case b @ Ior.Both(_, number) => programHelper(number, streetInput).map(s => b.flatMap(_ => s)) } ``` To keep some readability the program was split in two parts, otherwise code would be repeated. Note that when `parseNumberAsync` returns an `Ior.Both` it is necessary to combine it with the result of `programHelper`, otherwise some log messages would be lost. ```scala mdoc import scala.concurrent.Await import scala.concurrent.duration._ Await.result(addressProgramAsync("7", "Buckingham Palace Rd"), 1.second) Await.result(addressProgramAsync("SW1W", "Buckingham Palace Rd"), 1.second) Await.result(addressProgramAsync("SW1W", ""), 1.second) ``` ## IorT to the rescue The program of the previous section can be re-written using `IorT` as follows: ```scala mdoc:silent import cats.data.IorT def addressProgramAsyncIorT(numberInput: String, streetInput: String): IorT[Future, Logs, String] = for { number <- IorT(parseNumberAsync(numberInput)) street <- IorT(parseStreetAsync(streetInput)) sNumber <- IorT(numberToStringAsync(number)) } yield s"$sNumber, $street" ``` This version of `addressProgramAsync` is almost as concise as the non-asynchronous version. Note that when `F` is a monad, then `IorT` will also form a monad, allowing monadic combinators such as `flatMap` to be used in composing `IorT` values. ```scala mdoc Await.result(addressProgramAsyncIorT("7", "Buckingham Palace Rd").value, 1.second) Await.result(addressProgramAsyncIorT("SW1W", "Buckingham Palace Rd").value, 1.second) Await.result(addressProgramAsyncIorT("SW1W", "").value, 1.second) ``` Looking back at the implementation of `parseStreet` the return type could be `Either[Logs, String]` instead. Thinking of situations like this, where not all the types match, `IorT` provides factory methods of `IorT[F, A, B]` from: * `A`, `B` or both * `F[A]`, `F[B]` or both * `Ior[A, B]` or `F[Ior[A, B]]` * `Either[A, B]` or `F[Either[A, B]]` * `Option[B]` or `F[Option[B]]` * A `Boolean` test ## From `A` and/or `B` to `IorT[F, A, B]` To obtain a left version of `IorT` when given an `A` use `IorT.leftT`. When given a `B` a right version of `IorT` can be obtained with `IorT.rightT` (which is an alias for `IorT.pure`). Given both an `A` and a `B` use `IorT.bothT`. Note the two styles for providing the `IorT` missing type parameters. The first three expressions only specify not inferable types, while the last expression specifies all types. ```scala mdoc:nest val number = IorT.rightT[Option, String](5) val error = IorT.leftT[Option, Int]("Not a number") val weirdNumber = IorT.bothT[Option]("Not positive", -1) val numberPure: IorT[Option, String, Int] = IorT.pure(5) ``` ## From `F[A]` and/or `F[B]` to `IorT[F, A, B]` Similarly, use `IorT.left`, `IorT.right`, `IorT.both` to convert an `F[A]` and/or `F[B]` into an `IorT`. It is also possible to use `IorT.liftF` as an alias for `IorT.right`. ```scala mdoc:silent:nest val numberF: Option[Int] = Some(5) val errorF: Option[String] = Some("Not a number") val warningF: Option[String] = Some("Not positive") val weirdNumberF: Option[Int] = Some(-1) val number: IorT[Option, String, Int] = IorT.right(numberF) val error: IorT[Option, String, Int] = IorT.left(errorF) val weirdNumber: IorT[Option, String, Int] = IorT.both(warningF, weirdNumberF) ``` ## From `Ior[A, B]` or `F[Ior[A, B]]` to `IorT[F, A, B]` Use `IorT.fromIor` to a lift a value of `Ior[A, B]` into `IorT[F, A, B]`. An `F[Ior[A, B]]` can be converted into `IorT` using the `IorT` constructor. ```scala mdoc:silent:nest val numberIor: Ior[String, Int] = Ior.Right(5) val errorIor: Ior[String, Int] = Ior.Left("Not a number") val weirdNumberIor: Ior[String, Int] = Ior.both("Not positive", -1) val numberFIor: Option[Ior[String, Int]] = Option(Ior.Right(5)) val number: IorT[Option, String, Int] = IorT.fromIor(numberIor) val error: IorT[Option, String, Int] = IorT.fromIor(errorIor) val weirdNumber: IorT[Option, String, Int] = IorT.fromIor(weirdNumberIor) val numberF: IorT[Option, String, Int] = IorT(numberFIor) ``` ## From `Either[A, B]` or `F[Either[A, B]]` to `IorT[F, A, B]` Use `IorT.fromEither` or `IorT.fromEitherF` to create a value of `IorT[F, A, B]` from an `Either[A, B]` or a `F[Either[A, B]]`, respectively. ```scala mdoc:silent:nest val numberEither: Either[String, Int] = Right(5) val errorEither: Either[String, Int] = Left("Not a number") val numberFEither: Option[Either[String, Int]] = Option(Right(5)) val number: IorT[Option, String, Int] = IorT.fromEither(numberEither) val error: IorT[Option, String, Int] = IorT.fromEither(errorEither) val numberF: IorT[Option, String, Int] = IorT.fromEitherF(numberFEither) ``` ## From `Option[B]` or `F[Option[B]]` to `IorT[F, A, B]` An `Option[B]` or an `F[Option[B]]`, along with a default value, can be passed to `IorT.fromOption` and `IorT.fromOptionF`, respectively, to produce an `IorT`. For `F[Option[B]]` and default `F[A]`, there is `IorT.fromOptionM`. ```scala mdoc:silent:nest val numberOption: Option[Int] = None val numberFOption: List[Option[Int]] = List(None, Some(2), None, Some(5)) val number = IorT.fromOption[List](numberOption, "Not defined") val numberF = IorT.fromOptionF(numberFOption, "Not defined") val numberM = IorT.fromOptionM(numberFOption, List("Not defined")) ``` ## Creating an `IorT[F, A, B]` from a `Boolean` test `IorT.cond` allows concise creation of an `IorT[F, A, B]` based on a `Boolean` test, an `A` and a `B`. Similarly, `IorT.condF` uses `F[A]` and `F[B]`. ```scala mdoc:silent:nest val number: Int = 10 val informedNumber: IorT[Option, String, Int] = IorT.cond(number % 10 != 0, number, "Number is multiple of 10") val uninformedNumber: IorT[Option, String, Int] = IorT.condF(number % 10 != 0, Some(number), None) ``` ## Extracting an `F[Ior[A, B]]` from an `IorT[F, A, B]` Use the `value` method defined on `IorT` to retrieve the underlying `F[Ior[A, B]]`: ```scala mdoc:silent:nest val errorT: IorT[Option, String, Int] = IorT.leftT("Not a number") val error: Option[Ior[String, Int]] = errorT.value ``` --- ### Datatypes Kleisli (`docs/datatypes/kleisli.md`) # Kleisli API Documentation: @:api(cats.data.Kleisli) Kleisli enables composition of functions that return a monadic value, for instance an `Option[Int]` or a `Either[String, List[Double]]`, without having functions take an `Option` or `Either` as a parameter, which can be strange and unwieldy. We may also have several functions which depend on some environment and want a nice way to compose these functions to ensure they all receive the same environment. Or perhaps we have functions which depend on their own "local" configuration and all the configurations together make up a "global" application configuration. How do we have these functions play nice with each other despite each only knowing about their own local requirements? These situations are where `Kleisli` is immensely helpful. ## Functions One of the most useful properties of functions is that they **compose**. That is, given a function `A => B` and a function `B => C`, we can combine them to create a new function `A => C`. It is through this compositional property that we are able to write many small functions and compose them together to create a larger one that suits our needs. ```scala mdoc:silent val twice: Int => Int = x => x * 2 val countCats: Int => String = x => if (x == 1) "1 cat" else s"$x cats" val twiceAsManyCats: Int => String = twice andThen countCats // equivalent to: countCats compose twice ``` Thus. ```scala mdoc twiceAsManyCats(1) // "2 cats" ``` Sometimes, our functions will need to return monadic values. For instance, consider the following set of functions. ```scala mdoc:silent val parse: String => Option[Int] = s => if (s.matches("-?[0-9]+")) Some(s.toInt) else None val reciprocal: Int => Option[Double] = i => if (i != 0) Some(1.0 / i) else None ``` As it stands we cannot use `Function1.compose` (or `Function1.andThen`) to compose these two functions. The output type of `parse` is `Option[Int]` whereas the input type of `reciprocal` is `Int`. This is where `Kleisli` comes into play. ## Kleisli At its core, `Kleisli[F[_], A, B]` is just a wrapper around the function `A => F[B]`. Depending on the properties of the `F[_]`, we can do different things with `Kleisli`s. For instance, if `F[_]` has a `FlatMap[F]` instance (we can call `flatMap` on `F[A]` values), we can compose two `Kleisli`s much like we can two functions. ```scala mdoc:silent import cats.FlatMap import cats.syntax.all._ final case class Kleisli[F[_], A, B](run: A => F[B]) { def compose[Z](k: Kleisli[F, Z, A])(implicit F: FlatMap[F]): Kleisli[F, Z, B] = Kleisli[F, Z, B](z => k.run(z).flatMap(run)) } ``` Returning to our earlier example: ```scala mdoc:silent:nest // Bring in cats.FlatMap[Option] instance import cats.syntax.all._ val parse: Kleisli[Option,String,Int] = Kleisli((s: String) => if (s.matches("-?[0-9]+")) Some(s.toInt) else None) val reciprocal: Kleisli[Option,Int,Double] = Kleisli((i: Int) => if (i != 0) Some(1.0 / i) else None) val parseAndReciprocal: Kleisli[Option,String,Double] = reciprocal.compose(parse) ``` `Kleisli#andThen` can be defined similarly. It is important to note that the `F[_]` having a `FlatMap` (or a `Monad`) instance is not a hard requirement - we can do useful things with weaker requirements. Such an example would be `Kleisli#map`, which only requires that `F[_]` have a `Functor` instance (e.g. is equipped with `map: F[A] => (A => B) => F[B]`). ```scala mdoc:silent:nest import cats.Functor final case class Kleisli[F[_], A, B](run: A => F[B]) { def map[C](f: B => C)(implicit F: Functor[F]): Kleisli[F, A, C] = Kleisli[F, A, C](a => F.map(run(a))(f)) } ``` Below are some more methods on `Kleisli` that can be used as long as the constraint on `F[_]` is satisfied. ``` Method | Constraint on `F[_]` --------- | ------------------- andThen | FlatMap compose | FlatMap flatMap | FlatMap lower | Monad map | Functor traverse | Applicative ``` ### Type class instances The type class instances for `Kleisli`, like that for functions, often fix the input type (and the `F[_]`) and leave the output type free. What type class instances it has tends to depend on what instances the `F[_]` has. For instance, `Kleisli[F, A, B]` has a `Functor` instance as long as the chosen `F[_]` does. It has a `Monad` instance as long as the chosen `F[_]` does. The instances in Cats are laid out in a way such that implicit resolution will pick up the most specific instance it can (depending on the `F[_]`). An example of a `Monad` instance for `Kleisli` is shown below. *Note*: the example below assumes usage of the [kind-projector compiler plugin](https://github.com/typelevel/kind-projector) and will not compile if it is not being used in a project. ```scala mdoc:silent import cats.syntax.all._ // We can define a FlatMap instance for Kleisli if the F[_] we chose has a FlatMap instance // Note the input type and F are fixed, with the output type left free implicit def kleisliFlatMap[F[_], Z](implicit F: FlatMap[F]): FlatMap[Kleisli[F, Z, *]] = new FlatMap[Kleisli[F, Z, *]] { def flatMap[A, B](fa: Kleisli[F, Z, A])(f: A => Kleisli[F, Z, B]): Kleisli[F, Z, B] = Kleisli(z => fa.run(z).flatMap(a => f(a).run(z))) def map[A, B](fa: Kleisli[F, Z, A])(f: A => B): Kleisli[F, Z, B] = Kleisli(z => fa.run(z).map(f)) def tailRecM[A, B](a: A)(f: A => Kleisli[F, Z, Either[A, B]]) = Kleisli[F, Z, B]({ z => FlatMap[F].tailRecM(a) { f(_).run(z) } }) } ``` Below is a table of some of the type class instances `Kleisli` can have depending on what instances `F[_]` has. ``` Type class | Constraint on `F[_]` -------------- | ------------------- Functor | Functor Apply | Apply Applicative | Applicative FlatMap | FlatMap Monad | Monad Arrow | Monad Split | FlatMap Strong | Functor SemigroupK* | FlatMap MonoidK* | Monad ``` *These instances only exist for Kleisli arrows with identical input and output types; that is, `Kleisli[F, A, A]` for some type A. These instances use Kleisli composition as the `combine` operation, and `Monad.pure` as the `empty` value. Also, there is an instance of `Monoid[Kleisli[F, A, B]]` if there is an instance of `Monoid[F[B]]`. `Monoid.combine` here creates a new Kleisli arrow which takes an `A` value and feeds it into each of the combined Kleisli arrows, which together return two `F[B]` values. Then, they are combined into one using the `Monoid[F[B]]` instance. ## Other uses ### Monad Transformers Many data types have a monad transformer equivalent that allows us to compose the `Monad` instance of the data type with any other `Monad` instance. For instance, `OptionT[F[_], A]` allows us to compose the monadic properties of `Option` with any other `F[_]`, such as a `List`. This allows us to work with nested contexts/effects in a nice way (for example, in for-comprehensions). `Kleisli` can be viewed as the monad transformer for functions. Recall that at its essence, `Kleisli[F, A, B]` is just a function `A => F[B]`, with niceties to make working with the value we actually care about, the `B`, easy. `Kleisli` allows us to take the effects of functions and have them play nice with the effects of any other `F[_]`. This may raise the question, what exactly is the "effect" of a function? Well, if we take a look at any function, we can see it takes some input and produces some output with it, without having touched the input (assuming the function is pure, i.e. [referentially transparent](https://en.wikipedia.org/wiki/Referential_transparency_%28computer_science%29)). That is, we take a read-only value, and produce some value with it. For this reason, the type class instances for functions often refer to the function as a `Reader`. For instance, it is common to hear about the `Reader` monad. In the same spirit, Cats defines a `Reader` type alias along the lines of: ```scala mdoc:silent // We want A => B, but Kleisli provides A => F[B]. To make the types/shapes match, // we need an F[_] such that providing it a type A is equivalent to A // This can be thought of as the type-level equivalent of the identity function type Id[A] = A type Reader[A, B] = Kleisli[Id, A, B] object Reader { // Lifts a plain function A => B into a Kleisli, giving us access // to all the useful methods and type class instances def apply[A, B](f: A => B): Reader[A, B] = Kleisli[Id, A, B](f) } type ReaderT[F[_], A, B] = Kleisli[F, A, B] val ReaderT = Kleisli ``` The `ReaderT` value alias exists to allow users to use the `Kleisli` companion object as if it were `ReaderT`, if they were so inclined. The topic of functions as a read-only environment brings us to our next common use case of `Kleisli` - configuration. ### Configuration Functional programming advocates the creation of programs and modules by composing smaller, simpler modules. This philosophy intentionally mirrors that of function composition - write many small functions, and compose them to build larger ones. After all, our programs are just functions. Let's look at some example modules, where each module has its own configuration that is validated by a function. If the configuration is good, we return a `Some` of the module, otherwise a `None`. This example uses `Option` for simplicity - if you want to provide error messages or other failure context, consider using `Either` instead. ```scala mdoc:silent case class DbConfig(url: String, user: String, pass: String) trait Db object Db { val fromDbConfig: Kleisli[Option, DbConfig, Db] = ??? } case class ServiceConfig(addr: String, port: Int) trait Service object Service { val fromServiceConfig: Kleisli[Option, ServiceConfig, Service] = ??? } ``` We have two independent modules, a `Db` (allowing access to a database) and a `Service` (supporting an API to provide data over the web). Both depend on their own configuration parameters. Neither know or care about the other, as it should be. However our application needs both of these modules to work. It is plausible we then have a more global application configuration. ```scala mdoc:silent case class AppConfig(dbConfig: DbConfig, serviceConfig: ServiceConfig) class App(db: Db, service: Service) ``` As it stands, we cannot use both `Kleisli` validation functions together nicely - one takes a `DbConfig`, the other a `ServiceConfig`. That means the `FlatMap` (and by extension, the `Monad`) instances differ (recall the input type is fixed in the type class instances). However, there is a nice function on `Kleisli` called `local`. ```scala mdoc:silent:nest final case class Kleisli[F[_], A, B](run: A => F[B]) { def local[AA](f: AA => A): Kleisli[F, AA, B] = Kleisli(f.andThen(run)) } ``` What `local` allows us to do is essentially "expand" our input type to a more "general" one. In our case, we can take a `Kleisli` that expects a `DbConfig` or `ServiceConfig` and turn it into one that expects an `AppConfig`, as long as we tell it how to go from an `AppConfig` to the other configs. Now we can create our application config validator! ```scala mdoc:silent:nest final case class Kleisli[F[_], Z, A](run: Z => F[A]) { def flatMap[B](f: A => Kleisli[F, Z, B])(implicit F: FlatMap[F]): Kleisli[F, Z, B] = Kleisli(z => F.flatMap(run(z))(a => f(a).run(z))) def map[B](f: A => B)(implicit F: Functor[F]): Kleisli[F, Z, B] = Kleisli(z => F.map(run(z))(f)) def local[ZZ](f: ZZ => Z): Kleisli[F, ZZ, A] = Kleisli(f.andThen(run)) } case class DbConfig(url: String, user: String, pass: String) trait Db object Db { val fromDbConfig: Kleisli[Option, DbConfig, Db] = ??? } case class ServiceConfig(addr: String, port: Int) trait Service object Service { val fromServiceConfig: Kleisli[Option, ServiceConfig, Service] = ??? } case class AppConfig(dbConfig: DbConfig, serviceConfig: ServiceConfig) class App(db: Db, service: Service) def appFromAppConfig: Kleisli[Option, AppConfig, App] = for { db <- Db.fromDbConfig.local[AppConfig](_.dbConfig) sv <- Service.fromServiceConfig.local[AppConfig](_.serviceConfig) } yield new App(db, sv) ``` What if we need a module that doesn't need any config validation, say a strategy to log events? We would have such a module be instantiated from a config directly, without an `Option` - we would have something like `Kleisli[Id, LogConfig, Log]` (alternatively, `Reader[LogConfig, Log]`). However, this won't play nice with our other `Kleisli`s since those use `Option` instead of `Id`. We can define a `lift` method on `Kleisli` (available already on `Kleisli` in Cats) that takes a type parameter `G[_]` such that `G` has an `Applicative` instance and lifts a `Kleisli` value such that its output type is `G[F[B]]`. This allows us to then lift a `Reader[A, B]` into a `Kleisli[G, A, B]`. Note that lifting a `Reader[A, B]` into some `G[_]` is equivalent to having a `Kleisli[G, A, B]` since `Reader[A, B]` is just a type alias for `Kleisli[Id, A, B]`, and `type Id[A] = A` so `G[Id[A]]` is equivalent to `G[A]`. --- ### Datatypes Nel (`docs/datatypes/nel.md`) # NonEmptyList API Documentation: @:api(cats.data.NonEmptyList) ## Motivation We start with two examples of `NonEmptyList`s ### Usage in `Validated` and `Ior` If you have had the opportunity of taking a look to [Validated](validated.md) or [Ior](ior.md), you'll find that a common case is to use `NonEmptyList` with one of these data structures. Why? Because it fits nicely in the error reporting cases. As stated by its name, `NonEmptyList` is a _specialized_ data type that has at least *one* element. Otherwise it behaves like a normal `List`. For sum types like `Validated` (and `Ior`), it does not make sense to have a `Invalid` with no errors: no errors means it is a `Valid`! By using `NonEmptyList`, we explicitly say in the type that: *If* it is a `Invalid`, *then* there is at least one error. This is much more precise and we don't have to wonder whether the list of errors might be empty when reporting them later on. ### Avoiding `Option` by demanding more specific arguments As functional programmers, we naturally shy away from partial functions that can throw exceptions like the famous `head` method on, e.g., `List`. Let's take as an example a method that calculates the `average`: ```scala mdoc:silent def average(xs: List[Int]): Double = { xs.sum / xs.length.toDouble } ``` Clearly, this is not a valid definition for *empty* lists, because division by zero will throw an exception. To fix this, one way is to return an `Option` instead of a `Double` right away: ```scala mdoc:silent:nest def average(xs: List[Int]): Option[Double] = if (xs.isEmpty) { None } else { Some(xs.sum / xs.length.toDouble) } ``` That works and is safe, but this only masks the problem of *accepting invalid input*. By using `Option`, we extend the `average` function with the logic to handle empty lists. Additionally, all callers have to handle the `Option` cases, maybe over and over again. While better than failing with an exception, this is far from perfect. Instead what we would like to express is that `average` does not make sense at all for an empty list. Luckily, cats defines the `NonEmptyList`. As the name says, this represents a list that *cannot*, by construction, be empty. So given a `NonEmptyList[A]` you can be sure that there is at least one `A` in there. Let's see how that impacts your `average` method: ```scala mdoc:silent:nest import cats.data.NonEmptyList def average(xs: NonEmptyList[Int]): Double = { xs.reduceLeft(_+_) / xs.length.toDouble } ``` With that, `average` is free of any "domain invariant validation" and instead can focus on the actual logic of computing the average of the list. This ties in nicely with the recommendation of shifting your validation to the very borders of your program, where the input enters your system. ## Structure of a NonEmptyList `NonEmptyList` is defined as follows: ```scala final case class NonEmptyList[+A](head: A, tail: List[A]) { // Implementation elided } ``` The `head` of the `NonEmptyList` will be _non-empty_. Meanwhile, the `tail` can have zero or more elements contained in a `List`. ## Defined for all its elements An important trait of `NonEmptyList` is the totality. For `List` specifically, both `head` and `tail` are partial: they are only well-defined if it has at least one element. `NonEmptyList` on the other hand, _guarantees_ you that operations like `head` and `tail` are defined, because constructing an empty `NonEmptyList` is simply not possible! ## Constructing a NonEmptyList To construct a `NonEmptyList` you have different possibilities. ### Using `one` If you want to construct a collection with only one argument, use `NonEmptyList.one`: ```scala mdoc NonEmptyList.one(42) ``` ### Using `of` The `NonEmptyList.of` method on the companion of `NonEmptyList` as the signature: ```scala def of[A](head: A, tail: A*): NonEmptyList[A] ``` It accepts an argument list with at least one `A` followed by a *varargs* argument for the `tail`. Call it like this: ```scala mdoc NonEmptyList.of(1) NonEmptyList.of(1, 2) NonEmptyList.of(1, 2, 3, 4) ``` There also is `ofInitLast` which takes a normal `List[A]` for the prefix and a last element: ```scala mdoc NonEmptyList.ofInitLast(List(), 4) NonEmptyList.ofInitLast(List(1,2,3), 4) ``` ### Using `fromList` There is also `NonEmptyList.fromList` which returns an `Option[NonEmptyList[A]]`: ```scala mdoc NonEmptyList.fromList(List()) NonEmptyList.fromList(List(1,2,3)) ``` Last but not least, there is `.toNel` if you import the syntax for `list`: ```scala mdoc import cats.syntax.list._ List(1,2,3).toNel ``` ### Using `fromFoldable` and `fromReducible` Even more general, you can use `NonEmptyList.fromFoldable` and `NonEmptyList.fromReducible`. The difference between the two is that `fromReducible` can avoid the `Option` in the return type, because it is only available for non-empty datastructures. Here are some examples: ```scala mdoc import cats.syntax.all._ NonEmptyList.fromFoldable(List()) NonEmptyList.fromFoldable(List(1,2,3)) NonEmptyList.fromFoldable(Vector(42)) NonEmptyList.fromFoldable(Vector(42)) // Everything that has a Foldable instance! NonEmptyList.fromFoldable(Either.left[String, Int]("Error")) NonEmptyList.fromFoldable(Either.right[String, Int](42)) // Avoid the Option for things with a `Reducible` instance import cats.data.NonEmptyVector NonEmptyList.fromReducible(NonEmptyVector.of(1, 2, 3)) ``` --- ### Datatypes Nested (`docs/datatypes/nested.md`) # Nested API Documentation: @:api(cats.data.Nested) ## Motivation In day-to-day programming we quite often end up with data inside nested effects, e.g. an integer inside an `Either`, which in turn is nested inside an `Option`: ```scala mdoc:silent import cats.data.Validated import cats.data.Validated.Valid val x: Option[Validated[String, Int]] = Some(Valid(123)) ``` This can be quite annoying to work with, as you have to traverse the nested structure every time you want to perform a `map` or something similar: ```scala mdoc x.map(_.map(_.toString)) ``` `Nested` can help with this by composing the two `map` operations into one: ```scala mdoc:silent import cats.data.Nested import cats.syntax.all._ val nested: Nested[Option, Validated[String, *], Int] = Nested(Some(Valid(123))) ``` ```scala mdoc nested.map(_.toString).value ``` In a sense, `Nested` is similar to monad transformers like `OptionT` and `EitherT`, as it represents the nesting of effects inside each other. But `Nested` is more general - it does not place any restriction on the type of the two nested effects: ```scala final case class Nested[F[_], G[_], A](value: F[G[A]]) ``` Instead, it provides a set of inference rules based on the properties of `F[_]` and `G[_]`. For example: * If `F[_]` and `G[_]` are both `Functor`s, then `Nested[F, G, *]` is also a `Functor` (we saw this in action in the example above) * If `F[_]` and `G[_]` are both `Applicative`s, then `Nested[F, G, *]` is also an `Applicative` * If `F[_]` is an `ApplicativeError` and `G[_]` is an `Applicative`, then `Nested[F, G, *]` is an `ApplicativeError` * If `F[_]` and `G[_]` are both `Traverse`s, then `Nested[F, G, *]` is also a `Traverse` You can see the full list of these rules in the `Nested` companion object. ### A more interesting example (courtesy of [Channing Walton and Luka Jacobowitz via Twitter](https://twitter.com/LukaJacobowitz/status/1017355319473786880), slightly adapted) Say we have an API for creating users: ```scala mdoc:silent import scala.concurrent.Future case class UserInfo(name: String, age: Int) case class User(id: String, name: String, age: Int) def createUser(userInfo: UserInfo): Future[Either[List[String], User]] = Future.successful(Right(User("user 123", userInfo.name, userInfo.age))) ``` Using `Nested` we can write a function that, given a list of `UserInfo`s, creates a list of `User`s: ```scala mdoc:silent import scala.concurrent.Await import scala.concurrent.ExecutionContext.Implicits.global import scala.concurrent.duration._ import cats.Applicative import cats.data.Nested import cats.syntax.all._ def createUsers(userInfos: List[UserInfo]): Future[Either[List[String], List[User]]] = userInfos.traverse(userInfo => Nested(createUser(userInfo))).value val userInfos = List( UserInfo("Alice", 42), UserInfo("Bob", 99) ) ``` ```scala mdoc Await.result(createUsers(userInfos), 1.second) ``` Note that if we hadn't used `Nested`, the behaviour of our function would have been different, resulting in a different return type: ```scala mdoc:silent def createUsersNotNested(userInfos: List[UserInfo]): Future[List[Either[List[String], User]]] = userInfos.traverse(createUser) ``` ```scala mdoc Await.result(createUsersNotNested(userInfos), 1.second) ``` --- ### Datatypes Oneand (`docs/datatypes/oneand.md`) # OneAnd API Documentation: @:api(cats.data.OneAnd) The `OneAnd[F[_],A]` data type represents a single element of type `A` that is guaranteed to be present (`head`) and in addition to this a second part that is wrapped inside an higher kinded type constructor `F[_]`. By choosing the `F` parameter, you can model for example non-empty lists by choosing `List` for `F`, giving: ```scala mdoc:silent import cats.data.OneAnd type NonEmptyList[A] = OneAnd[List, A] ``` which used to be the implementation of non-empty lists in Cats but has been replaced by the `cats.data.NonEmptyList` data type. By having the higher kinded type parameter `F[_]`, `OneAnd` is also able to represent other "non-empty" data structures e.g. ```scala mdoc:silent type NonEmptyStream[A] = OneAnd[Stream, A] ``` --- ### Datatypes Optiont (`docs/datatypes/optiont.md`) # OptionT API Documentation: @:api(cats.data.OptionT) `OptionT[F[_], A]` is a light wrapper on an `F[Option[A]]`. Speaking technically, it is a monad transformer for `Option`, but you don't need to know what that means for it to be useful. `OptionT` can be more convenient to work with than using `F[Option[A]]` directly. ## Reduce map boilerplate Consider the following scenario: ```scala mdoc:silent import scala.concurrent.Future import scala.concurrent.ExecutionContext.Implicits.global val customGreeting: Future[Option[String]] = Future.successful(Some("welcome back, Lola")) ``` We want to try out various forms of our greetings. ```scala mdoc:silent val excitedGreeting: Future[Option[String]] = customGreeting.map(_.map(_ + "!")) val hasWelcome: Future[Option[String]] = customGreeting.map(_.filter(_.contains("welcome"))) val noWelcome: Future[Option[String]] = customGreeting.map(_.filterNot(_.contains("welcome"))) val withFallback: Future[String] = customGreeting.map(_.getOrElse("hello, there!")) ``` As you can see, the implementations of all of these variations are very similar. We want to call the `Option` operation (`map`, `filter`, `filterNot`, `getOrElse`), but since our `Option` is wrapped in a `Future`, we first need to `map` over the `Future`. `OptionT` can help remove some of this boilerplate. It exposes methods that look like those on `Option`, but it handles the outer `map` call on the `Future` so we don't have to: ```scala mdoc:silent:nest import cats.data.OptionT import cats.syntax.all._ val customGreetingT: OptionT[Future, String] = OptionT(customGreeting) val excitedGreeting: OptionT[Future, String] = customGreetingT.map(_ + "!") val withWelcome: OptionT[Future, String] = customGreetingT.filter(_.contains("welcome")) val noWelcome: OptionT[Future, String] = customGreetingT.filterNot(_.contains("welcome")) val withFallback: Future[String] = customGreetingT.getOrElse("hello, there!") ``` ## From `Option[A]` and/or `F[A]` to `OptionT[F, A]` Sometimes you may have an `Option[A]` and/or `F[A]` and want to *lift* them into an `OptionT[F, A]`. For this purpose `OptionT` exposes two useful methods, namely `fromOption` and `liftF`, respectively. E.g.: ```scala mdoc:silent val greetingFO: Future[Option[String]] = Future.successful(Some("Hello")) val firstnameF: Future[String] = Future.successful("Jane") val lastnameO: Option[String] = Some("Doe") val ot: OptionT[Future, String] = for { g <- OptionT(greetingFO) f <- OptionT.liftF(firstnameF) l <- OptionT.fromOption[Future](lastnameO) } yield s"$g $f $l" val result: Future[Option[String]] = ot.value // Future(Some("Hello Jane Doe")) ``` ## From `A` to `OptionT[F,A]` If you have only an `A` and you wish to *lift* it into an `OptionT[F,A]` assuming you have an [`Applicative`](../typeclasses/applicative.md) instance for `F` you can use `some` which is an alias for `pure`. There also exists a `none` method which can be used to create an `OptionT[F,A]`, where the `Option` wrapped `A` type is actually a `None`: ```scala mdoc:silent val greet: OptionT[Future,String] = OptionT.pure("Hola!") val greetAlt: OptionT[Future,String] = OptionT.some("Hi!") val failedGreet: OptionT[Future,String] = OptionT.none ``` ## Beyond map Sometimes the operation you want to perform on an `Future[Option[String]]` might not be as simple as just wrapping the `Option` method in a `Future.map` call. For example, what if we want to greet the customer with their custom greeting if it exists but otherwise fall back to a default `Future[String]` greeting? Without `OptionT`, this implementation might look like: ```scala mdoc:silent val defaultGreeting: Future[String] = Future.successful("hello, there") val greeting: Future[String] = customGreeting.flatMap(custom => custom.map(Future.successful).getOrElse(defaultGreeting)) ``` We can't quite turn to the `getOrElse` method on `OptionT`, because it takes a `default` value of type `A` instead of `Future[A]`. However, the `getOrElseF` method is exactly what we want: ```scala mdoc:silent:nest val greeting: Future[String] = customGreetingT.getOrElseF(defaultGreeting) ``` ## Getting to the underlying instance If you want to get the `F[Option[A]]` value (in this case `Future[Option[String]]`) out of an `OptionT` instance, you can simply call `value`: ```scala mdoc:silent:nest val customGreeting: Future[Option[String]] = customGreetingT.value ``` --- ### Datatypes State (`docs/datatypes/state.md`) # State API Documentation: @:api(cats.data.IndexedStateT) `State` is a structure that provides a functional approach to handling application state. `State[S, A]` is basically a function `S => (S, A)`, where `S` is the type that represents your state and `A` is the result the function produces. In addition to returning the result of type `A`, the function returns a new `S` value, which is the updated state. ## Robots Let's try to make this more concrete with an example. We have this `Robot` model: ```scala mdoc:silent final case class Robot( id: Long, sentient: Boolean, name: String, model: String) ``` We would like to generate some random `Robot` instances for test data. ## Pseudorandom values Scala's standard library has a built-in `Random` class that provides a (pseudo)random number generator (RNG). Let's use it to write a method that creates robots. ```scala mdoc:silent val rng = new scala.util.Random(0L) def createRobot(): Robot = { val id = rng.nextLong() val sentient = rng.nextBoolean() val isCatherine = rng.nextBoolean() val name = if (isCatherine) "Catherine" else "Carlos" val isReplicant = rng.nextBoolean() val model = if (isReplicant) "replicant" else "borg" Robot(id, sentient, name, model) } ``` ```scala mdoc val robot = createRobot() ``` We create a single `Random` instance, which is mutated as a side-effect each time that we call `nextLong` or `nextBoolean` on it. This mutation makes it more difficult to reason about our code. Someone might come along and see that we have `rng.nextBoolean` repeated three times within a single method. They might cleverly avoid repeated code and method invocations by extracting the common code into a variable: ```scala mdoc:silent:nest val rng = new scala.util.Random(0L) def createRobot(): Robot = { val id = rng.nextLong() val b = rng.nextBoolean() val sentient = b val isCatherine = b val name = if (isCatherine) "Catherine" else "Carlos" val isReplicant = b val model = if (isReplicant) "replicant" else "borg" Robot(id, sentient, name, model) } ``` ```scala mdoc val robot = createRobot() ``` But now the output of our program has changed! We used to have a replicant robot named Catherine, but now we have a borg robot named Carlos. It might not have been obvious, but the `nextBoolean` calls we were making had the side effect of mutating internal RNG state, and we were depending on that behavior. When we can't freely refactor identical code into a common variable, the code becomes harder to reason about. In functional programming lingo, one might say that such code lacks [referential transparency](https://en.wikipedia.org/wiki/Referential_transparency_(computer_science)). ## Purely functional pseudorandom values Since mutating state caused us trouble, let's create an RNG that is immutable. We'll use a simple RNG that can generate pseudorandom `Long` values based only on the previous "seed" value and some carefully chosen constants. You don't need to understand the details of this implementation for the purposes of this example, but if you'd like to know more, this is Knuth's 64-bit [linear congruential generator](https://en.wikipedia.org/wiki/Linear_congruential_generator). ```scala mdoc:silent final case class Seed(long: Long) { def next = Seed(long * 6364136223846793005L + 1442695040888963407L) } ``` Instead of mutating the existing `long` value, calling `next` returns a _new_ `Seed` instance with an updated `long` value. Since the RNG isn't updating state internally, we will need to keep track of state outside of the RNG. When we call `nextBoolean` we will want it to return a `Boolean` as it did before, but we will also want it to return an updated `Seed` that we can use to generate our next random value. ```scala mdoc:silent def nextBoolean(seed: Seed): (Seed, Boolean) = (seed.next, seed.long >= 0L) ``` Similarly, `nextLong` will return an updated `Seed` along with a `Long` value. ```scala mdoc:silent def nextLong(seed: Seed): (Seed, Long) = (seed.next, seed.long) ``` Now we need to explicitly pass in the updated state as we generate each new value. ```scala mdoc:silent:nest def createRobot(seed: Seed): Robot = { val (seed1, id) = nextLong(seed) val (seed2, sentient) = nextBoolean(seed1) val (seed3, isCatherine) = nextBoolean(seed2) val name = if (isCatherine) "Catherine" else "Carlos" val (seed4, isReplicant) = nextBoolean(seed3) val model = if (isReplicant) "replicant" else "borg" Robot(id, sentient, name, model) } val initialSeed = Seed(13L) ``` ```scala mdoc val robot = createRobot(initialSeed) ``` Now it is a bit more obvious that we can't extract the three `nextBoolean` calls into a single variable, because we are passing each one a different seed value. However, it is a bit cumbersome to explicitly pass around all of this intermediate state. It's also a bit error-prone. It would have been easy to accidentally call `nextBoolean(seed2)` for both the name generation and the model generation, instead of remembering to use `nextBoolean(seed3)` the second time. ## Cleaning it up with State State's special power is keeping track of state and passing it along. Recall the description of `State` at the beginning of this document. It is basically a function `S` => `(S, A)`, where `S` is a type representing state. Our `nextLong` function takes a `Seed` and returns an updated `Seed` and a `Long`. It can be represented as `Seed => (Seed, Long)`, and therefore matches the pattern `S => (S, A)` where `S` is `Seed` and `A` is `Long`. Let's write a new version of `nextLong` using `State`: ```scala mdoc:invisible:reset // re-define everything to make it easier to reset mdoc // as things like nextLong are conflicting with each other final case class Robot( id: Long, sentient: Boolean, name: String, model: String) final case class Seed(long: Long) { def next = Seed(long * 6364136223846793005L + 1442695040888963407L) } val initialSeed = Seed(13L) ``` ```scala mdoc:silent import cats.data.State val nextLong: State[Seed, Long] = State(seed => (seed.next, seed.long)) ``` The `map` method on `State` allows us to transform the `A` value without affecting the `S` (state) value. This is perfect for implementing `nextBoolean` in terms of `nextLong`. ```scala mdoc:silent val nextBoolean: State[Seed, Boolean] = nextLong.map(long => long >= 0) ``` The `flatMap` method on `State[S, A]` lets you use the result of one `State` in a subsequent `State`. The updated state (`S`) after the first call is passed into the second call. These `flatMap` and `map` methods allow us to use `State` in for-comprehensions: ```scala mdoc:silent val createRobot: State[Seed, Robot] = for { id <- nextLong sentient <- nextBoolean isCatherine <- nextBoolean name = if (isCatherine) "Catherine" else "Carlos" isReplicant <- nextBoolean model = if (isReplicant) "replicant" else "borg" } yield Robot(id, sentient, name, model) ``` At this point, we have not yet created a robot; we have written instructions for creating a robot. We need to pass in an initial seed value, and then we can call `value` to actually create the robot: ```scala mdoc:nest val (finalState, robot) = createRobot.run(initialSeed).value ``` If we only care about the robot and not the final state, then we can use `runA`: ```scala mdoc:nest val robot = createRobot.runA(initialSeed).value ``` The `createRobot` implementation reads much like the imperative code we initially wrote for the mutable RNG. However, this implementation is free of mutation and side-effects. Since this code is referentially transparent, we can perform the refactoring that we tried earlier without affecting the result: ```scala mdoc:silent:nest val createRobot: State[Seed, Robot] = { val b = nextBoolean for { id <- nextLong sentient <- b isCatherine <- b name = if (isCatherine) "Catherine" else "Carlos" isReplicant <- b model = if (isReplicant) "replicant" else "borg" } yield Robot(id, sentient, name, model) } ``` ```scala mdoc:nest val robot = createRobot.runA(initialSeed).value ``` This may seem surprising, but keep in mind that `b` isn't simply a `Boolean`. It is a function that takes a seed and _returns_ a `Boolean`, threading state along the way. Since the seed that is being passed into `b` changes from line to line, so do the returned `Boolean` values. ## Interleaving effects Let's expand on the above example; assume that our random number generator works asynchronously by fetching results from a remote server: ```scala mdoc:silent import scala.concurrent.Future import scala.concurrent.ExecutionContext.Implicits.global final case class AsyncSeed(long: Long) { def next = Future(AsyncSeed(long * 6364136223846793005L + 1442695040888963407L)) } ``` Using a proper `IO` data type would be ideal, but `Future` serves our purpose here. We now need to redefine our `nextLong` action: ```scala mdoc:fail val nextLong: State[AsyncSeed, Future[Long]] = State { seed => (seed.next, seed.long) } ``` Oops! That doesn't work: `State[S, A]` requires that we return a pure next `S` in the `State.apply` constructor. We could define `nextLong` as `State[Future[AsyncSeed], Future[Long]]`, but that would get us into even more trouble: ```scala mdoc:fail val nextLong: State[Future[AsyncSeed], Future[Long]] = State { seedF => seedF.map { seed => (seed.next, seed.long) } } ``` The `seed` that `State.apply` passes in is now a `Future`, so we must map it. But we can't return a `Future[(Future[S], [A])]`; so what do we do? Luckily, `State[S, A]` is an alias for `StateT[Eval, S, A]` - a monad transformer defined as `StateT[F[_], S, A]`. This data type represents computations of the form `S => F[(S, A)]`. If we plug in our concrete types, we get `AsyncSeed => Future[(AsyncSeed, A)]`, which is something we can work with: ```scala mdoc:silent import cats.data.StateT import cats.instances.future._ import scala.concurrent.ExecutionContext.Implicits.global val nextLong: StateT[Future, AsyncSeed, Long] = StateT { seed => seed.next zip Future.successful(seed.long) } ``` Now, what do we get back if we invoke `run` on our `nextLong` action? ```scala mdoc nextLong.run(AsyncSeed(0)) ``` Since every intermediate computation returns a `Future`, the composite computation returns a `Future` as well. To summarize, `StateT[F[_], S, A]` allows us to interleave effects of type `F[_]` in the computations wrapped by it. It should be noted that different combinators on `StateT` impose different constraints on `F`; for example, `map` only requires that `F` has a `Functor` instance, but `flatMap` naturally requires `F` to have a `FlatMap` instance. Have a look at the method signatures for the details. ## Changing States More complex, stateful computations cannot be easily modeled by a single type. For example, let's try to model a door's state: ```scala mdoc:silent sealed trait DoorState case object Open extends DoorState case object Closed extends DoorState case class Door(state: DoorState) def open: State[DoorState, Unit] = ??? def close: State[DoorState, Unit] = ??? ``` We would naturally expect that `open` would only operate on doors that are `Closed`, and vice-versa. However, to implement these methods currently, we have to pattern-match on the state: ```scala mdoc:silent val openImpl: State[DoorState, Unit] = State { doorState => doorState match { case Closed => (Open, ()) case Open => ??? // What now? } } ``` This puts us in the awkward situation of deciding how to handle invalid states; what's the appropriate behavior? Should we just leave the door open? Should we return a failure? That would mean that our return type needs to be modified. The most elegant solution would be to model this requirement statically using types, and luckily, `StateT` is an alias for another type: `IndexedStateT[F[_], SA, SB, A]`. This data type models a stateful computation of the form `SA => F[(SB, A)]`; that's a function that receives an initial state of type `SA` and results in a state of type `SB` and a result of type `A`, using an effect of `F`. So, let's model our typed door: ```scala mdoc:silent:nest import cats.Eval import cats.data.IndexedStateT def open: IndexedStateT[Eval, Closed.type, Open.type, Unit] = IndexedStateT.set(Open) def close: IndexedStateT[Eval, Open.type, Closed.type, Unit] = IndexedStateT.set(Closed) ``` We can now reject, at compile time, sequences of `open` and `close` that are invalid: ```scala mdoc:fail val invalid = for { _ <- open _ <- close _ <- close } yield () ``` While valid sequences will be accepted: ```scala mdoc val valid = for { _ <- open _ <- close _ <- open } yield () ``` Note that the inferred type of `valid` correctly models that this computation can be executed only with an initial `Closed` state. ```scala mdoc:fail valid.run(Open) ``` ```scala mdoc valid.run(Closed) ``` --- ### Datatypes Statet (`docs/datatypes/statet.md`) # StateT API Documentation: @:api(cats.data.IndexedStateT) `StateT[F[_], S, A]` is a data type that generalizes `State` with the ability to compose with effects in `F[_]`. Because `StateT` is defined in terms of `F`, it is a monad only if `F` is a monad. Additionally, `StateT` may acquire new capabilities via `F`: for example, if `F` is capable of error handling via `MonadThrow[F]`, then Cats derives an instance of `MonadThrow[StateT[F, S, *]]`. The type parameters are: - `F[_]` represents the effect in which the computation is performed. - `S` represents the underlying state, shared between each step of the state machine. - `A` represents the return value. It can be seen as a way to add the capability to manipulate a shared state to an existing computation in the context of `F`. This definition could be confusing, but it will become clear after learning the `State` monad and by the example below. ## StateT and State Relationship `StateT` is a monad transformer for [State](https://typelevel.org/cats/datatypes/state.html), in particular [State](https://typelevel.org/cats/datatypes/state.html) is defined as a `StateT` with [Eval](https://typelevel.org/cats/datatypes/eval.html) as the effect `F`. ```scala mdoc:silent import cats.data.StateT import cats.Eval type State[S, A] = StateT[Eval, S, A] ``` Therefore, `StateT` exposes the same methods of [State](https://typelevel.org/cats/datatypes/state.html), such as: `modify`, `get` and `set`. Plus additional methods, that handles effectful computations, eg: `modifyF`, `setF` and `liftF`. ## Example: Table Reservation System In this example, we are going to model a *Table Reservation System*. To do so, we need to keep track of the tables and the current reservations for each of them. The end-user can then try to insert a booking for a specific table and time. If such a table is available, then the booking is placed and the state is updated, otherwise, an error is returned. To simplify the logic, for each reservation we will just consider a single `LocalTime` starting at the beginning of the hour. Let's start with defining the type alias for the effect type: ```scala mdoc:silent:reset import cats.data.{StateT, NonEmptyList} import cats.syntax.all._ import java.time.LocalTime type ThrowableOr[A] = Either[Throwable, A] ``` We will use an `Either[Throwable, A]` to model either success `Right(a)` or failure `Left(ex)`. Now, we need to implement/define: - The type representing the reservation. - The type representing the state of the Table Reservation System. It will wrap around a collection of Reservations. - An initial state, that will be just empty (no reservations). - A custom `Throwable` to be used in case of an error. - The logic for the booking insertion. We can take advantage of the method `modifyF` later on to apply it to the system state. In addition, we can implement a simple function that will evaluate a `NonEmptyList` of reservations, inserting them one by one. ```scala mdoc:silent object TableReservationSystem { final case class ReservationId(tableNumber: Int, hour: LocalTime) final case class Reservation(id: ReservationId, name: String) final case class Reservations(reservations: List[Reservation]) { def insert(reservation: Reservation): ThrowableOr[Reservations] = if (reservations.exists(r => r.id == reservation.id)) Left(new TableAlreadyReservedException(reservation)) else Right(Reservations(reservations :+ reservation)) } final class TableAlreadyReservedException( reservation: Reservation ) extends RuntimeException( s"${reservation.name} cannot be added because table number ${reservation.id.tableNumber} is already reserved for the ${reservation.id.hour}" ) val emptyReservationSystem: Reservations = Reservations(List.empty) def insertBooking( reservation: Reservation ): StateT[ThrowableOr, Reservations, Unit] = StateT.modifyF[ThrowableOr, Reservations](_.insert(reservation)) def processBookings( bookings: NonEmptyList[Reservation] ): ThrowableOr[Reservations] = bookings .traverse_(insertBooking) .runS(emptyReservationSystem) } ``` That's it, we can finally test the code above providing a simple example input and print out the result. ``` /* Detailed source-code truncated for AI context efficiency. */ ``` The full source code of this example can be found at this [gist](https://gist.github.com/benkio/baa4fe1d50751cd602c4175f1bb39f4d) or [scastie](https://scastie.scala-lang.org/YhJmET6PRJKvrZTfrZJkbQ) --- ### Datatypes Validated (`docs/datatypes/validated.md`) # Validated API Documentation: @:api(cats.data.Validated) Imagine you are filling out a web form to signup for an account. You input your username and password and submit. Response comes back saying your username can't have dashes in it, so you make some changes and resubmit. Can't have special characters either. Change, resubmit. Passwords need to have at least one capital letter. Change, resubmit. Password needs to have at least one number. It would be nice to have all of these errors be reported simultaneously. That the username can't have dashes can be validated separately from it not having special characters, as well as from the password needing to have certain requirements. A misspelled (or missing) field in a config can be validated separately from another field not being well-formed. Enter `Validated`. ## A first approach You'll note firsthand that `Validated` is very similar to `Either` because it also has two possible values: errors on the left side or successful computations on the right side. Signature of the structure is as follows: ```scala sealed abstract class Validated[+E, +A] extends Product with Serializable { // Implementation elided } ``` And its _projections_: ```scala final case class Valid[+A](a: A) extends Validated[Nothing, A] final case class Invalid[+E](e: E) extends Validated[E, Nothing] ``` Before diving into `Validated`, let's take a look at an `Either` based first approach to address our validation necessity. Our data will be represented this way: ```scala mdoc:silent final case class RegistrationData(username: String, password: String, firstName: String, lastName: String, age: Int) ``` And our error model: ```scala mdoc:silent sealed trait DomainValidation { def errorMessage: String } case object UsernameHasSpecialCharacters extends DomainValidation { def errorMessage: String = "Username cannot contain special characters." } case object PasswordDoesNotMeetCriteria extends DomainValidation { def errorMessage: String = "Password must be at least 10 characters long, including an uppercase and a lowercase letter, one number and one special character." } case object FirstNameHasSpecialCharacters extends DomainValidation { def errorMessage: String = "First name cannot contain spaces, numbers or special characters." } case object LastNameHasSpecialCharacters extends DomainValidation { def errorMessage: String = "Last name cannot contain spaces, numbers or special characters." } case object AgeIsInvalid extends DomainValidation { def errorMessage: String = "You must be aged 18 and not older than 75 to use our services." } ``` We have our `RegistrationData` case class that will hold the information the user has submitted, alongside the definition of the error model that we'll be using for displaying the possible errors of every field. Now, let's explore the proposed implementation: ``` /* Detailed source-code truncated for AI context efficiency. */ ``` The logic of the validation process is as follows: **check every individual field based on the established rules for each one of them. If the validation is successful, then return the field wrapped in a `Right` instance; If not, then return a `DomainValidation` with the respective message, wrapped in a `Left` instance**. Note that we took advantage of the `.cond` method of `Either`, that is equivalent to do `if (cond) Right(value) else Left(error)`. Our service has the `validateForm` method for checking all the fields and, if the process succeeds it will create an instance of `RegistrationData`, right? Well, yes, but the error reporting part will have the downside of showing only the first error. Let's look this in detail: ```scala mdoc:silent:fail for { validatedUserName <- validateUserName(username) validatedPassword <- validatePassword(password) validatedFirstName <- validateFirstName(firstName) validatedLastName <- validateLastName(lastName) validatedAge <- validateAge(age) } yield RegistrationData(validatedUserName, validatedPassword, validatedFirstName, validatedLastName, validatedAge) ``` A for-comprehension is _fail-fast_. If some of the evaluations in the `for` block fails for some reason, the `yield` statement will not complete. In our case, if that happens we won't be getting the accumulated list of errors. If we run our code: ```scala mdoc FormValidator.validateForm( username = "fakeUs3rname", password = "password", firstName = "John", lastName = "Doe", age = 15 ) ``` We should have gotten another `DomainValidation` object denoting the invalid age. ### An iteration with `Validated` Time to do some refactoring! We're going to try a `Validated` approach: ```scala mdoc:silent import cats.data._ import cats.data.Validated._ import cats.syntax.all._ def validateUserName(userName: String): Validated[DomainValidation, String] = FormValidator.validateUserName(userName).toValidated def validatePassword(password: String): Validated[DomainValidation, String] = FormValidator.validatePassword(password).toValidated def validateFirstName(firstName: String): Validated[DomainValidation, String] = FormValidator.validateFirstName(firstName).toValidated def validateLastName(lastName: String): Validated[DomainValidation, String] = FormValidator.validateLastName(lastName).toValidated def validateAge(age: Int): Validated[DomainValidation, Int] = FormValidator.validateAge(age).toValidated ``` ```scala mdoc:fail def validateForm(username: String, password: String, firstName: String, lastName: String, age: Int): Validated[DomainValidation, RegistrationData] = { for { validatedUserName <- validateUserName(username) validatedPassword <- validatePassword(password) validatedFirstName <- validateFirstName(firstName) validatedLastName <- validateLastName(lastName) validatedAge <- validateAge(age) } yield RegistrationData(validatedUserName, validatedPassword, validatedFirstName, validatedLastName, validatedAge) } ``` What we've done here was to reuse the previously created validation functions and convert their output to a `Validated` instance with the `.toValidated` combinator. This one takes an `Either` and converts it to its equivalent `Validated`. This datatype, as with `Either` has two projections: `Valid` and `Invalid`, analogous to `Right` and `Left`, respectively. Remember that our goal is to get all the validation errors for displaying it to the user, but you'll find that this approach won't compile, as you can see in the previous snippet. Why? Without diving into details about monads, a for-comprehension uses the `flatMap` method for composition. Monads like `Either` can be composed in that way, but the thing with `Validated` is that it isn't a monad, but an [_Applicative Functor_](../typeclasses/applicativetraverse.md). That's why you see the message: `error: value flatMap is not a member of cats.data.Validated[DomainValidation,String]`. So, how do we do here? ### Meeting applicative We have to look into another direction: a for-comprehension plays well in a fail-fast scenario, but the structure in our previous example was designed to catch one error at a time, so, our next step is to tweak the implementation a bit. ```scala mdoc:silent sealed trait FormValidatorNec { type ValidationResult[A] = ValidatedNec[DomainValidation, A] private def validateUserName(userName: String): ValidationResult[String] = if (userName.matches("^[a-zA-Z0-9]+$")) userName.validNec else UsernameHasSpecialCharacters.invalidNec private def validatePassword(password: String): ValidationResult[String] = if (password.matches("(?=^.{10,}$)((?=.*\\d)|(?=.*\\W+))(?![.\\n])(?=.*[A-Z])(?=.*[a-z]).*$")) password.validNec else PasswordDoesNotMeetCriteria.invalidNec private def validateFirstName(firstName: String): ValidationResult[String] = if (firstName.matches("^[a-zA-Z]+$")) firstName.validNec else FirstNameHasSpecialCharacters.invalidNec private def validateLastName(lastName: String): ValidationResult[String] = if (lastName.matches("^[a-zA-Z]+$")) lastName.validNec else LastNameHasSpecialCharacters.invalidNec private def validateAge(age: Int): ValidationResult[Int] = if (age >= 18 && age <= 75) age.validNec else AgeIsInvalid.invalidNec def validateForm(username: String, password: String, firstName: String, lastName: String, age: Int): ValidationResult[RegistrationData] = { (validateUserName(username), validatePassword(password), validateFirstName(firstName), validateLastName(lastName), validateAge(age)).mapN(RegistrationData) } } object FormValidatorNec extends FormValidatorNec ``` Let's see what changed here: 1. In this new implementation, we're using a [NonEmptyChain](https://github.com/typelevel/cats/blob/main/core/src/main/scala/cats/data/NonEmptyChain.scala), a data structure that guarantees that at least one element will be present. In case that multiple errors arise, you'll get a chain of `DomainValidation`. 2. `ValidatedNec[DomainValidation, A]` is an alias for `Validated[NonEmptyChain[DomainValidation], A]`. When you use `ValidatedNec` you're stating that your accumulative structure will be a `NonEmptyChain`. With `Validated`, you have the choice about which data structure you want for reporting the errors (more on that soon). 3. We've declared the type alias `ValidationResult` that conveniently expresses the return type of our validation. 4. `.validNec` and `.invalidNec` combinators lets you _lift_ the success or failure in their respective container (either a `Valid` or `Invalid[NonEmptyChain[A]]`). 5. The [applicative](../typeclasses/applicative.md) syntax `(a, b, c, ...).mapN(...)` provides us a way to accumulatively apply the validation functions and yield a product with their successful result or the accumulated errors in the `NonEmptyChain`. Then, we transform that product with `mapN` into a valid instance of `RegistrationData`. **Deprecation notice:** since Cats `1.0.0-MF` the cartesian syntax `|@|` for applicatives is deprecated. If you're using `0.9.0` or less, you can use the syntax: `(a |@| b |@| ...).map(...)`. Note that, at the end, we expect to lift the result of the validation functions in a `RegistrationData` instance. If the process fails, we'll get our `NonEmptyChain` detailing what went wrong. For example: ```scala mdoc FormValidatorNec.validateForm( username = "Joe", password = "Passw0r$1234", firstName = "John", lastName = "Doe", age = 21 ) FormValidatorNec.validateForm( username = "Joe%%%", password = "password", firstName = "John", lastName = "Doe", age = 21 ) ``` Sweet success! Now you can take your validation process to the next level! ### A short detour As previously stated, `ValidatedNec[DomainValidation, A]` is an alias for `Validated[NonEmptyChain[DomainValidation], A]`. Typically, you'll see that `Validated` is accompanied by a `NonEmptyChain` when it comes to accumulation. The thing here is that you can define your own accumulative data structure and you're not limited to the aforementioned construction. For doing this, you have to provide a `Semigroup` instance. `NonEmptyChain`, by definition has its own `Semigroup`. For those who don't know what a `Semigroup` is, you can find out more [here](../typeclasses/semigroup.md). #### Accumulative Structures Let's take a look about how a `Semigroup` works in a `NonEmptyChain`: ```scala mdoc NonEmptyChain.one[DomainValidation](UsernameHasSpecialCharacters) |+| NonEmptyChain[DomainValidation](FirstNameHasSpecialCharacters, LastNameHasSpecialCharacters) ``` We're combining a couple of `NonEmptyChain`s. The first one has its mandatory element (note that we've built an instance of it with `.one`) and the second has a couple of elements. As you can see, the output of the combination, expressed by the `|+|` operator is another `NonEmptyChain` with the three elements. But, what about if we want _another_ way of combining? We can provide our custom `Semigroup` instance with the desired combining logic and pass it implicitly to your scope. ### Going back and forth Cats offers you a nice set of combinators for transforming your `Validated` based approach to an `Either` one and vice-versa. We've used `.toValidated` in our second example, now let's see how to use `.toEither`. #### From `Validated` to `Either` To do this, simply use `.toEither` combinator: ```scala mdoc // Successful case FormValidatorNec.validateForm( username = "Joe", password = "Passw0r$1234", firstName = "John", lastName = "Doe", age = 21 ).toEither // Invalid case FormValidatorNec.validateForm( username = "Joe123#", password = "password", firstName = "John", lastName = "Doe", age = 5 ).toEither ``` With this conversion, as you can see, we got an `Either` with a `NonEmptyChain` detailing the possible validation errors or our `RegistrationData` object. ## Another case Perhaps you're reading from a configuration file. One could imagine the configuration library you're using returns a `scala.util.Try`, or maybe a `scala.util.Either`. Your parsing may look something like: ```scala for { url <- config[String]("url") port <- config[Int]("port") } yield ConnectionParams(url, port) ``` You run your program and it says key "url" not found, turns out the key was "endpoint". So you change your code and re-run. Now it says the "port" key was not a well-formed integer. ## Parallel validation Our goal is to report any and all errors across independent bits of data. For instance, when we ask for several pieces of configuration, each configuration field can be validated separately from one another. How then do we enforce that the data we are working with is independent? We ask for both of them up front. As our running example, we will look at config parsing. Our config will be represented by a `Map[String, String]`. Parsing will be handled by a `Read` type class - we provide instances just for `String` and `Int` for brevity. ```scala mdoc:silent:reset trait Read[A] { def read(s: String): Option[A] } object Read { def apply[A](implicit A: Read[A]): Read[A] = A implicit val stringRead: Read[String] = new Read[String] { def read(s: String): Option[String] = Some(s) } implicit val intRead: Read[Int] = new Read[Int] { def read(s: String): Option[Int] = if (s.matches("-?[0-9]+")) Some(s.toInt) else None } } ``` Then we enumerate our errors - when asking for a config value, one of two things can go wrong: the field is missing, or it is not well-formed with regards to the expected type. ```scala mdoc:silent sealed abstract class ConfigError final case class MissingConfig(field: String) extends ConfigError final case class ParseError(field: String) extends ConfigError ``` We need a data type that can represent either a successful value (a parsed configuration), or an error. ```scala sealed abstract class Validated[+E, +A] object Validated { final case class Valid[+A](a: A) extends Validated[Nothing, A] final case class Invalid[+E](e: E) extends Validated[E, Nothing] } ``` Now we are ready to write our parser. ```scala mdoc:silent import cats.data.Validated import cats.data.Validated.{Invalid, Valid} case class Config(map: Map[String, String]) { def parse[A : Read](key: String): Validated[ConfigError, A] = map.get(key) match { case None => Invalid(MissingConfig(key)) case Some(value) => Read[A].read(value) match { case None => Invalid(ParseError(key)) case Some(a) => Valid(a) } } } ``` Everything is in place to write the parallel validator. Recall that we can only do parallel validation if each piece is independent. How do we enforce the data is independent? By asking for all of it up front. Let's start with two pieces of data. ```scala mdoc:silent def parallelValidateSimple[E, A, B, C](v1: Validated[E, A], v2: Validated[E, B])(f: (A, B) => C): Validated[E, C] = (v1, v2) match { case (Valid(a), Valid(b)) => Valid(f(a, b)) case (Valid(_), i@Invalid(_)) => i case (i@Invalid(_), Valid(_)) => i case (Invalid(e1), Invalid(e2)) => ??? } ``` We've run into a problem. In the case where both have errors, we want to report both. But we have no way of combining the two errors into one error! Perhaps we can put both errors into a `Chain`, but that seems needlessly specific - clients may want to define their own way of combining errors. How then do we abstract over a binary operation? The `Semigroup` type class captures this idea. ```scala mdoc:silent import cats.Semigroup def parallelValidate[E : Semigroup, A, B, C](v1: Validated[E, A], v2: Validated[E, B])(f: (A, B) => C): Validated[E, C] = (v1, v2) match { case (Valid(a), Valid(b)) => Valid(f(a, b)) case (Valid(_), i@Invalid(_)) => i case (i@Invalid(_), Valid(_)) => i case (Invalid(e1), Invalid(e2)) => Invalid(Semigroup[E].combine(e1, e2)) } ``` Perfect! But.. going back to our example, we don't have a way to combine `ConfigError`s. But as clients, we can change our `Validated` values where the error can be combined, say, a `Chain[ConfigError]`. It is more common however to use a `NonEmptyChain[ConfigError]` - the `NonEmptyChain` statically guarantees we have at least one value, which aligns with the fact that if we have an `Invalid`, then we most certainly have at least one error. This technique is so common there is a convenient method on `Validated` called `toValidatedNec` that turns any `Validated[E, A]` value to a `Validated[NonEmptyChain[E], A]`. Additionally, the type alias `ValidatedNec[E, A]` is provided. Time to parse. ```scala mdoc:silent:nest import cats.SemigroupK import cats.data.NonEmptyChain import cats.syntax.all._ case class ConnectionParams(url: String, port: Int) val config = Config(Map(("endpoint", "127.0.0.1"), ("port", "not an int"))) implicit val necSemigroup: Semigroup[NonEmptyChain[ConfigError]] = SemigroupK[NonEmptyChain].algebra[ConfigError] implicit val readString: Read[String] = Read.stringRead implicit val readInt: Read[Int] = Read.intRead ``` Any and all errors are reported! ```scala mdoc:nest val v1 = parallelValidate(config.parse[String]("url").toValidatedNec, config.parse[Int]("port").toValidatedNec)(ConnectionParams.apply) val v2 = parallelValidate(config.parse[String]("endpoint").toValidatedNec, config.parse[Int]("port").toValidatedNec)(ConnectionParams.apply) ``` ```scala mdoc:nest val config = Config(Map(("endpoint", "127.0.0.1"), ("port", "1234"))) val v3 = parallelValidate(config.parse[String]("endpoint").toValidatedNec, config.parse[Int]("port").toValidatedNec)(ConnectionParams.apply) ``` ## Apply Our `parallelValidate` function looks awfully like the `Apply#map2` function. ```scala def map2[F[_], A, B, C](fa: F[A], fb: F[B])(f: (A, B) => C): F[C] ``` Which can be defined in terms of `Apply#ap` and `Apply#map`, the very functions needed to create an `Apply` instance. Can we perhaps define an `Apply` instance for `Validated`? Better yet, can we define an `Applicative` instance? *Note*: the example below assumes usage of the [kind-projector compiler plugin](https://github.com/typelevel/kind-projector) and will not compile if it is not being used in a project. ```scala mdoc:silent import cats.Applicative implicit def validatedApplicative[E : Semigroup]: Applicative[Validated[E, *]] = new Applicative[Validated[E, *]] { def ap[A, B](f: Validated[E, A => B])(fa: Validated[E, A]): Validated[E, B] = (fa, f) match { case (Valid(a), Valid(fab)) => Valid(fab(a)) case (i@Invalid(_), Valid(_)) => i case (Valid(_), i@Invalid(_)) => i case (Invalid(e1), Invalid(e2)) => Invalid(Semigroup[E].combine(e1, e2)) } def pure[A](x: A): Validated[E, A] = Validated.valid(x) } ``` Awesome! And now we also get access to all the goodness of `Applicative`, which includes `map{2-22}`, as well as the `Semigroupal` tuple syntax. We can now easily ask for several bits of configuration and get any and all errors returned back. ```scala mdoc:silent:nest import cats.Apply import cats.data.ValidatedNec implicit def necSemigroup: Semigroup[NonEmptyChain[ConfigError]] = SemigroupK[NonEmptyChain].algebra[ConfigError] val personConfig = Config(Map(("name", "cat"), ("age", "not a number"), ("houseNumber", "1234"), ("lane", "feline street"))) case class Address(houseNumber: Int, street: String) case class Person(name: String, age: Int, address: Address) val personFromConfig: ValidatedNec[ConfigError, Person] = Apply[ValidatedNec[ConfigError, *]].map4(personConfig.parse[String]("name").toValidatedNec, personConfig.parse[Int]("age").toValidatedNec, personConfig.parse[Int]("house_number").toValidatedNec, personConfig.parse[String]("street").toValidatedNec) { case (name, age, houseNumber, street) => Person(name, age, Address(houseNumber, street)) } ``` ## Of `flatMap`s and `Either`s `Option` has `flatMap`, `Either` has `flatMap`, where's `Validated`'s? Let's try to implement it - better yet, let's implement the `Monad` type class. ```scala mdoc:silent:nest import cats.Monad implicit def validatedMonad[E]: Monad[Validated[E, *]] = new Monad[Validated[E, *]] { def flatMap[A, B](fa: Validated[E, A])(f: A => Validated[E, B]): Validated[E, B] = fa match { case Valid(a) => f(a) case i@Invalid(_) => i } def pure[A](x: A): Validated[E, A] = Valid(x) @annotation.tailrec def tailRecM[A, B](a: A)(f: A => Validated[E, Either[A, B]]): Validated[E, B] = f(a) match { case Valid(Right(b)) => Valid(b) case Valid(Left(a)) => tailRecM(a)(f) case i@Invalid(_) => i } } ``` Note that all `Monad` instances are also `Applicative` instances, where `ap` is defined as ```scala mdoc:compile-only trait Monad[F[_]] { def flatMap[A, B](fa: F[A])(f: A => F[B]): F[B] def pure[A](x: A): F[A] def map[A, B](fa: F[A])(f: A => B): F[B] = flatMap(fa)(f.andThen(pure)) def ap[A, B](fa: F[A])(f: F[A => B]): F[B] = flatMap(fa)(a => map(f)(fab => fab(a))) } ``` However, the `ap` behavior defined in terms of `flatMap` does not behave the same as that of our `ap` defined above. Observe: ```scala mdoc validatedMonad.tuple2(Validated.invalidNec[String, Int]("oops"), Validated.invalidNec[String, Double]("uh oh")) ``` This one short circuits! Therefore, if we were to define a `Monad` (or `FlatMap`) instance for `Validated` we would have to override `ap` to get the behavior we want. ```scala mdoc:silent:nest import cats.Monad implicit def accumulatingValidatedMonad[E: Semigroup]: Monad[Validated[E, *]] = new Monad[Validated[E, *]] { def flatMap[A, B](fa: Validated[E, A])(f: A => Validated[E, B]): Validated[E, B] = fa match { case Valid(a) => f(a) case i@Invalid(_) => i } def pure[A](x: A): Validated[E, A] = Valid(x) @annotation.tailrec def tailRecM[A, B](a: A)(f: A => Validated[E, Either[A, B]]): Validated[E, B] = f(a) match { case Valid(Right(b)) => Valid(b) case Valid(Left(a)) => tailRecM(a)(f) case i@Invalid(_) => i } override def ap[A, B](f: Validated[E, A => B])(fa: Validated[E, A]): Validated[E, B] = (fa, f) match { case (Valid(a), Valid(fab)) => Valid(fab(a)) case (i@Invalid(_), Valid(_)) => i case (Valid(_), i@Invalid(_)) => i case (Invalid(e1), Invalid(e2)) => Invalid(Semigroup[E].combine(e1, e2)) } } ``` But then the behavior of `flatMap` would be inconsistent with that of `ap`, and this will violate one of the [FlatMap laws](https://github.com/typelevel/cats/blob/main/laws/src/main/scala/cats/laws/FlatMapLaws.scala), `flatMapConsistentApply`: ```scala // the `<->` operator means "is equivalent to" and returns a data structure // `IsEq` that is used to verify the equivalence of the two expressions def flatMapConsistentApply[A, B](fa: F[A], fab: F[A => B]): IsEq[F[B]] = fab.ap(fa) <-> fab.flatMap(f => fa.map(f)) ``` ```scala mdoc:silent import cats.laws._ val flatMapLawsForAccumulatingValidatedMonad = FlatMapLaws[Validated[NonEmptyChain[String], *]](accumulatingValidatedMonad) val fa = Validated.invalidNec[String, Int]("oops") val fab = Validated.invalidNec[String, Int => Double]("Broken function") ``` ```scala mdoc flatMapLawsForAccumulatingValidatedMonad.flatMapConsistentApply(fa , fab) ``` Therefore, `Validated` has only an `Applicative` instance. ## `Validated` vs `Either` We've established that an error-accumulating data type such as `Validated` can't have a valid `Monad` instance. Sometimes the task at hand requires error-accumulation. However, sometimes we want a monadic structure that we can use for sequential validation (such as in a for-comprehension). This leaves us in a bit of a conundrum. Cats has decided to solve this problem by using separate data structures for error-accumulation (`Validated`) and short-circuiting monadic behavior (`Either`). If you are trying to decide whether you want to use `Validated` or `Either`, a simple heuristic is to use `Validated` if you want error-accumulation and to otherwise use `Either`. ## Sequential Validation If you do want error accumulation but occasionally run into places where you sequential validation is needed, then `Validated` provides a couple methods that may be helpful. ### `andThen` The `andThen` method is similar to `flatMap` (such as `Either.flatMap`). In the case of success, it passes the valid value into a function that returns a new `Validated` instance. ```scala mdoc val houseNumber = config.parse[Int]("house_number").andThen{ n => if (n >= 0) Validated.valid(n) else Validated.invalid(ParseError("house_number")) } ``` ### `withEither` The `withEither` method allows you to temporarily turn a `Validated` instance into an `Either` instance and apply it to a function. ```scala mdoc:silent def positive(field: String, i: Int): Either[ConfigError, Int] = { if (i >= 0) Right(i) else Left(ParseError(field)) } ``` Thus. ```scala mdoc:nest val houseNumber = config.parse[Int]("house_number").withEither{ either: Either[ConfigError, Int] => either.flatMap{ i => positive("house_number", i) } } ``` --- ### Datatypes Writer (`docs/datatypes/writer.md`) # Writer API Documentation: @:api(cats.data.WriterT) The `Writer[L, A]` datatype represents a computation that produces a tuple containing a value of type `L` and one of type `A`. Usually, the value `L` represents a description of the computation. A typical example of an `L` value could be a logging `String` and that's why from now on we will refer to it as the _Logging side_ of the datatype. Meanwhile, the value `A` is the actual output of the computation. The main features that `Writer` provides are: The flexibility regarding Log value management. It can be modified in multiple ways. See the [Operations section](#operations) When two functions are composed together, e.g. using `flatMap`, the logs of both functions will be combined using an implicit [Semigroup](../typeclasses/semigroup.md). ## Operations The `Writer` datatype provides a set of functions that are similar to the ones from the [Monad](../typeclasses/monad.md) typeclass. In fact, they share the same name and the same signature, but have an additional requirement of a [`Semigroup[L]`](../typeclasses/semigroup.md) that allows the log merging. `map` effects only the value, keeping the log side untouched. Plus, here we show `run` that just unwrap the datatype, returning its content. ```scala mdoc import cats.data.Writer import cats.instances._ val mapExample = Writer("map Example", 1).map(_ + 1) mapExample.run ``` `ap` allows applying a function, wrapped into a Writer. It works exactly like the `Applicative` as expected, but notice how the logs are combined using the [`Semigroup[String]`](../typeclasses/semigroup.md). ```scala mdoc val apExampleValue = Writer("ap value", 10) val apExampleFunc = Writer("ap function ", (i: Int) => i % 7) apExampleValue.ap(apExampleFunc).run ``` Same thing for `flatMap` ```scala mdoc val flatMapExample1 = Writer("flatmap value", 5) val flatMapExample2 = (x: Int) => Writer("flatmap function ", x * x) flatMapExample1.flatMap(flatMapExample2).run // We can use the for comprehension as well val flatMapForResult = for { value <- Writer("flatmap value", 5) result <- flatMapExample2(value) } yield result flatMapForResult.run ``` Apart from those, `Writer` comes with some specific functions to manage the log side of the computation: `tell` : Append a value to the log side. It requires a [`Semigroup[L]`](../typeclasses/semigroup.md). `swap` : Exchange the two values of the `Writer`. `reset` : Delete the log side. It requires a [`Monoid[L]`](../typeclasses/monoid.md) since it uses the `empty` value of the monoid. `value` : Returns only the value of the `Writer` `listen` : Transform the value of the `Writer` to a tuple containing the current value and the current log. ```scala mdoc val tellExample = Writer("tell example", 1).tell("log append") tellExample.run val swapExample = Writer("new value", "new log").swap swapExample.run val resetExample = Writer("long log to discard", 42).reset resetExample.run val valueExample = Writer("some log", 55).value valueExample val listenExample = Writer("listen log", 10).listen listenExample.run ``` ### Definition If we go looking at how `Writer` is actually defined, we will see it is just a type alias: ```scala mdoc:silent import cats.data.WriterT import cats.Id // cats/data/package.scala type Writer[L, V] = WriterT[Id, L, V] ``` So, all the [Operations](#operations) defined in the previous section are actually coming from the [WriterT datatype](writert.md) Most of the [`WriterT`](writert.md) functions require a [`Functor[F]`](../typeclasses/functor.md) or [`Monad[F]`](../typeclasses/monad.md) instance. However, Cats provides all the necessary instances for the [`Id`](id.md) type, therefore we don't have to worry about them. ## Example The example showed in here is taken from the [Rosetta Code site](https://rosettacode.org/wiki/Monads/Writer_monad). It simply applies a bunch of Math operations, logging each one of them. ```scala mdoc:silent:reset import cats.data.Writer import scala.math.sqrt val writer1: Writer[String, Double] = Writer.value[String, Double](5.0).tell("Initial value ") val writer2: Writer[String, Double => Double] = Writer("sqrt ", (i: Double) => sqrt(i)) val writer3: Double => Writer[String, Double] = (x: Double) => Writer("add 1 ", x + 1) val writer4: Writer[String, Double => Double] = Writer("divided by 2 ", (x: Double) => x / 2) val writer5: Writer[String, Double => Double] = Writer[String, Double => Double](writer3(0).written,(x: Double) => writer3(x).value) ``` ```scala mdoc // Pay attention on the ordering of the logs writer1 .ap(writer2) .flatMap(writer3(_)) .ap(writer4) .map(_.toString) .run import cats.syntax.compose._ (for { initialValue <- writer1 sqrt <- writer2 addOne <- writer5 divideBy2 <- writer4 } yield (sqrt >>> addOne >>> divideBy2)(initialValue) ).run ``` If you are interested in logging solutions, we recommend the library [log4cats](https://typelevel.org/log4cats/) --- ### Datatypes Writert (`docs/datatypes/writert.md`) # WriterT API Documentation: @:api(cats.data.WriterT) `WriterT[F[_], L, V]` is a type wrapper on an `F[(L, V)]`. Speaking technically, it is a monad transformer for [`Writer`](writer.md), but you don't need to know what that means for it to be useful. ## Composition `WriterT` can be more convenient to work with than using `F[Writer[L, V]]` directly, because it exposes operations that allow you to work with the values of the inner [`Writer`](writer.md) (`L` and `V`) abstracting both the `F` and [`Writer`](writer.md). For example, `map` allow you to transform the inner `V` value, getting back a `WriterT` that wraps around it. ```scala mdoc:nest import cats.data.{WriterT, Writer} WriterT[Option, String, Int](Some(("value", 10))).map(x => x * x) ``` Plus, when composing multiple `WriterT` computations, those will be composed following the same behaviour of a [`Writer`](writer.md) and the generic `F`. Let's see two examples with `Option` and `Either`: if one of the computations has a `None` or a `Left`, the whole computation will return a `None` or a `Left` since the way the two types compose typically behaves that way. Moreover, when the computation succeed, the logging side of the [`Writer`](writer.md)s will be combined. ```scala mdoc:silent val optionWriterT1 : WriterT[Option, String, Int] = WriterT(Some(("writerT value 1", 123))) val optionWriterT2 : WriterT[Option, String, Int] = WriterT(Some(("writerT value 1", 123))) val optionWriterT3 : WriterT[Option, String, Int] = WriterT.valueT(None) val eitherWriterT1 : WriterT[Either[String, *], String, Int] = WriterT(Right(("writerT value 1", 123))) val eitherWriterT2 : WriterT[Either[String, *], String, Int] = WriterT(Right(("writerT value 1", 123))) val eitherWriterT3 : WriterT[Either[String, *], String, Int] = WriterT.valueT(Left("error!!!")) ``` ```scala mdoc // This returns a Some since both are Some for { v1 <- optionWriterT1 v2 <- optionWriterT2 } yield v1 + v2 // This returns a None since one is a None for { v1 <- optionWriterT1 v2 <- optionWriterT2 v3 <- optionWriterT3 } yield v1 + v2 + v3 // This returns a Right since both are Right for { v1 <- eitherWriterT1 v2 <- eitherWriterT2 } yield v1 + v2 // This returns a Left since one is a Left for { v1 <- eitherWriterT1 v2 <- eitherWriterT2 v3 <- eitherWriterT3 } yield v1 + v2 + v3 ``` Just for completeness, we can have a look at the same example, but with [`Validated`](validated.md) since it as a slightly different behaviour than `Either`. Instead of short-circuiting when the first error is encountered, [`Validated`](validated.md) will accumulate all the errors. In the following example, you can see how this behaviour is respected when [`Validated`](validated.md) is wrapped as the `F` type of a `WriterT`. In addition, notice how `flatMap` and for comprehension can't be used in this case, since [`Validated`](validated.md) only extends [`Applicative`](../typeclasses/applicative.md), but not [`Monad`](../typeclasses/monad.md). ```scala mdoc:silent import cats.data.Validated import cats.data.Validated.{Invalid, Valid} import cats.syntax.all._ val validatedWriterT1 : WriterT[Validated[String, *], String, Int] = WriterT(Valid(("writerT value 1", 123))) val validatedWriterT2 : WriterT[Validated[String, *], String, Int] = WriterT(Valid(("writerT value 1", 123))) val validatedWriterT3 : WriterT[Validated[String, *], String, Int] = WriterT(Invalid("error 1!!!") : Validated[String, (String, Int)]) val validatedWriterT4 : WriterT[Validated[String, *], String, Int] = WriterT(Invalid("error 2!!!"): Validated[String, (String, Int)]) ``` ```scala mdoc // This returns a Right since both are Right (validatedWriterT1, validatedWriterT2 ).mapN((v1, v2) => v1 + v2) // This returns a Left since there are several Left (validatedWriterT1, validatedWriterT2, validatedWriterT3, validatedWriterT4 ).mapN((v1, v2, v3, v4) => v1 + v2 + v3 + v4) ``` ## Construct a WriterT A `WriterT` can be constructed in different ways. Here is the list of the main available constructors with a brief explanation and an example. `WriterT[F[_], L, V](run: F[(L, V)])` : This is the constructor of the datatype itself. It just builds the type starting from the full wrapped value. ```scala mdoc:nest // Here we use Option as our F[_] val value : Option[(String, Int)] = Some(("value", 123)) WriterT(value) ``` `liftF[F[_], L, V](fv: F[V])(implicit monoidL: Monoid[L], F: Applicative[F]): WriterT[F, L, V]` : This function allows you to build the datatype starting from the value `V` wrapped into an `F`. Notice how it requires: * [`Monoid[L]`](../typeclasses/monoid.md), since it uses the `empty` value from the typeclass. to fill the `L` value not specified in the input. * [`Applicative[F]`](../typeclasses/applicative.md) to modify the inner value. ```scala mdoc:nest import cats.instances.option._ val value : Option[Int] = Some(123) WriterT.liftF[Option, String, Int](value) ``` `put[F[_], L, V](v: V)(l: L)(implicit applicativeF: Applicative[F]): WriterT[F, L, V]` : As soon as there is an [`Applicative`](../typeclasses/applicative.md) instance of `F`, this function creates the datatype starting from the inner [`Writer`](writer.md)'s values. ```scala mdoc:nest WriterT.put[Option, String, Int](123)("initial value") ``` `putT[F[_], L, V](vf: F[V])(l: L)(implicit functorF: Functor[F]): WriterT[F, L, V]` : Exactly as `put`, but the value `V` is already wrapped into `F` ```scala mdoc:nest WriterT.putT[Option, String, Int](Some(123))("initial value") ``` ## Operations In the [Writer definition](writer.md#definition) section, we showed how it is actually a `WriterT`. Therefore, all the operations described into [Writer operations](writer.md#operations) are valid for `WriterT` as well. The only aspect we want to remark here is the following sentence from [`Writer`](writer.md)'s page: > Most of the `WriterT` functions require a `Functor[F]` or > `Monad[F]` instance. However, Cats provides all the necessary > instances for the `Id` type, therefore we don't have to worry about > them. In the case of `WriterT`, the user needs to ensure the required instances are present. Cats still provide a lot of default instances, so there's a high chance you could find what you are searching for with the right `import`. ## Example As an example, we can consider a simple naive console application that pings multiple HTTP well-known services and collect the time spent in each call, returning the total time of the whole execution at the end. We will simulate the calls by successful `Future` values. Using `WriterT` we can log each step of our application, compute, the time and work within the `Future` effect. ```scala mdoc:silent import cats.data.WriterT import scala.concurrent.{Await, Future} import scala.concurrent.duration.Duration import scala.concurrent.ExecutionContext.Implicits.global // Mocked HTTP calls def pingService1() : Future[Int] = Future.successful(100) def pingService2() : Future[Int] = Future.successful(200) def pingService3() : Future[Int] = Future.successful(50) def pingService4() : Future[Int] = Future.successful(75) def pingToWriterT(ping: Future[Int], serviceName: String) : WriterT[Future, String, Int] = WriterT.valueT[Future, String, Int](ping) .tell(s"ping to $serviceName ") .flatMap(pingTime => WriterT.put(pingTime)(s"took $pingTime \n")) val resultWriterT: WriterT[Future, String, Int] = for { ping1 <- pingToWriterT(pingService1(), "service #1") ping2 <- pingToWriterT(pingService2(), "service #2") ping3 <- pingToWriterT(pingService3(), "service #3") ping4 <- pingToWriterT(pingService4(), "service #4") } yield ping1 + ping2 + ping3 + ping4 val resultFuture: Future[String] = resultWriterT.run.map { case (log: String, totalTime: Int) => s"$log> Total time: $totalTime" } ``` And the final result as expected: ```scala mdoc Await.result(resultFuture, Duration.Inf) ``` --- ### Typeclasses Alternative (`docs/typeclasses/alternative.md`) # Alternative API Documentation: @:api(cats.Alternative) Alternative extends [`Applicative`](applicative.md) with a [`MonoidK`](monoidk.md). Let's stub out all the operations just to remind ourselves what that gets us. ```scala mdoc:silent import cats.{Applicative, MonoidK} trait Alternative[F[_]] extends Applicative[F] with MonoidK[F] { def ap[A, B](ff: F[A => B])(fa: F[A]): F[B] def pure[A](a: A): F[A] def empty[A]: F[A] def combineK[A](x: F[A], y: F[A]): F[A] } ``` As you might recall, `pure` wraps values in the context; `ap` allows us to do calculations in the context; `combineK` allows us to combine, for any given type `A`, any two contextual values `F[A]`; and `empty` provides the identity element for the combine operation. Like other type classes, `Alternative` instances must obey some laws, in addition to those otherwise applying to `MonoidK` and `Applicative instances`: * Right Absorption: Applying a contextual function `F[A => B]` to `empty [A]` should be `empty [B]`. * `ff ap F.empty[A] = F.empty[B]`. * Left Distributivity: Mapping over a combined element must be the combinations of the mapped elements. * `(fa <+> fb) map f = ((fa map f) <+> (fb map f))` where `fa: F[A]` and `fb: F[B]` and `f: A => B`. * Right Distributivity: Applying the combination of two functions must be the combination of their applications. * `(ff <+> fg) ap fa = (ff ap fa) <+> (fg ap fa)` where `ff: F[A => B]`, `fg: F[A => B]`, and `fa: F[A]`. These laws guarantee the compatibility of the otherwise possibly independent `Applicative` and `MonoidK` structures. ## Vector as an Alternative Let's examine an instance to get a feel for things. A useful instance of `Alternative` is that for `Vector`. The relevant imports: ```scala mdoc:reset:silent import cats.Alternative import cats.syntax.all._ ``` And what we can do with them: ```scala mdoc val empty = Alternative[Vector].empty[Int] val pureOfFive = 5.pure[Vector] val concatenated = 7.pure[Vector] <+> 8.pure[Vector] val double: Int => Int = _ * 2 val addFive: Int => Int = _ + 5 val apForVectors = (double.pure[Vector] <+> addFive.pure[Vector]) ap concatenated ``` ## Making choices with `Alternative` ### Alternative Parsing Suppose we have a simple parser library with an interface something like: ```scala mdoc:silent trait Decoder[A] { def decode(in: String): Either[Throwable, A] } object Decoder { def from[A](f: String => Either[Throwable, A]): Decoder[A] = new Decoder[A] { def decode(in: String) = f(in) } } ``` Then, we can implement an `Alternative` instance for this type like so: ```scala mdoc:silent implicit val decoderAlternative: Alternative[Decoder] = new Alternative[Decoder] { def pure[A](a: A) = Decoder.from(Function.const(Right(a))) def empty[A] = Decoder.from(Function.const(Left(new Error("No dice.")))) def combineK[A](l: Decoder[A], r: Decoder[A]): Decoder[A] = new Decoder[A] { def decode(in: String) = l.decode(in).orElse(r.decode(in)) } def ap[A, B](ff: Decoder[A => B])(fa: Decoder[A]): Decoder[B] = new Decoder[B] { def decode(in: String) = fa.decode(in) ap ff.decode(in) } } ``` The addition of the `Alternative` methods allows us to prioritize multiple strategies, compensating for inconsistencies in the source data. ```scala mdoc:silent def parseInt(s: String): Either[Throwable, Int] = Either.catchNonFatal(s.toInt) def parseIntFirstChar(s: String): Either[Throwable, Int] = Either.catchNonFatal(2 * Character.digit(s.charAt(0), 10)) // Try first parsing the whole, then just the first character. val decoder: Decoder[Int] = Decoder.from(parseInt _) <+> Decoder.from(parseIntFirstChar _) ``` This decoder correctly attempts each strategy in turn, as you can see: ```scala mdoc decoder.decode("555") decoder.decode("5a") ``` ### Partitioning Results `Alternative` gives us a notion of partitioning, as long as we also have a `Monad` available. This is what `separate` does. The trick here is that the inner type constructor (essentially the replacement for `Boolean` as the target of our "predicate") must have a `Bifoldable` available. A great example of a `Bifoldable` is `Either`, and another is `Tuple2`. Let's imagine that we're trying to make a bunch of independent possibly failure-prone calls with a bunch of different `Int` inputs (say it's the id of a resource), each returning `Either[String, Int]` where a left `String` is the code modeling the failure we're given (say it's the HTTP code returned by a remote API we're calling), while right of an integer is the output of the calculation. ```scala mdoc:silent // Resource holder returns (Request, Status) def requestResource(a: Int): Either[(Int, String), (Int, Long)] = { if (a % 4 == 0) Left((a, "Bad request")) else if (a % 3 == 0) Left((a, "Server error")) else Right((a, 200L)) } ``` We can use `separate` to pull apart the failures and successes zipped with the input, letting us log failures and proceed with successes intelligently. `separate` will pull the two different outcomes into different sides of a tuple. ```scala mdoc val partitionedResults = ((requestResource _).pure[Vector] ap Vector(5, 6, 7, 99, 1200, 8, 22)).separate ``` Alternatively (no pun intended), in a totally different version of the same idea, we can lean on the instance of `Bitraverse` for `Tuple2` to try and partition cleanly the outcomes of two different calculations that each depend pairwise on the same input (maybe we have a primary key and we want to select both the remote id of the resources with that key, and those that are linked via the foreign key to the input key). ```scala mdoc // Surprising regularity in this politico-geographical data model! def getRegionAndDistrict(pkey: Int): (Int, Vector[Int]) = (5 * pkey, (double.pure[Vector] <+> addFive.pure[Vector]) ap pkey.pure[Vector]) val regionsWithDistricts = (getRegionAndDistrict _).pure[Vector] ap Vector(5, 6, 7, 97, 1200, 8, 25) val regionIds = regionsWithDistricts.separate._1 val districtIds = regionsWithDistricts.separate._2.flatten ``` --- ### Typeclasses Applicative (`docs/typeclasses/applicative.md`) # Applicative API Documentation: @:api(cats.Applicative) `Applicative` extends [`Functor`](functor.md) with an `ap` and `pure` method. ```scala mdoc:silent import cats.Functor trait Applicative[F[_]] extends Functor[F] { def ap[A, B](ff: F[A => B])(fa: F[A]): F[B] def pure[A](a: A): F[A] def map[A, B](fa: F[A])(f: A => B): F[B] = ap(pure(f))(fa) } ``` `pure` wraps the value into the type constructor - for `Option` this could be `Some(_)`, for `Future` `Future.successful`, and for `List` a singleton list. `ap` is a bit tricky to explain and motivate, so we'll look at an alternative but equivalent formulation via `product`. ```scala mdoc:silent:nest trait Applicative[F[_]] extends Functor[F] { def product[A, B](fa: F[A], fb: F[B]): F[(A, B)] def pure[A](a: A): F[A] } // Example implementation for right-biased Either implicit def applicativeForEither[L]: Applicative[Either[L, *]] = new Applicative[Either[L, *]] { def product[A, B](fa: Either[L, A], fb: Either[L, B]): Either[L, (A, B)] = (fa, fb) match { case (Right(a), Right(b)) => Right((a, b)) case (Left(l) , _ ) => Left(l) case (_ , Left(l) ) => Left(l) } def pure[A](a: A): Either[L, A] = Right(a) def map[A, B](fa: Either[L, A])(f: A => B): Either[L, B] = fa match { case Right(a) => Right(f(a)) case Left(l) => Left(l) } } ``` Note that in this formulation `map` is left abstract, whereas in the previous one with `ap` `map` could be implemented in terms of `ap` and `pure`. This suggests that `ap` is equivalent to `map` and `product`, which is indeed the case. Such an `Applicative` must obey three laws: * Associativity: No matter the order in which you product together three values, the result is isomorphic * `fa.product(fb).product(fc) ~ fa.product(fb.product(fc))` * With `map`, this can be made into an equality with `fa.product(fb).product(fc) = fa.product(fb.product(fc)).map { case (a, (b, c)) => ((a, b), c) }` * Left identity: Zipping a value on the left with unit results in something isomorphic to the original value * `pure(()).product(fa) ~ fa` * As an equality: `pure(()).product(fa).map(_._2) = fa` * Right identity: Zipping a value on the right with unit results in something isomorphic to the original value * `fa.product(pure(())) ~ fa` * As an equality: `fa.product(pure(())).map(_._1) = fa` ## Applicatives for effect management If we view `Functor` as the ability to work with a single effect, `Applicative` encodes working with multiple **independent** effects. Between `product` and `map`, we can take two separate effectful values and compose them. From there we can generalize to working with any N number of independent effects. ```scala mdoc:reset:silent import cats.Applicative def product3[F[_]: Applicative, A, B, C](fa: F[A], fb: F[B], fc: F[C]): F[(A, B, C)] = { val F = Applicative[F] val fabc = F.product(F.product(fa, fb), fc) F.map(fabc) { case ((a, b), c) => (a, b, c) } } ``` ## What is ap? Let's see what happens if we try to compose two effectful values with just `map`. ```scala mdoc:silent import cats.syntax.all._ val f: (Int, Char) => Double = (i, c) => (i + c).toDouble val int: Option[Int] = Some(5) val char: Option[Char] = Some('a') ``` ```scala mdoc int.map(i => (c: Char) => f(i, c)) // what now? ``` We have an `Option[Char => Double]` and an `Option[Char]` to which we want to apply the function to, but `map` doesn't give us enough power to do that. Hence, `ap`. ## Applicatives compose Like [`Functor`](functor.md), `Applicative`s compose. If `F` and `G` have `Applicative` instances, then so does `F[G[_]]`. ```scala mdoc:silent import cats.data.Nested import cats.syntax.all._ import scala.concurrent.Future import scala.concurrent.ExecutionContext.Implicits.global val x: Future[Option[Int]] = Future.successful(Some(5)) val y: Future[Option[Char]] = Future.successful(Some('a')) ``` ```scala mdoc val composed = Applicative[Future].compose[Option].map2(x, y)(_ + _) val nested = Applicative[Nested[Future, Option, *]].map2(Nested(x), Nested(y))(_ + _) ``` ## Traverse The straightforward way to use `product` and `map` (or just `ap`) is to compose `n` independent effects, where `n` is a fixed number. In fact there are convenience methods named `apN`, `mapN`, and `tupleN` (replacing `N` with a number 2 - 22) to make it even easier. Imagine we have one `Option` representing a username, one representing a password, and another representing a URL for logging into a database. ```scala mdoc:silent import java.sql.Connection val username: Option[String] = Some("username") val password: Option[String] = Some("password") val url: Option[String] = Some("some.login.url.here") // Stub for demonstration purposes def attemptConnect(username: String, password: String, url: String): Option[Connection] = None ``` We know statically we have 3 `Option`s, so we can use `map3` specifically. ```scala mdoc Applicative[Option].map3(username, password, url)(attemptConnect) ``` Sometimes we don't know how many effects will be in play - perhaps we are receiving a list from user input or getting rows from a database. This implies the need for a function: ```scala mdoc:silent def sequenceOption[A](fa: List[Option[A]]): Option[List[A]] = ??? // Alternatively.. def traverseOption[A, B](as: List[A])(f: A => Option[B]): Option[List[B]] = ??? ``` Users of the standard library `Future.sequence` or `Future.traverse` will find these names and signatures familiar. Let's implement `traverseOption` (you can implement `sequenceOption` in terms of `traverseOption`). ```scala mdoc:silent:nest def traverseOption[A, B](as: List[A])(f: A => Option[B]): Option[List[B]] = as.foldRight(Some(List.empty[B]): Option[List[B]]) { (a: A, acc: Option[List[B]]) => val optB: Option[B] = f(a) // optB and acc are independent effects so we can use Applicative to compose Applicative[Option].map2(optB, acc)(_ :: _) } traverseOption(List(1, 2, 3))(i => Some(i): Option[Int]) ``` This works...but if we look carefully at the implementation there's nothing `Option`-specific going on. As another example let's implement the same function but for `Either`. ```scala mdoc:silent import cats.syntax.all._ def traverseEither[E, A, B](as: List[A])(f: A => Either[E, B]): Either[E, List[B]] = as.foldRight(Right(List.empty[B]): Either[E, List[B]]) { (a: A, acc: Either[E, List[B]]) => val eitherB: Either[E, B] = f(a) Applicative[Either[E, *]].map2(eitherB, acc)(_ :: _) } ``` ```scala mdoc traverseEither(List(1, 2, 3))(i => if (i % 2 != 0) Left(s"${i} is not even") else Right(i / 2)) ``` The implementation of `traverseOption` and `traverseEither` are more or less identical, modulo the initial "accumulator" to `foldRight`. But even that could be made the same by delegating to `Applicative#pure`! Generalizing `Option` and `Either` to any `F[_]: Applicative` gives us the fully polymorphic version. Existing data types with `Applicative` instances (`Future`, `Option`, `Either[E, *]`, `Try`) can call it by fixing `F` appropriately, and new data types need only be concerned with implementing `Applicative` to do so as well. ```scala mdoc:silent def traverse[F[_]: Applicative, A, B](as: List[A])(f: A => F[B]): F[List[B]] = as.foldRight(Applicative[F].pure(List.empty[B])) { (a: A, acc: F[List[B]]) => val fb: F[B] = f(a) Applicative[F].map2(fb, acc)(_ :: _) } ``` This function is provided by Cats via the `Traverse[List]` instance and syntax, which is covered in another tutorial. ```scala mdoc:silent import cats.syntax.all._ ``` ```scala mdoc List(1, 2, 3).traverse(i => Some(i): Option[Int]) ``` With this addition of `traverse`, we can now compose any number of independent effects, statically known or otherwise. ## Apply - a weakened Applicative A closely related type class is `Apply` which is identical to `Applicative`, modulo the `pure` method. Indeed in Cats `Applicative` is a subclass of `Apply` with the addition of this method. ```scala trait Apply[F[_]] extends Functor[F] { def ap[A, B](ff: F[A => B])(fa: F[A]): F[B] } trait Applicative[F[_]] extends Apply[F] { def pure[A](a: A): F[A] def map[A, B](fa: F[A])(f: A => B): F[B] = ap(pure(f))(fa) } ``` The laws for `Apply` are just the laws of `Applicative` that don't mention `pure`. In the laws given above, the only law would be associativity. One of the motivations for `Apply`'s existence is that some types have `Apply` instances but not `Applicative` - one example is `Map[K, *]`. Consider the behavior of `pure` for `Map[K, A]`. Given a value of type `A`, we need to associate some arbitrary `K` to it but we have no way of doing that. However, given existing `Map[K, A]` and `Map[K, B]` (or `Map[K, A => B]`), it is straightforward to pair up (or apply functions to) values with the same key. Hence `Map[K, *]` has an `Apply` instance. ## Syntax Syntax for `Applicative` (or `Apply`) is available under the `cats.implicits._` import. The most interesting syntax is focused on composing independent effects: it works just like the methods for composition we saw above (`map3`, `tuple3`, etc.), but achieves a slightly friendlier syntax by enriching Scala's standard tuple types. For example, we've already seen this code for mapping over three options together: ```scala mdoc Applicative[Option].map3(username, password, url)(attemptConnect) ``` With the applicative syntax, we can change this to the slightly shorter: ```scala mdoc import cats.syntax.all._ (username, password, url).mapN(attemptConnect) ``` We don't have to mention the type or specify the number of values we're composing together, so there's a little less boilerplate here. Another very useful `Apply` syntax is `tupled`, which allows a tuple of effectful values to be composed into a single effectful value containing a tuple. ```scala mdoc import cats.syntax.all._ val optPair: Option[(String, String)] = (username, password).tupled ``` ### See also Parallel variants Both `tupled` and `mapN` have [parallel](parallel.md) variant operations, named `parTupled` and `parMapN` respectively. Regular `tupled`/`mapN` evaluate their effects from left to right ("sequentially"), while `parTupled`/`parMapN` evaluate in an indeterminate order, or in parallel. The difference can be understood intuitively when the effect is an executable task, such as `IO` from [Cats Effect](https://typelevel.org/cats-effect/docs/concepts#concurrent). In this case, the parallel variants enable you to compose tuples of tasks into a single task that will run its sub-tasks concurrently. ## Further Reading * [Applicative Programming with Effects][applicativePaper] - McBride, Patterson. JFP 2008. [applicativePaper]: http://www.staff.city.ac.uk/~ross/papers/Applicative.html "Applicative Programming with Effects" --- ### Typeclasses Applicativemonaderror (`docs/typeclasses/applicativemonaderror.md`) # ApplicativeError and MonadError API Documentation: @:api(cats.ApplicativeError), @:api(cats.MonadError) ## Applicative Error ### Description `ApplicativeError` extends `Applicative` to provide handling for types that represent the quality of an exception or an error, for example, `Either[E, A]` ### TypeClass Definition `ApplicativeError` is defined by the following `trait` ```scala trait ApplicativeError[F[_], E] extends Applicative[F] { def raiseError[A](e: E): F[A] def handleErrorWith[A](fa: F[A])(f: E => F[A]): F[A] def handleError[A](fa: F[A])(f: E => A): F[A] def attempt[A](fa: F[A]): F[Either[E, A]] //More functions elided } ``` ### Use Case #### `Either` We can start with a less abstract way of performing a function. Here we will divide one number by another. ```scala mdoc:silent def attemptDivide(x: Int, y: Int): Either[String, Int] = { if (y == 0) Left("divisor is zero") else { Right(x / y) } } ``` While fine in the above approach, we can abstract the `Either` away to support any other kind of "error" type without having to create multiple functions with different "container" types. ```scala mdoc:silent import cats._ import cats.syntax.all._ def attemptDivideApplicativeError[F[_]](x: Int, y: Int)(implicit ae: ApplicativeError[F, String]): F[Int] = { if (y == 0) ae.raiseError("divisor is error") else { ae.pure(x/y) } } ``` The above method summons `ApplicativeError` to provide behavior representing an error where the end-user, based on type, will get their appropriate response. `ApplicativeError` is an `Applicative`, which means all `Applicative` functions are available for use. One such method is `pure`, which will return the `F[_]` representation, where `F` could represent `Either`. Another method that you will see is `raiseError`, which will generate the specific error type depending on what `F[_]` represents. If `F[_]` is an `Either`, then `ae.raiseError` will return `Left`. If `F[_]` represents a `Validation`, then `ae.raiseError` will return `Invalid`. For example, if we want to use an `Either` as our error representation, we can do the following: ```scala mdoc:silent type OnError[A] = Either[String, A] val e: OnError[Int] = attemptDivideApplicativeError(30, 10) ``` or simply via assignment ```scala mdoc:silent val f: Either[String, Int] = attemptDivideApplicativeError(30, 10) ``` #### `Validated` Given the same function `attemptDivideApplicativeError`, we can call that function again but with a different return type, since the `ApplicativeError` can support other "error" based types. Here we will use `cats.data.Validated` when calling `attemptDivideApplicativeError`. Notice that `attemptDivideApplicativeError` is the same as we defined above, so we make no other changes. ```scala mdoc:silent import cats.syntax.all._ import cats.data.Validated type MyValidated[A] = Validated[String, A] val g = attemptDivideApplicativeError[MyValidated](30, 10) ``` We can inline the right projection type alias, `MyValidated`, doing the following: ```scala mdoc:silent val h = attemptDivideApplicativeError[({ type T[A] = Validated[String, A]})#T](30, 10) ``` Or we can use [KindProjector](https://github.com/typelevel/kind-projector) to make this more refined and readable ```scala mdoc:silent val j = attemptDivideApplicativeError[Validated[String, *]](30, 10) ``` ### It is an `Applicative` after all As a Reminder, this is an [`Applicative`](applicative.md) so all the methods of `Applicative` are available to you to use in manipulating your values, `ap`, `mapN`, etc. In the following example, notice we are using `Applicative`'s `map2`, and of course, `pure` which also is a form of `Applicative`. ```scala mdoc:silent import cats.syntax.all._ def attemptDivideApplicativeErrorWithMap2[F[_]](x: Int, y: Int)(implicit ae: ApplicativeError[F, String]): F[_] = { if (y == 0) ae.raiseError("divisor is error") else { val fa = ae.pure(x) val fb = ae.pure(y) ae.map2(fa, fb)(_ / _) } } ``` ### Handling Errors `ApplicativeError` has methods to handle what to do when `F[_]` represents an error. In the following example, `attemptDivideApplicativeErrorAbove2` creates an error representation if the divisor is `0` or `1` with the message "Bad Math" or "Waste of Time". We will feed the result from `attemptDivideApplicativeErrorAbove2` into the `handler` method, where this method will pattern match on the message and provide an alternative outcome. ```scala mdoc:silent import cats.syntax.all._ def attemptDivideApplicativeErrorAbove2[F[_]](x: Int, y: Int)(implicit ae: ApplicativeError[F, String]): F[Int] = if (y == 0) ae.raiseError("Bad Math") else if (y == 1) ae.raiseError("Waste of Time") else ae.pure(x / y) def handler[F[_]](f: F[Int])(implicit ae: ApplicativeError[F, String]): F[Int] = { ae.handleError(f) { case "Bad Math" => -1 case "Waste of Time" => -2 case _ => -3 } } ``` Running the following will result in `Right(-1)` ```scala mdoc:silent handler(attemptDivideApplicativeErrorAbove2(3, 0)) ``` `handleErrorWith` is nearly the same as `handleError` but instead of returning a value `A`, we will return `F[_]`. This could provide us the opportunity to make it very abstract and return a value from a `Monoid.empty`. ```scala mdoc:silent def handlerErrorWith[F[_], M[_], A](f: F[A])(implicit F: ApplicativeError[F, String], M:Monoid[A]): F[A] = { F.handleErrorWith(f)(_ => F.pure(M.empty)) } ``` Running the following will result in `Right(0)` ```scala mdoc:silent handlerErrorWith(attemptDivideApplicativeErrorAbove2(3, 0)) ``` ### Handling Exceptions There will inevitably come a time when your nice `ApplicativeError` code will have to interact with exception-throwing code. Handling such situations is easy enough. ```scala mdoc def parseInt[F[_]](input: String)(implicit F: ApplicativeError[F, Throwable]): F[Int] = try { F.pure(input.toInt) } catch { case nfe: NumberFormatException => F.raiseError(nfe) } parseInt[Either[Throwable, *]]("123") parseInt[Either[Throwable, *]]("abc") ``` However, this can get tedious quickly. `ApplicativeError` has a `catchOnly` method that allows you to pass it a function, along with the type of exception you want to catch, and does the above for you. ```scala mdoc:nest def parseInt[F[_]](input: String)(implicit F: ApplicativeError[F, Throwable]): F[Int] = F.catchOnly[NumberFormatException](input.toInt) parseInt[Either[Throwable, *]]("abc") ``` If you want to catch all (non-fatal) throwables, you can use `catchNonFatal`. ```scala mdoc:nest def parseInt[F[_]](input: String)(implicit F: ApplicativeError[F, Throwable]): F[Int] = F.catchNonFatal(input.toInt) parseInt[Either[Throwable, *]]("abc") ``` ## MonadError ### Description Since a `Monad` extends an `Applicative`, there is naturally a `MonadError` that will extend the functionality of the `ApplicativeError` to provide `flatMap` composition. ### TypeClass Definition The Definition for `MonadError` extends `Monad` which provides the methods, `flatMap`, `whileM_`. `MonadError` also provides error handling methods like `ensure`, `ensureOr`, `adaptError`, `rethrow`. ```scala trait MonadError[F[_], E] extends ApplicativeError[F, E] with Monad[F] { def ensure[A](fa: F[A])(error: => E)(predicate: A => Boolean): F[A] def ensureOr[A](fa: F[A])(error: A => E)(predicate: A => Boolean): F[A] def adaptError[A](fa: F[A])(pf: PartialFunction[E, E]): F[A] def rethrow[A, EE <: E](fa: F[Either[EE, A]]): F[A] } ``` ### Use Case Given a method that accepts a tuple of coordinates, it finds the closest city. For this example we will hard-code "Minneapolis, MN," but you can imagine for the sake of In this example, you would either consult a database or a web service. ```scala mdoc:silent def getCityClosestToCoordinate[F[_]](x: (Int, Int))(implicit ae: ApplicativeError[F, String]): F[String] = { ae.pure("Minneapolis, MN") } ``` Next, let's follow up with another method, `getTemperatureByCity`, that given a city, possibly a city that was just discovered by its coordinates, we get the temperature for that city. Here, for the sake of demonstration, we are hardcoding a temperature of 78°F. ```scala mdoc:silent def getTemperatureByCity[F[_]](city: String)(implicit ae: ApplicativeError[F, String]): F[Int] = { ae.pure(78) } ``` With the methods that we will compose in place let's create a method that will compose the above methods using a for comprehension which interprets to a `flatMap`-`map` combination. `getTemperatureByCoordinates`'s parameterized type `[F[_]:MonadError[*[_], String]` injects `F[_]` into `MonadError[*[_], String]`; thus if the "error type" you wish to use is `Either[String, *]`, the `Either` would be placed in the hole of `MonadError`, in this case, `MonadError[Either[String, *], String]` `getTemperatureByCoordinates` accepts a `Tuple2` of `Int` and `Int` and returns `F`, which represents our `MonadError`, which can be a type like `Either` or `Validated`. In the method, since `getCityClosestToCoordinate` and `getTemperatureByCity` both return potential error types and they are monadic, we can compose them with a for comprehension. ```scala mdoc:silent def getTemperatureByCoordinates[F[_]: MonadError[*[_], String]](x: (Int, Int)): F[Int] = { for { c <- getCityClosestToCoordinate[F](x) t <- getTemperatureByCity[F](c) } yield t } ``` We can call `getTemperatureByCoordinates` with the following sample, which will return `78`. ```scala mdoc:silent type MyEither[A] = Either[String, A] getTemperatureByCoordinates[MyEither]((44, 93)) ``` With TypeLevel Cats, how you structure your methods is up to you: if you wanted to create `getTemperatureByCoordinates` without a Scala [context bound](https://docs.scala-lang.org/tutorials/FAQ/context-bounds.html) for `MonadError`, but create an `implicit` parameter for your `MonadError` you can have access to some additional methods. In the following example, we create an `implicit` `MonadError` parameter and call it `me`. Using the `me` reference, we can call any one of its specialized methods, like `raiseError`, to raise an error representation when things go wrong. ```scala mdoc:silent def getTemperatureByCoordinatesAlternate[F[_]](x: (Int, Int))(implicit me: MonadError[F, String]): F[Int] = { if (x._1 < 0 || x._2 < 0) me.raiseError("Invalid Coordinates") else for { c <- getCityClosestToCoordinate[F](x) t <- getTemperatureByCity[F](c) } yield t } ``` --- ### Typeclasses Applicativetraverse (`docs/typeclasses/applicativetraverse.md`) # Applicative and Traversable Functors ## An example from the standard library One of the most useful functions when working with `scala.concurrent.Future` is `Future.traverse`, presented below in a simplified form. ```scala mdoc:silent import scala.concurrent.{ExecutionContext, Future} def traverseFuture[A, B](as: List[A])(f: A => Future[B])(implicit ec: ExecutionContext): Future[List[B]] = Future.traverse(as)(f) ``` `traverseFuture` takes a `List[A]` and for each `A` in the list applies the function `f` to it, gathering results as it goes along. `f` is often referred to as an *effectful* function, where the `Future` effect is running the computation concurrently, presumably on another thread. This effect is apparent in the result of the function, which has gathered results inside `Future`. But what if the effect we wanted wasn't `Future`? What if instead of concurrency for our effect we wanted validation (`Option`, `Either`, `Validated`) or `State` ? It turns out we can abstract out the commonalities between all these data types and write a generic `traverse` function once and for all. We can even go further and abstract over data types that can be traversed over such as `List`, `Vector`, and `Option`. In this series we will build up the machinery needed to generalize the standard library's `Future.traverse` into its fully abstract and most reusable form. If you'd like to read the published literature on these ideas, some good starting points are "[Applicative Programming with Effects][applicativeProgramming]" by McBride and Patterson, and "[The Essence of the Iterator Pattern][iterator]" by Gibbons and Oliveira. [applicativeProgramming]: http://www.staff.city.ac.uk/~ross/papers/Applicative.html "Applicative Programming with Effects" [iterator]: https://www.cs.ox.ac.uk/jeremy.gibbons/publications/iterator.pdf "The Essence of the Iterator Pattern" --- ### Typeclasses Arrow (`docs/typeclasses/arrow.md`) # Arrow API Documentation: @:api(cats.arrow.Arrow) `Arrow` is a type class for modeling composable relationships between two types. One example of such a composable relationship is function `A => B`; other examples include `cats.data.Kleisli`(wrapping an `A => F[B]`, also known as `ReaderT`), and `cats.data.Cokleisli`(wrapping an `F[A] => B`). These type constructors all have `Arrow` instances. An arrow `F[A, B]` can be thought of as representing a computation from `A` to `B` with some context, just like a functor/applicative/monad `F[A]` represents a value `A` with some context. Having an `Arrow` instance for a type constructor `F[_, _]` means that an `F[_, _]` can be composed and combined with other `F[_, _]`s. You will be able to do things like: - Lifting a function `ab: A => B` into arrow `F[A, B]` with `Arrow[F].lift(ab)`. If `F` is `Function1` then `A => B` is the same as `F[A, B]` so `lift` is just the identity function. - Composing `fab: F[A, B]` and `fbc: F[B, C]` into `fac: F[A, C]` with `Arrow[F].compose(fbc, fab)`, or `fab >>> fbc`. If `F` is `Function1` then `>>>` becomes an alias for `andThen`. - Taking two arrows `fab: F[A, B]` and `fcd: F[C, D]` and combining them into `F[(A, C), (B, D)]` with `fab.split(fcd)` or `fab *** fcd`. The resulting arrow takes two inputs and processes them with two arrows, one for each input. - Taking an arrow `fab: F[A, B]` and turning it into `F[(A, C), (B, C)]` with `fab.first`. The resulting arrow takes two inputs, processes the first input and leaves the second input as it is. A similar method, `fab.second`, turns `F[A, B]` into `F[(C, A), (C, B)]`. ## Examples ### `Function1` `scala.Function1` has an `Arrow` instance, so you can use all the above methods on `Function1`. The Scala standard library has the `compose` and `andThen` methods for composing `Function1`s, but the `Arrow` instance offers more powerful options. Suppose we want to write a function `meanAndVar`, that takes a `List[Int]` and returns the pair of mean and variance. To do so, we first define a `combine` function that combines two arrows into a single arrow, which takes an input and processes two copies of it with two arrows. `combine` can be defined in terms of `Arrow` operations `lift`, `>>>` and `***`: ```scala mdoc:silent import cats.arrow.Arrow import cats.syntax.all._ def combine[F[_, _]: Arrow, A, B, C](fab: F[A, B], fac: F[A, C]): F[A, (B, C)] = Arrow[F].lift((a: A) => (a, a)) >>> (fab *** fac) ``` We can then create functions `mean: List[Int] => Double`, `variance: List[Int] => Double` and `meanAndVar: List[Int] => (Double, Double)` using the `combine` method and `Arrow` operations: ```scala mdoc:silent val mean: List[Int] => Double = combine((_: List[Int]).sum, (_: List[Int]).size) >>> {case (x, y) => x.toDouble / y} val variance: List[Int] => Double = // Variance is mean of square minus square of mean combine(((_: List[Int]).map(x => x * x)) >>> mean, mean) >>> {case (x, y) => x - y * y} val meanAndVar: List[Int] => (Double, Double) = combine(mean, variance) ``` ```scala mdoc meanAndVar(List(1, 2, 3, 4)) ``` Of course, a more natural way to implement `mean` and `variance` would be: ```scala mdoc:silent val mean2: List[Int] => Double = xs => xs.sum.toDouble / xs.size val variance2: List[Int] => Double = xs => mean2(xs.map(x => x * x)) - scala.math.pow(mean2(xs), 2.0) ``` However, `Arrow` methods are more general and provide a common structure for type constructors that have `Arrow` instances. They are also a more abstract way of stitching computations together. ### `Kleisli` A `Kleisli[F[_], A, B]` represents a function `A => F[B]`. You cannot directly compose an `A => F[B]` with a `B => F[C]` with functional composition, since the codomain of the first function is `F[B]` while the domain of the second function is `B`; however, since `Kleisli` is an arrow (as long as `F` is a monad), you can easily compose `Kleisli[F[_], A, B]` with `Kleisli[F[_], B, C]` using `Arrow` operations. Suppose you want to take a `List[Int]`, and return the sum of the first and the last element (if exists). To do so, we can create two `Kleisli`s that find the `headOption` and `lastOption` of a `List[Int]`, respectively: ```scala mdoc:silent import cats.data.Kleisli val headK = Kleisli((_: List[Int]).headOption) val lastK = Kleisli((_: List[Int]).lastOption) ``` With `headK` and `lastK`, we can obtain the `Kleisli` arrow we want by combining them, and composing it with `_ + _`: ```scala mdoc:silent val headPlusLast = combine(headK, lastK) >>> Arrow[Kleisli[Option, *, *]].lift(((_: Int) + (_: Int)).tupled) ``` ```scala mdoc headPlusLast.run(List(2, 3, 5, 8)) headPlusLast.run(Nil) ``` ### `FancyFunction` In this example let's create our own `Arrow`. We shall create a fancy version of `Function1` called `FancyFunction`, that is capable of maintaining states. We then create an `Arrow` instance for `FancyFunction` and use it to compute the moving average of a list of numbers. ```scala mdoc:silent case class FancyFunction[A, B](run: A => (FancyFunction[A, B], B)) ``` That is, given an `A`, it not only returns a `B`, but also returns a new `FancyFunction[A, B]`. This sounds similar to the `State` monad (which returns a result and a new `State` from an initial `State`), and indeed, `FancyFunction` can be used to perform stateful transformations. To run a stateful computation using a `FancyFunction` on a list of inputs, and collect the output into another list, we can define the following `runList` helper function: ```scala mdoc:silent def runList[A, B](ff: FancyFunction[A, B], as: List[A]): List[B] = as match { case h :: t => val (ff2, b) = ff.run(h) b :: runList(ff2, t) case _ => List() } ``` In `runList`, the head element in `List[A]` is fed to `ff`, and each subsequent element is fed to a `FancyFunction` which is generated by running the previous `FancyFunction` on the previous element. If we have an `as: List[Int]`, and an `avg: FancyFunction[Int, Double]` which takes an integer and computes the average of all integers it has seen so far, we can call `runList(avg, as)` to get the list of moving average of `as`. Next let's create an `Arrow` instance for `FancyFunction` and see how to implement the `avg` arrow. To create an `Arrow` instance for a type `F[A, B]`, the following abstract methods need to be implemented: ``` def lift[A, B](f: A => B): F[A, B] def id[A]: F[A, A] def compose[A, B, C](f: F[B, C], g: F[A, B]): F[A, C] def first[A, B, C](fa: F[A, B]): F[(A, C), (B, C)] ``` Thus the `Arrow` instance for `FancyFunction` would be: ```scala mdoc:silent implicit val arrowInstance: Arrow[FancyFunction] = new Arrow[FancyFunction] { override def lift[A, B](f: A => B): FancyFunction[A, B] = FancyFunction(lift(f) -> f(_)) override def first[A, B, C](fa: FancyFunction[A, B]): FancyFunction[(A, C), (B, C)] = FancyFunction {case (a, c) => val (fa2, b) = fa.run(a) (first(fa2), (b, c)) } override def id[A]: FancyFunction[A, A] = FancyFunction(id -> _) override def compose[A, B, C](f: FancyFunction[B, C], g: FancyFunction[A, B]): FancyFunction[A, C] = FancyFunction {a => val (gg, b) = g.run(a) val (ff, c) = f.run(b) (compose(ff, gg), c) } } ``` Once we have an `Arrow[FancyFunction]`, we can start to do interesting things with it. First, let's create a method `accum` that returns a `FancyFunction`, which accumulates values fed to it using the accumulation function `f` and the starting value `b`: ```scala mdoc:silent def accum[A, B](b: B)(f: (A, B) => B): FancyFunction[A, B] = FancyFunction {a => val b2 = f(a, b) (accum(b2)(f), b2) } ``` ```scala mdoc runList(accum[Int, Int](0)(_ + _), List(6, 5, 4, 3, 2, 1)) ``` To make the aformentioned `avg` arrow, we need to keep track of both the count and the sum of the numbers we have seen so far. To do so, we will combine several `FancyFunction`s to get the `avg` arrow we want. We first define arrow `sum` in terms of `accum`, and define arrow `count` by composing `_ => 1` with `sum`: ```scala mdoc:silent import cats.kernel.Monoid def sum[A: Monoid]: FancyFunction[A, A] = accum(Monoid[A].empty)(_ |+| _) def count[A]: FancyFunction[A, Int] = Arrow[FancyFunction].lift((_: A) => 1) >>> sum ``` Finally, we create the `avg` arrow in terms of the arrows we have so far: ```scala mdoc:silent def avg: FancyFunction[Int, Double] = combine(sum[Int], count[Int]) >>> Arrow[FancyFunction].lift{case (x, y) => x.toDouble / y} ``` ```scala mdoc runList(avg, List(1, 10, 100, 1000)) ``` --- ### Typeclasses Arrowchoice (`docs/typeclasses/arrowchoice.md`) # Arrow Choice API Documentation: @:api(cats.arrow.Choice) Usually we deal with function more often, we're so familiar with `A => B`. If we have two functions `A => C` and `B => C`, how can we compose them into a single function that can take either A or B and produce a C? So basically we just look for a function that has type `(A => C) => (B => C) => (Either[A, B] => C)`. This is exactly typeclass `Choice` provided, if we make `=>` more generic such as `F[_,_]`, you will get a `Choice` ```scala trait Choice[F[_, _]] { def choice[A, B, C](fac: F[A, C], fbc: F[B, C]): F[Either[A, B], C] } ``` Note the **infix** notation of `choice` is `|||`. ## Middleware A very useful case of `Choice` is middleware in HTTP server. Take Http4s for example: `HttpRoutes[F]` in Http4s is defined as [Kleisli](../datatypes/kleisli.md) ```scala type HttpRoutes[F[_]] = Kleisli[OptionT[F, *], Request[F], Response[F]] // defined type HttpRoutes def routes[F[_]]: HttpRoutes[F] = ??? // defined function routes ``` If we like to have an authentication middleware that composes the route, we can simply define middleware as: ```scala type Middleware[F[_]] = Kleisli[OptionT[F, *], Request[F], Either[Response[F], Request[F]]] // defined type Middleware def auth[F[_]]: Middleware[F] = ??? // defined function auth ``` Which means the `Request[F]` goes through the middleware, will become option of `Either[Response[F], Request[F]]`, where `Left` means the request is denied and return immediately, `Right` means the authentication is OK and request will get pass. Now we need to define what we should do when middleware returns `Left`: ```scala def reject[F[_]:Monad]: Kleisli[OptionT[F, *], Response[F], Response[F]] = Kleisli.ask[OptionT[F, *], Response[F]] // defined function reject ``` Now compose middleware with route ```scala def authedRoute[F[_]:Monad] = auth[F] andThen (reject[F] ||| routes[F]) // defined function authedRoute ``` You will then get a new route that has authentication ability by composing Kleisli. ## HTTP Response Another example will be HTTP response handler. ```scala val resp: IO[Either[Throwable, String]] = httpClient.expect[String](uri"https://google.com/").attempt ``` `attempt` is syntax from [`MonadError`](applicativemonaderror.md) When we need to handle error, without `Choice` the handler would be something like: ```scala resp.flatMap{ case Left => ??? case Right => ??? } ``` With `Choice` there will be more composable solution without embedded logic in pattern matching: ```scala def recover[A](error: Throwable): IO[A] = ??? def processResp[A](resp: String): IO[A] = ??? resp >>= (recover _ ||| processResp _) ``` # `ArrowChoice` `ArrowChoice` is an extended version of `Choice`, which has one more method `choose`, with syntax `+++` ```scala trait ArrowChoice[F[_, _]] extends Choice[F] { def choose[A, B, C, D](f: F[A, C])(g: F[B, D]): F[Either[A, B], Either[C, D]] } ``` With the middleware example, you can think `ArrowChoice` is middleware of middleware. For example if we want to append log to middleware of `auth`, that can log both when rejected response and pass request: ```scala def logReject[F[_]]: Kleisli[OptionT[F, *], Response[F], Response[F]] = ??? // defined function logReject def logThrough[F[_]]: Kleisli[OptionT[F, *], Request[F], Request[F]] = ??? // defined function logThrough ``` See how easy to compose log functionality into our `authedRoute`: ```scala def authedRoute[F[_]:Monad] = auth[F] andThen (logReject[F] +++ logThrough[F]) andThen (reject[F] ||| routes[F]) // defined function authedRoute ``` --- ### Typeclasses Bifoldable (`docs/typeclasses/bifoldable.md`) # Bifoldable API Documentation: @:api(cats.Bifoldable) `Bifoldable[F[_,_]]` instances identify data structures with two independent `Foldable` that fold to the same summary value. As a reminder `Foldable` is implemented in terms of `foldLeft` and `foldRight`; similarly `Bifoldable` is implemented in terms of: ```scala //eagerly performs a left-associative bi-fold over `fab` def bifoldLeft[A, B, C](fab: F[A, B], c: C)(f: (C, A) => C, g: (C, B) => C): C //lazily performs a right-associative bi-fold over `fab` def bifoldRight[A, B, C](fab: F[A, B], c: Eval[C])(f: (A, Eval[C]) => Eval[C], g: (B, Eval[C]) => Eval[C]): Eval[C] ``` and by implementing those 2 methods you also get: ```scala def bifold[A, B](fab: F[A, B])(implicit A: Monoid[A], B: Monoid[B]): (A, B) def bifoldMap[A, B, C](fab: F[A, B])(f: A => C, g: B => C)(implicit C: Monoid[C]): C ``` A lawful instance must have `bifoldLeft\Right` consistent with `bifoldMap`; left\right bi-folds based on associative functions should output similar results. ## Either and Validated as Bifoldable Assume multiple input paths and system requests that end up as `Either[Exception, *]` or `Validated[Exception, *]`. The requirement is to track the amount of errors and store the content of valid requests. First add the implicits: ```scala mdoc import cats._ import cats.data._ import cats.syntax.all._ ``` then let's define a summary class capable of storing this info: ```scala mdoc case class Report(entries: Chain[String], errors: Int) { def withEntries(entry: String): Report = this.copy(entries = entries :+ entry) def withError: Report = this.copy(errors = errors + 1) } ``` `Bifoldable` is useful to get a summary value from `F[_,_]` data types: ```scala mdoc def update[F[_, _]: Bifoldable](current: Report)(result: F[Exception, String]): Report = result.bifoldLeft(current)((acc, _) => acc.withError, (acc, user) => acc.withEntries(user)) ``` Here is the system input: ```scala mdoc val validated = List( Validated.valid[Exception, String]("valid request 1"), Validated.valid[Exception, String]("valid request 2"), Validated.invalid[Exception, String](new RuntimeException("Not a valid request")) ) val attempted = List( Either.right[Exception, String]("valid request 1"), Either.right[Exception, String]("valid request 2"), Either.left[Exception, String](new RuntimeException("Not a valid request")) ) ``` and bi-fold each value into the accumulator: ```scala mdoc val empty = Report(Chain.empty, 0) validated .foldl(empty)((acc, validation) => update(acc)(validation)) attempted .foldl(empty)((acc, attempt) => update(acc)(attempt)) ``` ## Tuple as Bifoldable Assume we have `(String, String, Int)` and to get our summary we need `_1` and `_3`. The existing implementations for `(*, *)`, `(T0, *, *)`, `(T0, T1, *, *)` .. aren't useful. Let's make a new `Bifoldable` instance: ```scala mdoc implicit def bifoldableForTuple3[A0]: Bifoldable[(*, A0, *)] = new Bifoldable[(*, A0, *)] { def bifoldLeft[A, B, C](fa: (A, A0, B), c: C)(f: (C, A) => C, g: (C, B) => C): C = g(f(c, fa._1), fa._3) def bifoldRight[A, B, C](fa: (A, A0, B), c: Eval[C])(f: (A, Eval[C]) => Eval[C], g: (B, Eval[C]) => Eval[C]): Eval[C] = g(fa._3, f(fa._1, c)) } ``` As we were saying in the beginning a lawful `Bifoldable` should have `bifoldLeft\Right` consistent with `bifoldMap`. Let's check our instance: ```scala mdoc //(name, age, occupation) val description = ("Niki", 22, "Developer") val expected = Bifoldable[(*, Int, *)].bifoldMap(description)(s => s, s => s) val left = Bifoldable[(*, Int, *)].bifoldLeft(description, Monoid[String].empty)( (acc, s) => acc |+| s, (acc, s) => acc |+| s ) val right = Bifoldable[(*, Int, *)].bifoldRight(description, Eval.later(Monoid[String].empty))( (s, acc) => acc.map(_ |+| s), (s, acc) => acc.map(_ |+| s) ) left === expected right.value === expected ``` **NOTE:** This instance would not be lawful if `bifoldRight` in particular would use a different ordering. Going from right to left as opposed to left to right means: ```scala mdoc def bifoldRight[A, B, C](fa: (A, Int, B), c: Eval[C])(f: (A, Eval[C]) => Eval[C], g: (B, Eval[C]) => Eval[C]): Eval[C] = f(fa._1, g(fa._3, c)) ``` and it would also reverse the output making the instance unlawful: ```scala mdoc val reversedRight = bifoldRight(description, Eval.later(Monoid[String].empty))( (s, acc) => acc.map(_ |+| s), (s, acc) => acc.map(_ |+| s) ) reversedRight.value === expected ``` ## Bifoldable `compose` If `F[_,_]` and `G[_,_]` have `Bifoldable` instances then so does `F[G[_,_],G[_,_]]`. Examples for `(Either[*, *], Either[*, *])` and `((*, *), (*, *))`: ```scala mdoc Bifoldable[(*, *)] .compose(Bifoldable[(*, *)]) .bifoldLeft((("name1 ", 1000),("name2 ", 2000)), List.empty[String])((acc, name) => acc :+ name, (acc, _) => acc) Bifoldable[(*, *)] .compose(Bifoldable[Either[*, *]]) .bifold((Left(1), Right(2))) ``` --- ### Typeclasses Bifunctor (`docs/typeclasses/bifunctor.md`) # Bifunctor API Documentation: @:api(cats.Bifunctor) `Bifunctor` takes two type parameters instead of one, and is a functor in both of these parameters. It defines a function `bimap`, which allows for mapping over both arguments at the same time. Its signature is as follows: ```scala def bimap[A, B, C, D](fab: F[A, B])(f: A => C, g: B => D): F[C, D] ``` ## Either as a Bifunctor Probably the most widely used Bifunctor instance is the Either data type. Say you have a value that is either an error or a `ZonedDateTime` instance. You also want to react to both possibilities - if there was a failure, you want to convert it to your own `DomainError`, and if the result was a success, you want to convert it to an UNIX timestamp. ```scala mdoc:silent import cats._ import cats.syntax.all._ import java.time._ case class DomainError(message: String) def dateTimeFromUser: Either[Throwable, ZonedDateTime] = Right(ZonedDateTime.now()) // Example definition ``` ```scala mdoc dateTimeFromUser.bimap( error => DomainError(error.getMessage), dateTime => dateTime.toEpochSecond ) ``` `Bifunctor` also defines a convenience function called `leftMap`, which is defined as follows: ```scala def leftMap[A, B, C](fab: F[A, B])(f: A => C): F[C, B] = bimap(fab)(f, identity) ``` There is no `rightMap` however - use `map` instead. The reasoning behind this is that in Cats, the instances of `Bifunctor` are also mostly instances of `Functor`, as it is the case with `Either`. ## Tuple2 as a Bifunctor Another very popular `Bifunctor` is that for the `Tuple2` data type, or `(A, B)` for types `A` and `B`. Let's say we have a list of balances and want divide them by the number of months in the lifetime of the account holder. The balances are given in cents. A bit contrived, but we want an average contribution per month to the given account. We want the result in dollars per month. The lifetime is given in the number of years the account has been active. ```scala mdoc val records: List[(Int, Int)] = List((450000, 3), (770000, 4), (990000, 2), (2100, 4), (43300, 3)) def calculateContributionPerMonth(balance: Int, lifetime: Int) = balance / lifetime val result: List[Int] = records.map( record => record.bimap( cents => cents / 100, years => 12 * years ) ).map((calculateContributionPerMonth _).tupled) ``` As you can see, this instance makes it convenient to process two related pieces of data in independent ways, especially when there is no state relationship between the two until processing is complete. Note that, just as with the bifunctor for `Either`, we do not have a `rightMap` function since the relevant instances of `Bifunctor` induce a `Functor` in the second argument, so we just use `map`. --- ### Typeclasses Bimonad (`docs/typeclasses/bimonad.md`) # Bimonad API Documentation: @:api(cats.Bimonad) The `Bimonad` trait directly extends `Monad` and `Comonad` without introducing new methods. `Bimonad` is different from other `Bi` typeclasses like `Bifunctor`, `Bifoldable` or `Bitraverse` where the prefix describes a `F[_, _]`. The `Bimonad` is a `F[_]` and the `Bi` prefix has a different meaning here: it's both a `Monad` and a `Comonad`. Keep in mind `Bimonad` has its own added laws so something that is both monadic and comonadic may not necessarily be a lawful `Bimonad`. If you use `Bimonad` as a convenience type such that: ```scala def f[T[_]: Monad: Comonad, S](fa: T[S]): S ``` is re-written to: ```scala def f[T[_]: Bimonad, S](fa: T[S]): S ``` then `T[_]` also needs to respect an extra set of laws. ### NonEmptyList as a Bimonad `NonEmptyList[_]` is a lawful `Bimonad` so you can chain computations (like a `Monad`) and `extract` the result at the end (like a `Comonad`). Here is a possible implementation: ```scala mdoc import cats._ import cats.data._ import cats.syntax.all._ implicit val nelBimonad: Bimonad[NonEmptyList] = new Bimonad[NonEmptyList] { // in order to have a lawful bimonad `pure` and `extract` need to respect: `nelBimonad.extract(nelBimonad.pure(a)) <-> a` override def pure[A](a: A): NonEmptyList[A] = NonEmptyList.one(a) override def extract[A](fa: NonEmptyList[A]): A = fa.head // use coflatMap from NonEmptyList override def coflatMap[A, B](fa: NonEmptyList[A])(f: NonEmptyList[A] => B): NonEmptyList[B] = fa.coflatMap(f) // use flatMap from NonEmptyList override def flatMap[A, B](fa: NonEmptyList[A])(f: A => NonEmptyList[B]): NonEmptyList[B] = fa.flatMap(f) // the tailRecM implementation is not the subject of this material // as an exercise try to implement it yourself override def tailRecM[A, B](a: A)(fn: A => NonEmptyList[Either[A, B]]): NonEmptyList[B] = ??? } ``` Note the equivalence: ```scala mdoc nelBimonad.pure(true).extract === NonEmptyList.one(true).head ``` Using generic bimonad syntax we could define a function that appends and extracts a configuration: ```scala mdoc def make[T[_]: Bimonad](config: T[String]): String = config .flatMap(c => Bimonad[T].pure(c + " with option A")) .flatMap(c => Bimonad[T].pure(c + " with option B")) .flatMap(c => Bimonad[T].pure(c + " with option C")) .extract ``` This works with one element non-empty lists: ```scala mdoc make(NonEmptyList.one("config")) ``` `Function0[_]` and `Eval[_]` are also lawful bimonads so the following calls are also valid: ```scala mdoc make(() => "config") make(Eval.later("config")) ``` --- ### Typeclasses Comonad (`docs/typeclasses/comonad.md`) # Comonad API Documentation: @:api(cats.Comonad) `Comonad` is a `Functor` and provides duals of the [`Monad`](monad.md) `pure` and `flatMap` functions. A dual to a function has the same types but the direction of the arrows are reversed. Whether or not that is useful, or even possible, depends on the particular type. For a more formal definition of duality, please refer to [https://ncatlab.org/nlab/show/duality](https://ncatlab.org/nlab/show/duality). ### extract Monads have `pure` from `Applicative` which gives you the ability to wrap a value `A` using the type constructor giving an `F[A]`. Comonad has `extract` which instead takes an `F[A]` and extracts the `A`. Therefore, to be able to implement extract we must have a type of which we are certain we can get an `A` from an `F[A]`. For example we cannot always get an `A` from a `List[A]` because if the list is empty there is nothing to get. For the same reason, `Option` doesn't have a `Comonad` instance, because we cannot always get an `A` from an Option, it may be empty too. Some examples that we can implement `Comonad` for include `OneAnd`, `Tuple2` and the "non empty" collections. First some imports. ```scala mdoc:silent import cats._ import cats.data._ import cats.syntax.all._ import cats.instances.list._ ``` `NonEmptyList` has a `Comonad` instance and its implementation of `extract` simply returns the head element of the list, which we know we will always have. ```scala mdoc NonEmptyList.of(1,2,3).extract ``` ### coflatMap `coflatMap` is the dual of Monad's `flatMap`. While `flatMap` allows us to chain together operations in a monadic context, `coflatMap` takes a value in some context `F[A]` and a function `F[A] => B` and returns a new value in a context `F[B]`. The default implementation of `coflatMap` for `NonEmptyList` will pass the supplied function with the whole list, then the tail of that, then the tail of that and so on. This is illustrated below. ```scala mdoc NonEmptyList.of(1,2,3,4,5).coflatMap(identity) ``` # CoflatMap While `FlatMap` is a weaker version of `Monad` that doesn't have the `pure` function, `CoflatMap` is a `Comonad` without the `extract` function. There are many instances of type classes in Cats that implement `CoflatMap` but not `Comonad`. For example we cannot write `extract` for `Option[A]` because there's no way to pull an `A` out of nowhere if the Option is empty. ```scala mdoc:silent def extract[A](fa : Option[A]): A = fa match { case Some(a) => a case None => ??? // What now? } ``` Another example is `List`. Remember we cannot write `extract` for list because lists can be empty, but we can implement the `coflatMap` and it works identically to the one shown above for `NonEmptyList`. ```scala mdoc List(1,2,3,4,5).coflatMap(identity) ``` --- ### Typeclasses Contravariant (`docs/typeclasses/contravariant.md`) # Contravariant API Documentation: @:api(cats.Contravariant) The `Contravariant` type class is for functors that define a `contramap` function with the following type: ```scala def contramap[A, B](fa: F[A])(f: B => A): F[B] ``` It looks like regular (also called `Covariant`) [`Functor`](functor.md)'s `map`, but with the `f` transformation reversed. Generally speaking, if you have some context `F[A]` for type `A`, and you can get an `A` value out of a `B` value — `Contravariant` allows you to get the `F[B]` context for `B`. Examples of `Contravariant` instances are [`Show`](show.md) and `scala.math.Ordering` (along with `cats.kernel.Order`). ## Contravariant instance for Show. Say we have a class `Money` with a `Show` instance, and a `Salary` class: ```scala mdoc:silent import cats._ import cats.syntax.all._ case class Money(amount: Int) case class Salary(size: Money) implicit val showMoney: Show[Money] = Show.show(m => s"$$${m.amount}") ``` If we want to show a `Salary` instance, we can just convert it to a `Money` instance and show that instead. Let's use `Show`'s `Contravariant`: ```scala mdoc implicit val showSalary: Show[Salary] = showMoney.contramap(_.size) Salary(Money(1000)).show ``` ## Contravariant instance for scala.math.Ordering. The `Show` example is trivial and quite far-fetched, let's see how `Contravariant` can help with orderings. The `scala.math.Ordering` type class defines comparison operations, e.g. `compare`: ```scala mdoc Ordering.Int.compare(2, 1) Ordering.Int.compare(1, 2) ``` There's also a method, called `by`, that creates new `Orderings` out of existing ones: ```scala def by[T, S](f: T => S)(implicit ord: Ordering[S]): Ordering[T] ``` In fact, it is just `contramap`, defined in a slightly different way! We supply `T => S` to receive `F[S] => F[T]` back. So let's use it to our advantage and get `Ordering[Money]` for free: ```scala mdoc // we need this for `<` to work import scala.math.Ordered._ implicit val moneyOrdering: Ordering[Money] = Ordering.by(_.amount) Money(100) < Money(200) ``` ## Subtyping Contravariant functors have a natural relationship with subtyping, dual to that of covariant functors: ```scala mdoc class A class B extends A val b: B = new B val a: A = b val showA: Show[A] = Show.show(a => "a!") val showB1: Show[B] = showA.contramap(b => b: A) val showB2: Show[B] = showA.contramap(identity[A]) val showB3: Show[B] = Contravariant[Show].narrow[A, B](showA) ``` Subtyping relationships are "lifted backwards" by contravariant functors, such that if `F` is a lawful contravariant functor and `B <: A` then `F[A] <: F[B]`, which is expressed by `Contravariant.narrow`. --- ### Typeclasses Contravariantmonoidal (`docs/typeclasses/contravariantmonoidal.md`) # Contravariant Monoidal API Documentation: @:api(cats.ContravariantMonoidal) The `ContravariantMonoidal` type class is for [`Contravariant`](contravariant.md) functors that can define a `product` function and a `unit` function. ```scala mdoc:silent import cats.Contravariant trait ContravariantMonoidal[F[_]] extends Contravariant[F] { def unit: F[Unit] def product[A, B](fa: F[A], fc: F[B]): F[(A, B)] def contramap2[A, B, C](fb: F[B], fc: F[C])(f: A => (B, C)): F[A] = contramap(product(fb, fc))(f) } ``` Notice that this allows us to define the `contramap2` function, much like the `map2` function and the `pure` function on the `Applicative` typeclass, but in reverse. Basically, if you have two contexts `F[B]` and `F[C]` for types `B` and `C`, as well as a way to produce types `B` and `C` simultaneously from a type `A`, then `ContravariantMonoidal` allows you to obtain a context `F[A]` for the type `A`. Examples of `ContravariantMonoidal` instances are [`Eq`](eq.md) and [`Const`](../datatypes/const.md), but there are also interesting instances for other types. ## Predicates Have `ContravariantMonoidal` An example application would be the case of predicates. Consider the type, ```scala mdoc:silent:reset import cats._ import cats.syntax.all._ case class Predicate[A](run: A => Boolean) ``` Then, we can exhibit a `ContravariantMonoidal` for `Predicate` by basing it on the `Monoid` for `Boolean` via `&&` as, ```scala mdoc:silent implicit val contravariantMonoidalPredicate: ContravariantMonoidal[Predicate] = new ContravariantMonoidal [Predicate] { def unit: Predicate[Unit] = Predicate[Unit](Function.const(true)) def product[A, B](fa: Predicate[A], fb: Predicate[B]): Predicate[(A, B)] = Predicate(x => fa.run(x._1) && fb.run(x._2)) def contramap[A, B](fa: Predicate[A])(f: B => A): Predicate[B] = Predicate(x => fa.run(f(x))) } ``` We could have also used `false` and `||`, but the "and" version tends to be a little more convenient for this application. Just like for `Contravariant`, we can `contramap` to pull `Predicates` back along functions. ```scala mdoc case class Money(value: Long) def isEven: Predicate[Long] = Predicate(_ % 2 == 0) def isEvenMoney: Predicate[Money] = isEven.contramap(_.value) isEvenMoney.run(Money(55)) ``` We can also lift functions contravariantly into the context instead of contramapping repeatedly. ```scala mdoc def times2Predicate: Predicate[Long] => Predicate[Long] = ContravariantMonoidal[Predicate].liftContravariant((x: Long) => 2*x) def liftMoney: Predicate[Long] => Predicate[Money] = ContravariantMonoidal[Predicate].liftContravariant(_.value) def trivial = times2Predicate(isEven) trivial.run(2) trivial.run(5) ``` More interestingly, we can combine multiple predicates using a `contramapN`. ```scala mdoc case class Transaction(value: Money, payee: String) def isEvan: Predicate[String] = Predicate(_ == "Evan") def isGreaterThan50Dollars: Predicate[Money] = liftMoney(Predicate(_ > 50)) def isEvenPaymentToEvanOfMoreThan50 = (isEvenMoney, isGreaterThan50Dollars, isEvan).contramapN( (trans: Transaction) => (trans.value, trans.value, trans.payee)) isEvenPaymentToEvanOfMoreThan50.run(Transaction(Money(56), "Evan")) ``` --- ### Typeclasses Eq (`docs/typeclasses/eq.md`) # Eq API Documentation: @:api(cats.kernel.Eq) Eq is an alternative to the standard Java `equals` method. It is defined by the single method `eqv`: ```scala def eqv(x: A, y: A): Boolean ``` In Scala it's possible to compare any two values using `==` (which desugars to Java `equals`). This is because `equals` type signature uses `Any` (Java's `Object`) to compare two values. This means that we can compare two completely unrelated types without getting a compiler error. The Scala compiler may warn us in some cases, but not all, which can lead to some weird bugs. For example this code will raise a warning at compile time: ```scala mdoc:warn 42 == "Hello" ``` While this code will compile without a hitch: ```scala mdoc "Hello" == 42 ``` Ideally, Scala shouldn't let us compare two types that can never be equal. As you can probably see in the type signature of `eqv`, it is impossible to compare two values of different types, eliminating these types of bugs altogether. The `Eq` syntax package also offers some handy symbolic operators: ```scala mdoc import cats.syntax.all._ 1 === 1 "Hello" =!= "World" ``` Implementing `Eq` instances yourself for every data type might seem like huge drawback compared to only slight gains of typesafety. Fortunately for us, we have two great options. One option is to use inbuilt helper functions. Another option is to use a small library called [kittens](https://github.com/typelevel/kittens), which can derive a lot of type class instances for our data types including `Eq`. The first option using `Eq.fromUniversalEquals` only defers to `==` and works like this: ```scala mdoc import cats.kernel.Eq import cats.syntax.all._ case class Foo(a: Int, b: String) implicit val eqFoo: Eq[Foo] = Eq.fromUniversalEquals Foo(10, "") === Foo(10, "") ``` For an example using Kittens check out the [kittens repo](https://github.com/typelevel/kittens). --- ### Typeclasses Foldable (`docs/typeclasses/foldable.md`) # Foldable API Documentation: @:api(cats.Foldable) Foldable type class instances can be defined for data structures that can be folded to a summary value. In the case of a collection (such as `List` or `Vector`), these methods will fold together (combine) the values contained in the collection to produce a single result. Most collection types have `foldLeft` methods, which will usually be used by the associated `Foldable[_]` instance. `Foldable[F]` is implemented in terms of two basic methods: - `foldLeft(fa, b)(f)` eagerly performs a left-associative fold over `fa`. - `foldRight(fa, b)(f)` lazily performs a right-associative fold over `fa`. Consider a simple list like `List(1, 2, 3)`. You could sum the numbers of this list using folds where `0` is the starting value (`b`) and integer addition (`+`) is the combination operation (`f`). Since `foldLeft` is left-associative, the execution of this fold would look something like `((0 + 1) + 2) + 3`. The execution of a similar `foldRight`-based solution would look something like `0 + (1 + (2 + 3))`. In this case, since integer addition is associative, both approaches will yield the same result. However, for non-associative operations, the two methods can produce different results. These form the basis for many other operations, see also: [A tutorial on the universality and expressiveness of fold](http://www.cs.nott.ac.uk/~gmh/fold.pdf) First some standard imports. ```scala mdoc:silent import cats._ import cats.syntax.all._ ``` And examples. ``` /* Detailed source-code truncated for AI context efficiency. */ ``` Hence when defining some new data structure, if we can define a `foldLeft` and `foldRight` we are able to provide many other useful operations, if not always the most efficient implementations, over the structure without further implementation. ------------------------------------------------------------------------------- Note that, in order to support laziness, the signature of `Foldable`'s `foldRight` is ```scala def foldRight[A, B](fa: F[A], lb: Eval[B])(f: (A, Eval[B]) => Eval[B]): Eval[B] ``` as opposed to ```scala def foldRight[A, B](fa: F[A], z: B)(f: (A, B) => B): B ``` which someone familiar with the `foldRight` from the collections in Scala's standard library might expect. This will prevent operations which are lazy in their right hand argument to traverse the entire structure unnecessarily. For example, if you have: ```scala mdoc val allFalse = LazyList.continually(false) ``` which is an infinite list of `false` values, and if you wanted to reduce this to a single false value using the logical and (`&&`). You intuitively know that the result of this operation should be `false`. It is not necessary to consider the entire list in order to determine this result, you only need to consider the first value. Using `foldRight` from the standard library *will* try to consider the entire list, and thus will eventually cause an out-of-memory error: ```scala // beware! throws OutOfMemoryError, which is irrecoverable allFalse.foldRight(true)(_ && _) ``` With the lazy `foldRight` on `Foldable`, the calculation terminates after looking at only one value: ```scala mdoc Foldable[LazyList].foldRight(allFalse, Eval.True)((a,b) => if (a) b else Eval.False).value ``` Unfortunately, since `foldRight` is defined on many collections - this extension clashes with the operation defined in `Foldable`. To get past this and make sure you're getting the lazy `foldRight` defined in `Foldable`, there's an alias `foldr`: ```scala mdoc allFalse.foldr(Eval.True)((a,b) => if (a) b else Eval.False).value ``` --- ### Typeclasses Functor (`docs/typeclasses/functor.md`) # Functor API Documentation: @:api(cats.Functor) `Functor` is a type class that abstracts over type constructors that can be `map`'ed over. Examples of such type constructors are `List`, `Option`, and `Future`. ```scala mdoc:silent trait Functor[F[_]] { def map[A, B](fa: F[A])(f: A => B): F[B] } // Example implementation for Option implicit val functorForOption: Functor[Option] = new Functor[Option] { def map[A, B](fa: Option[A])(f: A => B): Option[B] = fa match { case None => None case Some(a) => Some(f(a)) } } ``` A `Functor` instance must obey two laws: * Composition: Mapping with `f` and then again with `g` is the same as mapping once with the composition of `f` and `g` * `fa.map(f).map(g) = fa.map(f.andThen(g))` * Identity: Mapping with the identity function is a no-op * `fa.map(x => x) = fa` ## A different view Another way of viewing a `Functor[F]` is that `F` allows the lifting of a pure function `A => B` into the effectful function `F[A] => F[B]`. We can see this if we re-order the `map` signature above. ```scala mdoc:silent:nest trait Functor[F[_]] { def map[A, B](fa: F[A])(f: A => B): F[B] def lift[A, B](f: A => B): F[A] => F[B] = fa => map(fa)(f) } ``` ## Functors for effect management The `F` in `Functor` is often referred to as an "effect" or "computational context." Different effects will abstract away different behaviors with respect to fundamental functions like `map`. For instance, `Option`'s effect abstracts away potentially missing values, where `map` applies the function only in the `Some` case but otherwise threads the `None` through. Taking this view, we can view `Functor` as the ability to work with a **single** effect - we can apply a pure function to a single effectful value without needing to "leave" the effect. ## Functors compose If you've ever found yourself working with nested data types such as `Option[List[A]]` or `List[Either[String, Future[A]]]` and tried to `map` over it, you've most likely found yourself doing something like `_.map(_.map(_.map(f)))`. As it turns out, `Functor`s compose, which means if `F` and `G` have `Functor` instances, then so does `F[G[_]]`. Such composition can be achieved via the `Functor#compose` method. ```scala mdoc:reset:silent import cats.Functor import cats.syntax.all._ ``` ```scala mdoc val listOption = List(Some(1), None, Some(2)) // Through Functor#compose Functor[List].compose[Option].map(listOption)(_ + 1) ``` This approach will allow us to use composition without wrapping the value in question, but can introduce complications in more complex use cases. For example, if we need to call another function which requires a `Functor` and we want to use the composed `Functor`, we would have to explicitly pass in the composed instance during the function call or create a local implicit. ```scala mdoc:silent def needsFunctor[F[_]: Functor, A](fa: F[A]): F[Unit] = Functor[F].map(fa)(_ => ()) def foo: List[Option[Unit]] = { val listOptionFunctor = Functor[List].compose[Option] type ListOption[A] = List[Option[A]] needsFunctor[ListOption, Int](listOption)(listOptionFunctor) } ``` We can make this nicer at the cost of boxing with the `Nested` data type. ```scala mdoc:silent import cats.data.Nested import cats.syntax.all._ ``` ```scala mdoc val nested: Nested[List, Option, Int] = Nested(listOption) nested.map(_ + 1) ``` The `Nested` approach, being a distinct type from its constituents, will resolve the usual way modulo possible [SI-2712][si2712] issues (which can be addressed through [partial unification][partial-unification]), but requires syntactic and runtime overhead from wrapping and unwrapping. [partial-unification]: https://github.com/fiadliel/sbt-partial-unification "A sbt plugin for enabling partial unification" [si2712]: https://issues.scala-lang.org/browse/SI-2712 "SI-2712: implement higher-order unification for type constructor inference" --- ### Typeclasses Invariant (`docs/typeclasses/invariant.md`) # Invariant API Documentation: @:api(cats.Invariant) The `Invariant` type class is for functors that define an `imap` function with the following type: ```scala def imap[A, B](fa: F[A])(f: A => B)(g: B => A): F[B] ``` Every covariant (as well as [contravariant](contravariant.md)) functor gives rise to an invariant functor, by ignoring the `g` (or in case of contravariance, `f`) function. Examples for instances of `Invariant` are [`Semigroup`](semigroup.md) and [`Monoid`](monoid.md), in the following we will explain why this is the case using `Semigroup`, the reasoning for `Monoid` is analogous. ## Invariant instance for Semigroup Pretend that we have a `Semigroup[Long]` representing a standard UNIX timestamp. Let's say that we want to create a `Semigroup[Date]`, by *reusing* `Semigroup[Long]`. ### Semigroup does not form a covariant functor If `Semigroup` had an instance for the standard covariant [`Functor`](functor.md) type class, we could use `map` to apply a function `longToDate`: ```scala mdoc:silent import java.util.Date def longToDate: Long => Date = new Date(_) ``` But is this enough to give us a `Semigroup[Date]`? The answer is no, unfortunately. A `Semigroup[Date]` should be able to combine two values of type `Date`, given a `Semigroup` that only knows how to combine `Long`s! The `longToDate` function does not help at all, because it only allows us to convert a `Long` into a `Date`. Seems like we can't have an `Functor` instance for `Semigroup`. ### Semigroup does not form a contravariant functor On the other side, if `Semigroup` would form a *contravariant* functor by having an instance for [`Contravariant`](contravariant.md), we could make use of `contramap` to apply a function `dateToLong`: ```scala mdoc:silent import java.util.Date def dateToLong: Date => Long = _.getTime ``` Again we are faced with a problem when trying to get a `Semigroup[Date]` based on a `Semigroup[Long]`. As before consider the case where we have two values of `Date` at hand. Using `dateToLong` we can turn them into `Long`s and use `Semigroup[Long]` to combine the two values. We are left with a value of type `Long`, but we can't turn it back into a `Date` using only `contramap`! ### Semigroup does form an invariant functor From the previous discussion we conclude that we need both the `map` from (covariant) `Functor` and `contramap` from `Contravariant`. There already is a type class for this and it is called `Invariant`. Instances of the `Invariant` type class provide the `imap` function: ```scala def imap[A, B](fa: F[A])(f: A => B)(g: B => A): F[B] ``` Reusing the example of turning `Semigroup[Long]` into `Semigroup[Date]`, we can use the `g` parameter to turn `Date` into a `Long`, combine our two values using `Semigroup[Long]` and then convert the result back into a `Date` using the `f` parameter of `imap`: ```scala mdoc:silent:reset import java.util.Date // import everything for simplicity: import cats._ import cats.syntax.all._ def longToDate: Long => Date = new Date(_) def dateToLong: Date => Long = _.getTime implicit val semigroupDate: Semigroup[Date] = Semigroup[Long].imap(longToDate)(dateToLong) val today: Date = longToDate(1449088684104l) val timeLeft: Date = longToDate(1900918893l) ``` ```scala mdoc today |+| timeLeft ``` --- ### Typeclasses Invariantmonoidal (`docs/typeclasses/invariantmonoidal.md`) # Invariant Monoidal API Documentation: @:api(cats.InvariantMonoidal) `InvariantMonoidal` combines [`Invariant`](invariant.md) and `Semigroupal` with the addition of a `unit` methods, defined in isolation the `InvariantMonoidal` type class could be defined as follows: ```scala mdoc:compile-only trait InvariantMonoidal[F[_]] { def unit: F[Unit] def imap[A, B](fa: F[A])(f: A => B)(g: B => A): F[B] def product[A, B](fa: F[A], fb: F[B]): F[(A, B)] } ``` Practical uses of `InvariantMonoidal` appear in the context of codecs, that is interfaces to capture both serialization and deserialization for a given format. Another notable examples is [`Semigroup`](semigroup.md). This tutorial first shows how `Semigroup` is `InvariantMonoidal`, and how this can be used create `Semigroup` instances by combining other `Semigroup` instances. Secondly, we present a complete example of `Codec` for the CSV format, and show how it is `InvariantMonoidal`. Lastly, we present an alternative definition of `InvariantMonoidal` as a generalization of `Invariant`, and show that both definitions are equivalent. # `Semigroup` is `InvariantMonoidal` As explained in the [`Invariant` tutorial](invariant.md), `Semigroup` forms an invariant functor. Indeed, given a `Semigroup[A]` and two functions `A => B` and `B => A`, one can construct a `Semigroup[B]` by transforming two values from type `B` to type `A`, combining these using the `Semigroup[A]`, and transforming the result back to type `B`. Thus to define an `InvariantMonoidal[Semigroup]` we need implementations for `unit` and `product`. To construct a `Semigroup` from a single value, we can define a trivial `Semigroup` with a combine that always outputs the given value. A `Semigroup[(A, B)]` can be obtained from two `Semigroup`s for type `A` and `B` by deconstructing two pairs into elements of type `A` and `B`, combining these element using their respective `Semigroup`s, and reconstructing a pair from the results: ```scala mdoc:silent import cats.Semigroup def unit: Semigroup[Unit] = (_, _) => () def product[A, B](fa: Semigroup[A], fb: Semigroup[B]): Semigroup[(A, B)] = { case ((xa, xb), (ya, yb)) => fa.combine(xa, ya) -> fb.combine(xb, yb) } ``` Given an instance of `InvariantMonoidal` for `Semigroup`, we are able to combine existing `Semigroup` instances to form a new `Semigroup` by using the `Semigroupal` syntax: ```scala mdoc:silent import cats.syntax.all._ // Let's build a Semigroup for this case class case class Foo(a: String, c: List[Double]) implicit val fooSemigroup: Semigroup[Foo] = ( (implicitly[Semigroup[String]], implicitly[Semigroup[List[Double]]]) .imapN(Foo.apply)(Function.unlift(Foo.unapply)) ) ``` Our new Semigroup in action: ```scala mdoc Foo("Hello", List(0.0)) |+| Foo("World", Nil) |+| Foo("!", List(1.1, 2.2)) ``` # `CsvCodec` is `InvariantMonoidal` We define `CsvCodec`, a type class for serialization and deserialization of CSV rows: ```scala mdoc:silent type CSV = List[String] trait CsvCodec[A] { def read(s: CSV): (Option[A], CSV) def write(a: A): CSV } ``` The `read` method consumes columns from a CSV row and returns an optional value and the remaining CSV. The `write` method produces the CSV representation of a given value. Beside the composition capabilities illustrated later in this tutorial, grouping both serialization and deserialization in a single type class has the advantage to allows the definition of a law to capture the fact that both operations play nicely together: ```scala forAll { (c: CsvCodec[A], a: A) => c.read(c.write(a)) == ((Some(a), List())) ``` Let's now see how we could define an `InvariantMonoidal` instance for `CsvCodec`. Lifting a single value into a `CsvCodec` can be done "the trivial way" by consuming nothing from CSV and producing that value, and writing this value as the empty CSV: ```scala mdoc:silent trait CCUnit { def unit: CsvCodec[Unit] = new CsvCodec[Unit] { def read(s: CSV): (Option[Unit], CSV) = (Some(()), s) def write(u: Unit): CSV = List.empty } } ``` Combining two `CsvCodec`s could be done by reading and writing each value of a pair sequentially, where reading succeeds if both read operations succeed: ```scala mdoc:silent trait CCProduct { def product[A, B](fa: CsvCodec[A], fb: CsvCodec[B]): CsvCodec[(A, B)] = new CsvCodec[(A, B)] { def read(s: CSV): (Option[(A, B)], CSV) = { val (a1, s1) = fa.read(s) val (a2, s2) = fb.read(s1) ((a1, a2).mapN(_ -> _), s2) } def write(a: (A, B)): CSV = fa.write(a._1) ++ fb.write(a._2) } } ``` Changing a `CsvCodec[A]` to `CsvCodec[B]` requires two functions of type `A => B` and `B => A` to transform a value from `A` to `B` after deserialized, and from `B` to `A` before serialization: ```scala mdoc:silent trait CCImap { def imap[A, B](fa: CsvCodec[A])(f: A => B)(g: B => A): CsvCodec[B] = new CsvCodec[B] { def read(s: CSV): (Option[B], CSV) = { val (a1, s1) = fa.read(s) (a1.map(f), s1) } def write(a: B): CSV = fa.write(g(a)) } } ``` Putting it all together: ```scala mdoc:silent import cats.InvariantMonoidal implicit val csvCodecIsInvariantMonoidal: InvariantMonoidal[CsvCodec] = new InvariantMonoidal[CsvCodec] with CCUnit with CCProduct with CCImap ``` We can now define a few `CsvCodec` instances and use the methods provided by `InvariantMonoidal` to define `CsvCodec` from existing `CsvCodec`s: ```scala mdoc:silent val stringCodec: CsvCodec[String] = new CsvCodec[String] { def read(s: CSV): (Option[String], CSV) = (s.headOption, s.drop(1)) def write(a: String): CSV = List(a) } def numericSystemCodec(base: Int): CsvCodec[Int] = new CsvCodec[Int] { def read(s: CSV): (Option[Int], CSV) = (s.headOption.flatMap(head => scala.util.Try(Integer.parseInt(head, base)).toOption), s.drop(1)) def write(a: Int): CSV = List(Integer.toString(a, base)) } ``` ```scala mdoc:silent:nest case class BinDec(binary: Int, decimal: Int) val binDecCodec: CsvCodec[BinDec] = ( (numericSystemCodec(2), numericSystemCodec(10)) .imapN(BinDec.apply)(Function.unlift(BinDec.unapply)) ) case class Foo(name: String, bd1: BinDec, bd2: BinDec) val fooCodec: CsvCodec[Foo] = ( (stringCodec, binDecCodec, binDecCodec) .imapN(Foo.apply)(Function.unlift(Foo.unapply)) ) ``` Finally let's verify out CsvCodec law with an example: ```scala mdoc val foo = Foo("foo", BinDec(10, 10), BinDec(20, 20)) val fooCsv = fooCodec.write(foo) fooCodec.read(fooCsv) fooCodec.read(fooCodec.write(foo)) == ((Some(foo), List())) ``` # `InvariantMonoidal` as a generalization of `Invariant` To better understand the motivations behind the `InvariantMonoidal` type class, we show how one could naturally arrive to its definition by generalizing the concept of `Invariant` functor. This reflection is analogous to the one presented in [Free Applicative Functors by Paolo Capriotti](http://www.paolocapriotti.com/assets/applicative.pdf) to show how [`Applicative`](applicative.md) are a generalization of [`Functor`](functor.md). Given an `Invariant[F]` instance for a certain *context* `F[_]`, its `imap` method gives a way to lift two *unary* pure functions `A => B` and `B => A` into *contextualized* functions `F[A] => F[B]`. But what about functions of other arity? For instance, a value `a` of type `A` can be seen as a pair of nullary functions, one than given no input returns `a`, and the other than give `a` return no output, which we might want to lift them into a *contextualized* `F[A]`. Similarly, given two functions of type `(A, B) => C` and `C => (A, B)`, we might want to *contextualize* them as functions of type `(F[A], F[B]) => F[C]`. The `Invariant` instance alone does not provide either of these lifting, and it is therefore natural to define define a type class for generalizing `Invariant`s for functions of arbitrary arity: ```scala mdoc:silent trait MultiInvariant[F[_]] { def imap0[A](a: A): F[A] def imap1[A, B](f: A => B)(g: B => A)(fa: F[A]): F[B] def imap2[A, B, C](f: ((A, B)) => C)(g: C => (A, B))(fa: F[A], fb: F[B]): F[C] } ``` Higher-arity `imapN` can be defined in terms of `imap2`, for example for `N = 3`: ```scala mdoc:silent trait MultiInvariantImap3[F[_]] extends MultiInvariant[F] { def imap3[A, B, C, D]( f: ((A, B, C)) => D, g: D => (A, B, C), fa: F[A], fb: F[B], fc: F[C] ): F[D] = ( imap2[A, (B, C), D] (f compose { case (a, (b, c)) => (a, b, c) }) (g andThen { case (a, b, c) => (a, (b, c)) }) (fa, imap2[B, C, (B, C)](identity)(identity)(fb, fc)) ) } ``` We can observe that `MultiInvariant` is none other than an alternative formulation for `InvariantMonoidal`. Indeed, `imap1` and `imap` only differ by the order of their argument, and `imap2` can easily be defined in terms of `imap` and `product`: ```scala mdoc:silent trait Imap2FromImapProduct[F[_]] extends cats.InvariantMonoidal[F] { def imap2[A, B, C](f: ((A, B)) => C)(g: C => (A, B))(fa: F[A], fb: F[B]): F[C] = imap(product(fa, fb))(f)(g) } ``` --- ### Typeclasses Lawtesting (`docs/typeclasses/lawtesting.md`) # Law testing [Laws](../typeclasses.md#laws) are an important part of cats. Cats uses [discipline](https://github.com/typelevel/discipline) to define type class laws and the [ScalaCheck](https://github.com/rickynils/scalacheck) tests based on them. To test type class laws from Cats against your instances, you need to add a `cats-laws` dependency. ## Getting started First up, you will need to specify dependencies on `cats-laws` in your `build.sbt` file. ```scala libraryDependencies ++= Seq( "org.typelevel" %% "cats-laws" % "@VERSION@" % Test, ) ``` ## Example: Testing a Functor instance We'll begin by creating a data type and its Functor instance. ```scala mdoc import cats._ sealed trait Tree[+A] case object Leaf extends Tree[Nothing] case class Node[A](p: A, left: Tree[A], right: Tree[A]) extends Tree[A] object Tree { implicit val functorTree: Functor[Tree] = new Functor[Tree] { def map[A, B](tree: Tree[A])(f: A => B) = tree match { case Leaf => Leaf case Node(p, left, right) => Node(f(p), map(left)(f), map(right)(f)) } } } ``` ```scala mdoc:invisible import Tree._ //there is no real companion object in REPL ``` Cats defines all type class laws tests in `cats.laws.discipline.*` as `discipline`'s `RuleSet`s. Each `RuleSet` provides a `ScalaCheck` `Properties` through `ruleSet.all` to represent all the rules that it defines and inherits. For example, the `ScalaCheck` `Properties` for `Functor` can be retrieved using ```scala cats.laws.discipline.FunctorTests[Tree].functor[Int, Int, String].all ``` We will also need to create an `Eq` instance, as most laws will need to compare values of a type to properly test for correctness. For simplicity we'll just use `Eq.fromUniversalEquals`: ```scala mdoc implicit def eqTree[A: Eq]: Eq[Tree[A]] = Eq.fromUniversalEquals ``` ScalaCheck requires `Arbitrary` instances for data types being tested. We have defined an `Arbitrary` instance for `Tree` here, if you use Scala 2, you can avoid writing it manually with [scalacheck-shapeless](https://github.com/alexarchambault/scalacheck-shapeless). ```scala mdoc:silent import org.scalacheck.{Arbitrary, Gen} object arbitraries { implicit def arbTree[A: Arbitrary]: Arbitrary[Tree[A]] = Arbitrary(Gen.oneOf(Gen.const(Leaf), (for { e <- Arbitrary.arbitrary[A] } yield Node(e, Leaf, Leaf))) ) } ``` Now we can convert these `ScalaCheck` `Properties` into tests that the test framework can run. [discipline](https://github.com/typelevel/discipline) provides a helper `checkAll` function that performs this conversion for three test frameworks: `ScalaTest`, `Specs2` and `MUnit`. * If you are using `Specs2`, extend your test class with `org.typelevel.discipline.specs2.Discipline` (provided by `discipline-specs2`). * If you are using `ScalaTest`, extend your test class with `org.typelevel.discipline.scalatest.FunSuiteDiscipline` (provided by `discipline-scalatest`) and `org.scalatest.funsuite.AnyFunSuiteLike`. * If you are using `MUnit`, extend your test class with `munit.DisciplineSuite` (provided by `discipline-munit`). * For other test frameworks, you need to resort to their integration with `ScalaCheck` to test the `ScalaCheck` `Properties` provided by `cats-laws`. The following example is for MUnit. ```scala mdoc import cats.syntax.all._ import cats.laws.discipline.FunctorTests import munit.DisciplineSuite import arbitraries._ class TreeLawTests extends DisciplineSuite { checkAll("Tree.FunctorLaws", FunctorTests[Tree].functor[Int, Int, String]) } ``` * `cats.implicits._` imports the instances we need for `Eq[Tree[Int]]`, which the laws use to compare trees. * `FunctorTests` contains the functor laws. * `AnyFunSuite` defines the style of ScalaTest. * `FunSuiteDiscipline` provides `checkAll`, and must be mixed into `AnyFunSuite` * `arbitraries._` imports the `Arbitrary[Tree[_]]` instances needed to check the laws. Now when we run `test` in our sbt console, ScalaCheck will test if the `Functor` laws hold for our `Tree` type. You should see something like this: ``` [info] TreeLawTests: [info] - Tree.FunctorLaws.functor.covariant composition (58 milliseconds) [info] - Tree.FunctorLaws.functor.covariant identity (3 milliseconds) [info] - Tree.FunctorLaws.functor.invariant composition (19 milliseconds) [info] - Tree.FunctorLaws.functor.invariant identity (3 milliseconds) [info] Passed: Total 4, Failed 0, Errors 0, Passed 4 [success] Total time: 2 s, completed Aug 2, 2019 12:01:17 AM ``` And voila, you've successfully proven that your data type upholds the Functor laws! ### Testing `cats.kernel` instances For most of the type classes included in Cats, the above will work great. However, the law tests for the type classes inside `cats.kernel` are located in `cats.kernel.laws.discipline.*` instead. So we have to import from there to test type classes like `Semigroup`, `Monoid`, `Group` or `Semilattice`. Let's test it out by defining a `Semigroup` instance for our `Tree` type. ```scala mdoc import cats.syntax.all._ implicit def semigroupTree[A: Semigroup]: Semigroup[Tree[A]] = new Semigroup[Tree[A]] { def combine(x: Tree[A], y: Tree[A]) = (x, y) match { case (Leaf, _) => Leaf case (_, Leaf) => Leaf case (Node(xp, xLeft, xRight), Node(yp, yLeft, yRight)) => Node(xp |+| yp, combine(xLeft, yLeft), combine(xRight, yRight)) } } ``` Then we can add the Semigroup tests to our suite: ```scala mdoc:nest import cats.syntax.all._ import cats.kernel.laws.discipline.SemigroupTests import cats.laws.discipline.FunctorTests import munit.DisciplineSuite import arbitraries._ class TreeLawTests extends DisciplineSuite { checkAll("Tree.FunctorLaws", FunctorTests[Tree].functor[Int, Int, String]) checkAll("Tree[Int].SemigroupLaws", SemigroupTests[Tree[Int]].semigroup) } ``` --- ### Typeclasses Monad (`docs/typeclasses/monad.md`) # Monad API Documentation: @:api(cats.Monad) `Monad` extends the [`Applicative`](applicative.md) type class with a new function `flatten`. Flatten takes a value in a nested context (eg. `F[F[A]]` where F is the context) and "joins" the contexts together so that we have a single context (ie. `F[A]`). The name `flatten` should remind you of the functions of the same name on many classes in the standard library. ```scala mdoc Option(Option(1)).flatten Option(None).flatten List(List(1),List(2,3)).flatten ``` ### Monad instances If `Applicative` is already present and `flatten` is well-behaved, extending the `Applicative` to a `Monad` is trivial. To provide evidence that a type belongs in the `Monad` type class, cats' implementation requires us to provide an implementation of `pure` (which can be reused from `Applicative`) and `flatMap`. We can use `flatten` to define `flatMap`: `flatMap` is just `map` followed by `flatten`. Conversely, `flatten` is just `flatMap` using the identity function `x => x` (i.e. `flatMap(_)(x => x)`). ```scala mdoc:silent import cats._ implicit def optionMonad(implicit app: Applicative[Option]): Monad[Option] = new Monad[Option] { // Define flatMap using Option's flatten method override def flatMap[A, B](fa: Option[A])(f: A => Option[B]): Option[B] = app.map(fa)(f).flatten // Reuse this definition from Applicative. override def pure[A](a: A): Option[A] = app.pure(a) @annotation.tailrec def tailRecM[A, B](init: A)(fn: A => Option[Either[A, B]]): Option[B] = fn(init) match { case None => None case Some(Right(b)) => Some(b) case Some(Left(a)) => tailRecM(a)(fn) } } ``` ### flatMap `flatMap` is often considered to be the core function of `Monad`, and Cats follows this tradition by providing implementations of `flatten` and `map` derived from `flatMap` and `pure`. Part of the reason for this is that name `flatMap` has special significance in scala, as for-comprehensions rely on this method to chain together operations in a monadic context. ```scala mdoc import scala.reflect.runtime.universe universe.reify( for { x <- Some(1) y <- Some(2) } yield x + y ).tree ``` ### tailRecM In addition to requiring `flatMap` and `pure`, Cats has chosen to require `tailRecM` which encodes stack safe monadic recursion, as described in [Stack Safety for Free](http://functorial.com/stack-safety-for-free/index.pdf) by Phil Freeman. Because monadic recursion is so common in functional programming but is not stack safe on the JVM, Cats has chosen to require this method of all monad implementations as opposed to just a subset. All functions requiring monadic recursion in Cats do so via `tailRecM`. An example `Monad` implementation for `Option` is shown below. Note the tail recursive and therefore stack safe implementation of `tailRecM`. ```scala mdoc:silent:reset import cats.Monad import scala.annotation.tailrec implicit val optionMonad: Monad[Option] = new Monad[Option] { def flatMap[A, B](fa: Option[A])(f: A => Option[B]): Option[B] = fa.flatMap(f) def pure[A](a: A): Option[A] = Some(a) @tailrec def tailRecM[A, B](a: A)(f: A => Option[Either[A, B]]): Option[B] = f(a) match { case None => None case Some(Left(nextA)) => tailRecM(nextA)(f) // continue the recursion case Some(Right(b)) => Some(b) // recursion done } } ``` More discussion about `tailRecM` can be found in the [FAQ](../faq.md#what-is-tailrecm). ### ifM `Monad` provides the ability to choose later operations in a sequence based on the results of earlier ones. This is embodied in `ifM`, which lifts an `if` statement into the monadic context. ```scala mdoc import cats.syntax.all._ Monad[List].ifM(List(true, false, true))(ifTrue = List(1, 2), ifFalse = List(3, 4)) ``` ### Composition Unlike [`Functor`s](functor.md) and [`Applicative`s](applicative.md), not all `Monad`s compose. This means that even if `M[_]` and `N[_]` are both `Monad`s, `M[N[_]]` is not guaranteed to be a `Monad`. However, many common cases do. One way of expressing this is to provide instructions on how to compose any outer monad (`F` in the following example) with a specific inner monad (`Option` in the following example). *Note*: the example below assumes usage of the [kind-projector compiler plugin](https://github.com/typelevel/kind-projector) and will not compile if it is not being used in a project. ```scala mdoc:silent:reset import cats.Monad import cats.syntax.all._ case class OptionT[F[_], A](value: F[Option[A]]) implicit def optionTMonad[F[_]](implicit F: Monad[F]): Monad[OptionT[F, *]] = { new Monad[OptionT[F, *]] { def pure[A](a: A): OptionT[F, A] = OptionT(F.pure(Some(a))) def flatMap[A, B](fa: OptionT[F, A])(f: A => OptionT[F, B]): OptionT[F, B] = OptionT { F.flatMap(fa.value) { case None => F.pure(None) case Some(a) => f(a).value } } def tailRecM[A, B](a: A)(f: A => OptionT[F, Either[A, B]]): OptionT[F, B] = OptionT { F.tailRecM(a)(a0 => F.map(f(a0).value) { case None => Either.right[A, Option[B]](None) case Some(b0) => b0.map(Some(_)) }) } } } ``` This sort of construction is called a monad transformer. Cats has an [`OptionT`](../datatypes/optiont.md) monad transformer, which adds a lot of useful functions to the simple implementation above. ## FlatMap - a weakened Monad A closely related type class is `FlatMap` which is identical to `Monad`, minus the `pure` method. Indeed in Cats `Monad` is a subclass of `FlatMap` (from which it gets `flatMap`) and `Applicative` (from which it gets `pure`). ```scala trait FlatMap[F[_]] extends Apply[F] { def flatMap[A, B](fa: F[A])(f: A => F[B]): F[B] } trait Monad[F[_]] extends FlatMap[F] with Applicative[F] ``` The laws for `FlatMap` are just the laws of `Monad` that don't mention `pure`. One of the motivations for `FlatMap`'s existence is that some types have `FlatMap` instances but not `Monad` - one example is `Map[K, *]`. Consider the behavior of `pure` for `Map[K, A]`. Given a value of type `A`, we need to associate some arbitrary `K` to it but we have no way of doing that. However, given existing `Map[K, A]` and `Map[K, B]` (or `Map[K, A => B]`), it is straightforward to pair up (or apply functions to) values with the same key. Hence `Map[K, *]` has an `FlatMap` instance. --- ### Typeclasses Monoid (`docs/typeclasses/monoid.md`) # Monoid API Documentation: @:api(cats.kernel.Monoid) `Monoid` extends the power of `Semigroup` by providing an additional `empty` value. ```scala mdoc:silent trait Semigroup[A] { def combine(x: A, y: A): A } trait Monoid[A] extends Semigroup[A] { def empty: A } ``` This `empty` value should be an identity for the `combine` operation, which means the following equalities hold for any choice of `x`. ``` combine(x, empty) = combine(empty, x) = x ``` Many types that form a `Semigroup` also form a `Monoid`, such as `Int`s (with `0`) and `Strings` (with `""`). ```scala mdoc:reset:silent import cats.Monoid implicit val intAdditionMonoid: Monoid[Int] = new Monoid[Int] { def empty: Int = 0 def combine(x: Int, y: Int): Int = x + y } val x = 1 ``` ```scala mdoc Monoid[Int].combine(x, Monoid[Int].empty) Monoid[Int].combine(Monoid[Int].empty, x) ``` # Example usage: Collapsing a list In the `Semigroup` section we had trouble writing a generic `combineAll` function because we had nothing to give if the list was empty. With `Monoid` we can return `empty`, giving us ```scala mdoc:silent def combineAll[A: Monoid](as: List[A]): A = as.foldLeft(Monoid[A].empty)(Monoid[A].combine) ``` which can be used for any type that has a `Monoid` instance. ```scala mdoc:silent import cats.syntax.all._ ``` ```scala mdoc combineAll(List(1, 2, 3)) combineAll(List("hello", " ", "world")) combineAll(List(Map('a' -> 1), Map('a' -> 2, 'b' -> 3), Map('b' -> 4, 'c' -> 5))) combineAll(List(Set(1, 2), Set(2, 3, 4, 5))) ``` This function is provided in Cats as `Monoid.combineAll`. # The `Option` monoid There are some types that can form a `Semigroup` but not a `Monoid`. For example, the following `NonEmptyList` type forms a semigroup through `++`, but has no corresponding identity element to form a monoid. ```scala mdoc:silent import cats.Semigroup final case class NonEmptyList[A](head: A, tail: List[A]) { def ++(other: NonEmptyList[A]): NonEmptyList[A] = NonEmptyList(head, tail ++ other.toList) def toList: List[A] = head :: tail } object NonEmptyList { implicit def nonEmptyListSemigroup[A]: Semigroup[NonEmptyList[A]] = _ ++ _ } ``` How then can we collapse a `List[NonEmptyList[A]]` ? For such types that only have a `Semigroup` we can lift into `Option` to get a `Monoid`. ```scala mdoc:silent import cats.syntax.all._ implicit def optionMonoid[A: Semigroup]: Monoid[Option[A]] = new Monoid[Option[A]] { def empty: Option[A] = None def combine(x: Option[A], y: Option[A]): Option[A] = x match { case None => y case Some(xv) => y match { case None => x case Some(yv) => Some(xv |+| yv) } } } ``` This is the `Monoid` for `Option`: for any `Semigroup[A]`, there is a `Monoid[Option[A]]`. Thus: ```scala mdoc:reset:silent import cats.Monoid import cats.data.NonEmptyList import cats.syntax.all._ val list = List(NonEmptyList(1, List(2, 3)), NonEmptyList(4, List(5, 6))) val lifted = list.map(nel => Option(nel)) ``` ```scala mdoc Monoid.combineAll(lifted) ``` This lifting and combining of `Semigroup`s into `Option` is provided by Cats as `Semigroup.combineAllOption`. ----- N.B. Cats defines the `Monoid` type class in cats-kernel. The [`cats` package object](https://github.com/typelevel/cats/blob/main/core/src/main/scala/cats/package.scala) defines type aliases to the `Monoid` from cats-kernel, so that you can simply import `cats.Monoid`. --- ### Typeclasses Monoidk (`docs/typeclasses/monoidk.md`) # MonoidK API Documentation: @:api(cats.MonoidK) `MonoidK` is a universal monoid which operates on type constructors of one argument. This type class is useful when its type parameter `F[_]` has a structure that can be combined for any particular type, and which also has an "empty" representation. Thus, `MonoidK` is like a `Monoid` for type constructors (i.e. parametrized types). A `MonoidK[F]` can produce a `Monoid[F[A]]` for any type `A`. Here's how to distinguish `Monoid` and `MonoidK`: - `Monoid[A]` allows `A` values to be combined, and also means there is an "empty" A value that functions as an identity. - `MonoidK[F]` allows two `F[A]` values to be combined, for any `A`. It also means that for any `A`, there is an "empty" `F[A]` value. The combination operation and empty value just depend on the structure of `F`, but not on the structure of `A`. Let's compare the usage of `Monoid[A]` and `MonoidK[F]`. First some imports: ```scala mdoc:silent import cats.{Monoid, MonoidK} import cats.syntax.all._ ``` Just like `Monoid[A]`, `MonoidK[F]` has an `empty` method, but it is parametrized on the type of the element contained in `F`: ```scala mdoc Monoid[List[String]].empty MonoidK[List].empty[String] MonoidK[List].empty[Int] ``` And instead of `combine`, it has `combineK`, which also takes one type parameter: ```scala mdoc Monoid[List[String]].combine(List("hello", "world"), List("goodbye", "moon")) MonoidK[List].combineK[String](List("hello", "world"), List("goodbye", "moon")) MonoidK[List].combineK[Int](List(1, 2), List(3, 4)) ``` Actually the type parameter can usually be inferred: ```scala mdoc MonoidK[List].combineK(List("hello", "world"), List("goodbye", "moon")) MonoidK[List].combineK(List(1, 2), List(3, 4)) ``` `MonoidK` extends [`SemigroupK`](semigroupk.md), so take a look at the `SemigroupK` documentation for more examples. --- ### Typeclasses Nonemptytraverse (`docs/typeclasses/nonemptytraverse.md`) # NonEmptyTraverse API Documentation: @:api(cats.NonEmptyTraverse) `NonEmptyTraverse` is a non-empty version of the [Traverse](traverse.md) type class, just like [Reducible](reducible.md) is a non-empty version of [Foldable](foldable.md). As such, it extends both `Reducible` and `Traverse` in the type class hierarchy. It provides the `nonEmptyTraverse` and `nonEmptySequence` methods that require an instance of `Apply` instead of `Applicative`: ```scala def nonEmptyTraverse[G[_]: Apply, A, B](fa: F[A])(f: A => G[B]): G[F[B]] def nonEmptySequence[G[_]: Apply, A](fga: F[G[A]]): G[F[A]] ``` In the [Applicative tutorial](applicative.md) we learned of `Apply` as a weakened `Applicative` lacking the `pure` method. One example type lacking an `Applicative` instance is `Map[K, *]`, it's impossible to implement a `pure` method for it. Knowing this, we can make use of `NonEmptyTraverse`, to traverse over a sequence of `Map`s. One example application one could think of is, when we have a list of text snippets, count the occurrence of each word in each snippet and return all the common words and their occurrences in each snippet: ```scala mdoc import cats.syntax.all._ import cats.data.NonEmptyList val snippets = NonEmptyList.of("What do you do", "What are you doing") def countWords(text: String): Map[String, Int] = text.split(" ").groupBy(identity).view.mapValues(_.length).toMap snippets.nonEmptyTraverse(countWords) ``` Note that, just like `traverse`, `nonEmptyTraverse(f)` is equivalent to `map(f).nonEmptySequence`, so the above could be rewritten as: ```scala mdoc snippets.map(countWords).nonEmptySequence ``` `NonEmptyTraverse` also offers `flatNonEmptyTraverse` and `flatNonEmptySequence` methods that are analogous to `flatTraverse` and `flatSequence` in `Traverse`. Just like with `nonEmptyTraverse` these methods don't require a `Monad` instance, but only a `FlatMap`, which is the weakened version of `Monad` without `pure`. --- ### Typeclasses Parallel (`docs/typeclasses/parallel.md`) # Parallel API Documentation: @:api(cats.Parallel) When browsing the various `Monads` included in Cats, you may have noticed that some of them have data types that are actually of the same structure, but instead have instances of `Applicative`. E.g. `Either` and `Validated`. This is because defining a `Monad` instance for data types like `Validated` [would be inconsistent](../datatypes/validated.md#of-flatmaps-and-eithers) with its error-accumulating behaviour. In short, `Monads` describe dependent computations and `Applicatives` describe independent computations. Sometimes however, we want to use both in conjunction with each other. In the example of `Either` and `Validated` it is trivial albeit cumbersome to convert between the two types. Below is a short example of a situation where we might run into this. For simplicity, we'll use `String` as our type to represent errors. ```scala mdoc import cats.syntax.all._ import cats.data._ case class Name(value: String) case class Age(value: Int) case class Person(name: Name, age: Age) def parse(s: String): Either[NonEmptyList[String], Int] = { if (s.matches("-?[0-9]+")) Right(s.toInt) else Left(NonEmptyList.one(s"$s is not a valid integer.")) } def validateAge(a: Int): Either[NonEmptyList[String], Age] = { if (a > 18) Right(Age(a)) else Left(NonEmptyList.one(s"$a is not old enough")) } def validateName(n: String): Either[NonEmptyList[String], Name] = { if (n.length >= 8) Right(Name(n)) else Left(NonEmptyList.one(s"$n Does not have enough characters")) } ``` Now we want to parse two Strings into a value of `Person`: ```scala mdoc def parsePerson(ageString: String, nameString: String) = for { age <- parse(ageString) person <- (validateName(nameString).toValidated, validateAge(age).toValidated) .mapN(Person) .toEither } yield person ``` We had to convert to and from `Validated` manually. While this is still manageble, it gets worse the more `Eithers` we want to combine in parallel. To mitigate this pain, Cats introduces a type class `Parallel` that abstracts over `Monads` which also support parallel composition. It is simply defined in terms of conversion functions between the two data types: ```scala trait Parallel[M[_]] { type F[_] def sequential: F ~> M def parallel: M ~> F } ``` Where `M[_]` has to have an instance of `Monad` and `F[_]` an instance of `Applicative`. Recall that `~>` is just an alias for [`FunctionK`](../datatypes/functionk.md). This allows us to get rid of most of our boilerplate from earlier: ```scala mdoc:nest def parsePerson(ageString: String, nameString: String) = for { age <- parse(ageString) person <- (validateName(nameString), validateAge(age)).parMapN(Person) } yield person ``` We can also traverse over a `Traverse` using `Parallel`: ```scala mdoc List(Either.right(42), Either.left(NonEmptyList.one("Error 1")), Either.left(NonEmptyList.one("Error 2"))).parSequence ``` Parallel is also really useful for `zipping` collections. The standard `Applicative` instance for `List`, `Vector`, etc. behaves like the cartesian product of the individual collections: ```scala mdoc (List(1, 2, 3), List(4, 5, 6)).mapN(_ + _) ``` However often we will want to `zip` two or more collections together. We can define a different `ap` for most of them and use the `parMapN` syntax for that: ```scala mdoc (List(1, 2, 3), List(4, 5, 6)).parMapN(_ + _) ``` ## NonEmptyParallel - a weakened Parallel Some types cannot form a `Monad` or an `Applicative` because it's not possible to implement the `pure` function for them. However, these types can often have instances for `FlatMap` or `Apply`. For types like these we can use the `NonEmptyParallel` type class. An example for one of these is `ZipList`. With instances of `NonEmptyParallel` it's not possible to use the `parTraverse` and `parSequence` functions, but we can still use `parMapN` and also `parNonEmptyTraverse` and `parNonEmptySequence`, which are analogous to the functions defined on [`NonEmptyTraverse`](nonemptytraverse.md). --- ### Typeclasses Reducible (`docs/typeclasses/reducible.md`) # Reducible API Documentation: @:api(cats.Reducible) `Reducible` extends the `Foldable` type class with additional `reduce` methods. You may have come by one of the `reduce`, `reduceLeft` or `reduceOption` defined in Scala's standard collections. `Reducible` offers exactly these methods with the guarantee that the collection won't throw an exception due to a collection being empty, without having to reduce to an `Option`. This can be utilized effectively to derive `maximum` and `minimum` methods from `Reducible` instead of the `maximumOption` and `minimumOption` found on `Foldable`. In essence, `reduce` is like a non-empty `fold`, requiring no initial value. This makes `Reducible` very useful for abstracting over non-empty collections such as `NonEmptyList` or `NonEmptyVector`. Analogous to the `Foldable` type class, `Reducible[F]` is implemented in terms of two basic methods in addition to those required by `Foldable`: - `reduceLeftTo(fa)(f)(g)` eagerly reduces with an additional mapping function - `reduceRightTo(fa)(f)(g)` lazily reduces with an additional mapping function Now, because `Reducible` does not require an empty value, the equivalent of `fold` and `foldMap`, `reduce` and `reduceMap`, do not require an instance of `Monoid`, but of `Semigroup`. Furthermore, just like with `foldRight`, `reduceRight` uses the `Eval` data type to lazily reduce the collection. First some standard imports. ```scala mdoc:silent import cats._ import cats.data._ import cats.syntax.all._ ``` And examples. ```scala mdoc Reducible[NonEmptyList].reduce(NonEmptyList.of("a", "b", "c")) Reducible[NonEmptyList].reduceMap(NonEmptyList.of(1, 2, 4))(_.toString) Reducible[NonEmptyVector].reduceK(NonEmptyVector.of(List(1,2,3), List(2,3,4))) Reducible[NonEmptyVector].reduceLeft(NonEmptyVector.of(1,2,3,4))((s,i) => s + i) Reducible[NonEmptyList].reduceRight(NonEmptyList.of(1,2,3,4))((i,s) => Later(s.value + i)).value Reducible[NonEmptyList].reduceLeftTo(NonEmptyList.of(1,2,3,4))(_.toString)((s,i) => s + i) Reducible[NonEmptyList].reduceRightTo(NonEmptyList.of(1,2,3,4))(_.toString)((i,s) => Later(s.value + i)).value Reducible[NonEmptyList].nonEmptyIntercalate(NonEmptyList.of("a", "b", "c"), ", ") def countChars(s: String) = s.toCharArray.groupBy(identity).view.mapValues(_.length).toMap Reducible[NonEmptyList].nonEmptyTraverse_(NonEmptyList.of("Hello", "World"))(countChars) Reducible[NonEmptyVector].nonEmptyTraverse_(NonEmptyVector.of("Hello", ""))(countChars) Reducible[NonEmptyList].nonEmptySequence_(NonEmptyList.of(Map(1 -> 'o'), Map(1 -> 'o'))) ``` In addition to `reduce` requiring a `Semigroup` instead of a `Monoid`, `nonEmptyTraverse_` and `nonEmptySequence_` require `Apply` instead of `Applicative`. Also, `Reducible` offers a `reduceLeftM` method, that is just like `foldM`, but requires a `FlatMap` instance of a `Monad` instance. ```scala def reduceLeftM[G[_]: FlatMap, A, B](fa: F[A])(f: A => G[B])(g: (B, A) => G[B]): G[B] ``` --- ### Typeclasses Semigroup (`docs/typeclasses/semigroup.md`) # Semigroup API Documentation: @:api(cats.kernel.Semigroup) If a type `A` can form a `Semigroup` it has an **associative** binary operation. ```scala mdoc:silent trait Semigroup[A] { def combine(x: A, y: A): A } ``` Associativity means the following equality must hold for any choice of `x`, `y`, and `z`. ``` combine(x, combine(y, z)) = combine(combine(x, y), z) ``` A common example of a semigroup is the type `Int` with the operation `+`. ```scala mdoc:reset:silent import cats.Semigroup implicit val intAdditionSemigroup: Semigroup[Int] = _ + _ val x = 1 val y = 2 val z = 3 ``` ```scala mdoc Semigroup[Int].combine(x, y) Semigroup[Int].combine(x, Semigroup[Int].combine(y, z)) Semigroup[Int].combine(Semigroup[Int].combine(x, y), z) ``` Infix syntax is also available for types that have a `Semigroup` instance. ```scala mdoc import cats.syntax.all._ 1 |+| 2 ``` A more compelling example which we'll see later in this tutorial is the `Semigroup` for `Map`s. ```scala mdoc:silent import cats.syntax.all._ val map1 = Map("hello" -> 1, "world" -> 1) val map2 = Map("hello" -> 2, "cats" -> 3) ``` ```scala mdoc Semigroup[Map[String, Int]].combine(map1, map2) map1 |+| map2 ``` # Example instances Cats provides many `Semigroup` instances out of the box such as `Int` (`+`) and `String` (`++`)... ```scala mdoc:reset:silent import cats.Semigroup import cats.syntax.all._ ``` ```scala mdoc Semigroup[Int] Semigroup[String] ``` Instances for type constructors regardless of their type parameter such as `List` (`++`) and `Set` (`union`)... ```scala mdoc Semigroup[List[Byte]] Semigroup[Set[Int]] trait Foo Semigroup[List[Foo]] ``` And instances for type constructors that depend on (one of) their type parameters having instances such as tuples (pointwise `combine`). ```scala mdoc Semigroup[(List[Foo], Int)] ``` # Example usage: Merging maps Consider a function that merges two `Map`s that combines values if they share the same key. It is straightforward to write these for `Map`s with values of type say, `Int` or `List[String]`, but we can write it once and for all for any type with a `Semigroup` instance. ```scala mdoc:silent import cats.syntax.all._ def optionCombine[A: Semigroup](a: A, opt: Option[A]): A = opt.map(a |+| _).getOrElse(a) def mergeMap[K, V: Semigroup](lhs: Map[K, V], rhs: Map[K, V]): Map[K, V] = lhs.foldLeft(rhs) { case (acc, (k, v)) => acc.updated(k, optionCombine(v, acc.get(k))) } ``` ```scala mdoc val xm1 = Map('a' -> 1, 'b' -> 2) val xm2 = Map('b' -> 3, 'c' -> 4) val x = mergeMap(xm1, xm2) val ym1 = Map(1 -> List("hello")) val ym2 = Map(2 -> List("cats"), 1 -> List("world")) val y = mergeMap(ym1, ym2) ``` It is interesting to note that the type of `mergeMap` satisfies the type of `Semigroup` specialized to `Map[K, *]` and is associative - indeed the `Semigroup` instance for `Map` uses the same function for its `combine`. ```scala mdoc Semigroup[Map[Char, Int]].combine(xm1, xm2) == x Semigroup[Map[Int, List[String]]].combine(ym1, ym2) == y ``` # Exploiting laws: associativity Since we know `Semigroup#combine` must be associative, we can exploit this when writing code against `Semigroup`. For instance, to sum a `List[Int]` we can choose to either `foldLeft` or `foldRight` since all that changes is associativity. ```scala mdoc val leftwards = List(1, 2, 3).foldLeft(0)(_ |+| _) val rightwards = List(1, 2, 3).foldRight(0)(_ |+| _) ``` Associativity also allows us to split a list apart and sum the parts in parallel, gathering the results in the end. ```scala mdoc:silent val list = List(1, 2, 3, 4, 5) val (left, right) = list.splitAt(2) ``` ```scala mdoc val sumLeft = left.foldLeft(0)(_ |+| _) val sumRight = right.foldLeft(0)(_ |+| _) val result = sumLeft |+| sumRight ``` However, given just `Semigroup` we cannot write the above expressions generically. For instance, we quickly run into issues if we try to write a generic `combineAll` function. ```scala def combineAll[A: Semigroup](as: List[A]): A = as.foldLeft(/* ?? what goes here ?? */)(_ |+| _) ``` `Semigroup` isn't powerful enough for us to implement this function - namely, it doesn't give us an identity or fallback value if the list is empty. We need a more powerfully expressive abstraction, which we can find in the `Monoid` type class. N.B. Cats defines the `Semigroup` type class in cats-kernel. The [`cats` package object](https://github.com/typelevel/cats/blob/main/core/src/main/scala/cats/package.scala) defines type aliases to the `Semigroup` from cats-kernel, so that you can simply import `cats.Semigroup`. --- ### Typeclasses Semigroupk (`docs/typeclasses/semigroupk.md`) # SemigroupK API Documentation: @:api(cats.SemigroupK) `SemigroupK` has a very similar structure to [`Semigroup`](semigroup.md), the difference is that `SemigroupK` operates on type constructors of one argument. So, for example, whereas you can find a `Semigroup` for types which are fully specified like `Int` or `List[Int]` or `Option[Int]`, you will find `SemigroupK` for type constructors like `List` and `Option`. These types are type constructors in that you can think of them as "functions" in the type space. You can think of the `List` type as a function which takes a concrete type, like `Int`, and returns a concrete type: `List[Int]`. This pattern would also be referred to having `kind: * -> *`, whereas `Int` would have kind `*` and `Map` would have kind `*,* -> *`, and, in fact, the `K` in `SemigroupK` stands for `Kind`. First some imports. ```scala mdoc:silent import cats._ import cats.syntax.all._ ``` For `List`, the `Semigroup` instance's `combine` operation and the `SemigroupK` instance's `combineK` operation are both list concatenation: ```scala mdoc SemigroupK[List].combineK(List(1,2,3), List(4,5,6)) == Semigroup[List[Int]].combine(List(1,2,3), List(4,5,6)) ``` However for `Option`, the `Semigroup`'s `combine` and the `SemigroupK`'s `combineK` operation differ. Since `Semigroup` operates on fully specified types, a `Semigroup[Option[A]]` knows the concrete type of `A` and will use `Semigroup[A].combine` to combine the inner `A`s. Consequently, `Semigroup[Option[A]].combine` requires an implicit `Semigroup[A]`. In contrast, `SemigroupK[Option]` operates on `Option` where the inner type is not fully specified and can be anything (i.e. is "universally quantified"). Thus, we cannot know how to `combine` two of them. Therefore, in the case of `Option` the `SemigroupK[Option].combineK` method has no choice but to use the `orElse` method of Option: ```scala mdoc Semigroup[Option[Int]].combine(Some(1), Some(2)) SemigroupK[Option].combineK(Some(1), Some(2)) SemigroupK[Option].combineK(Some(1), None) SemigroupK[Option].combineK(None, Some(2)) ``` There is inline syntax available for both `Semigroup` and `SemigroupK`. Here we are following the convention from scalaz, that `|+|` is the operator from semigroup and that `<+>` is the operator from `SemigroupK` (called `Plus` in scalaz). ```scala mdoc:silent import cats.syntax.all._ val one = Option(1) val two = Option(2) val n: Option[Int] = None ``` Thus. ```scala mdoc one |+| two one <+> two n |+| two n <+> two n |+| n n <+> n two |+| n two <+> n ``` You'll notice that instead of declaring `one` as `Some(1)`, we chose `Option(1)`, and we added an explicit type declaration for `n`. This is because the `SemigroupK` type class instances is defined for `Option`, not `Some` or `None`. If we try to use `Some` or `None`, we'll get errors: ```scala mdoc:fail Some(1) <+> None None <+> Some(1) ``` --- ### Typeclasses Show (`docs/typeclasses/show.md`) # Show API Documentation: @:api(cats.Show) Show is an alternative to the Java `toString` method. It is defined by a single function `show`: ```scala def show(a: A): String ``` You might be wondering why you would want to use this, considering `toString` already serves the same purpose and case classes already provide sensible implementations for `toString`. The difference is that `toString` is defined on `Any`(Java's `Object`) and can therefore be called on anything, not just case classes. Most often, this is unwanted behaviour, as the standard implementation of `toString` on non case classes is mostly gibberish. Consider the following example: ```scala mdoc (new {}).toString ``` The fact that this code compiles is a design flaw of the Java API. We want to make things like this impossible, by offering the `toString` equivalent as a type class, instead of the root of the class hierarchy. In short, `Show` allows us to only have String-conversions defined for the data types we actually want. To make things easier, Cats defines a few helper functions to make creating `Show` instances easier. ```scala /** creates an instance of Show using the provided function */ def show[A](f: A => String): Show[A] /** creates an instance of Show using object toString */ def fromToString[A]: Show[A] ``` These can be used like this: ```scala mdoc import cats.Show case class Person(name: String, age: Int) implicit val showPerson: Show[Person] = Show.show(person => person.name) case class Department(id: Int, name: String) implicit val showDep: Show[Department] = Show.fromToString ``` This still may not seem useful to you, because case classes already automatically implement `toString`, while `show` would have to be implemented manually for each case class. Thankfully with the help of a small library called [kittens](https://github.com/typelevel/kittens) a lot of type class instances including `Show` can be derived automatically! Cats also offers `Show` syntax to make working with it easier. This includes the `show` method which can be called on anything with a `Show` instance in scope: ```scala mdoc import cats.syntax.all._ val john = Person("John", 31) john.show ``` It also includes a String interpolator, which works just like the standard `s"..."` interpolator, but uses `Show` instead of `toString`: ```scala mdoc val engineering = Department(2, "Engineering") show"$john works at $engineering" ``` --- --- METRICS --- - Files Extracted: 65 - Estimated Token Budget: ~125530 tokens - Canonical Reference: https://codewiki.google/github.com/typelevel/cats