1. Project Overview & Quickstart (arrow-kt/arrow)
File: README.md
Ξrrow is a library for Typed Functional Programming in Kotlin.
Arrow aims to provide a lingua franca of interfaces
and abstractions across Kotlin libraries. For this, it includes the most popular data types such
as Option, and Either, functional operators such as zipOrAccumulate, and computation
blocks to empower users to write pure FP apps and libraries built atop higher order abstractions.
[Documentation](http://arrow-kt.io)
- Quickstart and setup
- Typed errors
- Coroutines and resources
- Resilience
- Immutable data
- Collections and functions
The documentation is hosted in a separate repository.
Join Us
Arrow is an inclusive community powered by awesome individuals like you. As an actively growing
ecosystem, Arrow and its associated libraries and toolsets are in need of new contributors! We have
issues suited for all levels, from entry to advanced, and our maintainers are happy to provide 1:1
mentoring. All are welcome in Arrow.
If youβre looking to contribute, have questions, or want to keep up-to-date about whatβs happening,
please follow us here and say hello!
Find more details in CONTRIBUTING.
License
Copyright (C) 2017 The Ξrrow Authors
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.File: arrow-libs/stack/README.md
BOM file
Thanks to @ps-feng for creating #2129 proposal.
BOM file is useful to include several dependencies without the specification of their versions.
Use
dependencies {
implementation platform("io.arrow-kt:arrow-stack:$ARROW_VERSION")
// Versions are no longer required here
implementation "io.arrow-kt:arrow-core"
implementation "io.arrow-kt:arrow-fx"
implementation "io.arrow-kt:arrow-mtl"
...
}Links
File: arrow-libs/resilience/arrow-resilience/README.md
Module main
Arrow Fx Resilience
Ξrrow Fx Resilience is part of Arrow Fx Ξrrow.
File: arrow-libs/fx/arrow-fx-stm/README.md
Module main
Arrow Fx STM
Ξrrow Fx STM is part of Arrow Fx Ξrrow, and the public documentation and information on how to use the library can be found here.
File: arrow-libs/fx/arrow-fx-coroutines/README.MD
Module main
Arrow Fx Coroutines
Ξrrow Fx Coroutines is part of Arrow Fx Ξrrow, and the public documentation and information on how to use the library can be found there.
Contributing and understanding the internals of Arrow Fx Coroutines
To understand how Arrow Fx Coroutines work, we need to take a step back and look at how Coroutines work in Kotlin.
We have several core types that are exposed from Kotlin Std, and the code rewrite performed by the compiler.
It's not very important to understand how the code rewrite works, the most important thing to understand is that it rewrites your suspend code into a very efficient state machine that allows flattening callbacks. In other words, when a method suspends it means that it will not return immediately but at a later point in time through a callback. This is important to know, since this concept also leaks into public API of coroutines.
Kotlin Std Coroutines
CoroutineContext
CoroutineContext is a immutable key-value map, where the keys are the types and the values are the instances of the type. Meaning that for every type, there can only be a single instance entry.
This is useful to for example ensure there is only a single ContinuationInterceptor, or a single Job in the case of KotlinX Coroutines or a single SuspendConnection in the case of Arrow Fx Coroutines.
A suspend function/program always has a CoroutineContext, this restriction is made in the Continuation interface.
This type-instance relationship is achieved by two interfaces CoroutineContext.Element and CoroutineContext.Key<E: CoroutineContext.Element>. Where CoroutineContext.Element has a Key<*>, which needs to point to a Key for it's concrete type.
A simple example:
class CoroutineName(val name: String): CoroutineContext.Element {
companion object Key : CoroutineContext.Key<CoroutineName>
override val key: CoroutineContext.Key<*> = Key
}Continuation
interface Continuation<A> {
val ctx: CoroutineContext
fun resume(result: Result<A>)
}As mentioned above, a suspend programs can return immediately or suspend and return it's value at a later point in time. Thus this can be modeled with a Contination<A> where both the immediate result or a suspended result is always returned to resume, this ensures that the result is always consumed in the same manner no matter how the suspend program behaves.
Important to note here is that the resume callback works with Result<A> which is similar to Either but with a fast inlined implementation. This allows us to safely run any code inside suspend, whilst keeping our programs from blowing up since it will be safely rewired to resume.
Additionally it always has a CoroutineContext, which could be EmptyCoroutineContext (~ emptyMap()), or any other CoroutineContext.
Such a Continuation is always required to consume any suspend program through the runners exposed by the Kotlin Standard Library.
startCoroutine
The most high level runner in the standard library is startCoroutine, and it's the first method we'll discuss that creates a Coroutine and runs it. Kotlinβs standard library defines a Coroutine as an instance of a suspendable computation. In other words, a Coroutine is a compiled suspend () -> T program wired to a Continuation.
Letβs take a quick look at an example with the 3 types we've seen and discussed above.
import kotlin.coroutines.Continuation
import kotlin.coroutines.EmptyCoroutineContext
import kotlin.coroutines.startCoroutine
suspend fun one(): Int = 1
suspend fun fail(): Int = TODO()
fun main() {
::one.startCoroutine(Continuation(EmptyCoroutineContext) {
println(it) // Success(1)
})
::fail.startCoroutine(Continuation(EmptyCoroutineContext) {
println(it) // Failure(kotlin.NotImplementedError: An operation is not implemented.)
})
}Here you can see that even if our program blows up, it will be correctly wired into the Continuation callback without blowing up main.
If we take a look at its very short implementation, we can see that it exists of a composition of 3 functions. createCoroutineUnintercepted with the completion Continuation, intercepted and resume. In the sections below we'll dig into what that means, and how that works.
All you really need to remember here is that suspend () -> T will run on a given Continuation<T>, on its CoroutineContext. If that CoroutineContext is capable of interception, it will intercept first and then it will start with resume(Unit).
public fun <T> (suspend () -> T).startCoroutine(completion: Continuation<T>) {
createCoroutineUnintercepted(completion).intercepted().resume(Unit)
}ContinuationInterceptor
Before we can move into what createCoroutineUnintercepted is or how it works, we should first understand what interception or intercepted is in Kotlin Coroutines.
One thing that is important to understand is that Coroutines by themselves have nothing to do with concurreny or asynchronity, but rather they can be used to build concurrent frameworks or libraries.
So let's take a quick look at its definition
public interface ContinuationInterceptor : CoroutineContext.Element {
companion object Key : CoroutineContext.Key<ContinuationInterceptor>
public fun <T> interceptContinuation(continuation: Continuation<T>): Continuation<T>
public fun releaseInterceptedContinuation(continuation: Continuation<*>) {
/* do nothing by default */
}
}Here we can see that ContinuationInterceptor is a CoroutineContext.Element, which means we can put into an immutable CoroutineContext map. It has an abstract interceptContinuation method, which is called upon calling intercepted. It can be called upon the Continuation returned by createCoroutineUnintercepted, and in that case will call interceptContinuation in the case the CoroutineContext contains a ContinuationInterceptor. This operation is idempotent, since the result is cached.
When the Continuation is finished running, then before returning it will call releaseInterceptedContinuation, but only in the case interceptContinuation returned a different Continuation than it received.
Let's take a look at a simple example.
import kotlin.coroutines.AbstractCoroutineContextElement
import kotlin.coroutines.Continuation
import kotlin.coroutines.ContinuationInterceptor
import kotlin.coroutines.CoroutineContext
import kotlin.coroutines.startCoroutine
suspend fun helloWorld(): String = "Hello World!"
fun main() {
::helloWorld.startCoroutine(Continuation(Printer) {
println("Received result: $it")
})
}
object Printer : AbstractCoroutineContextElement(ContinuationInterceptor), ContinuationInterceptor {
override fun <T> interceptContinuation(continuation: Continuation<T>): Continuation<T> {
val newContinuation = object : Continuation<T> {
override val context: CoroutineContext = continuation.context
override fun resumeWith(result: Result<T>) {
println("PrinterContinuation resume: pass $result to original continuation")
continuation.resumeWith(result)
}
}
println("interceptContinuation($continuation)\nand returning a new one: $newContinuation\n")
return newContinuation
}
override fun releaseInterceptedContinuation(continuation: Continuation<*>) {
super.releaseInterceptedContinuation(continuation)
println("releaseInterceptedContinuation($continuation)")
}
}1. interceptContinuation(Continuation at kotlin.coroutines.intrinsics.IntrinsicsKt__IntrinsicsJvmKt$createCoroutineUnintercepted$$inlined$createCoroutineFromSuspendFunction$IntrinsicsKt__IntrinsicsJvmKt$2)
and returning a new one: Printer$interceptContinuation$newContinuation$1@448139f0
2. PrinterContinuation resume: pass Success(kotlin.Unit) to original continuation
3. releaseInterceptedContinuation(Printer$interceptContinuation$newContinuation$1@448139f0)
4. Received result: Success(Hello World!)Let's analyse the output:
In the first line of the output we can clearly see that our original Continuation created by startCoroutine is one created by the Kotlin Compiler through createCoroutineUnintercepted (and some other internal machinery inside the Kotlin Std library which creates a Coroutine from a suspending function). We wrap it, and return our own Printer continuation.
Secondly, our Printer continuation is started with resume(Unit), which we see reflected as receiving Success(kotlin.Unit) in resume and then simply passing it along to our original continuation. The original continuation, created by the Kotlin Compiler, will then start our helloWorld suspend program.
After our program is finished, we see that our
Printercontinuation is released.Finally, we see that our result is finally returned to our own
Continuation.
So you might be wondering why this is useful? This isn't very useful for printing anything obviously, because we only get intercepted before anything starts running and released when it's finished. But it gives us the perfect entry point to schedule the coroutine on a Thread or ExecutorService.
Intercepting to schedule/dispatch
Scheduling something on a ExecutorService is simply a call to the function ExecutorService#submit with a Runnable argument. Which is exactly what we can do with interception, looking at the previous log output, we can intercept the start signal of Result before we pass it to the original continuation. This means we can intercept the start signal, and submit it on an ExecutorService before starting the original continuation.
So let's see the most basic example where we do this for the ForkJoinPool.
object CommonPoolContext : AbstractCoroutineContextElement(ContinuationInterceptor), ContinuationInterceptor {
val pool: ForkJoinPool = ForkJoinPool()
override fun <T> interceptContinuation(continuation: Continuation<T>): Continuation<T> =
object : Continuation<T> {
override val context: CoroutineContext = continuation.context
override fun resumeWith(result: Result<T>) {
pool.submit { continuation.resumeWith(result) }
}
}
}We create a singleton ContinuationInterceptor just like we did for our Printer, and we when we get intercepted we can then wrap the original Continuation with a Continuation that submits the start signal onto the ForkJoinPool.
A small example to see this in action, we need the Thread.sleep in main since when we reach pool.submit in the Continuation main will finish and we want to wait a bit so we can see the ouput printed.
import java.util.concurrent.ForkJoinPool
import kotlin.coroutines.AbstractCoroutineContextElement
import kotlin.coroutines.Continuation
import kotlin.coroutines.ContinuationInterceptor
import kotlin.coroutines.CoroutineContext
import kotlin.coroutines.startCoroutine
suspend fun helloWorld(): String =
"Hello from ${Thread.currentThread().name}"
object CommonPoolInterceptor : AbstractCoroutineContextElement(ContinuationInterceptor), ContinuationInterceptor {
val pool: ForkJoinPool = ForkJoinPool()
override fun <T> interceptContinuation(continuation: Continuation<T>): Continuation<T> =
object : Continuation<T> {
override val context: CoroutineContext = continuation.context
override fun resumeWith(result: Result<T>) {
pool.execute { continuation.resumeWith(result) }
}
}
}
fun main() {
::helloWorld.startCoroutine(Continuation(CommonPoolInterceptor) {
println(it)
})
Thread.sleep(100)
}Now that we know what it means to intercept and why it's crucial to build effect systems on top of Kotlin Coroutines, let's investigate some more of the lower level combinators. Those will allow us to by-pass intercepted from the higher level combinators to avoid scheduling all over the place, or allowing us to change CoroutineContext without having to explicitly schedule. This is important in relation to cancellation, which we'll see in great detail below.
createCoroutineUnintercepted
createCoroutineUnintercepted is the lowest level function of creating (and thus subsequently starting) a Coroutine, and you'll find it is used in most other function with either createCoroutineXXX or startCoroutineXXX.
It's found in the kotlin.coroutines.intrinsics package, which we saw 1 method of so far intercepted. And we'll cover the other content below. As we've already mentioned above, this method turns a suspend () -> T function into a Coroutine, which is an instance of a suspendable computation. This Coroutine hasn't started running yet, and we can run it as many times as we want. As you might've guessed from above, we can start the Coroutine by calling resume(Unit) on it.
For a given program suspend () -> T we pass a Continuation<A>, and createCoroutineUnintercepted returns us a Continuation. This Continuation models a Coroutine, which is the program suspend () -> T wired to it's context and callback resume: (Result<T>) -> Unit.
Let's see a quick example of the creation of a simple Coroutine which is started multiple times.
import kotlin.coroutines.intrinsics.createCoroutineUnintercepted
import kotlin.coroutines.Continuation
import kotlin.coroutines.EmptyCoroutineContext
import kotlin.coroutines.resume
suspend fun one(): String = "Hello World!"
val cont: Continuation = ::one
.createCoroutineUnintercepted(Continuation(EmptyCoroutineContext, ::println))
fun main() {
cont.resume(Unit)
println("Fired once")
cont.resume(Unit)
println("Fired Twice")
cont.resume(Unit)
println("Fired three times")
}If we look back at startCoroutine, we see that is simply creates a Coroutine, always invokes its interceptor and then immediately starts it. So the highest level runner, combines the lowest level functions.
public fun <T> (suspend () -> T).startCoroutine(completion: Continuation<T>) {
createCoroutineUnintercepted(completion).intercepted().resume(Unit)
}suspendCoroutineUninterceptedOrReturn / COROUTINE_SUSPENDED
As you might've noticed in all examples above, we haven't actually used any suspension so far. But what does it mean to suspend? Most simply put it means the function doesn't immediately returns, but is going to return the result through the callback (Result<T>) -> Unit.
At the lowest guts of writing suspend functions you can find this function, is capable of returning immediately or returning in a suspended way. We however have to signal to Kotlin's Coroutine system that we're returning immediately or we're suspending and returning at a later point in time. This is done with the marker COROUTINE_SUSPENDED.
COROUTINE_SUSPENDED is a simply marker that can be compared to check if it's the users result, or we should wait for Continuation#resume to be called. In the Kotlin Standard Library we can find following definition.
val COROUTINE_SUSPENDED: AnySo in combination with suspend inline fun <T> suspendCoroutineUninterceptedOrReturn(crossinline block: (Continuation<T>) -> Any?): T, we can now write suspend fun which suspend or return immediately.
suspend fun immediateHelloWorld(): String =
suspendCoroutineUninterceptedOrReturn { cont ->
"Hello World!"
}
suspend fun suspendingHelloWorld(): String =
suspendCoroutineUninterceptedOrReturn { cont ->
cont.resumeWith(Result.success("Hello World!"))
COROUTINE_SUSPENDED
}Here you can see that in immediateHelloWorld we immediately return "Hello World!" to the suspendCoroutineUninterceptedOrReturn function, and in suspendingHelloWorld we return the value through the Continuation#resume callback and signal to the Coroutine that we returned the result through the callback.
This is useful when you need to access the CoroutineContext, but you know you don't actually need to suspend since it allows accesing the Continuation and its CoroutineContext without having to suspend.
Or when you want to suspend the coroutine but you don't want to explicitely intercept again, this is a useful optimisation when for example the CoroutineContext contains a ContinuationInterceptor that dispatches/schedules which is an expensive operation when it's unnecessary.
suspendCoroutine
So what is the difference with the higher level and better known suspendCoroutine function? As the name of the lower level function might have hinted, this function always calls intercepted but additionally it also removes the ability to immediately return whilst still allowing for this optimisation underneath at a relative small cost!.
The implementation of suspendCoroutine looks like this:
@SinceKotlin("1.3")
@InlineOnly
public suspend inline fun <T> suspendCoroutine(crossinline block: (Continuation<T>) -> Unit): T {
contract { callsInPlace(block, InvocationKind.EXACTLY_ONCE) }
return suspendCoroutineUninterceptedOrReturn { c: Continuation<T> ->
val safe = SafeContinuation(c.intercepted())
block(safe)
safe.getOrThrow()
}
}As you can see here it's build on top of the lower level function suspendCoroutineUninterceptedOrReturn, which we've discussed above, but it calls intercepted on the Continuation. This is done to add some additional safety, which only works in the case the CoroutineContext contains a dispatching ContinuationInterceptor.
Let's say that inside suspendCoroutine you wrap some Java API that runs on an unbounded cached pool (commonly known as IO pools) to run blocking IO operations such as JDBC, it would be dangerous to remain on this pool since we could unknownlingly exhaust the JDBC pool deadlocking calls to JDBC. For this reason it's much safer to always call intercepted so that after suspension we return to the original context. Similarly to how evalOn or withContext return to their original context after running an operation on a different pool. This is a safety measure commonly used in effect systems in other languages as well such as IO.async from Cats Effects in Scala. Important to note is that this only works if a dispatching ContinuationInterceptor is available in the CoroutineContext.
So what is SafeContinuation? SafeContinuation runs the code provide by the block lambda, and in case that it calls resumeWith in an immediate manner we want to return that result immmediately to suspendCoroutineUninterceptedOrReturn. In the case it for example forks to a different context, for example for JDBC or a network call, we cannot return immediately and we need to return COROUTINE_SUSPENDED such as explained above. This machinery and state machine is hidden inside SafeContinuation. Simply put, getOrThrow is raced with resumeWith to see which one is reached first and depending on who wins it returns the result or COROUTINE_SUSPENDED. You can check the implementation here.
startCoroutineUninterceptedOrReturn
The last method we haven't discussed yet is startCoroutineUninterceptedOrReturn, which is similar to startCoroutine but as you might've guessed doesn't call intercepted. Additionally, it's more raw bones since it either returns us the result immediately or it returns us the result in the Continuation callback.
If we look at the implementation, we can also clearly see how it works in a desugared way on the JVM.
@SinceKotlin("1.3")
@InlineOnly
public actual inline fun <T> (suspend () -> T).startCoroutineUninterceptedOrReturn(
completion: Continuation<T>
): Any? = (this as Function1<Continuation<T>, Any?>).invoke(completion)The suspend () -> T function is casted to a Function1<Continuation<T>, Any?>, and we simply pass the completion callback as an argument. If you think back to suspendCoroutineUninterceptedOrReturn which either passed the result to the callback or returned immediately it's clear how these two could be wired together.
So let's take back our examples from suspendCoroutineUninterceptedOrReturn and see how they behave when we run them with startCoroutineUninterceptedOrReturn.
import kotlin.coroutines.Continuation
import kotlin.coroutines.EmptyCoroutineContext
import kotlin.coroutines.intrinsics.suspendCoroutineUninterceptedOrReturn
import kotlin.coroutines.intrinsics.startCoroutineUninterceptedOrReturn
import kotlin.coroutines.intrinsics.COROUTINE_SUSPENDED
suspend fun immediateHelloWorld(): String =
suspendCoroutineUninterceptedOrReturn { cont ->
"Hello World!"
}
suspend fun suspendingHelloWorld(): String =
suspendCoroutineUninterceptedOrReturn { cont ->
cont.resumeWith(Result.success("Hello World!"))
COROUTINE_SUSPENDED
}
fun main() {
::immediateHelloWorld.startCoroutineUninterceptedOrReturn(Continuation(EmptyCoroutineContext) {
println("I am never called")
}).also { println("Immediately returned: $it") }
::suspendingHelloWorld.startCoroutineUninterceptedOrReturn(Continuation(EmptyCoroutineContext) {
println("I am called with result: $it")
}).also { println("suspending returned: $it") }
}Output:
Immediately returned: Hello World!
I am called with result: Success(Hello World!)
suspending returned: COROUTINE_SUSPENDEDHere in the output we can clearly see that in the first case the callback is never invoked, and we get the result immediately returned to us.
In contrast to the second case where the callback is invoked, and the immediate returned value is COROUTINE_SUSPENDED.
This means that if we use startCoroutineUninterceptedOrReturn you always need to check the returned value to check if it's COROUTINE_SUSPENDED or a user returned value (which in that case needs to be casted back from Any? to T).
This might all seem very obscure, but very powerful as we'll discuss below how we use these primitives to implement some of Arrow Fx Coroutines core operations.
Arrow Fx Coroutines
When building an effect system it's crucial that we take into account cancellation, since in todays high traffic back-ends or fast paced web/mobile users its important we can cancel request when a user or device is not interested in receiving the result anymore.
SuspendConnection
At the basis of this cancellation is a sealed class SuspendConnection, which is also a CoroutineContext and is defined as follows:
sealed class SuspendConnection : AbstractCoroutineContextElement(SuspendConnection) {
companion object Key : CoroutineContext.Key<SuspendConnection>
abstract suspend fun cancel(): Unit
abstract fun isCancelled(): Boolean
abstract fun push(token: CancelToken): Unit
// ...
}A CancelToken is a inline class wrapper around suspend () -> Unit which is used to model cancellation token. These are used to define cancellable operations in a typed manner.
It has two implementations, Uncancellable and DefaultConnection
Uncancellable
The uncancellable implementation has a no-op cancel method, and always returns false upon checking isCancelled.
Which subsequently means that all cancellable operations become uncancellable when run on this CoroutineContext.
Some optimisations could be implemented when an uncancellable context is detected.
DefaultConnection
The DefaultConnection keeps track of all CancelToken registered for a given SuspendConnection.
It does so by keeping all CancelToken in a FIFO stack, and running them in order when cancelled.
Since SuspendConnection is a FIFO stack, we can register a token before running a cancellable operation,
and pop the token after the cancellable operation has finished so
we don't run a CancelToken when it's not necessary anymore since that could lead to (undefined) weird behavior. This is useful for example in parMap2 where we want to launch two parallel operations, and push their cancellation system onto to the original SuspendConnection and pop it when both finish.
Cancellation
So what does it actually mean to cancel? Currently in Arrow 0.11.0, cancellation means that the code stops running (or in other words suspends forever and never returns).
This is implemented as a cancelBoundary, and with all the understanding of the low-level Kotlin machinery we have now can be implemented as follows.
suspend fun cancelBoundary(): Unit =
suspendCoroutineUninterceptedOrReturn { cont ->
if (cont.context[SuspendConnection]?.isCancelled() == true) COROUTINE_SUSPENDED
else Unit
}So a cancelBoundary checks if the SuspendConnection#isCancelled and if so it suspends forever and never invokes the callback, effectively this means it stops running the code.
In the case the SuspendConnection isn't cancelled, it immediately returns Unit. Here we use suspendCoroutineUninterceptedOrReturn since we can return immediately when no cancellation signal was triggered, and we don't want to intercept or schedule but we have to check the Continuation#context to be able to access the SuspendConnection.
So this is how from any user or library code we can add checks for cancellation that automatically exit when a cancellation signal was trigerred.
When isCancelled returns true, this means that all the registered CancelToken's inside SuspendConnection have run, or are running.
NOTE:
In Kotlin 1.4, CancellationException was added to the Kotlin Std as an ExperimentalApi. The documentation states that a cancellable coroutine should throw it to signal that it has been canccelled. In a future version of Arrow Fx Coroutines we can follow this pattern by instead of suspend upon cancellation like we did in Arrow Fx IO, and as is known from Cats Effects IO, we can throw CancellationException. This would also only happen when we're running on a cancellable SuspendConnection, and we'd still be able to control cancellation in the same way as you might be familiar with from Cats Effects IO or Arrow Fx IO.
startCoroutineCancellable
So how do we start a Coroutine with SuspendConnection, so we can start suspend () -> T in a cancellable way. Using SuspendConnection is no different than using any other CoroutineContext, so let's take a look at the most simple example we can come up with.
import arrow.fx.coroutines.CancelToken
import arrow.fx.coroutines.SuspendConnection
import arrow.fx.coroutines.seconds
import arrow.fx.coroutines.sleep
import kotlin.coroutines.Continuation
import kotlin.coroutines.startCoroutine
suspend fun main() {
val connection = SuspendConnection()
connection.push(CancelToken { println("CancelToken ran") })
suspend { sleep(1.seconds) }.startCoroutine(Continuation(connection) {
println("This never runs, since sleep was cancelled.")
})
connection.cancel()
Thread.sleep(2000) // Sleep longer than suspend program
}Output
CancelToken ranAs we can see in the output, it only prints the println from the CancelToken. The reason that we also don't see the output from the Continuation callback is because it was cancelled before reaching it, since the sleep operation gets cancelled and thus never resumed.
So one of the final questions is, how do we build cancellable operators such as sleep.
cancellable (suspendCoroutineCancellable)
To understand how cancellable operations are build, let's take a quick look at the implementation of sleep. If you're familiar with Arrow's IO, or another IO implementation the code below might look familiar.
internal val scheduler: ScheduledExecutorService = TODO()
suspend fun sleep(duration: Duration): Unit =
if (duration.amount <= 0) Unit
else cancellable { resumeWith: (Result) -> Unit ->
val cancelRef = scheduler.schedule(
{ resumeWith(Result.success(Unit)) },
duration.amount,
duration.timeUnit
)
CancelToken { cancelRef.cancel(false) }
}We open a cancellable block and get a callback which we can invoke with either a pure value, or a Throwable failure. We can then schedule a task to resume the callback after the desired Duration has passed, and to the cancellable block we return the CancelToken that we want to execute if the resulting suspend fun gets cancelled.
So the signature of cancellable looks like:
suspend fun <A> cancellable(cb: ((Result<A>) -> Unit) -> CancelToken): A
Where we create a cancellable suspend fun by consuming a callback of the shape ((Result<A>) -> Unit) and returning CancelToken.
So let's take a look at what the implementation of cancellable looks like.
suspend fun <A> cancellable(cb: ((Result<A>) -> Unit) -> CancelToken): A =
suspendCoroutine { cont ->
val conn = cont.context[SuspendConnection] ?: SuspendConnection.Uncancellable
val cancellable = ForwardCancellable()
conn.push(cancellable.cancel())
val token = try {
cb(cont::resumeWith)
} catch (throwable: Throwable) {
cont.resumeWith(Result.failure(throwable.nonFatalOrThrow()))
CancelToken.unit
}
cancellable.complete(token)
}We suspendCoroutine to access the Continuation, and since we use suspendCoroutine here it will also intercept and thus behave in the same safe way as Kotlin's high-level combinators.
We retrieve the SuspendConnection, and default back to an Uncancellable connection.
Since we don't know how long it will take before cb: ((Result<A>) -> Unit) -> CancelToken returns the CancelToken we need a mechanism to back-pressure conn, when it gets cancelled, until we receive the CancelToken. Otherwise conn might get cancelled whilst CancelToken is not registered yet creating a race-condition and possibly leaking resources.
ForwardCancellable allows us to defer or back-pressure the cancellation signal from the original conn, until we receive the complete signal with the resulting CancelToken.
So here we create a ForwardCancellable and push its cancel token onto the SuspendConnection. We run the foreign user code with the resumeWith callback, and complete the ForwardCancellable with the returned CancelToken.
In case of an exception we resumeWith the exception and return a no-op CancelToken.
ForwardCancellable
ForwardCancellable is useful to forward cancellation when you want to inject an uncancellable piece of logic,
or defer cancellation and wire two different SuspendConnection's together.
ForwardCancellable can be cancelled and completed with a CancelToken.
The following 4 scenarios can occur:
When
cancelis called beforecomplete, thencancel's token is back-pressured untilcompleteis called.
Thencancel's token will subsequently call the CancelToken that was passed tocomplete.When
cancelis called aftercomplete, thencancelimmediately runs the cancellation token passed tocomplete.When the
ForwardCancellableis completed, beforecancel's token is invoked.
The completedCancelTokenwill be invoked immediately whencancel's token is invoked.Calling
completetwice will result in anArrowInternalException, and is considered an internal developer bug.
File: generator-scripts/README.md
Generators
This directory is for source code generators.
To use generator:
- Open Arrow in IntelliJ
- Right click on file
- Select "Run
"
2. Official Technical Reference & Guides (arrow-kt/arrow-website)
File: README.md
arrow-website
This website is built using Docusaurus 2, a modern static website generator.
Requirements
The site is built through node and npm (included in Node.js), and the minimum version required for it to work is:
"node": ">=16.14"You can check your version by going to your terminal and performing:
node -vDepending on your OS and your OS apps/package managers, you could find different methods of having the proper node version installed. Please visit the Node.js website to see the best way for you to have it available in your system.
We recommend the use of a tool like NVM which could ease the installation, and will also allow you to have different node versions coexisting in your system in case you need it. Using a LTS Node.js version is recommended.
Installation
$ npm installLocal Development
$ npm run startThis command starts a local development server and opens up a browser window. Most changes are reflected live without having to restart the server.
Build
$ npm run buildThis command generates static content into the build directory and can be served using any static contents hosting service.
Deployment
Using SSH:
$ USE_SSH=true npm run deployNot using SSH:
$ GIT_USER=<Your GitHub username> npm run deployIf you are using GitHub pages for hosting, this command is a convenient way to build the website and push to the gh-pages branch.
File: content/docs/learn/immutable-data/index.md
title: Immutable Data
import DocCardList from '@theme/DocCardList';
Immutable data
Arrow Optics provide concise ways to handle immutable data.
:::note Where to find it
Immutable data manipulation with optics is available through the arrow-optics library,
and the corresponding plug-ins.
:::
File: content/docs/learn/immutable-data/intro.md
sidebar_position: 1
Introduction
Data classes, sealed hierarchies, and above all, immutable data is
an excellent recipe for domain modeling. If we want
to model a domain meticulously, we often end up with a large number of (nested)
classes, each of them representing a particular kind of object.
data class Person(val name: String, val age: Int, val address: Address)
data class Address(val street: Street, val city: City)
data class Street(val name: String, val number: Int?)
data class City(val name: String, val country: String)Alas, Kotlin doesn't provide great tooling out of the box to transform those
values. Data classes have a built-in copy method, but we need to repeat the
name of the fields and perform iterated copies even if we only want to touch
one single field.
fun Person.capitalizeCountry(): Person =
this.copy(
address = address.copy(
city = address.city.copy(
country = address.city.country.capitalize()
)
)
):::tip
We often use the word transform even though we are talking about immutable
data. In most cases, we refer to creating a copy of the value where some of
the fields are modified.
:::
Meet optics
Arrow provides a solution to this problem in the form of optics. Optics
are values that represent access to a value (or values) inside a larger
value. For example, we may have an optic focusing (that's the term we use)
on the address field of a Person. By combining different optics, we can
concentrate on nested elements, like the city field within the address field
within a Person. But code speaks louder than words, so let's see how the
example above might be improved using optics.
:::note Where to find it
Immutable data manipulation with optics is available through the arrow-optics library, and the corresponding arrow-optics-ksp-plugin compiler plug-in.
:::
The easiest way to start with Arrow Optics is through its compiler
plug-in. After adding it to your build
you just need to mark each class for which you want optics generated
with the @optics annotation.
import arrow.optics.*
@optics data class Person(val name: String, val age: Int, val address: Address) {
companion object
}
@optics data class Address(val street: Street, val city: City) {
companion object
}
@optics data class Street(val name: String, val number: Int?) {
companion object
}
@optics data class City(val name: String, val country: String) {
companion object
}:::caution Annoying companion object
You need to have a companion object declaration in each class, even if it's empty.
This is due to limitations in KSP,
the compiler plug-in framework used to implement the Arrow Optics plug-in.
:::
The plug-in generates optics for each field, available under the class name.
For example, Person.address is the optic focusing on the address field.
Furthermore, you can create optics that focus on nested fields using the
same dot notation you're used to. In this case,Person.address.city.country represents the optic focusing precisely on
the field we want to transform. By using it, we can reimplement capitalizeCountry
in two ways:
Optic-first: the
modifyoperation of an optic takes an entire value
(thisin the example) and the transformation to apply to the focused element.kotlinfun Person.capitalizeCountryModify(): Person = Person.address.city.country.modify(this) { it.capitalize() }Copy builder: Arrow Optics provides an overload of
copythat, instead
of named arguments, takes a block. Inside that block, you can use the syntaxoptic transform operationto modify a focused element.kotlinfun Person.capitalizeCountryCopy(): Person = this.copy { Person.address.city.country transform { it.capitalize() } }
Many optics to rule them all
You might have noticed that we speak about optics. In fact, there are a few
important kinds that differ in the amount of elements they
can potentially focus on. All the optics in the example above are lenses,
which have precisely one focus. At the other end of the spectrum, we have
traversals, which focus on any amount of elements; they can be used to
uniformly modify all the elements in a list, among other operations.
Optics form a hierarchy that we can summarize in the diagram below.
graph TD;
traversal{{"[**Traversal**](../traversal/) (0 .. β)<tt>getAll</tt> (return a list)<tt>modify</tt> and <tt>set</tt>"}};
optional{{"[**Optional**](../optional/) (0 .. 1)<tt>getOrNull</tt> (return a nullable)"}};
lens{{"[**Lens**](../lens/) (exactly 1)<tt>get</tt>"}};
prism{{"[Prism](../prism-iso/)<tt>reverseGet</tt>"}};
iso{{"[Iso](../prism-iso/#isomorphisms)"}}
traversal-->optional;
optional-->lens;
optional-->prism;
lens-->iso;
prism-->iso;The "main line" of optics is Traversal β Optional β Lens, which differ
only in the number of elements they focus on. Prism and Iso add a slight
twist: they allow not only modifying, but also creating new values and
matching over them.
:::info Even more optics
Arrow 1.x features a larger hierarchy of optics because the operations of
"getting" values and "modifying" them live in different interfaces.
Arrow 2.x simplifies the hierarchy to the five optics described in this section.
graph TD;
subgraph "<h4>only access</h4>"
fold{{"**Fold** (0 .. β)<tt>getAll</tt>"}};
optionalFold{{"**OptionalFold** (0 .. 1)<tt>getOrNull</tt>"}};
getter{{"**Getter** (exactly 1)<tt>get</tt>"}};
end
setter{{"**Setter**<tt>modify</tt> and <tt>set</tt>"}};
subgraph "<h4>Arrow 2.x</h4>"
traversal{{"**Traversal**"}};
optional{{"**Optional**"}};
lens{{"**Lens**"}};
prism{{"Prism<tt>reverseGet</tt>"}};
iso{{"Iso"}}
end
fold-->traversal;
setter-->traversal;
fold-->optionalFold;
optionalFold-->getter;
optionalFold-->optional;
traversal-->optional;
getter-->lens;
optional-->lens;
optional-->prism;
lens-->iso;
prism-->iso;:::
File: content/docs/learn/immutable-data/lens.md
sidebar_position: 2
Lenses
Lenses are the most common type of optic you work with. This section discusses
them at length.
:::info In a rush?
- Lenses represent references to fields.
- To access the value, use
get. - To modify the value, use
setandmodify. - To modify several elements at once, use
copy.
:::
The `Lens` type
We've mentioned in the introduction that optics are values that
represent access to data. You can draw parallels with how function values
represent behavior.
Let's introduce a few data classes and kindly ask the Arrow Optics plug-in to
generate lenses for every field by having an @optics annotation:
import arrow.optics.*
@optics data class Person(val name: String, val age: Int, val address: Address) {
companion object
}
@optics data class Address(val street: Street, val city: City) {
companion object
}
@optics data class Street(val name: String, val number: Int?) {
companion object
}
@optics data class City(val name: String, val country: String) {
companion object
}The lenses are generated in the companion object, so you can think of yourPerson being extended as follows:
data class Person(val name: String, val age: Int, val address: Address) {
companion object {
val name: Lens = TODO()
val age: Lens = TODO()
val address: Lens = TODO()
}
}Notice that lenses in Arrow are typed, which means that they "know"
both the type of the larger value and the type of the element we focus on.
β± this lens operates on 'Person'
val address: Lens
β³ this lens gives access to an 'Address' valueOperations
Lenses provide three primary operations:
getobtains the elements focused on with a lens.setchanges the value of the focus to a new one.modifytransforms the value of the focus by applying a given function.
Code speaks louder than words (well, sometimes). Here's a small snippet showcasing
the three operations applied to an instance of our Person class. Notice how
the three operations live on the lens and get the value they operate on as an
argument.
fun example() {
val me = Person(
"Alejandro", 35,
Address(Street("Kotlinstraat", 1), City("Hilversum", "Netherlands"))
)
Person.name.get(me) shouldBe "Alejandro"
val meAfterBirthdayParty = Person.age.modify(me) { it + 1 }
Person.age.get(meAfterBirthdayParty) shouldBe 36
val newAddress = Address(Street("Kotlinplein", null), City("Amsterdam", "Netherlands"))
val meAfterMoving = Person.address.set(me, newAddress)
Person.address.get(meAfterMoving) shouldBe newAddress
}Composition
The power of lenses (and optics in general) lies in the ability to compose
them to get to nested values. The type parameters in Lens ensure that the
composition accesses values that are really there. For example, here's a lens
that focuses on the city where a Person lives:
val personCity: Lens =
Person.address compose Address.city compose City.name
fun example() {
val me = Person(
"Alejandro", 35,
Address(Street("Kotlinstraat", 1), City("Hilversum", "Netherlands"))
)
personCity.get(me) shouldBe "Hilversum"
val meAtTheCapital = personCity.set(me, "Amsterdam")
}The compose infix function is an integral part of the library, but you hardly
will ever see it mentioned explicitly. As part of its job, the Arrow Optics compiler
plug-in introduces additional extension functions that allow you to use the
regular dot operation to access composed lenses. The code above can be rewritten
in that form:
fun example() {
val me = Person(
"Alejandro", 35,
Address(Street("Kotlinstraat", 1), City("Hilversum", "Netherlands"))
)
Person.address.city.name.get(me) shouldBe "Hilversum"
val meAtTheCapital = Person.address.city.name.set(me, "Amsterdam")
}More powerful `copy`
Everything we've discussed to this point is enough to make the transformation
of nested data much nicer without the nesting of nested copy calls. However,
if we need to modify more than one field, we must nest calls to set or modify.
fun Person.moveToAmsterdamModify(): Person =
Person.address.city.name.set(
Person.address.city.country.set(this, "Netherlands"),
"Amsterdam"
)Arrow Optics provides a copy function that replicates the
built-in copy ability to modify more than one field. The syntax is slightly different,
though. After the copy, you need to start a block. And within that block, you
can use the name of a lens to perform an operation.
fun Person.moveToAmsterdamCopy(): Person = copy {
Person.address.city.name set "Amsterdam"
Person.address.city.country set "Netherlands"
}You can also use transform if you need access to an original field value.
fun Person.capitalizeNameAndCountry(): Person = copy {
Person.address.city.name transform { it.capitalize() }
Person.address.city.country transform { it.capitalize() }
}Another nicety is that you can condense those operations that share
part of the journey to their focus. In our case, we are modifying two elements
in address.city, which we can join using inside.
fun Person.moveToAmsterdamInside(): Person = copy {
inside(Person.address.city) {
City.name set "Amsterdam"
City.country set "Netherlands"
}
}Sealed class hierarchies
If you have a set of classes with a common sealed parent, then lenses
can be generated for those properties shared by all of them.
Those properties must appear already in the common parent.
For example, the plug-in generates a lens of the name field given
the code below. This lens complements the prisms
that are generated to focus on each of the two subclasses.
@optics sealed interface SUser {
val name: String
companion object
}
@optics data class SPerson(
override val name: String,
val age: Int
): SUser {
companion object
}
@optics data class SCompany(
override val name: String,
val vat: VATNumber
): SUser {
companion object
}Integration with Compose
If you are using Compose, either in Android
or Multiplaftorm
flavors, you often need to update a MutableState
by applying some modification to the previous value.
The arrow-optics-compose package provides a version ofcopy useful in those situations.
class AppViewModel: ViewModel() {
private val _personData = mutableStateOf(...)
fun updatePersonalData(
newName: String, newAge: Int
) {
_personData.updateCopy {
Person.name set newName
Person.age set newAge
}
}
}:::note Compose and Snapshots
updateCopy uses the snapshot system
in Compose to ensure that all the modifications in the
block happen atomically.
:::
File: content/docs/learn/immutable-data/optional.md
sidebar_position: 3
Optionals
Optionals allow focusing on elements that may not be present. This includes
nullable values and elements on indexed collections such as List or Map.
:::info In a rush?
- Optionals represent potentially missing values.
- Prisms extend optionals to represent class hierarchies.
- To access the value, use
getOrNull. - To modify the value (only if present), use
setandmodify.
:::
Indexed collections
To exemplify why optionals are helpful, let's introduce a few domain classes
that model a small in-memory database mapping person names to the city in
which they live.
import arrow.optics.*
import arrow.optics.dsl.*
import arrow.optics.typeclasses.*
@optics data class Db(val cities: Map<String, City>) {
companion object
}
@optics data class City(val name: String, val country: String) {
companion object
}There's a notion of elements within a map, which we refer to by their key.
However, we cannot model them as lenses because we don't know whether
a particular key is present in the map. Optionals come to the rescue: they are
optics whose focus may not exist for a specific value.
As a result, the get operation is replaced by getOrNull, where null
indicates that the value is not present. The following code snippet provides
an example of that behavior using the index provided by Arrow Optics.
val db = Db(mapOf(
"Alejandro" to City("Hilversum", "Netherlands"),
"Ambrosio" to City("Ciudad Real", "Spain")
))
fun example() {
Db.cities.index("Alejandro").country.getOrNull(db) shouldBe "Netherlands"
Db.cities.index("Jack").country.getOrNull(db) shouldBe null
}One important (and sometimes surprising) behavior of optionals is that usingset or modify only transforms the value if it is already present. That means
that we cannot use index to add elements to the database, only to modify
those already present.
fun example() {
val dbWithJack = Db.cities.index("Jack").set(db, City("London", "UK"))
// Jack was not really added to the database
("Jack" in dbWithJack.cities) shouldBe false
}If you want to perform a change over the collection, use modify over the
lens that corresponds to that field.
fun example() {
val dbWithJack = Db.cities.modify(db) { it + ("Jack" to City("London", "UK")) }
// now Jack is finally in the database
("Jack" in dbWithJack.cities) shouldBe true
}:::tip More indexed collections
The first parameter to the index optional represents the type of collection
you are accessing. Currently, this argument can be Index.list, Index.map,Index.sequence, or Index.string. The choice defines the type of keys
and values expected by each operation.
:::
Nullable types
:::danger Breaking change in Arrow 2.x
The Arrow Optics plug-in in Arrow 1.x creates optionals for fields with nullable
types. This has sometimes led to surprises because, with an optional, you cannot
modify that value if it's null. In Arrow 2.x, every field gives rise to a lens
instead. The old behavior is available via the notNull extension function.
:::
Kotlin supports the notion of nullable types,
which clearly specify when a value may be absent. The compiler prevents you from
calling a method or function that requires a non-null value with a potentially
absent one. These checks are also in place when working with Arrow Optics;
if you have a nickname: Lens, you must account for
nullability in each potential modification.
It's also possible to turn a lens over a nullable type into an optional of the
unwrapped type using notNull. In the example above, nickname.notNull
has the type Optional (notice the lack of ? at the end
of the second type parameter). However, you should be aware that, following the same logic
as with indexed collections where you could not add or remove elements, withnotNull, you cannot change whether the value is null or not; only modify
it if it's already not null.
File: content/docs/learn/immutable-data/prism-iso.md
sidebar_position: 5
Prisms & Isos
Prisms extend the capabilities of optics from merely inspecting or modifying
values to constructing them. This is very useful when using sealed hierarchies
or value classes.
:::info In a rush?
- Prisms extend optionals to represent class hierarchies.
- Isos extend prisms (and lenses) to represent lossless conversion between types.
- One important case is given by value classes.
- To build a value use
reverseGet.
:::
(Sealed) class hierarchies
The following is an example of User where
we have two options: a person or a company.
import arrow.optics.*
@optics sealed interface User {
companion object
}
@optics data class Person(val name: String, val age: Int): User {
companion object
}
@optics data class Company(val name: String, val country: String): User {
companion object
}The Arrow Optics plug-in generates two optics within User, namelyUser.person and User.company. These optics only focus on a value when
it has the corresponding type. This is often used to modify a value only
for a specific type in the hierarchy, leaving the rest untouched. This is
precisely what happens in the function below:Persons get their age incremented, but Companys remains unchanged.
fun List.happyBirthday() =
map { User.person.age.modify(it) { age -> age + 1 } }Several of the types in Arrow Core fit this pattern of sealed hierarchy, and
Arrow Optics contains optics matching those. One example is Either, with
the corresponding left and right.
Constructing values
The optics we're discussing in this section provide an added feature: they can
be used to create new values in addition to inspecting or modifying existing
ones. Optionals with this power are called prisms, and this power is
available as the reverseGet operation.
For example, we can build a Left value using the corresponding prism
instead of the constructor.
fun example() {
val x = Prism.left().reverseGet(5)
x shouldBe Either.Left(5)
}Isomorphisms
Prisms allow you to construct values, but still the top of the hierarchy
may have different subclasses, so access still required as optional.
For example, you can construct an Either from Right, but when you
inspect an Either, Left is also a possibility. There are some cases
when the conversion between two types is lossless: we can go back and forth
without any chance of failure on inspection. We say that there is an
isomorphism between those types; for that reason the corresponding optic
is called an iso.
For example, we can move between Option<String> and Either
without loss of information. We convert from Some to Right, and fromNone to Left, and vice versa. There's no loss of information becauseUnit is an object, so there's a single instance which may appear in theLeft value.
:::tip Iso = Prism + Lens
You can see an iso as a prism where get always succeeds,
or as a lens which also supports the reverseGet operation.
:::
Value classes
One important case in which lossless conversion is possible is given
by value (or inline) classes,
which wrap a single value as a distinct type. This kind of classes are
very useful to model your domain accurately.
@optics data class Person(val name: String, val age: Age) {
companion object
}
@JvmInline @optics value class Age(val age: Int) {
companion object
}Since isos are also lenses, you can still use the syntax from the latter
to access the value contained in the class.
fun Person.happyBirthday(): Person =
Person.age.age.modify(this) { it + 1 }
fun example() {
val p = Person("me", Age(29))
p.happyBirthday().age shouldBe Age(30)
}File: content/docs/learn/immutable-data/reflection.md
sidebar_position: 6
Reflection
Although we strongly recommend generating optics using the DSL and @optics attribute, sometimes this is impossible. We provide the small utility package arrow-optics-reflect for those scenarios, which bridges Arrow Optics with Kotlin's reflection capabilities.
Kotlin provides a simple way to obtain a reference to a class member by using ClassName::memberName. For example, given the following class definition:
data class Person(val name: String, val friends: List<String>)we can use Person::name and Person::friends to refer to each of the fields in the class. Those references are very similar to optics.arrow-optics-reflect provides extension methods that turn those references into optics. You can obtain a lens for the name field in Person by writing:
Person::name.lenswhich you can later use as any other lens:
fun example() {
val p = Person("me", listOf("pat", "mat"))
val m = Person::name.lens.modify(p) { it.capitalize() }
m.name shouldBe "Me"
}:::danger
This only works on data classes with a public copy method (which is the default). Remember that, as opposed to a mutable variable, optics will always create a new copy when asking for modification.
:::
Nullables and collections
Sometimes it's preferable to expose a field using a different optic:
- When the field type is nullable, you can use
optionalto obtain an optional instead of a lens. - When the field type is a
ListorMap, you can useeveryandvaluesto obtain a traversal to the elements.
fun example() {
val p = Person("me", listOf("pat", "mat"))
val m = Person::friends.every.modify(p) { it.capitalize() }
m.friends shouldBe listOf("Pat", "Mat")
}Prisms
A typical pattern in Kotlin programming is to define a sealed abstract class (or interface) with subclasses representing choices in a union.
sealed interface Cutlery
object Fork: Cutlery
object Spoon: CutleryWe provide an instance method that creates a prism focused only on a specific subclass of a parent class. Both ends are essential and must be provided when creating the optic:
instance<Cutlery, Fork>()You can compose this optic freely with others. Here's an example in which we obtain the number of forks in a list of cutlery using optics:
fun example() {
val things = listOf(Fork, Spoon, Fork)
val forks = Every.list<Cutlery>() compose instance<Cutlery, Fork>()
val noOfForks = forks.size(things)
noOfForks shouldBe 2
}File: content/docs/learn/immutable-data/traversal.md
sidebar_position: 4
Traversals
The framework laid out by optics extends very nicely to values like lists, which
may contain any potential number of elements. Traversals is the name of that kind of
optic.
:::info In a rush?
- Traversals focus on an indefinite number of values.
- To access all the values, use
getAll. - Traversals provide an API similar to those of collections.
- To modify every value focused by the traversal, use
modify.
:::
`Every` element in a collection
As with other kinds of optics, Traversal<T, A> represents a reference to elements
of type A within a larger structure of type T. As hinted above, most
traversals arise from focusing on elements in a collection; in Arrow Optics,
those basic traversals live in the Every object.
Let's introduce a small data class for our examples:
@optics data class Person(val name: String, val age: Int, val friends: List) {
companion object
}If we want to update the age of every Person in a list, the usual way is to
use the map function from the standard library.
fun List.happyBirthdayMap(): List =
map { Person.age.modify(it) { age -> age + 1 } }The same code can be rewritten using only optics by leveraging a traversal for
lists.
fun List.happyBirthdayOptics(): List =
Every.list().age.modify(this) { age -> age + 1 }Traversals become more useful when composed with other optics. Imagine that we
want to update the age of all the friends of a Person; the usual implementation
with map and copy becomes quite complex.
fun Person.happyBirthdayFriends(): Person =
copy(
friends = friends.map { friend -> friend.copy(age = friend.age + 1) }
)The implementation using optics hides all the complexity related to mapping
and copying, and simply focuses on the path to access the values.
fun Person.happyBirthdayFriendsOptics(): Person =
Person.friends.every.age.modify(this) { it + 1 }:::note Every(List)
In versions prior to 2.0, .every required an additional argument specifying
the type of traversal. So one would write Person.friends.every(Every.list()).
This problem was related to type erasure,
but fortunately, a new encoding was found without this caveat.
:::
More than `getAll`
The counterpart to get and getOrNull when speaking about traversals is calledgetAll, and returns a list of focused elements. But you don't always have to
go through an intermediate list to obtain information about the elements focused
by a Traversal, we provide a large API
based on Kotlin's collections in the standard library.
For example, you can call isEmpty to check whether the traversal matches any
element. Or, more generally, you can call size to obtain the number of elements
it matches. Note that, in any case, these operations are "optics-first", so
you need to provide the value they operate on as an argument.
File: content/docs/learn/design/domain-modeling.md
title: Domain modeling sidebar_position: 1
:::note This article was originally published in the 47 Degrees blog.
:::
The goal of functional domain modeling is to describe your business domain as accurately as possible to achieve more type-safety, maximize the use of the compiler with our domain and, thus, prevent bugs and reduce unit testing. Additionally, it makes communicating about the domain easier since the domain is the touchpoint with the real world. Kotlin is a good fit for functional domain modeling. It offers us data class, sealed class, enum class, and value class. And we have Arrow, which provides some interesting generic data types such as Either and Ior.
In some codebases, you can find the following primitive type-based implementation of an Event:
data class Event(
val id: Long,
val title: String,
val organizer: String,
val description: String,
val date: LocalDate
)The types used here have little or no meaning since title, organizer, and description all share the same type. This makes our code prone to subtle bugs where we might rely on title instead of description, and the compiler would not be able to help us. Let's look at an example where things go wrong without the compiler being able to help us.
Event(
0L,
"Simon Vergauwen",
"In this blogpost we dive into functional DDD...",
"Functional Domain Modeling",
LocalDate.now()
)Here, we have mixed up organizerand description, but the compiler is happy and constructs the Event object. There are more cases where you can fall into this trap; for example, when destructuring.
So how do we prevent this from happening, or how can we improve our domain model to be better typed? We use value class, Kotlin's feature to disguise an already-existing type under a new name. Doing this causes no additional overhead since value class is erased at runtime. At the moment of writing, every appearance of value class requires a corresponding @JvmInline annotation.
@JvmInline value class EventId(val value: Long)
@JvmInline value class Organizer(val value: String)
@JvmInline value class Title(val value: String)
@JvmInline value class Description(val value: String)
data class Event(
val id: EventId,
val title: Title,
val organizer: Organizer,
val description: Description,
val date: LocalDate
)If we go back to our previous example, the compiler now fails to compile since we pass Organizer where Title is expected, Description where Organizer is expected, and so on.
Event(
EventId(0L),
Organizer("Simon Vergauwen"),
Description("In this blogpost we dive into functional DDD..."),
Title("Functional Domain Modeling"),
LocalDate.now()
)In functional programming, this type of data composition is also known as a product type or a record, which models an and relationship.
So we can say that an Event exists out of an EventId and a Title and an Organizer and a Description and a LocalDate, which tells us much more than an Event that exists out of a Long and a String and a String and a String and a LocalDate.
Let's say we must evolve our Event model to keep track of any age restrictions. We could model this with String again, but that would only worsen our original problem. So let's say we follow the MPAA film ratings, which is an enumeration of 5 different cases. Since we're clearly talking about a fixed set of cases, or enumeration, we use an enum class.
enum class AgeRestriction(val description: String) {
General("All ages admitted. Nothing that would offend parents for viewing by children."),
PG("Some material may not be suitable for children. Parents urged to give \"parental guidance\""),
PG13("Some material may be inappropriate for children under 13. Parents are urged to be cautious."),
Restricted("Under 17 requires accompanying parent or adult guardian. Contains some adult material."),
NC17("No One 17 and Under Admitted. Clearly adult.")
}Using an enum class is much more powerful than String for reasons beyond the problems we already explained above. A String has infinite possible values, but now we only have five possible ones.
So it's much easier to reason about AgeRestriction than to reason and work with String.
In functional programming, this type of data composition is also known as a sum type, which models an or relationship.
So we can say that an AgeRestriction is either General or PG or PG13 or Restricted or NC17.
This tells us much more than if it was just a String. A String would have infinite values, while AgeRestriction modeled as an enum class only has five different values.
So using sum types can drastically reduce the complexity of our types.
With online events on the rise, we have a different kind of event that doesn't occur at an Address, but rather at a certain Url. So, depending on what kind of Event it is, the data inside will be slightly different. Naively we could implement this as follows:
@JvmInline value class Url(val value: String)
@JvmInline value class City(val value: String)
@JvmInline value class Street(val value: String)
data class Address(val city: City, val street: Street)
data class Event(
val id: EventId,
val title: Title,
val organizer: Organizer,
val description: Description,
val date: LocalDate,
val ageRestriction: AgeRestriction,
val isOnline: Boolean,
val url: Url?,
val address: Address?
)This is a common encoding, but it can be pretty problematic. If isOnline is true, url will be non-null and vice-versa for address. However, after checking isOnline, both url and address is still null, so we'll end up with code like this.
fun printLocation(event: Event): Unit =
if(event.isOnline) {
event.url?.value?.let(::println)
} else {
event.address?.let(::println)
}But, even worse, we can also easily break the intended contract, like in the example below.
Event(
Id(0L),
Title("Functional Domain Modeling"),
Organizer("47 Degrees"),
Description("Building software with functional DDD..."),
LocalDate.now(),
AgeRestriction.General,
true,
null,
null
)The compiler is happy with the below definition, even though our intended contract said that, if it's isOnline, then url would be non-null.
We can prevent this issue by introducing a sealed class to combine Event.Online and Event.AtAddress in a typed way.
sealed class Event {
abstract val id: EventId
abstract val title: Title
abstract val organizer: Organizer
abstract val description: Description
abstract val ageRestriction: AgeRestriction
abstract val date: LocalDate
data class Online(
override val id: EventId,
override val title: Title,
override val organizer: Organizer,
override val description: Description,
override val ageRestriction: AgeRestriction,
override val date: LocalDate,
val url: Url
) : Event()
data class AtAddress(
override val id: EventId,
override val title: Title,
override val organizer: Organizer,
override val description: Description,
override val ageRestriction: AgeRestriction,
override val date: LocalDate,
val address: Address
) : Event()
}This solves our previous issue, where we can instantiate an online Event without an Url, and offers a much nicer way of working with the data. Instead of if(event.isOnline), we can now use an exhaustive when to pattern match on Event, and due to Kotlin's smart casting, we can safely access url in the case that it's Event.Online.
fun printLocation(event: Event): Unit =
when(event) {
is Online -> println(event.url.value)
is AtAddress -> println("${event.address.city}: ${event.address.street}")
}This type of data composition is also known as a sum type, which models an or relationship, but sealed class offers us more powerful capabilities than enum class.
A sealed class allows cases to exist out of object, data class, or even another sealed class. An enum class cannot extend another class, so it cannot be a case of a sealed class.
Here, our sealed class exists out of 2 cases, an Online or AtAddress Event, where Online and AtAddress are product types of several other types.
A rule of thumb in Kotlin is to use an enum class when the cases don't contain any data or, in other words, if all cases can be modeled as object.
As we've already seen in the examples above, modeling your domain precisely has many benefits. It can eliminate specific bugs, such as instantiating data incorrectly. It makes our code/model easier to reason about by eliminating invalid values, and it can improve code relying on our models.
Let's look at how we can use Arrow's data types to clarify our code's domain problems further. Our program has some EventService that can fetch an upcoming Event based on an EventId.
interface EventService {
suspend fun fetchUpcomingEvent(id: EventId): Event
}What is completely missing from our EventService is the different kinds of error scenarios we could encounter. It's only modeled through Throwable in suspend. So if we'd want to model the error domain explicitly, we could use any of the techniques we've already seen above.
Here, we model 2 different cases:
- An event is not found.
- An event is no longer upcoming but has already happened.
sealed class Error {
data class EventNotFound(val id: EventId): Error()
data class EventPassed(val event: Event): Error()
}We can compose these separate domains, Error and Event, using Either from Arrow Core. This allows us to model an or relationship, meaning that fetchUpcomingEvent either returns an Error or an Event, but never both. So Either is a generic sum type, which allows us to generically compose two separate domains with each other in an or relationship.
So, if we update our EventService:
interface EventService {
suspend fun fetchUpcomingEvent(id: EventId): Either<Error, Event>
}Since Either is defined as a sealed class in Arrow Core, we can use the same technique we used above with when to extract the Error or Event in a safe way.
In this article, we've seen how we can improve our domain by:
- Eliminating primitive types in our domain definition and using
value classto prevent runtime overhead. - Using
enum classandsealed classto model disjunctions in our domain, such as certain data being available depending on the type ofEvent. - Utilizing Arrow's
Eitherto compose two different domains with an or relationship.
File: content/docs/learn/design/effects-contexts.md
title: Effects and contexts sidebar_position: 2
:::note This article was originally published in the 47 Degrees blog, and subsequently updated to replace context receivers with context parameters.
:::
How you model data is an essential part of the design of your software. The other important side is how you model your behaviors. Today, we will talk about how to simplify your code with effects, avoiding heavyweight dependency injection frameworks on the go.
For every function you write, there's often some main data you require and many other ancillary pieces, the context. The typical example is a function to write some user information to the database: apart from the User itself, you need a DatabaseConnection, and maybe a way to log potential issues. One possibility is to make everything a parameter, as follows.
suspend fun User.saveInDb(conn: DatabaseConnection, logger: Logger) {
// prepare something
val result = conn.execute(update)
if (result.isFailure) logger.log("Big problem!")
// keep doing stuff
}This has an advantage: everything is explicit. You know precisely what you need to run this function. But there's a significant disadvantage too: everything is explicit. This means you must manually thread all this context in your functions. This also means a lot of boilerplate mudding your code and decreased maintainability, as any change in the context required for some piece of code could create a domino effect in the rest of the code.
This problem isn't new, of course; this is our old friend dependency injection (DI). In short, we want to have all the pieces we need, but don't want to thread them. The world has no shortage of DI frameworks, but almost all of them suffer from two disadvantages:
- Dependencies are not reflected as part of the type, so they become part of a hidden contract. We prefer explicitness for two reasons: (1) the rest of the team can discover the dependencies without looking in the function's body or how it's used, and (2) the compiler can perform some sanity checks for us.
- Dependency injection does not use the same vocabulary as the rest of the code. We prefer ways to model this problem using language features instead of relying on magic, such as annotation processing.
Dependencies as receivers
The technique we want to present today takes a similar starting point to many others: define each of your dependencies as interfaces, which we usually call effects. Remember to mark all of your functions with suspend. This allows for better control of both the effect and the runtime levels. In the case of a database connection, the interface can be as easy as a function returning it.
interface Database {
suspend fun connection(): DatabaseConnection
}We always suggest exposing a more constrained API, though. In this case, instead of giving up all your control of the connection, we may provide a way to execute a query. This opens the door to execute queueing the queries, for example.
interface Database {
suspend fun <A> execute(q: Query<A>): Result<A>
}Regardless of your choice, the final step is to make this interface your receiver type. Consequently, your User must now appear as an argument, and you can directly call execute, since it comes from the receiver Database interface.
suspend fun Database.saveUserInDb(user: User) {
// prepare something
val result = execute(update)
// do more things
}This solution is explicit, as Database appears as part of the type of saveUserInDb, but avoids lots of boilerplate by using one Kotlin feature: receivers.
Injecting
We can provide different incarnations of Database by creating new objects implementing the interface above. The most straightforward is simply wrapping a DatabaseConnection, as we were doing previously:
class DatabaseFromConnection(conn: DatabaseConnection) : Database {
override suspend fun <A> execute(q: Query<A>): Result<A> =
conn.execute(q)
}This is not the only one: the execution may work in a pool of connections, or we could use an in-memory database for testing.
class DatabaseFromPool(pool: ConnectionPool) : Database { ... }
class InMemoryDatabase() : Database { ... }The scope function with is one of Kotlin's idiomatic ways to provide the value of a receiver. In most cases, we would use a block right after it, in which Database is already made available.
suspend fun example() {
val conn = openDatabaseConnection(connParams)
with(DatabaseFromConnection(conn)) {
saveUserInDb(User("Alex"))
}
}Some teams prefer to bundle the whole create the wrapping object + call with into specialized runner functions. In that case, a clear pattern appears, in which the last parameter of such runner is a function that consumes that receiver.
suspend fun <A> db(
params: ConnectionParams,
f: suspend Database.() -> A): A {
val conn = openDatabaseConnection(connParams)
return with(DatabaseFromConnection(conn), f)
}
suspend fun example() {
db(connParams) { saveUserInDb(User("Alex")) }
}Always `suspend`
Strong statements require strong justifications, and we made one when we said that we always mark the functions in our effects with the suspend modifier. To understand the justification, we need to dive into the difference between describing some computation and actually executing it.
Most of the time, we think of computation as happening "on the spot." Whenever the program arrives at my println("Hello!"), it's going to execute it immediately. This model is simple; on the other hand, it's also problematic for a couple of reasons:
- If the
printlnwas in code we do not control, it's imposing on us a so-called side effect that we cannot easily handle or get rid of. - Immediate executing the code denies us the possibility of modifying the parameters of this execution: maybe we wanted it to happen in a different thread. For example, because this is part of a graphical interface requiring some operations to occur in a specific "UI thread."
The suspend modifier in Kotlin avoids these problems because such a function is not immediately executed. We can still compose those functions β in fact, Kotlin makes it so easy that it looks as if we were writing "normal" code β. But suspend functions themselves are only descriptions of what should be done. As a very simplistic approximation, think of a function () -> Int: such a function is not yet an integral number, but a recipe to obtain one.
Once you've built the full description, you can execute it in many different ways. The simplest one is runBlocking. By marking all the functions in our effects as suspend, we give freedom to both implementors of instances (who may choose to introduce additional threading, for example) and end users of the effects (who can ultimately decide how to spawn the initial computation).
If you squint your eyes a bit more, the type of runBlocking actually looks suspiciously similar to the db function defined above.
fun <T> runBlocking(
context: CoroutineContext = EmptyCoroutineContext,
block: suspend CoroutineScope.() -> T): THere, CoroutineScope is the effect related to handling concurrency since it provides functions like cancel or launch.
This notion of separating description from the execution of code is by no means new. The Scala community has been playing with this idea for a long time, as witnessed by Cats Effect or ZIO. Those libraries, however, cannot get to the same level of integration as Kotlin's built-in suspend. For deeper insight into the use of suspend as effects, check the Coroutines section on this site.
More than one dependency
I know what you're thinking: "where's the Logger?" Indeed, this technique requires a bit of refinement if your context comprises more than one effect. Let's start by defining the interface for logging:
interface Log {
suspend fun log(message: String): Unit
}Here comes the trick: since contexts are interfaces, we can represent requiring both effects as a subclass of both. For this, we bring in a not-so-well-known feature in Kotlin: where clauses. Simply stated, where clauses allow us to define more than one upper bound for a generic type. Let's see it in action:
suspend fun <Ctx> Ctx.saveUserInDb(user: User)
where Ctx : Database, Ctx : Log {
// prepare something
val result = execute(update)
if (result.isFailure) log("Big problem!")
// keep doing stuff
}This pattern slowly grows on you, and at the end, you always define your functions using a context receiver Ctx, whose effects you define later as upper bounds using where.
The main problem now is injecting the required instances of Database and Log so you can call saveUserInDb. Unfortunately, the following does not work, even though we have those two instances at hand.
suspend fun example() {
db(connParams) { stdoutLogger {
saveUserInDb(User("Alex"))
} }
}The problem is that saveUserInDb expects both to be packed into a single context object. We can work around it by creating an object on the spot and using delegation to refer to the previously-created instances.
suspend fun example() {
db(connParams) { stdoutLogger {
with(object : Database by this@db, Logger as this@stdoutLogger) {
saveUserInDb(User("Alex"))
}
} }
}The main restriction is that you cannot inject two values whose type only depends on different type parameters due to JVM erasure. For example, imagine we had defined a generic "environment value" effect.
interface Environment<A> {
suspend fun get(): A
}Then we cannot define a context requiring Environment<ConnectionParams> and Environment<AppConfig>. In practice, this is not so problematic, as you often want to introduce more specialized interfaces and constrain your usage as outlined above for Database.
Looking at the future
The future looks quite bright for Kotliners in this respect. For a few versions now, the language includes _context parameters, which would allow a cleaner design for what we are describing using subtyping. Using context receiver, we're able to state:
context(db: Database, logger: Logger)
fun User.saveInDb() { ... }and inject the values by simply nesting the calls to db and stdoutLogger, or simply using the context function.
Contexts, effects, algebras
Kotlin's documentation often discusses contexts when describing this usage of receivers. We often use effect instead; this term is used in functional programming to describe everything done by a function outside data manipulation. Some usages in that respect are "effect handlers" in many Haskell libraries, or Cats effect in the Scala community.
This technique is also related to tagless final as approached in Scala. In that case, we would refer to our Database interface as an algebra. There's an important distinction though: whereas tagless final uses heavyweight-type machinery available only in Scala (and Haskell), the technique in this blog post uses simpler mechanisms offered by Kotlin.