mathjs Documentation

1. Project Overview & Quickstart (josdejong/mathjs)

File: README.md

Math.js is an extensive math library for JavaScript and Node.js. It features a flexible expression parser with support for symbolic computation, comes with a large set of built-in functions and constants, and offers an integrated solution to work with different data types like numbers, big numbers, complex numbers, fractions, units, and matrices. Powerful and easy to use.

Features

Supports numbers, bignumbers, bigints, complex numbers, fractions, units, strings, arrays, and matrices.
Is compatible with JavaScript's built-in Math library.
Contains a flexible expression parser.
Does symbolic computation.
Comes with a large set of built-in functions and constants.
Can be used as a command line application as well.
Runs on any JavaScript engine.
Is easily extensible.
Open source.

Usage

Math.js can be used in both node.js and in the browser.

Install math.js using npm:

text

npm install mathjs

Or download mathjs via one of the CDN's listed on the downloads page:

https://mathjs.org/download.html

Math.js can be used similar to JavaScript's built-in Math library. Besides that,
math.js can evaluate
expressions
and supports
chained operations.

import {
  atan2, chain, derivative, e, evaluate, log, pi, pow, round, sqrt
} from 'mathjs'

// functions and constants
round(e, 3)                    // 2.718
atan2(3, -3) / pi              // 0.75
log(10000, 10)                 // 4
sqrt(-4)                       // 2i
pow([[-1, 2], [3, 1]], 2)      // [[7, 0], [0, 7]]
derivative('x^2 + x', 'x')     // 2 * x + 1

// expressions
evaluate('12 / (2.3 + 0.7)')   // 4
evaluate('12.7 cm to inch')    // 5 inch
evaluate('sin(45 deg) ^ 2')    // 0.5
evaluate('9 / 3 + 2i')         // 3 + 2i
evaluate('det([-1, 2; 3, 1])') // -7

// chaining
chain(3)
    .add(4)
    .multiply(2)
    .done()  // 14

See the Getting Started for a more detailed tutorial.

Browser support

Math.js works on any ES2020 compatible JavaScript engine, including node.js, Chrome, Firefox, Safari, and Edge.

Documentation

Build

First clone the project from github:

text

git clone [email protected]:josdejong/mathjs.git
cd mathjs

Install the project dependencies:

text

npm install

Then, the project can be build by executing the build script via npm:

text

npm run build

This will build ESM output, CommonJS output, and the bundle math.js
from the source files and put them in the folder lib.

Develop

When developing new features for mathjs, it is good to be aware of the following background information.

Code

The code of mathjs is written in ES modules, and requires all files to have a real, relative path, meaning the files must have a *.js extension. Please configure adding file extensions on auto import in your IDE.

Architecture

What mathjs tries to achieve is to offer an environment where you can do calculations with mixed data types,
like multiplying a regular number with a Complex number or a BigNumber, and work with all of those in matrices.
Mathjs also allows to add a new data type with little effort.

The solution that mathjs uses has two main ingredients:

Typed functions. All functions are created using typed-function. This makes it easier to (dynamically) create and extend a single function with new data types, automatically do type conversions on function inputs, etc. So, if you create function multiply for two numbers, you can extend it with support for multiplying your own data type, say MyDecimal. If you define a conversion from MyDecimal to number, the typed-function will automatically allow you to multiply a MyDecimal with a number.
Dependency injection. When we have a function multiply with support for MyDecimal, thanks to the dependency injection, other functions using multiply under the hood, like prod, will automatically support MyDecimal too. This also works the other way around: if you don't need the heavyweight multiply (which supports BigNumbers, matrices, etc), and you just need a plain and simple number support, you can use a lightweight implementation of multiply just for numbers, and inject that in prod and other functions.

At the lowest level, mathjs has immutable factory functions which create immutable functions. The core function math.create(...) creates a new instance having functions created from all passed factory functions. A mathjs instance is a collection of created functions. It contains a function like math.import to allow extending the instance with new functions, which can then be used in the expression parser.

Implementing a new function

A common case is to implement a new function. This involves the following steps:

Implement the function in the right category, for example ./src/function/arithmetic/myNewFunction.js, where you can replace arithmetic with the proper category, and myNewFunction with the name of the new function. Add the new function to the index files ./src/factoriesAny.js and possibly ./src/factoriesNumber.js.
Write documentation on the function in the source code comment of myNewFunction.js. This documentation is used to auto generate documentation on the website. It should include a History section with one line, indicating the upcoming version number in which the function will be created.
Write embedded documentation for the new function in ./src/expression/embeddedDocs/function/arithmetic/myNewFunction.js. Add the new documentation to the index file ./src/expression/embeddedDocs/embeddedDocs.js.
Write unit tests for the function in ./test/unit-tests/function/arithmetic/myNewFunction.test.js.
Write the necessary TypeScript definitions for the new function in ./types/index.d.ts, and write tests for it in ./test/typescript-tests/testTypes.ts. This is described in ./types/EXPLANATION.md -- make sure to read that page, as Typescript definitions must be added in multiple places in the code.
Ensure the code style is ok by running npm run lint (run npm run format to fix the code style automatically).

Build scripts

The build script currently generates two types of output:

any, generate entry points to create full versions of all functions
number: generating and entry points to create lightweight functions just supporting number

For each function, an object is generated containing the factory functions of all dependencies of the function. This allows to just load a specific set of functions, and not load or bundle any other functionality. So for example, to just create function add you can do math.create(addDependencies).

Test

To execute tests for the library, install the project dependencies once:

text

npm install

Then, the tests can be executed:

text

npm test

To test the type definitions:

text

npm run test:types

Additionally, the tests can be run on FireFox using headless mode:

text

npm run test:browser

To run the tests remotely on LambdaTest, first set the environment variables LT_USERNAME and LT_ACCESS_KEY with your username and access key and then execute:

text

npm run test:lambdatest

You can separately run the code linter, though it is also executed with npm test:

text

npm run lint

To automatically fix linting issue, run:

text

npm run format

To test code coverage of the tests:

text

npm run coverage

To see the coverage results, open the generated report in your browser:

text

./coverage/lcov-report/index.html

Continuous integration testing

Continuous integration tests are run on GitHub Actions and TestMu AI (formerly LambdaTest) every time a commit is pushed to GitHub. GitHub Actions runs the tests for different versions of node.js, and TestMu AI runs the tests on all major browsers.

Thanks, GitHub Actions and TestMu AI for the generous free hosting of this open source project!

License

mathjs is published under the Apache 2.0 license:

text

Copyright (C) 2013-2026 Jos de Jong <[email protected]>

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

   https://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

mathjs contains a JavaScript port of the CSparse library, published under the LGPL-2.1+ license:

text

CSparse: a Concise Sparse matrix package.
Copyright (c) 2006, Timothy A. Davis.
http://www.suitesparse.com

--------------------------------------------------------------------------------

CSparse is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation; either
version 2.1 of the License, or (at your option) any later version.

CSparse is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
Lesser General Public License for more details.

You should have received a copy of the GNU Lesser General Public
License along with this Module; if not, write to the Free Software
Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA

File: docs/index.md

Documentation

Math.js can be used in the browser, in node.js and in any JavaScript engine. Installation and download instructions are available on the Download page of the website.

Getting Started

Documentation

Core
Expressions
Data Types
- Numbers
- BigNumbers
- bigints
- Fractions
- Complex Numbers
- Matrices
- Units
Reference
Custom bundling
Command Line Interface
History

2. Official Technical Reference & Guides (josdejong/josdejong.github.io)

File: README.md

blog

Personal blog

https://josdejong.com

Theme based on https://github.com/mismith0227/hugo_theme_pickles

Start server locally

text

hugo server -D

Here -D is to also render drafts.

Create a new article like:

text

hugo new posts/2020-06-20-my-new-post.md

When the article is finished and ready to publish, remove the line draft = true.

How to publish

Write a new article, once done, set draft=false, commit the new article with git.
Commit and push the main branch to GitHub, the website is published via GitHub Actions: https://github.com/josdejong/blog/actions.

File: content/posts/2013-07-19-welcome.md

categories: [] comments: true date: "2013-07-19T00:00:00Z" title: Welcome! url: "blog/2013/07/19/welcome/"

Welcome at my blog. I just installed it freshly using
octopress.
Like everybody, I'm very unique and have the world a lot of interesting things
to tell ;-). So here we go.

File: content/posts/2013-07-27-details-matter.md

categories:

application
details
quality
user experience
comments: true
date: "2013-07-27T00:00:00Z"
title: Details matter
url: "blog/2013/07/27/details-matter/"

This is a theme which pops up again and again at the software projects I am working on.
I really liked this blog by Jeff Atwood in this regard, explaining
all your app is, is a collection of tiny details.
Software which has not taken care of details right can be extremely annoying,
and vice versa, an application having its details right can be real joy to work with.

This article explains the importance of paying attention to details
in application development. This holds true for different different application
levels. The article discusses this in two sections:

The first section explains the importance for the end users of your applications,
and gives some practical tips to help you to achieve a better end user experience.
The second section focuses on developing usable software libraries.
This highlights the need to write readable and well documented code.

End user experience

A decade ago people where used to adapting themselves to
the vagaries of software applications - they had no choice. This is kind of
the world up-side down: computers and applications should make peoples life
easier and more fun, rather than get into their way and regularly raise their
irritation level to dangerous heights.

Nowadays people have more choice, for example when it comes to smartphone apps.
The barrier to install and remove apps is very low, and the selection is huge.
If your app doesn't work exactly as a user wants or expects,
it will be immediately replaced by some other app who better understands what
the user wants. Users are very critical in that, which is healthy.
This forces app builders to deliver quality if they want to attract a large
user base.

Now what kind of details am I talking about?
Well, consider your own experience with applications.
Both desktop, web, and mobile apps.
What kind of things annoy you in the applications that you use?
Here a list with examples of details that strongly influence the user experience.
They are all very logic, but I see these kind of details being overlooked a lot,
even in professional applications.

Feedback.
When pressing a button, you want immediate feedback that the
application is working on it. Pressing a button and having the application
"do nothing" or even block completely is a terrible user experience.
Most applications do not need to be blazing fast and be finished without
missing a beat, but it is very important that they inform the user being busy.
For actions that really take a lot of time, the application
should also inform the user of the progress of the action.

When developing an application, you typically work in a development environment
with fast hardware and a fast internet connection.
It is important to anticipate which actions may get slow when in a real world
environment without fast hardware, fast internet, and with loaded servers.
To be robust against this, it is wise to always give the user immediate
feedback. Disable the clicked button, inform the user via a "loading..."
message, and perform actions in an asynchronous, non-blocking way such that
the user interface stay responsive all the time.
Navigation.
Applications are often build with one type of navigation
in mind. However, more and more people are using differing devices
to work with the same applications.
Especially websites should reckon with this and make it easy to navigate
in different ways: mouse, touch, keyboard.
Keep in mind to have scrolling work smoothly both for touch as well as using
the mouse wheel. Built in shortcut keys, so you can navigate with arrows
for example. Things like quick keys can make it much more comfortable and
efficient for users who use your application intensively.
Loosing data.
What is more annoying than filling out a large
form, press the "Submit" button, getting back some weird error,
and... having to fill out everything again?
Or losing data from a document which you where editing when the application
crashes, or when your internet connection goes down at an unfortunate moment.
And what about accidentally doing something in an app and having no way to
undo the action?

I suppose everyone has encountered these situation.
We are getting better at this though: more and more web applications take care
of this now. Forms stay neatly filled and more and more applications
automatically save your input, no longer relying on you pressing the "Save"
button every now and then.
A user should never have to worry about losing data.
It is necessary to take this into account when designing your application.
Documentation.
Ideally, an application is self explanatory. A new user immediately knows
how to use the application. Mobile applications typically have limited
functionality, leaving no doubt on how to use the application.
For larger applications, it is important to provide users with documentation
in the form of tooltips explaining the behavior of buttons and menu items,
helping the user along the way.
As a last resort, an application should always have documentation explaining
the functionality and the workflow in detail.
Design.
Design is not directly influencing the functionality which your application
offers. It plays a large role though in how a user feels about using the
application. Ensuring the application looks good makes the user feel that
this is a professional, robust application.
Which is probably is, because you appear to care for details.

All of these points of attention may sound trivial,
so why don't we just apply this?
Well, because it takes more time and is more complicated to reckon with all
of this when building an application.

Keep in mind though that you build an application to serve the user.
Force yourself to put a little more effort in ensuring a good user
experience. This can really make the difference,
resulting in lots of users actually enjoying using your application.

Developer experience

On a different level, developers themselves are also encountering user
experience issues when working with libraries and frameworks.

Most programmers don't like commenting code and documenting their applications.
It is boring and takes a lot of time.
However, in order to make your work usable for others,
either as code or in the form of a library or framework.
You really need to pay attention to this.

Structure.
Your code should be well structured and neatly commented. Regularly take
time to clean up redundant parts of the code.
Ensure your API's stay consistent.
Don't be afraid to refactor code to improve readability and clarity.
Don't use dirty tricks which are hard to understand for others.
Keep in mind that you only write the code once, but it will be read often.
Spending a little extra time to make your code better readable and
maintainable will most likely pay of, not only for others but also for
yourself.
Documentation.
The more developers use your work, the more it will pay of to spend some
effort on documenting your library.
Suppose you don't document your work. Then each developer starting to
use your library may have to spend a couple of hours (and a lot of frustration)
to dig in your undocumented code and search the web for examples and help.
When spending a few hours to document your code, you will save each of these
developers a couple of hours work. This, ultimately, will save us together
a lot of time which we can spend doing actual work.

And no, now and then exporting your code as JavaDoc is not enough.
Good documentation consists of the following parts:
- A high level overview of how your library works, how all parts fit together,
  explaining the flow of the program.
- A set tutorials, helping a developer to get up and running with your stuff.
- A set of examples, demonstrating how to use your library in practice.
- Detailed documentation describing the API's of your library.
  These docs should be easy to generate when you neatly document your code.
Setup.
A library or framework will not gain much traction when the process to get it
up and running is a 14 step process which takes hours and requires a lot of
knowledge and pre-conditions.
It should be a no-brainer to get your library up and running.
If not, developers will give up and find themselves an alternative which
actually does come with good documentation and an easy setup.
Writing a getting started tutorial helps a lot making clear what an
installation exactly needs.

When your library is complicated to get up and running,
this may be a sign that something is wrong. If that is the case,
take a step back and figure out what makes it so complicated.
Eliminate these complex requirements and ensure a smooth first experience
with your software.

You can be a brilliant programmer and write genius pieces of code.
However, if you don't make your work accessible and usable for others, your
code will most likely have a limited use, and will die as soon as you're no
longer in the neighborhood to baby-sit and maintain it.
Put a little extra effort in documenting your code and make it easy for other
developers to get started with your code. This will drastically increase the
applicability of your code, and makes the life of developers much more fun.

I'm sure you prefer to use applications and libraries which are well documented
and supported, so take these as an example when writing your own.

File: content/posts/2013-09-18-multithreading-is-todays-goto.md

categories:

multithreading
concurrency
asynchronousity
parallelization
comments: true
date: "2013-09-18T00:00:00Z"
title: Multithreading is today's GOTO
url: "blog/2013/09/18/multithreading-is-todays-goto/"

Ask any programmer to the worst bug he ever encountered and you will very likely hear a terrible story concerning multithreading and data locks. Or maybe an issue around memory corruption or memory leakage. Two topics which are apparently hard to master for us programmers.

The latter case, memory corruption, is becoming less and less of a problem, as most modern languages come with automatic memory management. Most people see this as a step forward: it saves us a lot of trouble and results in more robust applications. Multithreading on the other hand is seen as something obvious and essential. Even more, programming languages which don't support multithreading are seen as inferior and immature.

Why is it that we hold on so strongly to this multithreading paradigm while it gives us so much trouble? Are we too proud to acknowledge that we are not capable of taming this multithreaded beast?

What is wrong with multithreading

I work a lot with Java. Java is very proud on the solutions it offers for concurrency, multithreading, and resource locking. And yes, they work great... if you apply them correctly and consistently everywhere in your application. And that is where the fundamental problem with multithreading lies: when you write code, it is not thread-safe by default. You have to take explicit actions to make your classes robust against a multithreaded environment. And when doing maintenance in a later stage, you need to be careful to keep everything thread-safe. Making a class thread safe requires extra effort and worsens performance, so there is no incentive to apply thread-safe patterns by default.

Doing multithreading right requires in-depth knowledge of all components that you interact with. In large, real-world applications, it can be hard to figure out which code and which third party libraries are thread-safe and which don't. It isn't trivial to make the whole thread-safe without introducing dead locks or other issues.

Multithreading is very error prone. It goes wrong unless you take careful safety measures. And when it goes wrong, it's incredibly hard to debug. Multithreading is easy in theory, but practice shows differently.

The case for singlethreaded applications

When starting to program JavaScript, coming from a Java or C background, you may notice a deafening absence of complexity. Part of this is because JavaScript is singlethreaded. You just don't-have-to-worry about data corruption as your application just can't manipulate the same data simultaneously from multiple threads - there are no multiple threads. It's an incredible relief not having to reckon with these issues. Instead, you can focus on the actual logic you want to build around your data.

Why is it that JavaScript is so strong in dealing with asynchronousity? Common JavaScript and Node.js patterns use asynchronous callback functions, timeouts, and alike. The reason that this works so easy and naturally is because JavaScript is singlethreaded. It wouldn't have worked if JavaScript had been multithreaded. That would result in a terrible mess of locks on the data shared between all these callbacks. This may be the reason why these kind of powerful concurrency patterns are not used a lot in languages like C++ and Java: these languages just aren't very suitable for asynchronous programming.

The point here is: a singlethreaded application just works without you having to take safety measures to protect your data. You can't screw it up. You can do as complicated things as you like, even so complicated that you don't understand it yourself anymore. But you simply can't get your data corrupted.

Yeah very funny, but I just need multithreading

It's just plain stupid to have an application doing all work in a single thread. Any single CPU-heavy request will completely block the application. You remember these websites where you press a button and the complete browser freezes because JavaScript needs to do some work?! That's thanks to JavaScript being singlethreaded you know...

Well, what you need isn't multithreading per se. What you do need are solutions for concurrency. However there are other models to deal with concurrency besides multithreading and resource locking. Typical models used in JavaScript and Node.js are:

Message passing between isolated processes.
JavaScript supports web workers in the browser to do CPU intensive tasks in the background.
In Node.js you typically spawn child processes which interact with the main process via message passing.
Note that the client/server model itself is a message passing solution as well: the client offloads work to the server side, and they interact with each via other HTTP requests.
Splitting a long running computation in small pieces.
To keep a singlethreaded application responsive, you can partition long running computations into many small parts. The parts are executed one by one with a short timeout between them, to give the application space to process other events. The size of the parts can be determined by a fixed amount of work, or by a maximum amount of time which the calculation may block the application.

These solutions require a different way of thinking and modelling than multithreading and resource locking. My experience though is that modelling your application in separate, isolated parts (or as actors) results in a very clear and natural separation of concerns. This is beneficial to the overall architecture of your application.

How to utilize multi-core processors then?

One concern with single-threaded applications is that they can't utilize a multi-core processor. This is a currently active discussion: what programming paradigms are suitable to utilize multi-core architectures? And no, multithreading is not the solution. Multithreaded applications don't magically solve the multi-core dilemma. The solution involves modelling your application in a way that allows for parallel processing.

I don't have a ready made solution for this multi-core problem. I'm convinced however that dealing with multi-cores should be abstracted away from the developer. Similar to GPU's which do most graphics processing in parallel without us having to worry about that.
All the developer should have to worry about is modelling his application in a parallelizable way, and the programming language should ideally facilitate this. Not you, but the virtual machine running your application should worry about the (multi-core) hardware.

An analogy with the beloved GOTO statement

There have been long and heated discussions in the past on whether the GOTO statement is good or bad. It's incredible powerful, and quite a logic statement for everybody with a background in assembly. At the same time, using GOTO can very easily lead to spagetti code. Nowadays, GOTO is considered harmful by most, but it wasn't at that time. Luckily, smart people came up with a completely different paradigm: structured programming. Modelling your application in separate subroutines. This offered a solution on a different abstraction level than the GOTO statement. One which was even more powerful, and made the GOTO statement redundant.

I believe that you can compare multitreading with the GOTO statement. At least how multithreading is implemented right now. It solves a serious need: we need solutions for concurrency. However, it is not the right solution for the problem. It introduces so many troubles that, as with GOTO, we can consider this a harmful paradigm which must be avoided. We are a little bit late to fulfill Steve Yegge's prediction #3, but better late than never.

We should start thinking about better solutions for concurrency instead of multithreading. No, maybe not. The solutions are probably already there. Obviously, they will require a completely different way of modelling our applications. But all we have to do is realize that multithreading really sucks, stop torturing ourselves, and start using some of the other excellent solutions out there.

File: content/posts/2014-01-13-shaped-by-the-language-you-use.md

categories:

language
paradigms
comments: true
date: "2014-01-13T00:00:00Z"
title: Shaped by the language you use
url: "blog/2014/01/13/shaped-by-the-language-you-use/"

A few months ago I did an astonishing discovery, namely how strong your solutions are shaped by the programming language you use, the language that lives in your head. The interesting thing is that I already knew this. But despite that, I hadn't expected this to be so incredibly influencing until I saw such a clear case of this in my own work.

The right tool for the right job

First a short explanation of my programming background. I've experience with various Basics, C++, Java, Python, JavaScript, Matlab, and others. Experience with procedural languages, object oriented languages, prototypal languages. Strict and loosly typed languages. Compiled languages and scripting languages. Desktop, web, server, mobile. I have explored quite some corners of the programming universe, and the more I learn, the more I discover just how little I know...

When gaining experience with different languages, you discover that each language has its strong and weak sides. There is not a single language which is the ultimate solution for everything. That's why programmers tell you to 'use the right tool for the right job' rather than strive to find that one super-language. Which language is the best choice for a certain project depends on things like functional requirements, platform support, team experience, and more.

So far so good.

Consequences of choosing a language

The choice for a programming language is quite determinative for how the application code will be structured. The language may be procedural, object oriented, event driven, prototype-oriented, functional, etc. Every language has its own paradigms, patterns, and best practices.

For example the strict nature of Java encourages you structure your code very well, create abstraction layers and use patterns like getters/setters and interfaces. This helps in keeping large projects scalable and maintainable directly from the start. On the other hand, the focus on structure has annoying side effects: lots of classes consisting mainly of empty getters and setters, over-abstraction which makes it hard to understand what is actually going on, a lot of marshalling of objects, etc. All of this can cause a lot of overhead over the actual contents of an application. it can make an application rigid and adjustments cumbersome.

The opposite of the Java may be JavaScript, which is extremely unstructured and dynamic by nature. This encourages you to program in a flexible, defensive, and robust way, acting friendly and forgiving for (partially) wrong inputs. JavaScript applications are typically built up in a loosely-coupled and modular way, perfectly fitting in the continuously changing, dynamic environment of today's world. On the other hand, the lack of support for structure can easily result in archaic and unmaintainable code. A lot of discipline is required from the programmer to create and guard the needed structure and abstractions.

It is best to adhere to a language's best practices. When working in a team, one need to agree on code style and structure to ensure consistent code, and need to document the code. This helps making the application understandable and maintainable by others than just you.

The choice of language plays an important role in how your application will be structured. However, the choice of language may influence a lot more than just that.

Shaped by your language

Where it is really becoming interesting is when realizing that a language not only is a matter of differing syntax, organization, library support, and tooling. It also strongly influences the way you solve problems and the type of solutions that you see.

You are not using a programming language 'just' to express your logic and your solutions. Rather, the solutions that you come up with are influenced and limited by the language that lives in your head. If you are used to an object-oriented language, you will think in a different way than when you are used to functional languages. These and other paradigms all offer a different solution-space. Some problems are easily described in an object-oriented way, others by taking an event-driven or functional approach.

How influencing a language can be became clear to me while developing a math application in my spare time. This application is basically capable of parsing and evaluating expressions. It supports complex numbers, matrices, units, and other advanced data types. I have built this application first in a Basic language, then in C++, then in Java, and finally in JavaScript.

The third iteration of the application, written in Java, had become really neat. It was well structured and had logical abstractions. All data types where classes extending a single base class MathObject. Functions too where created as classes extending a specific base class MathFunction. The function classes had an eval method with a specific signature, accepting instances of MathObject as arguments. Implementing a new function was really easy and robust: just implement eval functions for the supported data types, unsupported types where automatically dealt with by the extended base class. The expression parser generated a node tree linking these functions and arguments together. It was a closed and well controlled solution.

But then I decided to rewrite the whole application in JavaScript. Now that is quite a different story. JavaScript encourages you to create an open solution rather than closed, controlled solution. JavaScript is no strictly typed language, and you can call any function with any type and number of parameters. You can't control that. At first I thought this would complicate the application, having to do a lot of type checking in every function. But it turned out to give amazing opportunities. Rather than creating my own data types, I let the functions accept both JavaScript's primitives (Booleans, Numbers, Strings, and Arrays), as well as advanced data types (such Matrix and Complex). Having first class functions available was really an enormous advantage.

This open nature of JavaScript gave a great synergy between my application and JavaScript itself. The Java application was built on top of Java in a kind of closed, sandboxed environment, only accepting my own data types. The JavaScript version on the other hand was seamlessly integrated with JavaScript itself, embracing JavaScripts own data types and first class functions. This resulted in strongly reduced complexity, and an amazingly open solution, which because of this openness can be integrated seamlessly with other applications and libraries (no marshalling horror...).

I was really perplexed to see how influential a language can be on the architecture of an application. I have been thinking: what did I do wrong with the Java version? Well, I did too much in an object oriented fashion. And I wanted to control everything with my own classes. This object-oriented paradigm is the core of what Java is, but it turns out not to be the right solution for everything. Now, with this lesson learned, could I rewrite the application in Java with the simplicity it has in JavaScript? I think for an important part: yes. This is mainly a matter of thinking outside of the (Java) box, and not primarily about a limitation in the language itself (though I really miss lambdas...). A language seriously influences your way of thinking.

Conclusion

There are many programming languages available. Non of them is perfect, each has their strong and weak sides. When starting a project, it is important to choose the right tool for the job: the language which best fits given functional requirements, platform support, and team experience.

The choice of language does not only determine syntax and tool chain though. It also strongly influences the way you solve problems and the type of solutions that you see.

When you are familiar with one language and one way of working, you may be blind for other types of solutions, which might have resulted in much simpler solutions. Even when you are familiar with multiple languages it may be the case that you - like me - always tend to stick to a few familiar patterns.

Be aware of the fact that the language that lives in your mind plays an important role in how you solve problems. Improve yourself by learning new languages, paradigms, and patterns. Dare to experiment, take time to build and compare prototypes following different paradigms and see how that works out for your problem. Every now and then this will lead to surprising results. It's just awesome to see a complicated problem being reduced to an elegant, straight forward solution when applying a well suited paradigm!

File: content/posts/2014-10-18-the-art-of-creating-simple-but-flexible-apis.md

categories:

api
complexity
flexibility
simplicity
comments: true
date: "2014-10-18T00:00:00Z"
title: The art of creating simple but flexible APIs
url: "blog/2014/10/18/the-art-of-creating-simple-but-flexible-apis/"

A while ago I read the blog post Ranging Near and Far by Scott Sauyet. On Echo JS, Scott titled a link to this blog "Choosing simple APIs over flexible ones". This title suggests that you have to choose between either simple or flexible APIs, which is a false dilemma. This got me thinking: What makes a good API?

Flexibility

Scott discusses the API of the range function of Ramda compared to that of Underscore. Where Underscores range supports optional start (defaulting to zero), custom step size, and a negative step, Ramda's range "only" supports a step size of 1 and does not allow to omit the start value. While Underscores implementation is more flexible, Scott argues that Ramda's more limited implementation may be the better choice because of its simplicity. Scott suggests that it is way more complicated to use Underscores range because of it being more flexible. Scott has an argument here, but there are more factors determining how easy it is to use an API.

Simple != limited

It is important to distinguish simplicity from being limited. Simplicity is the opposite of complexity. The complexity of an API can be measured by the cognitive load it requires to use the API. This cognitive load is determined by:

Compactness
Count the number of functions and the number of parameters per function.
This may be the hardest to get right. You have to find the right primitives to cover the APIs functionality in a compact way. The functions should be as "orthogonal" to each other as possible, not overlapping in functionality.
Verbosity
Common scenarios should be easy to do, verbosity should be minimal there. Ideally, the verbosity and complexity should grow linear when using more advanced features.
Consistency
All functions of the API should have consistent naming, and the function signatures should be consistent.
Appropriate function naming
Shorter function names are easier to remember, though too short function names can become meaningless. Finding good names may be one of the most difficult things in software development and it's importance should not be underestimated.
Adhere to common practice
The API should ideally work similar to other (comparable) APIs where the audience is familiar with.
The number of different ways you can call an individual function
The less different signatures a function accepts, the easier it is to recall it's functioning.

The importance of these factors depends on the application and the situation. In established environments, adhering to common practice is very important, even at the expense of consistency. In others cases, consistency or compactness may be decisive. Together, these factors determine the complexity of the API as a whole, and thus how easy it will be for a user to work with it.

Practical tips

Creating simple, powerful APIs requires a lot of practice. There is no magic formula, but here are some practical tips:

Limit the number of parameters of a function to 3 or maximum 4, use an object if you have more
If you have to pass more than 4 parameters to a function, it's not easy to see which argument corresponds to which parameter. You can accidentally forget one argument, causing all arguments matching the wrong parameter.
Take a at the signature of the CreatWindowEx function of the C++ Win32 API, which requires 12 (!) parameters:
c++
```
// BAD: way too many parameters
HWND WINAPI CreateWindowEx(DWORD dwExStyle, LPCTSTR lpClassName, LPCTSTR lpWindowName, DWORD dwStyle, int x int y, int nWidth, int nHeight, HWND hWndParent, HMENU hMenu, HINSTANCE hInstance, LPVOID lpParam);
```
Instead of having an endless list with parameter, pass a single object with named parameters instead. Take a look at the ajax function of jQuery:
js
```
$.ajax({
  type: 'POST',
  url: '/rest/user',
  data: { name: 'John', location: 'Boston' },
  dataType: 'application/json'
  success: function () {...},
  error: function () {...},
  // ... and much more (optional) parameters ...
});
```
This function has 30+ parameters, but this is no problem at all: normally you only need to provide a few of them. Passing an object holding all parameters has a couple of advantages:
- By looking at the code, you immediately see which parameter the arguments represent.
- No need to pass null or undefined for optional parameters. Instead, most parameters are optional and have a good default value, so for common scenarios you will have to provide only a few parameters.

Support granularity in options
Sometimes a specific feature has a set of options related to just this feature. These options can be put in a nested object. For example, one parameter can support both a boolean to enable/disable the feature as a whole, or accepts an object allowing to specify detailed options for this feature.

// completely disable a feature
simulation.run({
  animate: false
});

// enable a feature, use default configuration
simulation.run({
  animate: true
});

// enable a feature, custom configuration
simulation.run({
  animate: {
    duration: 3000,
    renderer: 'webgl',
    ...
  }
});

Or another example:

// specify one color for all headers
draw({
  color: 'gray'
})

// specify different colors for individual components
draw({
  color: {
    h1: 'gray',
    h2: 'darkred',
    h3: 'gray',
    default: 'black'
  } 
})

Too long function names may indicate a lack of abstraction
If you find yourself giving functions long names, and cannot shorten them without loosing their meaning, this may be a sign that you have to rethink your API. It can be the case that the API has too many functions and needs to be split out in multiple small APIs on different layers:

// BAD: a verbose API due to lack of abstraction
app.createUser(...)
app.updateUser(...)
app.deleteUser(...)
app.createProduct(...)
app.updateProduct(...)
app.deleteProduct(...)
...

// GOOD: split in multiple smaller (and trivial) APIs
app.user: User
app.product: Product

User.create(...)
User.update(...)
User.delete(...)

Product.create(...)
Product.update(...)
Product.delete(...)

It may also be the case that functions can be bundled together into a single one with an extra option:

// BAD: too specific functions
app.createCustomer(...)
app.createEnterpriseCustomer(...)
app.createStudentCustomer(...)

// GOOD:
app.createCustomer({
  type: 'normal'    // Choose from 'normal', 'enterprise', 'student'
  // ... other arguments
})

Create an open, flexible API
It's tempting to create "controlled" APIs where you control exactly what a user can input. Trying to control this easily results in a limited, closed API. For example the following function may look easy and quite flexible:
js
```
customers.find({
  name: string, 
  profession: string, 
  city: string
})
```
But this API is limited to a few hardcoded properties. The following approach is much more flexible, and probably easier to remember:
js
```
// functional approach: callback function to filter customers
customers.find(function (customer) {
  // ... full freedom to match a customer against any criteria
  return customer.name == 'John';
});
```
Note that this second approach requires Customer to have a public API, which may be a good thing anyway.
Adhere to common practice
If you create a function range(start, end), it is a good idea to exclude end. If you create string or array manipulation functions, it's a good idea to keep the indexes zero-based. Why? Because all existing (JavaScript) native functions and modules do it like that. You will cause a lot of trouble for your users when deviating from these kind of common practice.
Internal and external API
Typically, a module exposes a small part of the internal API as public API. If you see that the public API is totally different from the API used internally, you may need to rethink the API. Users will start asking for features resembling the internal API that you use yourself. Users want to be just as close to the metal as you yourself.
Prevent feature creep
All modules start out with a cozy and simple API. But as the module matures, the API grows, and there is the danger of "feature creep". In the end, an API may be so bloated that it's hardly usable for the average user. To prevent this, at some point the module needs get an extensible architecture: A limited core, enriched with a set of extensions. This extensibility allows users to build custom solutions for their specific needs on top of it, instead of demanding all kind of exotic features to be built in the main product.
Write documentation and examples
By writing docs you will see whether the API is consistent and easy to explain. Writing examples helps a lot in figuring out whether your API works as nicely as you thought beforehand. Main purpose of docs and examples are of course getting your users up and running with your software as smoothly as possible.
Look at APIs of popular modules
Study the APIs of popular modules to see how they do things and why. One of the reasons they have become the most popular of their kind will be that they have a great API.

Conclusion

The complexity of an API can be measured by the cognitive load it requires to use the API. Factors determining the cognitive load are compactness and consistency of the API, verbosity in common use cases, right function naming, and whether they follow common practice. Depending on the goals of a library, authors may come to different conclusions for similar functionality, like different choices made for the range function by either Ramda and Underscore.

Learning to write APIs which are both simple and flexible requires a lot of practice and takes a life time. Learn from the APIs of popular modules. Pay attention to details in your own APIs. Keep rethinking your API, keep trying to get things more and more logic, consistent, and concise. Watch for feature creep. Aim for an open, extensible API.

File: content/posts/2015-01-06-code-reuse.md

categories:

reuse
genericity
modularity
composability
coupling
cohesion
comments: true
date: "2015-01-06T00:00:00Z"
title: Reusable software? Just don't write generic code
url: "blog/2015/01/06/code-reuse/"

It may be the holy grail of programming: writing reusable code. Not getting stuck by inventing the wheel again and again, but moving forward and building on top of what has been done. Looking at what we do today, we still haven't really figured this out.

So you are writing a software library. You had a great idea for a simple, broadly applicable solution. You have cooked up a nice API and are happy with the result. Then, new features are needed. The library grows and grows, and becomes extremely versatile. To account for more and more specific cases though, the API grows at a disproportional rate. Due to this feature creep, soon there isn't much left from your initial, beautiful API. At some point it becomes almost too complicated to work with the library due to all configuration options and edge cases.

Then, a new, fresh alternative arises and moves your library into oblivion. This new library is awesome. It doesn't suffer from a bloated API, it just works. It does more or less the same as your library but is faster and easier to use. Over time though, this new library will go through the same cycle as yours. It gets bloated too and will be replaced with something fresh at some time.

What went wrong?

Generic solutions

In the quest for writing reusable software, it's a common mistake to make software too generic. To build a single, uber flexible, monolithic solution which is applicable in an extremely broad range of use cases and environments. Being generic is a basic requirement for reusable software. However, making software too generic causes it's usability to suffer.

When a piece of software grows too large, it becomes bloated. Besides having a negative impact on the performance, the software typically becomes more complicated to use. The component will have a more extensive API and configuration. This requires more knowledge on the clients side, and makes debugging harder as more things can go wrong. Developing and debugging the component itself becomes harder as one needs to account for more and more edge cases. Summarizing, we can say:

Maximizing genericity complicates use

Besides negatively affecting the usability, being too generic can also hurt the reusability. Having a broader API and requiring more configuration results in a higher "contact surface" or coupling between the component and the application. This makes it harder to move the component to an other application for reuse.

Making your software too generic can make it too complicated to use. This can result in the software not being used at all, defeating the original goal of writing reusable software.

Reusable software

How to write reusable software then? Don't write a single monolithic solution that will solve all problems. Instead, write small, composable, focused components.
Components that just do one thing and stick with it (single responsibility principle). Components that are loosely coupled and have a concise API. This is one of the basic fundamentals in software engineering:

Reusable software requires loose coupling and strong cohesion

Writing small components will give your software a high chance of survival: all individual components are easy to use and understand, and are usable on their own in various use cases. When one of the component gets outdated, it can be replaced relatively easy as the components doesn't have a high coupling with each other.

It's hard to find the right boundaries and granularity for components. You don't always get it right the first time. And when software grows over time, it will be necessary to reorganize, split, and refactor components in order to keep them focused. Don't be afraid of refactoring. This is necessary to keep your software healthy in the long run, as opposed to killing it off because of overweight.

A great source of inspiration to me is the Node.js community, which seems to have small, composable components in their DNA. It's like LEGO. It's the Unix philosophy:

Favor composability over monolithic design

The flexibility and non-strictness of JavaScript helps a lot in this regard. It's interesting to see the recent interest in functional and reactive programming in the JavaScript community. These paradigms can help a lot in writing loosely coupled, reusable code. Also, the trend towards micro service architectures looks promising.

Pitfalls

There are a couple of common patterns which are good in itself, but can easily be used the wrong way, adversely affecting the (re)usability of code. Let's have a closer look at two of them: abstraction layers and inheritance.

Abstraction layers everywhere

An important ingredient for reusability is separating interface from implementation. That way, implementations can be replaced without the need to change the clients code. The Gang of Four writes in Design patterns:

"Program to an interface, not an implementation."

Abstracting from implementations is a great solution when applied in the right cases. For example when using an ODBC driver to communicate with different types of SQL databases without having to write database specific code.

Especially in the Java world, I see a tendency to overuse and misuse this type of abstraction via interfaces. It's as if some Java programmers don't dare to make any concrete choice, and want to be able to replace everything without having to change anything in their code. Therefore, they introduce lots of abstractions and plenty of configuration. This results in very generic code, not bound to specific implementations. It may look like a great, generic, reusable solution. In practice however, these solutions are easily too complicated to be usable at all.

As Mark Seeman puts it in his blog Interfaces are not abstractions:

"Do you extract interfaces from your classes to enable loose coupling? If so, you probably have a 1:1 relationship between your interfaces and the concrete classes that implement them. That's probably not a good sign, and violates the Reused Abstractions Principle (RAP)."

Do not introduce an abstraction layer unless it is clear that you will have multiple implementations (YAGNI principle).

Excessive use of inheritance

Inheritance is a great way to extend a class with extra functionality. In the Java community, inheritance is used excessively. Java offers a single inheritance model, where classes can only inherit from a single super class. It's not uncommon to see inheritance levels of six or even more classes, which is absurd. De Gang of Four describes the danger of overusing inheritance in Design Patterns:

"Because inheritance exposes a subclass to details of its parent's implementation, it's often said that 'inheritance breaks encapsulation'."

Inheritance causes a high coupling, as the superclass exposes it's internals to subclasses, and the subclasses are depending on external functionality offered by the superclass. This makes it hard to change the superclass without breaking the functionality of its subclasses.

Daniel Pietraru clearly explains the difficulty with inheritance in No, Inheritance is not the way to achieve code reuse:

"Inheritance, while a very powerful tool, it is very hard to use right. While the rule of thumb - use inheritance for 'is a' relationships and composition for 'has a' relationships - is very true, applying it in non trivial situations is difficult and wanders close to philosophy. Think about this question: 'Is an CircusDog a Dog or maybe CircusDog is just a role a Dog plays?'."

In general, inheritance should be used only when:

One class is an extension of another class, reusing most of the code of this class and adding some new features to it.
Two or more classes are very similar and share a large part of code, which can then be moved into a superclass used by all of them.

Conclusions

Writing reusable code is not about developing generic, monolithic all-rounders. The key to writing reusable code is to write focused, composable components with a high cohesion and loose coupling.

Keep your code sanitized. Replace outdated components. Don't be afraid to refactor when needed. Be careful to use patterns like interfaces and inheritance proportionally, do not overuse them. Strive for writing focused components with a low complexity: easy to understand and easy to maintain.

File: content/posts/2015-03-28-a-broader-view-on-isomorphic-javascript.md

categories:

isomorphic
api
cross environment
comments: true
date: "2015-03-28T00:00:00Z"
title: A broader view on isomorphic JavaScript
url: "blog/2015/03/28/a-broader-view-on-isomorphic-javascript/"

tl;dr In order to ease full stack JavaScript development, we need to take away the hurdles keeping these worlds separated. We need unified, isomorphic solutions for common functionality like module loading, networking, background processes, logging, and others.

Ever had to use statements like the following?

if (typeof window !== 'undefined') {
  // browser
}
else {
  // node.js
}

It's a simple check to see whether a script is being executed in in a browser, and if not, assume that it's a node.js process. I noticed that I need them more and more. I mostly develop libraries. As soon as I need functionality which is not standard available in both browser and node.js, like fetching urls, I have to create two different implementations and switch depending on the environment. It's annoying and I don't yet have a seamless solution for dealing with environment specific code.

There has been quite some some discussion lately about isomorphic JavaScript. Most discussion focuses on client or server side rendering, spurred by the recent popularity of React. Isomorphic JavaScript however is a much broader topic. It's basically about writing code which can run in different environments such as the browser and node.js. In this article we will have a look at isomorphic JavaScript in a broader sense, and explore ways to deal with environment specific code in a seamless way.

Types of isomorphic code

JavaScript can run in different environments. One can think of client-side in the browser, server-side with node.js, on mobile devices using Cordova or PhoneGap, and on microcontrollers using Espruino. These environments have in common that they all support the JavaScript language. What they don't have in common is environment specific functionality like the DOM and AJAX calls in the browsers, http servers and disk access in node.js, and access to mobile phone sensors in Cordova.

Spike Brehm distinguishes two types of isomorphic modules in this talk Master Class: Isomorphic JavaScript: environment agnostic or shimmed per environment.

Environment agnostic modules.
These modules use only pure JavaScript functionality, and no environment specific properties like window and process. Examples are lodash, async, moment.js, numeral.js, and math.js.
Modules shimmed per environment.
They offer a single API, and different implementations per environment. For example superagent, socket.io, debug, and pubnub.

Environment agnostic modules just work fine everywhere. Shimmed modules however require selecting the right implementation depending on the current environment. This is no problem when the module is consumed by an application. In that case it's clear in which environment the application will run, like server side or client side. But when the module is consumed by another module, this choice can't be made. For example a library for currency conversion (which makes network requests to a REST API), or a library using a proper debug library. The consuming module will have to distribute implementations for the different environments too. This results in a cascading effect, forcing all modules down the chain to create builds for various environments.

Shimmable functionality

In the previous section we discussed isomorphic modules offering a single API and different implementations per environment. This is only possible when the underlying functionality is available on each of the targeted environments. Let's for now focus on the browser and node.js. In both, networking is a core functionality: the browser offers AJAX technology, and node.js has HTTP (and other protocols) in it's genes. Both environments offer similar functionality, but with totally different API's.

The following table shows an (incomplete) list with similar functionality available on both browser and node.js:

Functionality	Browser	Node.js
Module system	none	CommonJS
HTTP requests	XMLHttpRequest (AJAX)	http
Sockets	WebSocket	net
WebRTC	WebRTC, peer.js	node-webrtc
Background processes	Web Worker	child_process
Rendering	DOM	React, jsdom, PhantomJS
Debugging	Web Console: Chrome, FireFox	console, Winston, Bunyan

JavaScript land still lacks de facto abstractions on top of these overlapping functionalities, allowing to use the same API's in both browser and node.js environments.

Notable here is the approach of browserify, which offers browser shims for many node.js specific modules like http and console. In practice though, the size of these shims can be quite large, which can be a serious problem for browser applications.

The browser and node.js worlds are currently still largely separated. There is a lot of room to bring them together on multiple levels, easing full stack JavaScript development. One step in the right direction is taken by npm for example, which encourages us to use npm for both front-end and back-end development, instead of using different package managers for different environments. An other important step is a universal module system for JavaScript, available in both node.js and the browser: ES6 modules. The third step is up to us module developers: build more isomorphic libraries which can be used in both browser and node.js without any hassle.

How to distribute isomorphic libraries?

There are basically two ways for distributing isomorphic libraries:

Distribute different implementations per environment. The library consumer will have to select the right implementation depending on the environment.
Distribute a single, universal implementation, which automatically detects the environment at runtime and loads the needed environment specific code.

The problem with the first approach is that it does not solve the problem but passes it on to the library consumer. This is fine when consumed by an application, but not when consumed by an other library. This library in turn has to check the environment, load the correct implementation of the consumed library, and has to distribute multiple versions of itself too. This results in a cascading effect.

The problem with the second approach is that the isomorphic library will contain the code for all supported environments. For node.js modules this is no big deal, but for the browser it's important to minimize the amount of bundled code. It's simply not acceptable to send precious kB's over the wire containing unused, node.js specific code.

The only scalable, long term solution here is the second one. The exposure of environment specific code should be minimized. Library consumers should not have to bother about which implementation to pick, nor should they be bothered with distributing their own library in an isomorphic way due to isomorphic dependencies. This means we have to tackle the problem of distributing bloated application code containing code for multiple environments. We need build tools which can strip away non-relevant, environment specific code, like removing node.js specific code when bundling a browser app.

Tooling and conventions

In order to facilitate universally usable isomorphic libraries, two things are needed:

Tooling to detect the current environment at runtime.
Tooling to strip node.js specific code from application bundles when redundant.

Environment detection

In order to be able to let an isomorphic library switch to the right implementation, the library needs a way to detect in which environment it is running. This does not need to be very complicated. There may exist already exist solutions for this without me knowing it. We need detection and handling of environments. It could look like:

environment.isBrowser = function () {
  return typeof window !== 'undefined';
};
environment.isNode = function () {
  return typeof window === 'undefined' && typeof process !== 'undefined';
};
environment.isWebWorker = function () {
  return typeof self !== 'undefined' && typeof postMessage === 'function';
};
// ... detect whether running in a node.js child_process
// ... detect whether running a native web application like Cordova
// ... detect whether running on desktop or on a mobile device
// ... detect what type of browser (Chrome, FF, IE, ...)
// ... detect the browser version or node version

This module should of course be published as an UMD module. Suppose we have an http library with an implementation for the browser (myModule.browser.js) and node.js (myModule.node.js), the module could expose it's main file, index.js, like:

module.exports = environment.isNode()
    ? require('./myModule.node.js')
    : require('./myModule.browser.js');

Bundling for the browser

Suppose that modules are distributed as a single, universal implementation, containing multiple implementations for different environments. When bundling an application for use in the browser, the parts of the code not relevant for the browser (i.e. implementations for node.js) should be removed.

Tools like browserify, webpack, and uglifyjs are typically used for bundling and minification of browser apps. When bundling an application, it is possible to configure specific modules to be excluded from the bundle. This mechanism could be used to exclude environment specific modules without forcing the end-user to specify them one by one. When all environment specific code uses a suffix, it would be easy to filter away these files from an application bundle. Files could be named like:

text

// apply suffixes for files only relevant for a specific environment:
myModule.node.js      // node.js, io.js
myModule.browser.js   // browsers: Chrome, FF, IE, ...
myModule.native.js    // native web apps, Cordova, PhoneGap, ...
// ... more

To bundle an application with browserify, all you have to do is ignore all *.node.js files:

text

browserify myApp.js --outfile ./dist/myApp.js --exclude **/*.node.js

Note that such a solution will only work if every isomorphic module uses the same environment suffixes. There is a convention needed to let this be successful.

Conclusion

There is a lot going on in the JavaScript world. Innovation is taking place at an incredible speed. It's hard to keep up with the latest trends. With the rise of node.js, JavaScript is becoming more and more ubiquitous. Two worlds, client and server side, are both powered by JavaScript. But common functionality such as networking requires the use of environment specific modules. Code cannot just be executed in both environments, which causes a split between the two worlds.

In order to ease full stack JavaScript development, we need to take away the hurdles keeping these worlds separated. We need to come up with unified, isomorphic solutions for common functionality like module loading (ES6 to the rescue), networking, background processes, logging, and others.

There are opportunities here. JavaScript is already going full speed. Let's ignite the after burners and go crazy.

Update 2018-01-10: Replaced a broken link to an article about the CommonJS module system.

File: content/posts/2015-08-08-architectural-changes-in-mathjs-v2.md

categories:

mathjs
modularization
typed-function
comments: true
date: "2015-08-08T00:00:00Z"
title: Architectural changes in math.js v2
url: "blog/2015/08/08/architectural-changes-in-mathjs-v2/"

tl;dr Math.js v2 has been released a week ago. It has a completely reworked, modular architecture which allows creating custom builds. All functions have been rewritten into typed-functions. This takes away a lot of overhead for type checking and type conversions. These changes create a lot of flexibility in both extending and consuming the library.

A week ago math.js v2 has been released, a big milestone! You can download it here. Math.js is an extensive math library for JavaScript and Node.js. It features a flexible expression parser and offers an integrated solution to work with numbers, fractions, big numbers, complex numbers, units, and matrices. It has been around for two and a half years now and has a steadily growing community around it and number of dedicated contributors.

Math.js v2 has a completely reworked architecture, and it adds support for fractions, sparse matrices, a linear equation solver, improved LaTeX support, and more. Currently there are contributors working on algebraic differentiation and derived units. A special thanks to Favian, Max, Rogelio, and Eric, who have put a lot of effort in the library lately.

Growing pains

As you can imagine, the library is growing. The point where it becomes too large to just mindlessly bundle it with your application comes closer. You typically only need a small part of the library, and it's just a waste of CPU, memory, and KB's when you need to bundle the complete library with your application. The architecture of math.js v1 however wasn't suitable for custom bundling. It was also missing ways to optimize functions for known input types (very important for matrix operations). Lastly, dealing with new data types and conversions between data types was quite cumbersome.

For this reason the architecture of math.js v2 was drastically changed. The main architectural changes are related to modularization, dynamic type checking, and lazy loading. These changes will be covered in the in the next sections.

Modularization

To enable custom loading and custom bundling, a modular structure and explicit dependencies are required. This wasn't trivial to achieve. Math.js does not consist of a set of isolated functions like say lodash. There are a couple of challenges here:

The functions are not static but need access to a configuration object. This means that the functions need to be created via a factory function, and need to be loaded via some loader which passes the config object to the function. This makes it impossible to just reference individual functions like mathjs/lib/arithmetic/add: the functions must be constructed via a loader.
The architecture of v1 had a shared math namespace containing all functions. This was done for a couple of reasons: there where circular references between functions, functions needed access to a shared configuration, and some functions depended on other functions in the namespace. Problem was that this shared namespace resulted in implicit dependencies, which makes it impossible to determine dependencies when bundling a subset of the functions. In v2, this shared namespace is removed and the circular references are resolved. A function has to load it's dependencies explicitly. This allows to bundle only a subset of the available functions and data types.
The most difficult issue is the following. There are currently about ten different data types, and each function contains an implementation for each data type. Suppose you only need one or two of these data types, how can you exclude the other data types and exclude their implementations from all individual functions? We do not have a solution for this yet. Possible directions for solutions are:
- Smart dead-code elimination of bundlers/minifiers, possibly using conditional compilation.
- Writing a browserify transform or webpack loader which is capable of removing code related to excluded data types. This may be relatively easy because of the easily recognizable structure of typed-functions.
Math.js v1 contained a couple of large helper files. If you only need one of the functions in a helper file, you need to bundle the whole file. In v2, most helper files are split into one-file-per-function. This gives a more fine grained way to define dependencies, and allows to bundle only the utility functions that are actually used.

Dynamic type checking logic

Each function of math.js handles multiple data types like numbers, complex number, and matrices. In v1, these functions consisted of a single function body, which did type checking of the input types, and evaluated accordingly (see for example exp.js of v1). Problems with this approach are:

all type checking and conversion logic has to be written explicitly in the functions themselves. This gives quite some code overhead, and distracts from the actual function logic.
it is also a "closed" solution: no easy way to extend existing functions with support for new data types.
it is not possible to access low level implementations of a function handling a specific data type (like add for numbers). This is a necessity for performance optimizations with for example matrix operations.

In v2, functions are written as typed-functions using the typed-function library. See for example exp.js of v2). Using typed-function, functions can be defined as a set of signatures such as a function add having signatures add(number, number) and add(unit, unit). typed-function dynamically constructs the functions from the given signatures. Conversions from non-supported types is done automatically where applicable. Typed functions can be extended at run time with more signatures. Individual signatures can be addressed, allowing for performance optimizations when an input type is known beforehand. For example when you have a matrix containing just numbers, there is no need to type check every entry when performing an operation on the matrix.

The typed functions in v2 give us a lot of flexibility and decoupling, and ensures that when writing new functions, you only have to care about the implementation logic, not about type checking or type conversion logic.

Lazy loading

A third chane worth mentioning is lazy loading of functions. Loading all functions and all data types in memory costs time and, well, memory. In v2, functions and data types are by default lazy loaded: only as soon as you use a function, it will be loaded. This is achieved by using Object properties which replace their value with the lazy-loaded function on the first read. The result is a much faster loading library.

Conclusion

All in all I'm very happy with what we have achieved with v2. It was a bumpy road and a lot of boring refactoring. And there have been difficulties getting a good performance with typed-functions. But it turns out that the final v2 has even better performance than v1 in most cases, and the lazy-loading solution ensures that loading times remain limited: v2 loads about as fast as v1, whilst the library has grown considerably. It was definitely worth it.

There is still a lot of work to do regarding modularization. There is a lot to be optimized there. And while it's awesome that we can finally do custom builds, one thing I don't quite like is the need to write your own custom index file for custom bundling. Ideally, users should not have to bother about custom bundles: this should be done automatically by our bundling and minifying tools. I'm following initiatives like rollup by Rich Harris with great interest.

Math.js v2 takes away the bottlenecks for further growth, and gives a lot of flexibility for both extending as well as consuming the library. The future looks bright.

File: content/posts/2016-06-05-static-typing-the-good-parts.md

categories:

interoperability
scalability
static typing
dynamic typing
structural typing
nominal typing
comments: true
date: "2016-06-05T00:00:00Z"
title: 'Static typing: the good parts'
url: "blog/2016/06/05/static-typing-the-good-parts/"

tl;dr So far I've been hesitant to embrace static type checkers like TypeScript. It's because I associated static type checking with the restrictions and interoperability issues that I know from Java and C++. However, it turns out that TypeScript is different: it uses a structural type system rather than a nominal one, which eliminates these downsides. It gives us a way to describe data types without sacrificing flexibility, interoperability, or scalability.

An uneasy feeling

Sometimes I have the feeling that I'm supposed to be averse of static typing in order to be a proper JavaScript citizen. I regularly hear people bragging about not needing it at all. And I have to say, I barely miss static type checking in JavaScript either: the cons are way bigger than the pros. Still, I miss static typing every now and then. That made me wonder, what is it about static typing that I miss?

The smell of static typing gives me an uncomfortable feeling. It reminds my of my hard-core Java time which was a continuous fight against the system: trying to get classes to play nice together. In Java, a lot of time goes into bookkeeping of classes and marshalling between them rather than developing actual application logic. So far I have been hesitant to embrace static type checkers like TypeScript or Flow which are built on top of JavaScript. I was afraid that the down sides of static typing would leak back into my applications again.

My first thought was: when I feel I need static typing in my application... it's probably a code smell. It means that the application has grown to large and complicated to easily keep in my head. I can "solve" this by using a statically typed language which allows me to reach a higher level of complexity before my head explodes. Or I can acknowledge that the application is just too complicated and refactor it into smaller, easier to understand chunks.

However, there is more to it. Static typing and dynamic typing are typically pitched against each other as two opposites. You have to choose one or the other. However, I'm convinced that there are parts of static typing which are harmful and parts that are useful. Let's figure this out.

The bad parts

Let me try to explain the problem with static typing with a Java example. Suppose we create a function which calculates the distance between two points:

java

class Point {
	double x = 0;
	double y = 0;

	public Point (double x, double y) {
		this.x = x;
		this.y = y;
	}
}

static double distance (Point a, Point b) {
	double dx = b.x - a.x;
	double dy = b.y - a.y;
	return Math.sqrt(dx * dx + dy * dy);
}

We can use it like:

java

// OK
Point a = new Point(0, 0);
Point b = new Point(3, 4);
double d = distance(a, b);

Instead of a class Point we could better define it as an interface, that makes the distance function more flexible as we can create multiple classes implementing the same interface. In Java you can't define properties on an interface though, only methods, so we have to define getters getX() and getY() on the interface and the implementing classes:

java

interface Point {
  double getX()
  double getY()
}

class Point2D extends Point {
	double x = 0;
	double y = 0;

	public Point (double x, double y) {
		this.x = x;
		this.y = y;
	}

	double getX () {
	    return this.x;
	}

	double getY() {
	    return this.y;
	}
}

// OK
Point2D a = new Point2D(0, 0);
Point2D b = new Point2D(3, 4);
double d = distance(a, b);

However, suppose we are using an external library which deals with points too. It will probably have it's own classes and/or interfaces for points, like:

java

class ColoredPoint {
	double x = 0;
	double y = 0;
	String color = "black";

	ColoredPoint (double x, double y, String color) {
		this.x = x;
		this.y = y;
		this.color = color;
	}

	double getX () {
	    return this.x;
	}

	double getY() {
	    return this.y;
	}

	string getColor() {
	    return this.color;
	}
}

// FAIL
ColoredPoint a = new ColoredPoint(0, 0, "red");
ColoredPoint b = new ColoredPoint(3, 4, "blue");
double d = distance(a, b);

We know this class is compatible with our Point class. But the compiler doesn't agree.
We will have to write code to convert a ColoredPoint into Point, else the distance function doesn't accept it:

java

Point2D convertToPoint2D (ColoredPoint point) {
    return new Point2D(point.getX(), point.getY())
}

// OKISH
ColoredPoint a = new ColoredPoint(0, 0, "red");
ColoredPoint b = new ColoredPoint(3, 4, "blue");
double d = distance(convertToPoint2D(a), convertToPoint2D(b));

Another common case is that we're just dealing with JSON data fetched from a server or database:

java

// FAIL
JSONArray points = new JSONArray("[{\"x\":0,\"y\":0},{\"x\":3,\"y\":4}]");
JSONObject a = points.getJSONObject(0);
JSONObject b = points.getJSONObject(1);
double d = distance(a, b);

From a logical point of view you could argue that something like this could work. But from a Java point of view you of course know that it won't. Java strongly discourages working with "unstructured" data like JSON objects or arbitrary maps. The normal way of doing things is to parse JSON into Java classes like our Point2D.

Wrapping up, you will probably say that I just shouldn't do such things in Java. You are right, Java isn't made for that. What I observe here though is:

Java cannot deal with data in a flexible way, it can only handle Java classes with methods (having data encapsulated). In practice, the most straightforward solutions are often to operate on data directly in a functional way, without class ceremony. Java is no good fit there.
Java has serious cross-cutting concerns, which strongly hinders its scalability. If you have two different libraries each having their own classes and interfaces dealing with similar entities (like Point), they can't work together. You will have to do quite some plumbing to convert from one type to the other everywhere the libraries have to work together.

If you look at the original goal: a function which calculates the distance between two points {x,y}, you see that we have to jump through a lot of hoops and have to deal with serious limitations. This demonstrates the bad parts of static typing: it harms interoperability and scalability.

The good parts

The good part of static typing in my opinion is that data types describe the intent of functions. They describe the input types that a function expect, and the output type that a function will return. This is very helpful and convenient, especially when working on a large application with a large team. For me, the most value is in communicating interfaces of functions and components between developers, not in statically analyzing correctness of an application or detecting type related bugs.

Let's see how the same code example we did for Java works out in TypeScript:

interface Point {
  x: number
  y: number
}

function distance(a: Point, b: Point) {
  const dx = a.x - b.x
  const dy = a.y - b.y
  return Math.sqrt(dx * dx + dy * dy)
}

We don't need getters and setters in JavaScript, and can keep the interface a simple, straightforward data structure instead. Of course we can use getters and setters and other methods when needed, but we're not restricted here. You will see the impact of of using a simple data structure later on.

Next, we can create a class that implements the Point interface:

class Point2D implements Point {
  x: number
  y: number

  constructor (x: number, y: number) {
    this.x = x
    this.y = x
  }
}

// OK
const a = new Point2D(0, 0)
const b = new Point2D(3, 4)
const d = distance(a, b)

But we can also use a class that implicitly implements the Point interface, without even knowing it. There are no interoperability issues, the following class ColoredPoint is compatible with the interface Point:

// Note that the class ColoredPoint is totally unaware of `Point` and `distance`
class ColoredPoint {
  x: number
  y: number
  color: string

  constructor (x: number, y: number, color: string) {
    this.x = x
    this.y = x
    this.color = color
  }
}

// OK
const a = new ColoredPoint(0, 0, 'red')
const b = new ColoredPoint(3, 4, 'blue')
const d = distance(a, b)

We can also use plain data:

// OK
const a = {x: 0, y: 0}
const b = {x: 0, y: 0}
const d = distance(a, b)

The reason that our function distance works with both Point2D, ColoredPoint, as well as with plain objects is because TypeScript uses a structural type system:

Type compatibility in TypeScript is based on structural subtyping. Structural typing is a way of relating types based solely on their members. This is in contrast with nominal typing.

Not only TypeScript uses structural typing, similar initiatives like Flow, rtype (and JSDoc for that matter) do the same. This is awesome! This is the best of both worlds: we can communicate interfaces with our fellow developers, get warnings when trying to call functions in incompatible ways, without sacrificing flexibility, interoperability, or scalability.

You might wonder why languages like Java don't use structural typing since nominal typing causes such interoperability issues. Well, the answer is simple: it's way more expensive to deep compare classes property by property than it is to compare a single memory pointer. Since Java has runtime type checking, the performance penalty is just too big. Languages like TypeScript only do type checking at compile time, so it's a non-issue for them.

Conclusion

The experience I have with statically typed languages such as Java and C++ is that it gets in your way. It restricts you, forces you to write lots of boilerplate for simple classes and conversions between them. The reason is that these languages use a nominal type system, which compares classes by name. TypeScript on the other hand uses a structural type system, which compares types by property. This does not cause interoperability issues between similar data types, and provides what we need most: a way to communicate the interface of our functions.

Of course there are more reasons to prefer a statically typed language. One is that "it saves you from a lot of bugs". Well, I don't recognize that advantage: as soon as you unit test your code, static type checking becomes redundant. More on this in a article published yesterday by Eric Elliot: The Shocking Secret About Static Types.

I'm no longer in doubt whether I can happily use a static type checker like TypeScript without being dragged back into the restrictive, limiting environment that I know all too well from statically typed languages like Java. TypeScript and it's companions offer the best of both worlds.

1. Project Overview & Quickstart (josdejong/mathjs) Copy Chapter

File: README.md Copy Chapter

Features Copy Chapter

Usage Copy Chapter

Browser support Copy Chapter

Documentation Copy Chapter

Build Copy Chapter

Develop Copy Chapter

Code Copy Section

Architecture Copy Section

Implementing a new function Copy Section

Build scripts Copy Section

Test Copy Chapter

Continuous integration testing Copy Section

License Copy Chapter

File: docs/index.md Copy Chapter

Documentation

Getting Started Copy Chapter

Documentation Copy Chapter

2. Official Technical Reference & Guides (josdejong/josdejong.github.io) Copy Chapter

File: README.md Copy Chapter

blog

Start server locally Copy Chapter

How to publish Copy Chapter

File: content/posts/2013-07-19-welcome.md Copy Chapter

categories: [] comments: true date: "2013-07-19T00:00:00Z" title: Welcome! url: "blog/2013/07/19/welcome/" Copy Chapter

File: content/posts/2013-07-27-details-matter.md Copy Chapter

End user experience Copy Chapter

Developer experience Copy Chapter

File: content/posts/2013-09-18-multithreading-is-todays-goto.md Copy Chapter

What is wrong with multithreading Copy Chapter

The case for singlethreaded applications Copy Chapter

Yeah very funny, but I just need multithreading Copy Chapter

How to utilize multi-core processors then? Copy Chapter

An analogy with the beloved GOTO statement Copy Chapter

File: content/posts/2014-01-13-shaped-by-the-language-you-use.md Copy Chapter

The right tool for the right job Copy Chapter

Consequences of choosing a language Copy Chapter

Shaped by your language Copy Chapter

Conclusion Copy Chapter

File: content/posts/2014-10-18-the-art-of-creating-simple-but-flexible-apis.md Copy Chapter

Flexibility Copy Chapter

Simple != limited Copy Chapter

Practical tips Copy Chapter

Conclusion Copy Chapter

File: content/posts/2015-01-06-code-reuse.md Copy Chapter

Generic solutions Copy Chapter

Reusable software Copy Chapter

Pitfalls Copy Chapter

Abstraction layers everywhere Copy Section

Excessive use of inheritance Copy Section

Conclusions Copy Chapter

File: content/posts/2015-03-28-a-broader-view-on-isomorphic-javascript.md Copy Chapter

Types of isomorphic code Copy Chapter

Shimmable functionality Copy Chapter

How to distribute isomorphic libraries? Copy Chapter

Tooling and conventions Copy Chapter

Environment detection Copy Section

Bundling for the browser Copy Section

Conclusion Copy Chapter

File: content/posts/2015-08-08-architectural-changes-in-mathjs-v2.md Copy Chapter

Growing pains Copy Chapter

Modularization Copy Chapter

Dynamic type checking logic Copy Chapter

Lazy loading Copy Chapter

Conclusion Copy Chapter

File: content/posts/2016-06-05-static-typing-the-good-parts.md Copy Chapter

An uneasy feeling Copy Chapter

The bad parts Copy Chapter

The good parts Copy Chapter

Conclusion Copy Chapter

Explore Related Technologies & Topics

1. Project Overview & Quickstart (josdejong/mathjs)

File: README.md

Features

Usage

Browser support

Documentation

Build

Develop

Code

Architecture

Implementing a new function

Build scripts

Test

Continuous integration testing

License

File: docs/index.md

Getting Started

Documentation

2. Official Technical Reference & Guides (josdejong/josdejong.github.io)

File: README.md

Start server locally

How to publish

File: content/posts/2013-07-19-welcome.md

categories: [] comments: true date: "2013-07-19T00:00:00Z" title: Welcome! url: "blog/2013/07/19/welcome/"

File: content/posts/2013-07-27-details-matter.md

End user experience

Developer experience

File: content/posts/2013-09-18-multithreading-is-todays-goto.md

What is wrong with multithreading

The case for singlethreaded applications

Yeah very funny, but I just need multithreading

How to utilize multi-core processors then?

An analogy with the beloved GOTO statement

File: content/posts/2014-01-13-shaped-by-the-language-you-use.md

The right tool for the right job

Consequences of choosing a language

Shaped by your language

Conclusion

File: content/posts/2014-10-18-the-art-of-creating-simple-but-flexible-apis.md

Flexibility

Simple != limited

Practical tips

Conclusion

File: content/posts/2015-01-06-code-reuse.md

Generic solutions

Reusable software

Pitfalls

Abstraction layers everywhere

Excessive use of inheritance

Conclusions

File: content/posts/2015-03-28-a-broader-view-on-isomorphic-javascript.md

Types of isomorphic code

Shimmable functionality

How to distribute isomorphic libraries?

Tooling and conventions

Environment detection

Bundling for the browser

Conclusion

File: content/posts/2015-08-08-architectural-changes-in-mathjs-v2.md

Growing pains

Modularization

Dynamic type checking logic

Lazy loading

Conclusion

File: content/posts/2016-06-05-static-typing-the-good-parts.md

An uneasy feeling

The bad parts

The good parts

Conclusion