Math, Machine Learning, and JavaScript

Survey

Overview

Big Picture
Why JavaScript?
State of JavaScript
State of Ecosystem
A Standard Library
The Future

Going to be a roller coaster of sorts. An emotional journey with highs and lows, some more lows, and even lower lows, but a journey which ends on a positive note and leaves you feeling good about the future! So bear with me. :)

First, I will talk about the big picture and why this point in time is special.

Next, I am going to motivate why you should care about numeric computing in JavaScript. :)

Then, I'll talk about the State of Math in JavaScript. :(

Next, I'll discuss the JavaScript numeric computing ecosystem. :|

I'll follow by talking about a project in which I am involved which seeks to drive the conversation forward. :)

And I'll end by telling you how you can get involved in the JavaScript computing revolution. :)

Big Picture

1. Edge Computing

2. Browser Virtual Machine

3. Modularity

JavaScript's evolution is guided by a set of constraints which are unique among the thousands of programming languages which exist today. While each constraint considered in isolation may find parallels in any number of languages, it is their amalgamation which sets JavaScript apart.

One of those constraints is need for radical modularity: the ability to send exactly what is needed when it is needed. We see this in the past with UMD/AMD/CommonJS module loading specifications and module loaders (like RequireJS) and, now, in the modern day, in the form of bundlers (like browserify, Webpack, and rollup) and the ongoing standards work around JavaScript imports.

All of this history and continued work is centered around how we can efficiently organize, transform, and shape our code at the speed of thought. We are forced to embrace patterns and code principles which may be possible, but are not existential in other languages.

In short, in their current form, other programming languages are ill-suited for the web, as a platform.

To take an example from numeric computing, even if, say, Julia (and I use Julia because I have seen chatter in their message boards) could "compile" to run in a web browser, they would need to provide a means for decoupling much of their codebase, as well as provide reference implementations for all of BLAS, LAPACK, and other vendor dependencies which are hardware optimized. While not impossible, such an effort would entail considerable work with dubious upside, especially for a group of developers where the web as a platform is outside of their domain of expertise.

4. Artificial Intelligence

The deluge of data, the ascent of data science as an overarching discipline, and the rise of artificial intelligence as a pervasive technology are coalescing to drive technological advances and innovation. Couple these advances with new web APIs and capabilities and you have a fertile playground for creating new types of applications and experiences. Whether that is combining the browser camera API with computer vision to generate quotes for movers or leveraging WebGL to create CAD-like applications or using local storage to create a virtual filesystem for data analysis and offline access.

The principle driver behind the rise of artificial intelligence is automation. And the driver for automation is scale. Developers, and the companies they work for, will continue to seek increases in operational efficiency and volume. As part of their efforts, the more performant their applications, the more tailored the consumer experience, and the more intelligent their technology, the more value they will capture and the faster they will grow.

5. Community

Always Bet on JavaScript

JavaScript

Language of the Web

JavaScript is THE language of the web, whether you like it or not.

JavaScript has pros and cons, just as every language does. But while much maligned, JavaScript is a survivor. Time and time again pundits have prognosticated the death of JavaScript (Java, CoffeeScript, Dart, LiveScript, TypeScript, and now WebAssembly), and, yet, JavaScript continues to live. Given its burgeoning community, importance on the web, and its steady path to global domination, JavaScript seems to get stronger with time. Each time JavaScript detractors have sounded the death knell, they have been wrong.

Yet, despite JavaScript's upward trajectory, the language has yet to make inroads in numeric computing, machine learning, and more generally data science.

MATLAB

R

Python

Mathematica

Julia

Why?

Dynamically compiled
U/Int64
BigInt/BigNumber
Single-threaded
C/C++/Fortran bindings
Performance

Dynamically compiled: so are R, Python, Julia, etc.
Int64: but neither does R (https://www.r-bloggers.com/r-in-a-64-bit-world/; requires 3rd party package); in Python v2, had a "long" type (arbitrary-precision integer, which depending on how implemented can be a bit slower than dedicated machine types), which is now true for all integers in Python v3
No native BigInt/BigNumber support: neither does R (need to use a package), MATLAB requires a toolbox, Python uses arbitrary-precision integers
Single-threaded: so is R
C/C++/Fortran bindings: Node.js native add-ons and WebAssembly
Perf: R is notoriously "slow" (slow being a relative term); JavaScript is faster (cue a couple benchmarks)

	C	R	Python	JavaScript
sin	35,054,483	281,001	5,838,843	9,598,538
expm1	40,191,109	274,317	6,161,389	9,139,959
erf	77,101,176	170,354	4,824,532	10,463,968

Just to put performance in context, here are cross-language benchmarks...

Here are benchmarks for three functions. The first two are standard math library functions across four languages: C, R, Python, and JavaScript.

The C benchmarks are the baseline. In general, C is roughly 5-10x faster than JavaScript. JavaScript is between 2-5x faster than Python. And JavaScript is between 10-100x faster than R.

Note that the last function, the error function, is implemented in pure vanilla JavaScript. Meaning, we are not calling down to a system library. And even in JavaScript, this function is 2x faster than Python.

Now, I cannot confirm that this is generally true, but...it is illustrative that the notion that JavaScript is "slow" is nonsense. You can write fast JavaScript so long as you know what you are doing. :)

Community.

Libraries.

Libraries: lack of comparable libraries (e.g., NumPy for JavaScript does not exist). This is a fair criticism.

Lack of libraries may be a reason to not use JavaScript for a particular application requiring such libraries here and now; however, this does not preclude us from creating those libraries so that future applications are not limited by a lack of available libraries in the first place.

But...what about first mover advantage? Those libs took years to develop and optimize? True. However, people working on this now have "last" mover advantage. The R&D has been done. Bringing to JavaScript is a matter of putting in the time and effort to read, understand, and implement comparable implementations. Not a piece of cake, but also not a herculean challenge.

Why?

Web APIs
Rendering
Visualization/Computation
Ubiquity
Distribution
Package Management

Applications

Edge Computing
Cross-platform
Compute Intensive
Interactive Data Analysis
Integrated ML
AI Powered

Mad science.

State of Math in JavaScript

acos
asin
atan
atan2
cos
sin
tan
acosh
asinh
atanh
cosh
sinh
tanh

abs
exp
log (ln)
pow
sqrt
sign
cbrt
expm1
log10
log1p
log2

ceil
floor
round
fround
trunc

max
min
random
hypot
clz32
imul

In ES5, the standard math library consisted of 18 functions and 8 constants. These functions can be categorized as trigonometric, other special functions, including rounding, and miscellaneous, including a PRNG.

ES2015/ES6 added 17 more functions to the standard math library. Again, these functions are broken down into trigonometric, other special functions, including rounding, and miscellaneous.

The addition of these 17 functions means the JavaScript standard math library has 35 functions.

In short, the JavaScript standard math library is quite small, especially when compared to other languages and dedicated numeric computing environments.

In comparison, the Golang standard math library has 54 functions (JS has 35), 8+ PRNGs, as well as support for 64-bit and 128-bit complex numbers. Notably, Golang has equivalents for most C/C++ standard math library functions.

Besides being small, the JavaScript standard math library has other problems.

No standard algorithms.
No minimum precision.
Portability.
No common codebase.
Slow pace of innovation.
Bugs.

ECMA-262 does not specify particular math algorithms (only recommends); hence implementors are free to choose algos according to speed/precision trade-offs.
ECMA-262 does not require a minimum precision; hence, no guarantee that a given JavaScript environment has a precise implementation.
Portability is non-existent for numeric computing applications due to potential cross-browser variability and potential variability from one browser version to the next.
No common codebase, so resolving issues takes considerable time and effort, especially when different vendors use varying implementations in varying languages with varying organization and structure.
Specification-level changes require cross-browser vendor consensus, causing the pace of innovation at the standards level to be slow (e.g., U/Int64 support has been on the table, off the table for more than a decade, and is most recently on the table as part of a BigInt proposal).
Bugs are common (cue bug or two; e.g., `pow`).

Math.pow


var x = Math.pow( 10, 308 );
// returns:  1.0000000000000006e+308
// expected: 1.0e+308

Math.exp


var y = Math.exp( 100.0 );
// returns:  2.6881171418161485e+43
// expected: 2.6881171418161356e+43

What's being done?

Not much.

The fragmentation problem.
The portability problem.
The versioning problem.
The flexibility problem.
The backward compatibility problem.
The globals problem.
The testing problem.
The trust problem.

Updating the specification does not solve the "fragmentation" problem. Notably, even if you agree on a particular algorithm for computing e^x, asking 5 or more different vendors to implement said spec, each in their own way in accordance with their own code base and conventions increases the likelihood that at least one of them will make a mistake, thus leading to the "portability" problem.
The "portability" problem is exacerbated by the lack of implementation versioning. Meaning, I cannot independently specify what version of a particular vendor implementation I want in a given runtime, which may be important if I want to guarantee exact reproducibility of my results. In an "evergreen" world, each browser restart means I might have a different underlying implementation.
On the flip side, mandating a particular algorithm at the specification level coupled with the "versioning" problem would lead to the "flexibility" problem and a corollary "backward compatibility" problem. Currently, because vendors are not required to either use a particular algorithm or meet precision requirements, they have been free to choose algorithms which, for example, favor speed over accuracy, and because no guarantees have been made in the specification, users cannot expect or rely on consistent implementations, thus allowing vendors to avoid backward compatibility requirements when changing underlying implementations. In short, "we can change our algo and even intentionally sacrifice precision for speed because the spec allows us to." There are reasonable arguments in favor of flexibility, but the domains to which they apply are NOT numeric computing!
Other problems include the "globals" problem (i.e., you cannot trust that the host environment has not been hijacked by some other library), the "testing" problem (i.e., browser vendors have historically failed to rigorously test their implementations), and the "trust" problem, which underscores all the other problems.

State of the Ecosystem

False assumptions.
Low hanging fruit.
Poor implementations.
Insufficient scope.
Lack of ambition.

First, most libraries assume a good standard math library. As we already discussed, this is false. Any library which makes this assumption is at an immediate inherent disadvantage.
Next, much of what you see in JavaScript and on npm are the same types of libraries all addressing the same low hanging fruit (e.g., calculating the mean, variance, and standard deviation of an array), with minor differences and stylistic variations. And while the idea is fine, most people writing these libraries are either not numerically trained or simply have not done their homework as to how to write robust algorithms, which means the implementations of even the most basic functions are poor.
One of my basic litmus tests for any math library is how a library computes the variance. The classic definition involving the sum of squared differences is ill-suited for numeric computation due to the risk of overflow and catastrophic cancellation. Unfortunately, a majority of libraries fail this test.
Next, a few libraries have pushed the boundaries of what is possible. For instance, Andrew Karpathy's ConvNet.js, which does "deep learning" in a browser, or Weblas, which tries to implement BLAS algorithms via WebGL to run on graphic cards. However, these libraries are too often more proof-of-concepts than general frameworks which provide a solid foundation on top of which we can build more advanced numeric computing libraries.
Next, there is the work of Mikola Lysenko, who showed how the same multidimensional data structures used in NumPy, Julia, R, and MATLAB, and required for interfacing with BLAS libraries can be implemented in JavaScript (`ndarray`). However, despite this trailblazing work, the community has yet to effectively harness the concept of an ndarray within a general purpose computing framework.

A lot of noise.

1500+ Packages

A standard library.

So what's needed?

Your help.

Thank you!

https://github.com/stdlib-js/stdlib
https://www.patreon.com/athan

Math, Machine Learning, and JavaScript

Survey

Overview

Big Picture

1. Edge Computing

2. Browser Virtual Machine

3. Modularity

4. Artificial Intelligence

5. Community

Always Bet on JavaScript

JavaScript

Language of the Web

Why?

Community.

Libraries.

Why?

Applications

Mad science.

State of Math in JavaScript

Math.pow

Math.exp

What's being done?

Not much.

State of the Ecosystem

A lot of noise.

1500+ Packages

A standard library.

So what's needed?

Your help.

Thank you!

Appendix

The End