Ideas are elusive, slippery things. Best to keep a pad of paper and a pencil at your bedside, so you can stab them during the night before they get away.

State of the Art Technology

Writing with pen is the first skill you learn in school. It is also one of the most important skills you learn in life, because writing enables you to record your thoughts and send them to another human. Your thought may effect a person that is thousands of kilometres away, and even hundreds years in the future. Recording thoughts is the most powerful technology our civilization ever created. And without writing, well, there wouldn't be any human civilization. Writing is here to stay, so we might look at how we can improve it.

Let's look at one specific mode of writing that people often do. You have a problem that can be expressed as an algebraic expression. For example you are a carpenter and you want to know volume of piece of wood that is 13cm by 13cm by 1m. Or you want to know price of 3kg of bananas if 1kg is for 2.44€. Or you want to know how much money you'll have in 10 years when you start with 1000€ and 3.5% annual growth. This is basic math that you can do with pen and paper without even using a calculator. However, calculator is equally useful tool as pen and paper because you can think faster by automating the boring parts. Okay, so let's look at some concrete example of this workflow:

Writing a math expression with pen on paper

You write down the expression and then you have to type it in the calculator. You'll probably do it by writing down partial results and typing them in the calculator to get the final result. This is a slow process that can be eased by using calculator with parentheses – you retype the expression char-by-char and you have your answer. Nowadays people don't really carry around physical calculators, but we have something more powerful – a computer. Computers have many forms – one can be a handheld calculator with touchscreen, which you could use in tandem with pen and paper to solve our math problem. Or you know, use pen and paper that are computers.

Apple iPad Pro with Apple Pencil – a modern pen and paper (source)

So let's write our problem again. This time with modern versions of our beloved tools.

Writing a math expression with modern pen on modern paper

We have black paper and white ink. Such innovation! But it looks identical as our first try. Even though we use computerized pen and paper, the new medium acts the same way as one that is 5000 years old. Thanks to machine learning, computers are now able to understand our handwriting, so we can enhance our pen and paper to be smart!

Writing a math expression with smart pen and smart paper

As you can see, this isn't great. The transcribed text has errors. We tried to write it in one shot and the handwriting looks ugly, so the recognition result wasn't as good. Maybe if we try to write it by chunks, we might get it right.

Writing a math expression with smart pen and smart paper and dumb caret navigation

Entering text on computer is usually done with caret navigation – a blinking vertical line designates an insertion point – if you press a letter on a keyboard, it will be inserted at that position. With this mode we are mixing in different input system – keyboard – into our pen and paper land, but at least we can write what we wanted... However we lost the original form of our handwritten thought and we are left only with printed characters. We can do a bit better.

Writing a math expression with smart pen and smart paper and smart design

This was much better experience than the previous one. And we have both representations at the same time – original handwritten form, and also derived printed form that was generated after each stroke. Even though there was a lot of computation going on during the scribbling, we still don't know the answer of our math question. So let's try one more time.

Writing a math expression with smart pen and smart paper and smarter design

Finally, this is it — we have everything we need. We have our original scribbles, which get translated into printed letters. On top of that, the computer recognized it is an algebraic formula and highlighted numbers and operators with different colors. Not only printed characters, but also individual strokes. And along the scribbling, we had formula evaluated the whole time. As soon as we finished last stroke on our problem statement, we already had the solution.

Pen, paper and calculator working seamlessly together to ease our thinking.

From Past To The Future