Changing the Way We Write with AI-powered Handwriting Recognition

AI continues to drive human-machine interactions to new heights. Discover how touchless technology combined with machine learning makes it possible to write in thin air and convert handwriting into digital text for any application.

For most of us, there’s nothing more personal and natural than handwriting, and it’s no surprise. We’ve been scribbling down our thoughts since 3,500 BCE, and the craft of writing has been linked to improved creativity, focus, memory recall and problem-solving skills compared to typing.

But today we live in a world of keyboards and buttons. What if we could take this uniquely human skill that we’ve finessed over millennia, and incorporate it into our daily digital touchpoints? After all, shouldn’t machines be adapting to us, not the other way around?

This is exactly what’s happening thanks to a new Neonode collaboration with MyScript – the world leader in real-time handwriting-recognition technology. Using deep machine learning, MyScript’s easy-to-install software can identify natural handwriting in 72 languages, including logographic characters, math, and musical notation – and convert it to digital text and symbols. Cool, right? But it gets better.

When MyScript’s technology is integrated with Neonode’s Touch Sensor Modules, it’s possible to write on any surface (digital or physical) with any input (think finger, stylus or even magic wand), or use in-air hand gestures to write on interactive holographic displays and convert words and symbols in real-time. The fusion of these two pioneering technologies opens up a huge array of applications for many industry sectors. But before we take a technical deep-dive, let’s look at three day-to-day examples that have the potential to improve the digital experience for millions of people.

Rewrite the Rules for In-car Interactions

In the automotive world, one word trumps all: safety. Distracted driving is frequently cited as the number one cause of all road accidents – and as our cars get smarter, the number of interaction points inside them increases.

But while safety is always the number one priority, it doesn’t mean that convenience needs to take a backseat. With Neonode technology, it’s already possible to add multi-touch functionalities to in-car displays and controls, adjusting anything from the sound system to the air-conditioning using mid-air hand gestures. Throw handwriting recognition into the mix and you’ve got a different kind of game-changer.

Instead of scrolling through a menu or selecting buttons to issue commands, you can write your input using your finger – either in thin air, on a digital screen, or on any interior surface – all without taking your eyes off the road. For example, you could handwrite a street name on the GPS screen instead of looking for letters on a keyboard or spinning dials. This allows drivers to keep their eyes on the traffic by switching complex interfaces for muscle memory. MyScript software can also be pre-loaded with street names and other data to make smart predictive suggestions.

With an easier input method, vehicles can be both sleeker and safer and car manufacturers gain a competitive edge. With the help of Epicnpoc, the developers of BOWL, a multimodal HMI development platform for automotive manufacturers, creating protypes with in-car HMI script recognition is easy. BOWL combines a set of functional software and hardware including Neonode and MyScript tech that enables OEMs and tier-one suppliers to rapidly create, test, and validate new and innovative in-vehicle applications.

Audi A7 *MMI touch response system powered with MyScript technology.

Make Every Hotel Stay Noteworthy

In a post-pandemic boom, forward-thinking hotels are pushing the boundaries when it comes to reinventing the guest experience and are finding increasingly creative solutions to make their guests feel safe and special.

During any hotel stay, there are multiple touchpoints when guests need to communicate who they are and what they need. Digital handwriting can make these experiences much more personal.

For example, when guests check in and out of a hotel, they can simply write their name on any screen or surface and follow the steps provided. And if a hospitality provider wants to go the extra mile, they can remove physical touch all together. With holographic technology, people can safely pinch, scroll, and rotate any image in mid-air – and write on it too. Imagine writing your room service order and any other requests on a floating 3D display, or even on a wall, and having a system accurately interpret and respond to your handwritten communication. This is entirely possible with Neonode and MyScript. Hotels have a new way to wow their customers and increase both satisfaction and loyalty.

Holographic check-in systems can turn in-air handwriting into digital text.

Enable Swifter Retail Experiences

We’ve all had frustrating moments where we are flicking through different categories at a self-service kiosk, searching for our favorite fast-food snack, or searching for a non-scannable item at the grocery store checkout and it’s no where in sight. By integrating handwriting recognition with Neonode’s touch or touchless solutions, customers can now write exactly what they’re looking for on or near the screen with their finger, saving them time and frustration.

The same goes for holographic point-of-sale systems, advertising and information boards, and in-store wayfinders. Thanks to MyScript’s handwriting-recognition technology, people from all over the world can simply write their order preference in whichever language they prefer. This brings many benefits to the retail and quick-service restaurant sectors, making the ordering process faster, more fun, and in touchless instances, more hygienic.

Using Advanced AI to Make Handwriting Digital

So how is this possible? Handwriting recognition poses significant technical challenges due to the huge variability of handwriting formats and styles. Cursive writing means the system must identify and segment individual characters from a single pen stroke, while for Chinese characters it must distinguish a single character from over 30,000 possible ideograms. The challenge is amplified when introducing bi-directional languages like Arabic or Hebrew, where the system must recognize that the input is coming from right to left – and from left to right when it includes English loan words.

At MyScript, the team has overcome these challenges through their progressive use of neural networks to identify patterns in vast datasets. MyScript’s handwriting research team uses AI and machine-learning techniques to solve problems that can be formulated as sequence-to-sequence conversion problems – in order to accurately convert handwritten text into its constituent characters.

With all the hard work done, OEMs can now easily leverage the full power of accurate handwriting recognition from any surface (or hologram!) via MyScript’s OS-agnostic SDK, which is fully compatible with Neonode Touch Sensor Modules.

Together, Neonode and MyScript make it easy to transform the human-machine experience for the better.

About MyScript

MyScript is the market pioneer and leader in handwriting recognition and digital ink management. The company’s proprietary AI interprets all kinds of handwritten content in over 72 languages, from text and math to diagrams and musical notation It is available via the cloud, across the main operating systems and can be ported to more specific environments. MyScript Interactive Ink (iink) technology puts handwriting at the core of the digital world, allowing users to create and interact with handwritten content without sacrificing the flexibility of digital. The power of the tech can be experienced in MyScript’s consumer apps, MyScript Calculator and Nebo. MyScript serves customers worldwide from its headquarters in Nantes, France, and regional offices located in China, Japan, Korea and the U.S.

For more information, see www.myscript.com