Felix Rieseberg

Using Native Windows Features from Electron

Felix Rieseberg -

Electron combines the convenience of coding in JavaScript with the unlimited power of native development, allowing users to mix and match between simple JavaScript, C, C++, Objective C, and even Rust.

Most developers are intimately familiar with macOS - many of us use Macs as our daily drivers. That said, there is a sizable number of users who might still need a USB port and are likely to run Windows.

This post is a primer on how your Electron app can interact with Windows to deliver a user experience that feels truly native and integrates with the operating system.

Windows in 2017

When we talk about modern Windows development, we usually talk about Windows 10, which just received its first major update (called the Anniversary Update). If you were to build a typical Electron app as a native Windows application, you would most likely build a Universal Windows App. Most applications in the Windows Store are universal Windows apps, communicating with Windows through a set of APIs found in the Windows Runtime (usually just called WinRT).

WinRT is the home for anything interesting happening in Windows today. If you'd like to interact with hardware, lock screens, payments, notifications, or the Siri-like Cortana, you're talking to WinRT to make it happen. Logically, this post will now summarize the ways to call WinRT from Electron.

Calling Windows with Electron's APIs

You don't have to get your hands dirty and touch WinRT yourself - Electron already comes with a number of integrated Windows features enabling you to provide a more native Windows user experience.

Consider making full use of the task bar - Electron allows you to set a custom thumbnail region, buttons, and tool tips.

Using Electron's own modules, you can send primitive notifications, add entries to the recent document list, or set a little icon overlay.

That said, beyond those features, Electron does not go any deeper. That's fair, too - it's a cross-platform framework that gives you all the tools to implement the "rest of the way" yourself.

Native Node Addons (without C++)

If you want to go deeper, consider the power of native Node Addons. They are dynamically-linked shared objects, written in C or C++, and can be used just as if they were an ordinary Node.js module. They are used primarily to provide an interface between JavaScript running in Node.js and C/C++ libraries.

Nadav Bar over at Microsoft built a tool that automatically generates native Node addons for each WinRT module, enabling JavaScript developers to simply require() native Windows modules. The end result is an impressive number of modules available of npm.

Let's look at an example and use one of the more powerful Windows APIs - Windows.Media.FaceAnalysis, which contains tools to track and detect faces in images and videos. In particular, we can use the FaceDetector class to scan images for faces. According to the documentation, we can use the method DetectFacesAsync() to scan a SoftwareBitmap for faces, as long as that SoftwareBitmap is in the right color mode. Working backwards, we need to do the following four things:

  • Get a Windows.Storage.Streams.IRandomAccessStream from a file path (like C:\images\myImage.jpg)
  • Turn that file stream into a Windows.Graphics.Imaging.SoftwareBitmap
  • Convert that bitmap into a color mode the computer can check for faces, like gray8
  • Instantiate a FaceDetector and use it to scan for faces

That sounds like quite the heavy lifting, but using NodeRT, it really becomes quite simple. Let's load the file first:

const {FileAccessMode, StorageFile} = require('@nodert-win10/windows.storage')

function getFileStream (file) {
  return new Promise((resolve, reject) => {
    StorageFile.getFileFromPathAsync(file, (err, storageFile) => {
      storageFile.openAsync(FileAccessMode.read, (err, fileStream) => {
        resolve(fileStream)
      })
    })
  })
}

Now that we got a fileStream, let's turn it into a SoftwareBitmap:

const {BitmapDecoder} = require('@nodert-win10/windows.graphics.imaging')

function getSoftwareBitmap (fileStream) {
  return new Promise((resolve, reject) => {
    BitmapDecoder.createAsync(fileStream, (err, decoder) => {
      decoder.getSoftwareBitmapAsync((err, bitmap) => {
        resolve(bitmap)
      })
    })
  })
}

You should see a pattern emerging here: All the native WinRT classes and methods are turned into JavaScript classes and methods, with asynchronous methods using Node.js callbacks. I'd personally prefer Promises, but this way, you're not tied to a later version of Node.js. Let's continue by instantiating our face detector and using it to detect some faces:

const {FaceDetector} = require('@nodert-win10/windows.media.faceanalysis')
const {SoftwareBitmap, BitmapPixelFormat} = require('@nodert-win10/windows.graphics.imaging')

function detectFacesInBitmap (bitmap) {
  return new Promise((resolve, reject) => {
    const inputPixelFormat = BitmapPixelFormat.gray8
    const grayBitmap = SoftwareBitmap.convert(bitmap, inputPixelFormat)

    FaceDetector.createAsync((err, detector) => {
      detector.detectFacesAsync(grayBitmap, (err, result) => {
        resolve(result.length)
      })
    })
  })
}

Given how easy this was, I even went ahead and added some error handling and released a little Node module that you can go ahead and use for yourself - electron-face-detection-windows.

There are already a few modules out there offering native WinRT features as convenient Node modules - notable mentions are electron-windows-notifications, which enables you to send complex Windows Notifications (with buttons, text input, and all kinds of bells and whistles) - or electron-windows-interactive-notifications, which uses straight-up C++ to integrate Electron deeply with Windows.

Native Node Addons (with C++)

If you're ready to jump into the deep end of the pool, you can communicate with WinRT directly from the native C++ code of a native Node Addon. Many APIs are already accessible, but if you're interested in the whole corpus of WinRT, cppwinrt might be able to help you out: The official Microsoft project is a standard C++ language projection for the Windows Runtime implemented solely in header files. It allows you to both author and consume Windows Runtime APIs using any standards-compliant C++ compiler - which includes node-gyp and all its friends.

This allows you to use some of the most modern C++ methods. Let's look at doing some OCR recognition on a bitmap without NodeRT, directly from C++:

  future<hstring> OcrOperation() {
    auto file = co_await StorageFile::GetFileFromPathAsync(L"…");
    auto stream = co_await file.OpenAsync(FileAccessMode::Read);
    auto decoder = co_await BitmapDecoder::CreateAsync(stream);
    auto bitmap = co_await decoder.GetSoftwareBitmapAsync();
    auto engine = OcrEngine::TryCreateFromUserProfileLanguages();
    auto result = co_await engine.RecognizeAsync(bitmap);

    return result.Text();
  }

In summary: Look at all the cool stuff in WinRT - and use it, if it enhances your application!