Skip to main content

3 posts tagged with "building"

View All Tags

· 4 min read

Those of you who are keeping an eye on the nut.js repository might have noticed something — quite a big change is on the horizon!

As I already mentioned in my previous post, I'm trying to reduce the amount of work required to keep nut.js compatible with upcoming node and Electron version. This became even more pressing since the Electron team switched to an eight weeks release cycle, so nut.js would require a new release every 8 weeks as well to keep up. The single reason for this is the opencv4nodejs-prebuilt dependency required to provide all the image matching functionality. Even though I'm making progress on building a replacement that does not suffer this version dependence, it'll take some more time to get it released. So, while working on solving this issue, I also decided to tackle two things mentioned in the nut.js user survey I posted a while ago. Turns out, nut.js users are undecided when it comes to image processing. Half of you are big fans, the others are unhappy about the drawbacks like node version dependence or package size. That, plus the wish for a plug-in system in nut.js, lead to the following strategy towards the next major release:

  1. Establish a simple plug-in system to make underlying provider packages configurable
  2. Pull existing image processing code and dependencies out of nut.js into a dedicated package that utilises the newly created plug-in system, making image matching an explicit opt-in feature
  3. Keep working on a proper replacement package for image matching that will seamlessly replace the existing plug-in package

What does this mean for nut.js users?#

Unfortunately, there will be breaking changes. At least for those of you using image matching functionality. The good thing is, it should be relatively simple to fix these.

Instead of only installing nut.js via e.g. npm i @nut-tree/nut-js, you'll have to install an additional image matching plug-in package.

Let's look at an example:#

  1. Create a new project and install nut.js
npm init -ynpm i @nut-tree/nut-js@next
  1. Create an index.js file with the following content:
const { screen } = require("@nut-tree/nut-js");
(async () => {  try {    await screen.find("img.png");  } catch (e) {    console.error(e);  }})();
  1. Run the example and check its output
node index.js
Searching for img.png failed. Reason: 'Error: No ImageFinder registered'

The error output tells you that nut.js cannot search for your template image since no suitable ImageFinder has been registered.

A similar error appears when trying to save a screenshot to disk:

const { screen } = require("@nut-tree/nut-js");
(async () => {  try {    await screen.capture("foo");  } catch (e) {    console.error(e);  }})();
Error: No ImageWriter registered

This again tells us that a plug-in suitable for writing image files to disk is missing.

How to we solve this?#

As a first step, let's install the newly created plug-in containing the image processing code extracted from nut.js.

npm i @nut-tree/template-matcher

Once installed, all we have to do is import it in our code:

const { screen } = require("@nut-tree/nut-js");require("@nut-tree/template-matcher"); // THIS IS NEW
(async () => {  try {    await screen.find("img.png");  } catch (e) {    console.error(e);  }})();

That's all it takes to put the previous image matching code back in place:

Searching for img.png failed. Reason: 'Error: No match with required confidence 0.99. Best match: 0.9249920099973679 at (384, 26, 409.5, 62)'

I'm not using any image matching functionality, do I have to do anything?#

Short answer: No! If you haven't used any image matching functionality in your code, you won't encounter any problems. Actually, you'll even profit from it!

By extracting the image matching code into a separate plug-in the nut.js base package itself:

  • Lost around 70 MB of required disk space
  • Is no longer dependent on a certain node version, so you're able to use it with e.g. node 17 right away

Awesome, can I start using it?#

Currently, everything I just showed you is available in the latest @next release.

But please be warned! This is still under active development and since it's a snapshot release, things might change/break at any time until it gets released as 2.0.0!

The main reason for this blog post is to inform users early on, so hopefully there won't be too many surprises once it's stable!

With that in mind, feel free to test it and let me know what you think about it!

Best regards,

Simon

· One min read

First steps have been made! My endeavour to get rid of nan-type bindings in nut.js, opencv4nodejs, to be precise, is making progress. Slow, but steady.

So far I redesigned how OpenCV itself is built and distributed for further use, solving one of the issues which are bothering me with the existing setup. This redesign eliminates a whole codebase I had to think through and follows a pretty lean approach for providing precompiled libs.

The setup is already automated for all three platforms and I started a new project which uses these new artefacts. The process required some tweaking but right now everything fits together really smooth.

As I already said, first steps have been made and I'm looking forward to take the next ones!

Best regards

Simon

· 2 min read

Sometimes you've got to ask this simple question. Does something spark joy for you? In my case, something that definitely does not spark joy is updating opencv4nodejs-prebuilt. The reason being is the fact that opencv4nodejs relies on  Native Abstractions for Node.js , which in turn means that on every new node or Electron release, I'd have to provide additional prebuilds. In theory, this sound rather straightforward. Add a new node / Electron version to your CI config and off we go, but reality looks a bit different. Hands down, opencv4nodejs is great. It's configurable, compatible with recent OpenCV versions and async. However, it also has its peculiarities.

It relies on a custom npm package which provides a wrapper around the whole OpenCV CMake build. This way it’s possible to configure and build OpenCV without leaving the well known npm ecosystem. While this approach definitely has its benefits, it’s an additional burden for me. When I started building nut.js I just used opencv4nodejs. But since I envisioned the installation of nut.js as straight forward as possible the whole „we build everything for you on install" just didn’t suite me. That’s how I ultimately ended up adopting two codebases by forking both the npm OpenCV build and opencv4nodejs itself to provide prebuilt binaries.

As time went by, I noticed a few things I definitely wanted to change:

  • node ABI dependence: Every new node release requires new prebuilds which is the sole reason nut.js is running a bit late with support latest node versions
  • node-gyp: I don’t know why this build system is still so popular. I’m not a fan
  • Way too much functionality: opencv4nodejs aims to provide general purpose bindings for OpenCV while I’m only requiring a very small set of features
  • Mental load: I’m not particularly interested in deep diving another two codebases
  • Complexity: Even tough the setup works I’m the only one who knows why

To put it in a nutshell (haha): I’m constantly looking for alternative solutions which solve the above points. I recently started digging into something which might be promising, I’ll let you know once I know!

Best regards

Simon