Image Matching

nl-matcher

Next-level image matching for nut.js with multi-scale search, alpha masking, and match validation.

Overview

The @nut-tree/nl-matcher plugin provides a next-level image matching module for nut.js. Compared to the template-matcher, it offers Apple Silicon support, multiple instance detection via screen.findAll(), alpha-channel masking, and improved performance (~1.7x faster).

Multi-Scale Search

Search at multiple scales to handle resolution differences

searchMultipleScales: true

Alpha Masking

Use the alpha channel to mask irrelevant image regions

applyAlphaMask: true

Match Validation

Verify matches to reduce false positives

validateMatches: true

Installation

typescript
npm i @nut-tree/nl-matcher

Subscription Required

This package is included in nl-matcher, Solo, and Team subscription plans.

Quick Reference

useNlMatcher

useNlMatcher()
void

Activate the nl-matcher image matching provider

Usage

Activate nl-matcher by calling useNlMatcher() before performing any image searches:

typescript
import { screen, imageResource } from "@nut-tree/nut-js";
import { useNlMatcher } from "@nut-tree/nl-matcher";

useNlMatcher();

const img = await screen.findAll(imageResource("..."));

Configuration

nl-matcher accepts a NlMatcherProviderData object via the providerData option to control matching behavior:

searchMultipleScales

Default: true. When enabled, nl-matcher searches for images at multiple scales. Disabling this searches only at the original scale, which improves speed when you know the scale is constant:

typescript
const img = await screen.findAll(imageResource("..."), {
    providerData: {
        searchMultipleScales: false,
    },
});

scaleSteps

Default: [0.9, 0.8, 0.7, 0.6, 0.5]. Defines the scale factors used when searchMultipleScales is enabled. The matcher always searches at scale 1.0 (original size) plus these additional factors:

typescript
const img = await screen.find(imageResource("..."), {
    providerData: {
        scaleSteps: [0.9, 0.8, 0.7],
    },
});

useGrayScale

Default: false. Converts images to grayscale before matching. This can improve search speed at the cost of potentially reduced accuracy:

typescript
const img = await screen.find(imageResource("..."), {
    providerData: {
        useGrayScale: true,
    },
});

applyAlphaMask

Default: false. Uses the alpha channel of the reference image to mask out irrelevant regions during matching. Useful for icons or elements on varying backgrounds:

typescript
const img = await screen.find(imageResource("icon-with-transparency.png"), {
    providerData: {
        applyAlphaMask: true,
    },
});

Performance Note

Alpha masking may slow searches when used together with multi-scale search, as the mask must be applied at each scale step.

validateMatches

Default: false. Performs an additional verification step on each match to reduce false positives. This may slightly increase processing time and can affect confidence values:

typescript
const img = await screen.find(imageResource("submit-button.png"), {
    providerData: {
        validateMatches: true,
    },
});

Combining Options

All provider options can be combined in a single search:

typescript
const img = await screen.find(imageResource("button.png"), {
    confidence: 0.9,
    providerData: {
        searchMultipleScales: true,
        scaleSteps: [0.9, 0.8],
        useGrayScale: false,
        applyAlphaMask: true,
        validateMatches: true,
    },
});

Options Reference

searchMultipleScales

searchMultipleScales?: boolean
default: true

Search at multiple scales. Disabling searches only at original scale.

scaleSteps

scaleSteps?: number[]
default: [0.9, 0.8, 0.7, 0.6, 0.5]

Scale factors for multi-scale search. The original scale (1.0) is always included.

useGrayScale

useGrayScale?: boolean
default: false

Convert images to grayscale before matching. Faster but potentially less accurate.

applyAlphaMask

applyAlphaMask?: boolean
default: false

Use the alpha channel to mask irrelevant image regions during matching.

validateMatches

validateMatches?: boolean
default: false

Perform additional verification to reduce false positives. May slightly affect confidence values.

Best Practices

Performance

  • Disable searchMultipleScales when the target scale is known
  • Reduce scaleSteps to only the sizes you expect
  • Use searchRegion to narrow the search area
  • Enable useGrayScale when color is not important

Accuracy

  • Enable validateMatches for critical interactions
  • Use applyAlphaMask for icons on varying backgrounds
  • Capture reference images at the same resolution as the target screen

Image Search

Learn about all image search capabilities in nut.js

All Plugins

Browse the full list of nut.js plugins

Was this page helpful?