Skip to main content

Screen Control

nut.js allows searching and waiting for images on your screen to either verify certain conditions, or use it for further processing.

Configuration#

The nut.js screen comes with a config object which allows to configure its behaviour.

confidence#

screen.config.confidence specifies the required matching percentage required to mark a possible candidate for a given template image a match.

autoHighlight#

screen.config.autoHighlight is a boolean toggle which enables automated highlighting of image search results. This will highlight the matching Region by showing an opaque window.

highlightDurationMs#

screen.config.highlightDurationMs configures the duration in milliseconds a highlight window is shown.

highlightOpacity#

screen.config.highlightOpacity configures the opacity of highlight windows. Ranges from 0 (fully transparent) to 1 ( fully opaque).

resourceDirectory#

screen.config.resourceDirectory configures the location to load assets from via imageResource. This allows configuring resource locations depending on e.g. the current operating system.

One could provide multiple folders containing platform specific template images and chose the correct resource directory at runtime. Following this scheme loading of platform specific images would be possible without changes to the source.

capture#

capture allows you to capture a screenshot and store it to your filesystem.

captureRegion#

captureRegion allows you to capture a screenshot of a desktop region and store it to your filesystem.

grab#

grab allows you to retrieve an Image containing the current screen content.

grabRegion#

grabRegion allows you to an Image containing the current content of a desktop region.

find#

find takes a template Image and tries to find a match on the main screen. It is possible to override the configured matching confidence and search region providing OptionalSearchParameters. In case of a match, the corresponding Region on screen is returned.

await mouse.move(straightTo(centerOf(screen.find(imageResource("image.png")))));

findAll#

In contrast to find returning only the most probable match, findAll takes a template Image and returns a list of all matched occurrences on the main screen. It is possible to override the configured matching confidence and search region providing OptionalSearchParameters. In case of a match, the corresponding Region on screen is returned.

const matches = await screen.findAll(imageResource("image.png"));
for (const match of matches) {
await mouse.move(straightTo(centerOf(match)));
}

highlight#

When working with template images to e.g. move the mouse to certain positions it can be quite cumbersome to follow along without visual clues.

highlight allows you to display an opaque window overlay which makes it easier to visually follow detection / movement.

await screen.highlight(screen.find("image.png"));

on#

on allows you to register callbacks which will be executed once find returns a match for a given template image.

This way it's possible to repeatedly execute actions whenever a certain image is detected on screen.

waitFor#

Similar to find, waitFor will search for a template image on a systems main screen.

While find will fail immediately if no match is found, waitFor allows to configure a timeout in milliseconds during which the screen will repeatedly be scanned for the template image. The interval in milliseconds, in which the image search is carried out, is configurable as well. Its default value is set to 500ms. Once the configured timeout is reached with no match, waitFor will fail.

await mouse.move(straightTo(centerOf(screen.waitFor(imageResource("image.png"), 3000, 500))));

colorAt#

colorAt will return RGBA color information at a specified pixel location.

For example, a black pixel would be represented as

RGBA { R: 0, G: 0, B: 0, A: 255 }

width#

width returns the main screen's width in pixels.

height#

height returns the main screen's height in pixels.