Searching your screen for images
nut.js allows searching and waiting for images on your screen to either verify certain conditions, or use them for further
processing.
The nut.js screen comes with
a config object which allows configuration of its behaviour.
confidence
screen.config.confidence
specifies the required matching percentage required to mark a possible candidate for a given
template image a match.
autoHighlight
screen.config.autoHighlight
is a boolean toggle which enables automated highlighting of image search results. This
will highlight the matching Region by showing an opaque window.
highlightDurationMs
screen.config.highlightDurationMs
configures the millisecond duration a highlight window is shown.
highlightOpacity
screen.config.highlightOpacity
configures the opacity of highlight windows. Ranges from 0 (fully transparent) to 1 (
fully opaque).
resourceDirectory
screen.config.resourceDirectory
configures the location to load assets
via imageResource
. This allows
configuration of resource locations depending on e.g. the current operating system.
One could provide multiple folders containing platform specific template images and choose the correct resource directory
at runtime. Following this scheme, loading of platform specific images would be possible without changes to the source.
capture
allows you to capture a screenshot and store it to your filesystem.
captureRegion
allows you to capture a screenshot of a desktop region and store it to your filesystem.
grab
allows you to retrieve an Image
containing the current screen content.
grabRegion
allows you to retrieve an Image
containing the current content of a desktop region.
find
takes a template Image
and tries to find a match on the main screen. It is possible to
override the configured matching confidence and search region
providing OptionalSearchParameters. In case of a match,
the corresponding Region on screen is returned.
1await mouse.move(straightTo(centerOf(screen.find(imageResource("image.png")))));
In contrast to find
returning only the most probable match, findAll
takes a
template Image
and returns a list of all matched occurrences on the main screen. It is
possible to override the configured matching confidence and search region
providing OptionalSearchParameters. In case of a match,
the corresponding Regions on screen is returned.
1const matches = await screen.findAll(imageResource("image.png")); 2for (const match of matches) { 3 await mouse.move(straightTo(centerOf(match))); 4}
When working with template images to e.g. move the mouse to certain positions it can be quite cumbersome to follow along
without visual clues.
highlight
allows you to display an opaque window overlay which makes it easier to visually follow detection /
movement.
1await screen.highlight(screen.find(imageResource("image.png")));
on
allows you to register callbacks which will be
executed once find returns a match for a given template image.
This way it's possible to repeatedly execute actions whenever a certain image is detected on screen.
Similar to find, waitFor
will search for a template image on a
system's main screen.
While find will fail immediately if no match is found, waitFor
allows configuration of a timeout in milliseconds during which the screen will repeatedly be scanned for the template image.
The interval in milliseconds, in which the image search is carried out, is configurable as well. Its default value is set to 500ms.
Once the configured timeout is reached with no match, waitFor
will fail.
1await mouse.move(straightTo(centerOf(screen.waitFor(imageResource("image.png"), 3000, 500))));
colorAt
will return RGBA color information at a specified pixel location within the system's main screen.
For example, a black pixel would be represented as:
1RGBA { R: 0, G: 0, B: 0, A: 255 }
width
returns the main screen's width in pixels.
height
returns the main screen's height in pixels.
© 2023