First steps with nut.js - from installation to automating mouse movement
nut.js comes with a pre-built version of OpenCV for your respective target platform.
In order to use these pre-compiled bindings, certain runtime conditions have to be met.
In case you're running Windows 10 N and want to use ImageFinder plugins, please make sure to have the Media Feature Pack installed.
On macOS, Xcode command line tools are required.
You can install them by running:
1xcode-select --install
Attention:
In case you're experiencing problems like your mouse not moving or your keyboard not typing,
please make sure to give the process you're executing your tests with accessibility permissions.
nut.js will give you a subtle hint in case permissions are lacking:
##### WARNING! The application running this script is not a trusted process! Please visit https://github.com/nut-tree/nut.js#macos #####
When an application wants to use accessibility features, a permission pop-up should be shown.
If not, you could try to manually add the application you're running the script from.
Settings -> Security & Privacy -> Privacy tab -> Accessibility -> Add...
For example, if you want to execute your node script in e.g. iTerm2
, you'd have to add iTerm.app
to the list.
When running your script from a built-in terminal in e.g. VSCode
or IntelliJ
, you'd have to add the respective IDE.
Figure: accessibility permissions screen
Depending on your distribution, Linux setups may differ.
In general, nut.js requires
Installation on *buntu
distributions:
1sudo apt-get install build-essential libxtst-dev
Setups on other distributions might differ.
nut.js is built and tested against various versions of node.
However, for best compatibility, it's recommended to run the latest available LTS version of node (lts/gallium
at the time of writing) and using a version manager like nvm.
With our prerequisites met, let’s continue installing.
The following steps are meant to be carried out in a dedicated directory of your choice.
We will simply refer to this directory as working directory
.
Let’s first initialize a new npm project in our working directory
by executing:
npm init
Feel free to fill out the interactive dialogue, but it’s not a hard requirement to continue the initial setup.
(You could even accept all defaults by running npm init -y
)
Running
1npm i @nut-tree/nut-js
or
1yarn add @nut-tree/nut-js
in our newly created npm project will install nut.js and its required dependencies.
nut.js also provides snapshot releases which allow testing of upcoming features.
Running
1npm i @nut-tree/nut-js@next
or
1yarn add @nut-tree/nut-js@next
will install the most recent development release of nut.js.
Attention: While snapshot releases are great to work with upcoming features before a new stable release, it is still a snapshot release.
Please bear in mind that things might change and / or break on snapshot releases, so it is not recommended using them in production.
Now that we're all set up it's time to get things moving!
In our working directory
, let's create a new file, index.js
.
Open it in your favourite editor and add the following lines to get started:
1const { mouse } = require("@nut-tree/nut-js"); 2 3(async () => { 4 5})();
mouse
gives you control over your, well, mouse, so let's play around with it a bit!
Attention: nut.js is fully async, so in most examples you will see something like the above snippet, which is an async IIFE used as a workaround to use async
/ await
at the top-level.
1const { mouse, left, right, up, down } = require("@nut-tree/nut-js"); 2 3(async () => { 4 await mouse.move(left(500)); 5 await mouse.move(up(500)); 6 await mouse.move(right(500)); 7 await mouse.move(down(500)) 8})();
nut.js provides a declarative API, so instead of explicitly stating where we want our cursor to be, we can use the MovementApi functions to move our cursor relative to our current position.
When executed via node index.js
you'll see that your cursor moves along a square and ends up at its initial position.
Moving our cursor in up, down, left or right direction is a good start, but we're not sitting in front of an Etch A Sketch.
Let's see how we can target specific points on our screen.
1const { mouse, straightTo, Point } = require("@nut-tree/nut-js"); 2 3(async () => { 4 const target = new Point(500, 350); 5 6 await mouse.move(straightTo(target)); 7})();
straightTo
is another MovementApi
function which takes a target Point and computes a straight line towards it, starting at our current cursor position.
In case you want to configure mouse movement speed to go faster / slower, every instance exposed by nut.js provides a config
object.
The mouse config object allows you to configure movement speed measured in pixels per second.
1const { mouse, straightTo, Point } = require("@nut-tree/nut-js"); 2 3(async () => { 4 mouse.config.mouseSpeed = 2000; 5 const fast = new Point(500, 350); 6 await mouse.move(straightTo(fast)); 7 mouse.config.mouseSpeed = 100; 8 const slow = new Point(100, 150); 9 await mouse.move(straightTo(slow)); 10})();
Sometimes we don't want to move along a path to reach a certain point.
In such cases, we can rely on setPosition to immediately change our cursor position to the provided Point.
1const { mouse, Point } = require("@nut-tree/nut-js"); 2 3(async () => { 4 const target = new Point(500, 350); 5 await mouse.setPosition(target); 6})();
mouse
instance to control your cursor.config
object.© 2023