Mouse

A mouse is still the most prominent pointer device for "conventional" computers (like notebooks or desktop computers).

nut.js resembles this via its mouse object. It gives you the possibility to set your cursor position, move the cursor along paths, click, scroll, drag or do everything in combination with keys to select multiple items.

You can build up your own function to combine multiple steps, like clicking at a certain position:

import {mouse, Point, straightTo} from "@nut-tree/nut-js";

async function clickAtPosition(position: Point): void {
    await mouse.move(straightTo(position));
    await mouse.leftClick();
}

or you could combine it with on-screen image or text search:

import {centerOf, imageResource, mouse, straightTo} from "@nut-tree/nut-js";
import "@nut-tree/nl-matcher";

const sourceLocation = await screen.find(imageResource("source_image.png"));
await mouse.move(straightTo(centerOf(sourceLocation)));
const targetLocation = await screen.find(imageResource("target_image.png"));
await mouse.drag(straightTo(centerOf(targetLocation)));

A combination of mouse and keyboard allows you to e.g. select multiple images:

import {centerOf, imageResource, Key, keyboard, mouse, straightTo} from "@nut-tree/nut-js";
import "@nut-tree/nl-matcher";

// Locate all instances of our image on screen
const allIcons = await screen.findAll(imageResource("icon.png"));
// Press Cmd key to select multiple elements
await keyboard.pressKey(Key.LeftCmd);

// Iterate over all located images and select them via mouse click
for (const icon of allIcons) {
    await mouse.move(straightTo(centerOf(icon)));
    await mouse.click(Button.LEFT);
}
await keyboard.releaseKey(Key.LeftCmd);

// Press delete
await keyboard.pressKey(Key.Delete);
await keyboard.releaseKey(Key.Delete);

// Confirm deletion by pressing Return
await keyboard.pressKey(Key.Return);
await keyboard.releaseKey(Key.Return);

Movement itself is also very flexible. It simple follows a path of points, no matter how it looks.

The two built-in function straightTo and centerOf allow you to easily move towards the center of a region that might've been returned from a call to screen.find.

But maybe you want something totally different? No problem! It all boils down to a series of points to follow, it's up to you to build up your own path!

Take this example. A custom function which let's the cursor spiral towards a certain point.

import {mouse} from "@nut-tree/nut-js";

async function alongSpiralPath(end, circles, steps) {
    const current = await mouse.getPosition();
    const startRadius = Math.sqrt(
        Math.pow(end.x - current.x, 2) + Math.pow(end.y - current.y, 2),
    );
    const path = [];
    const fullCircle = 2 * Math.PI;
    const angleIncrement = (circles * fullCircle) / steps;
    let angle = 0;
    let radiusDecrement = startRadius / steps;

    for (let i = 0; i < steps; i++) {
        // Calculate the radius for this step
        const radius = startRadius - i * radiusDecrement;

        // Convert polar coordinates to Cartesian coordinates
        const x = end.x + radius * Math.cos(angle);
        const y = end.y + radius * Math.sin(angle);

        path.push({x, y});

        // Increment the angle for next point
        angle += angleIncrement;
    }

    return path;
}

mouse.config.mouseSpeed = 500;

const screenWidth = await screen.width();
const screenHeight = await screen.height();

const target = {
    x: screenWidth / 3,
    y: screenHeight / 4,
};

await mouse.move(alongSpiralPath(target, 5, 1000));

It's up to you to decide wich path to take!