Custom Mouse Movement Functions

nut.js provides several mouse movement functions our of the box. It supports relative movement using one of the relative movement functions left, right, up or down or directed movment using straightTo.

These are included in the framework, but it's also possible to provide a custom movement function to guide your mouse around. Let's take a look at an example which makes your mouse move towards a target point in a squiggly line!

Attention: The following code sample is using top-level await, so to follow along, please add "type": "module" to your package.json.

A custom movement function

Let's start our little experiment with some basic setup:

import {mouse} from "@nut-tree/nut-js";

async function alongSineWaveTo(endPoint, amplitude = 50, frequency = 4, numPoints = 1000) {
}

Next, we'll determine the current mouse position which will be the starting position for our custom mouse path:

import {mouse} from "@nut-tree/nut-js";

async function alongSineWaveTo(endPoint, amplitude = 50, frequency = 4, numPoints = 1000) {
    const source = await mouse.getPosition();
}

Preparations

With this setup up we can now start to build our movement path. In general, mouse movement in nut.js works by simply following a path of points. To do so, we determine the slope and length of the line connecting our start and end points.

import {mouse} from "@nut-tree/nut-js";

async function alongSineWaveTo(endPoint, amplitude = 50, frequency = 4, numPoints = 1000) {
    const source = await mouse.getPosition();
    const target = await endPoint;
    // Calculate slope and length of the line segment
    const dx = target.x - source.x;
    const dy = target.y - source.y;
    const length = Math.sqrt(dx * dx + dy * dy);
}

Next, we can calculate the normalized normal vector (the vector that is perpendicular to our line and has a length of 1):

import {mouse} from "@nut-tree/nut-js";

async function alongSineWaveTo(endPoint, amplitude = 50, frequency = 4, numPoints = 1000) {
    const source = await mouse.getPosition();
    const target = await endPoint;
    // Calculate slope and length of the line segment
    const dx = target.x - source.x;
    const dy = target.y - source.y;
    const length = Math.sqrt(dx * dx + dy * dy);

    // Normal vector (perpendicular to the line segment)
    const nx = -dy / length;
    const ny = dx / length;
}

Calculating the path

Using these to pieces of information we can loop over the amount of points we want to generate and calculate the x and y coordinates.

async function alongSineWaveTo(endPoint, amplitude = 50, frequency = 4, numPoints = 1000) {
    const source = await mouse.getPosition();
    const target = await endPoint;
    // Calculate slope and length of the line segment
    const dx = target.x - source.x;
    const dy = target.y - source.y;
    const length = Math.sqrt(dx * dx + dy * dy);

    // Normal vector (perpendicular to the line segment)
    const nx = -dy / length;
    const ny = dx / length;

    const sineWavePoints = [];
    for (let idx = 0; idx <= numPoints; idx++) {
        const t = idx / numPoints;
        const waveAmplitude = amplitude * Math.sin(2 * Math.PI * frequency * t);

        // Projecting the sine wave on the line
        const x = source.x + dx * t + nx * waveAmplitude;
        const y = source.y + dy * t + ny * waveAmplitude;

        sineWavePoints.push({x: x, y: y});
    }

    return sineWavePoints;
}

The full sample

And we're done! We can now use our custom movement function as a replacement for e.g. straightTo and our mouse cursor will no longer move in a straight line towards a point, but in a squiggly sine wave.

import {mouse, Point} from "@nut-tree/nut-js";

async function alongSineWaveTo(endPoint, amplitude = 50, frequency = 4, numPoints = 1000) {
    const source = await mouse.getPosition();
    const target = await endPoint;
    // Calculate slope and length of the line segment
    const dx = target.x - source.x;
    const dy = target.y - source.y;
    const length = Math.sqrt(dx * dx + dy * dy);

    // Normal vector (perpendicular to the line segment)
    const nx = -dy / length;
    const ny = dx / length;

    const sineWavePoints = [];
    for (let idx = 0; idx <= numPoints; idx++) {
        const t = idx / numPoints;
        const waveAmplitude = amplitude * Math.sin(2 * Math.PI * frequency * t);

        // Projecting the sine wave on the line
        const x = source.x + dx * t + nx * waveAmplitude;
        const y = source.y + dy * t + ny * waveAmplitude;

        sineWavePoints.push({x: x, y: y});
    }

    return sineWavePoints;
}

await mouse.move(alongSineWaveTo(new Point(10, 10)));

Not only does this work with Point inputs, you can seamlessly use it with centerOf and e.g. screen.find as well:

await mouse.move(alongSineWaveTo(centerOf(screen.find(imageResource("an_image.png")))));

This gives you full power to customize mouse movement to your requirements. Moving to your target over a fixed set of waypoints, moving in circles or something totally different, it's up to you!

All the best

Simon