nut.js logonutjs.dev

Tutorials

Reacting on User Input - nut.js Input Monitoring

Unlock the potential of real-time user interaction in your Node.js applications with nut.js! In this guide, we'll delve into how to monitor and react to user input - such as keyboard strokes and mouse movements — using nut.js input monitoring.

Setup

Currently only @nut-tree/bolt implements an input monitor.

So in order to use the input monitor, you need to install @nut-tree/bolt:

npm i @nut-tree/bolt

Once that's done, you can start using the input monitor in your application.

import {useBoltInputMonitor} from '@nut-tree/bolt';

useBoltInputMonitor();

This registers and enables the @nut-tree/bolt InputMonitor implementation to be used by nut.js.

But so far, this doesn't get us anywhere. We need to start monitoring the input events.

import {useBoltInputMonitor} from '@nut-tree/bolt';
import {system} from '@nut-tree/nut-js';

useBoltInputMonitor();

system.startMonitoringInputEvents();

setTimeout(() => {
    system.stopMonitoringInputEvents();
}, 30000);

system.startMonitoringInputEvents(); actually starts the monitoring. If we would call system.stopMonitoringInputEvents(); right after it, monitoring would stop immediately.

In this example, we stop monitoring after 30 seconds.

Input Event Loop

If you're curious, you can try yourself what happens if you do not stop monitoring input events.

Have you tried it? No? Then let me tell you: Your script will never terminate, because the monitoring loop will keep running.

Event monitoring happens in a separate thread which continuously emits events to the main thread. So if you don't stop the monitoring, the main thread will not terminate.

Reacting to User Input

There are two major categories of input events: keyboard and mouse events. Each event comes with a type property as well as a timestamp.

Keyboard Events

export interface KeyDownEvent extends InputEvent {
    type: "keyDown",
    key: Key,
    modifiers: Key[],
    character?: string,
}

export interface KeyUpEvent extends InputEvent {
    type: "keyUp",
    key: Key,
    modifiers: Key[],
    character?: string,
}

Keyboard events are emitted when a key is pressed or released. The key property contains the key that was pressed or released, along with a list of possible modifier keys like Shift, Ctrl, or Alt.

In case you are wondering why there is an optional character property: nut.js assumes a US ANSI keyboard layout when detecting key presses. But depending on the actual keyboard layout and/or input language, the character that is printed on the screen might be something totally different. Thus, nut.js tries to detect the actual character that is printed on the screen and if possible, provides it in the optional character property.

Mouse Events

Mouse events in nut.js provide a comprehensive way to monitor various interactions with the mouse device, such as movements, clicks, drags, and scrolls. Each mouse event comes with detailed information that can help you create responsive and interactive applications.

Compared to keyboard events, the mouse can emit a wider variety of events:

export interface MouseWheelEvent extends InputEvent {
    type: "mouseWheel",
    deltaX: number,
    deltaY: number,
    modifiers: Key[],
}

The MouseWheelEvent is emitted when the user scrolls the mouse wheel. It includes:

  • type: Indicates the event type, which is "mouseWheel".
  • deltaX and deltaY: Represent the amount scrolled horizontally and vertically, respectively.
  • modifiers: An array of modifier keys (like Shift, Ctrl, or Alt) that are pressed during the event.
export interface MouseDownEvent extends InputEvent {
    type: "mouseDown",
    targetPoint: Point,
    button: Button,
    modifiers: Key[],
}

export interface MouseUpEvent extends InputEvent {
    type: "mouseUp",
    targetPoint: Point,
    button: Button,
    modifiers: Key[],
}

MouseDownEvent is emitted when a mouse button is pressed. MouseUpEvent is emitted when a mouse button is released.

Both events include:

  • type: Either "mouseDown" or "mouseUp".
  • targetPoint: The cursor's position at the time of the event.
  • button: Specifies which mouse button was pressed or released (e.g., left, right, middle).
  • modifiers: Any modifier keys pressed during the event.
export interface MouseMoveEvent extends InputEvent {
    type: "mouseMove",
    targetPoint: Point,
    modifiers: Key[],
    deltaX: number,
    deltaY: number,
}

The MouseMoveEvent is emitted whenever the mouse is moved. It includes:

  • type: "mouseMove".
  • targetPoint: The new cursor position after the movement.
  • deltaX and deltaY: The change in the cursor's position along the X and Y axes since the last event.
  • modifiers: Modifier keys pressed during the movement.
export interface MouseDragEvent extends InputEvent {
    type: "mouseDrag",
    targetPoint: Point,
    button: Button,
    modifiers: Key[],
    deltaX: number,
    deltaY: number,
}

The MouseDragEvent is emitted when the mouse moves while a button is held down (dragging). It contains:

  • type: "mouseDrag".
  • targetPoint: The cursor's position during the drag.
  • button: The mouse button being held down.
  • deltaX and deltaY: The change in position since the last drag event.
  • modifiers: Any modifier keys pressed.

Utilizing Mouse Events

By handling these mouse events, you can create applications that respond dynamically to user interactions, such as:

  • Implementing drag-and-drop functionality.
  • Tracking mouse movement for drawing applications or games.
  • Responding to scroll events for custom scrolling behavior.
  • Detecting combined inputs like Shift + mouse click for enhanced controls.

Each event's deltaX and deltaY properties are particularly useful for calculating the speed or direction of the mouse movement, enabling more nuanced responses in your application.

Example: Building a "hot corner" application

In this example, we'll demonstrate how to use mouse events in nut.js to implement a "hot-corner" functionality. A hot-corner is a specific area of the screen (usually a corner) that triggers an action when the cursor moves into it. We'll create a script that minimizes the active window when the mouse is moved to the top-left corner of the screen while holding down the Ctrl key.

import {
    keyboard,
    mouse,
    Key,
    Point,
    isKeyEvent,
    withModifiers,
    isAtPosition,
    system,
    getActiveWindow,
} from "@nut-tree/nut-js";
import {useBoltInputMonitor, useBoltWindows} from "@nut-tree/bolt";

// Enable input monitoring and window control features
useBoltInputMonitor();
useBoltWindows();

// Listen for the "Ctrl+Q" key combination to exit the script
keyboard.on("keyDown", async (evt) => {
    if (
        isKeyEvent(evt, Key.Q) &&
        (withModifiers(evt, [Key.LeftControl]) || withModifiers(evt, [Key.RightControl]))
    ) {
        console.log("Ctrl+Q pressed, exiting");
        system.stopMonitoringInputEvents();
    }
});

// Listen for mouse movements to the top-left corner with the "Ctrl" key pressed
mouse.on("mouseMove", async (evt) => {
    if (isAtPosition(evt, new Point(0, 0)) && withModifiers(evt, [Key.LeftControl])) {
        console.log("Ctrl+Mouse at (0, 0), minimizing active window");
        const activeWindow = await getActiveWindow();
        await activeWindow.minimize();
    }
});

// Start monitoring input events
system.startMonitoringInputEvents();

Set Up Keyboard Listener for Exiting

keyboard.on("keyDown", async (evt) => {
    if (
        isKeyEvent(evt, Key.Q) &&
        (withModifiers(evt, [Key.LeftControl]) || withModifiers(evt, [Key.RightControl]))
    ) {
        console.log("Ctrl+Q pressed, exiting");
        system.stopMonitoringInputEvents();
    }
});
  • We listen for the keyDown event on the keyboard.
  • The isKeyEvent utility checks if the Q key was pressed.
  • The withModifiers utility checks if either the left or right Ctrl key is held down.
  • If both conditions are met, we log a message and stop monitoring input events, effectively exiting the script.

Set Up Mouse Movement Listener for Hot-Corner

mouse.on("mouseMove", async (evt) => {
    if (isAtPosition(evt, new Point(0, 0)) && withModifiers(evt, [Key.LeftControl])) {
        console.log("Ctrl+Mouse at (0, 0), minimizing active window");
        const activeWindow = await getActiveWindow();
        await activeWindow.minimize();
    }
});
  • We listen for the mouseMove event on the mouse.
  • The isAtPosition utility checks if the mouse is at the coordinates (0, 0), which is the top-left corner of the screen.
  • The withModifiers utility checks if the Ctrl key is held down.
  • If both conditions are satisfied, we retrieve the active window using getActiveWindow() and minimize it using await activeWindow.minimize().

Useful helper functions

Previous
Get Active Window