@nut-tree/plugin-ai-sdk

VercelAiSdkVisionFinder

Vision finder implementation powered by Vercel AI SDK.

The provider uses an LLM vision model to decide whether screen content matches
a natural language description and optionally returns one or more matching
regions.

UseAnthropicVisionProviderOptions

Options for useAnthropicVisionProvider.
Accepts all provider config options plus an optional model override.

UseOllamaVisionProviderOptions

Options for useOllamaVisionProvider.
Accepts all provider config options plus an optional model override.

UseOpenAICompatibleVisionProviderOptions

Options for useOpenAICompatibleVisionProvider.

Targets self-hosted servers that implement the OpenAI Chat Completions API
(e.g. vLLM, LMStudio, llama.cpp, LocalAI). Both the model ID and the
openaiCompatible server config are required because there is no sensible
default for either.

UseOpenAIVisionProviderOptions

Options for useOpenAIVisionProvider.
Accepts all provider config options plus an optional model override.

UseVisionProviderOptions

Configuration for the Vercel AI SDK vision finder, including default model
selection, confidence thresholds, and provider-specific connection options.

VercelAiSdkVisionModelConfiguration

Identifies a specific vision model by its provider and model ID.

VercelAiSdkVisionProviderConfig

Configuration for the Vercel AI SDK vision finder, including default model
selection, confidence thresholds, and provider-specific connection options.

VercelAiSdkVisionProviderData

Per-request overrides passed via the providerData field of a match request.

VisionMatchingSettings

Tuning parameters for vision match scoring.

useAnthropicVisionProvider

useOllamaVisionProvider

useOpenAICompatibleVisionProvider

useOpenAIVisionProvider

VercelAiSdkVisionCoordinateSpace

Coordinate space the model is asked to report regions in.

- "pixels" (default): absolute pixel coordinates of the transmitted
image. The image dimensions are stated in the prompt. Matches models
trained on absolute coordinates (e.g. the Qwen2.5-VL family).
- "normalized": coordinates relative to the image dimensions, in the
range [0, coordinateScale] (default scale: 1000; left/width
relative to the image width, top/height to the image height). The
finder converts them back to pixels deterministically. Matches the
grounding convention of e.g. Gemma and Qwen3-VL, and is invariant under
uniform image rescaling by the serving runtime.

Model families differ in their trained convention, so benchmark both
settings for a given model (e.g. with @nut-tree/vision-evals).

VercelAiSdkVisionModelProvider

Supported LLM provider backends for vision model requests.

Use "openai-compatible" for self-hosted servers that implement the
OpenAI Chat Completions API (e.g. vLLM, LMStudio, llama.cpp, LocalAI).