Watch external storage — PexarBolt Docs

External-storage filesystem observationThe host SDK ctx.fs.watchExternal watches current/future external storage recursively by default. Desktop and the plugin must run, but Dock USB is unnecessary. This is a host capability, not standalone device execution. It is an additive SDK/contract 1.0.0 capability requiring updated App/tools.This is the single developer guide for external storage. The plugin contract is the normative reference; this guide explains the same API with examples, and reliability requirements cover transfer-state handling and live verification.Status and scopeImplemented: fs.watch with ctx.fs.watchExternal, recursive defaults, native directory notifications, snapshots/revisions/overflow recovery, mount lifecycle handling, explicit watcher.closed failures, and CLI/MCP virtual volumes. Supported platforms are macOS external physical volumes and Windows USB storage. Later increments added fs.transfers (macOS published NSProgress; Windows reports unsupported), fs.read (whole-file and streaming reads without an SDK total-size cap), fs.eject (non-forced safe removal of the physical device), live I/O rates, optional file indexing (fileChanges:false) and automatic discovery recovery. The authoring tools expose matching simulated content, progress, rates and ejection.Not implemented: manually selected ctx.fs.watch folders, Dock-only scope, volume allowlists, directory grant selectors, native mount notifications and dedicated sleep recovery. Discovery polls every five seconds; files use Node fs.watch plus 60-second reconciliation (not Chokidar); unavailable indexed roots retry every 60 seconds. names matches literally, and exceeding eight volumes fails the collection. The original 2026-09-17 target design is archived in the client repository at docs/design/2026-09-17-filesystem-watch.md; it is not a description of current behavior.Declaration and grants"host": { "entry": "host.mjs", "sdk": "1.0.0", "permissions": ["fs.watch"], "externalStorage": { "scope": "external", "recursive": true } }externalStorage is optional and defaults to that declaration. Permissions cover names, types, sizes and modification times on current/future external volumes. Package upgrades require renewed authorization. recursive:false restricts access to mount roots; call options cannot expand grants. fs.watch makes deviceExecution return host.fs.watch alone exposes metadata. fs.transfers, fs.read and fs.eject separately grant transfer observation, content reading and safe ejection. No writes or arbitrary paths are exposed. Plugins receive opaque volumeId, volume label, mountId and slash-separated relative paths, never host absolute paths. With network permission, they may send metadata to the authorized origin; the permission UI discloses this.APIconst watcher = await ctx.fs.watchExternal({ recursive: true, // Inherit the manifest by default extensions: ['.json'], // Optional: at most 16, case-insensitive, file filtering only names: ['status.json'], // Optional: at most 32 exact relative paths debounceMs: 250, // 100..2000 signal: ctx.signal, }, async event => { /* Handle events */ }); const page = await watcher.snapshot(volumeId, mountId, { cursor, limit: 200, signal }); void watcher.closed.catch(error => ctx.log.warn(String(error))); await watcher.close();Combined filters intersect; empty arrays match no files, but directories still appear. names forbids absolute/hidden paths, parent traversal, backslashes and globs. Plugins cannot supply roots. Subscription returns before asynchronous discovery; callbacks start only after the caller receives the watcher. One collection subscription per plugin is permitted; close before replacing it.

Event

Meaning

volume / mounted

New mount with volume and mountId; its snapshot may not be ready

volume / unmounted

Remove the local mount view without synthesizing per-file deletions

volume-event / reset

Index ready or resynchronization needed; supplies a new epoch

volume-event / changes

upsert/remove delta from fromRevision to revision, with observedAt

volume-event / status

Temporary root_missing/access_denied/quota_exceeded/backend_error

transfer

Publisher-reported current progress or terminal state when enabled

transfer-status

Observation backend status, separate from indexing

transfer-reset

Queue overflow; recover using transferSnapshot

Rename is remove(old)+upsert(new). Coalesced notifications describe observed final state, not an audit log or write-completion proof. Transient files may never appear; native hints can upsert unchanged size/mtime. Keys include volumeId+mountId, and remount changes mountId.Snapshots freeze the host index, not the physical filesystem atomically. Pages retain their revision, with at most 200 entries/48 KiB. One snapshot per volume; starting another replaces it. Cursors are single-use and expire after 60 seconds. Wrong/expired cursor, mount or subscription returns stale_event; initialization/unavailability returns unavailable.Synchronization:

Create an empty view on mounted. On reset, page a full snapshot and retain epoch/revision.

Callbacks are serial; changes queue on the host during paging. Drop batches at or below the snapshot revision afterward.

Apply deltas only when epoch matches and fromRevision equals the local revision; otherwise resnapshot.

Removing a directory removes its subtree. Unmounted/status clears stale local state.

Expired cursors/new epochs restart synchronization. A rejected closed promise ends use of the subscription and must show an error.

Pace snapshot pages by at least 20 ms to avoid exhausting the generic 100 SDK requests/second budget. Callbacks must finish within ten seconds; perform expensive work outside them with bounded queues and generation checks.Minimal notification exampleA change-count view need not read file lists but must handle reset. Its state schema contains integer changes and string status. Counts represent observed changes, not an audit of filesystem operations.export async function activate(ctx) { let changes = 0, status = 'Waiting for external storage', timer, stopped = false; const publish = () => { if (timer || stopped) return; timer = setTimeout(() => { timer = undefined; void ctx.publish({ changes, status }).catch(error => { if (!stopped) ctx.log.warn(String(error)); }); }, 300); }; const watcher = await ctx.fs.watchExternal({}, event => { if (event.type === 'volume') status = event.action; else if (event.type === 'volume-event') { if (event.event.type === 'reset') status = 'Watching'; else if (event.event.type === 'status') status = event.event.reason; else { changes += event.event.changes.length; status = 'Changed'; } } publish(); }); void watcher.closed.catch(error => { status = 'Watcher stopped'; ctx.log.warn(String(error)); publish(); }); publish(); return async () => { stopped = true; clearTimeout(timer); await watcher.close(); }; }Do not await closed inside activate: it represents the subscription lifetime. Throwing or exceeding ten seconds closes the subscription and rejects closed. Never wait for a later event from inside its callback. Close, abort, stop and revocation release watchers, queues, snapshots and pending requests. An entered callback cannot be forcibly cancelled, but no new callback begins after close.Copy progress from external applicationsHandle pause/resume, cancellation, interruption and unknown outcomes; see Reliability requirements. Optional transfer.reason distinguishes publisher_cancelled, observation_ended, volume_unmounted and backend_unavailable. Non-running speed/ETA is null. A 100% fraction never overrides cancelled/unknown. Active tasks become unknown before unmount, while confirmed terminal events remain. Close/abort stops observation, not Finder copying.Declare fs.watch and fs.transfers and use watchExternal({transfers:true},listener). Observation defaults off, preserving older behavior.macOS subscribes to publicly published cross-process NSProgress. Finder-to-external-disk copying has been verified, but not every application or copy method publishes progress. Silence does not prove no transfer exists. Coverage includes discovered, validated directories; new directories, rapid tasks and paths beyond scan quotas may be missed. Windows reports unsupported while ordinary file observation/reading remains available.type FsTransfer = { id: string; path: string; source: 'macos-nsprogress' | 'fixture'; operation: 'copy' | 'move' | 'download' | 'upload' | 'other'; state: 'running' | 'paused' | 'completed' | 'cancelled' | 'unknown'; completedUnits: number | null; totalUnits: number | null; fractionCompleted: number | null; bytesPerSecond: number | null; estimatedSecondsRemaining: number | null; observedAt: number; }; const snapshot = await watcher.transferSnapshot(volumeId, mountId, { signal }); // { status: { state, coverage: 'published-only', reason? }, transfers: FsTransfer[] }

fractionCompleted is the publisher's 0..1 ratio or null. Never estimate totals from file size. completedUnits/totalUnits are publisher units, not necessarily bytes: a measured Finder 256 MiB copy reported 1,000 total units. Speed/ETA appear only when supplied; ETA is an estimate.

completed requires publisher finished; cancellation takes precedence. Unpublication, process exit or backend failure without completion evidence yields unknown. Directory tasks need not emit per-file progress; completion proves neither validation, flush nor safe removal.

Backend states are starting/available/unsupported/unavailable, with platform/helper_missing/backend_error/quota_exceeded/root_unavailable reasons. Available means ready, still with published-only coverage. Backend failure may require resubscription.

The helper samples published progress every 250 ms; sandbox delivery remains coalesced and limited to four events/second, not per-byte updates. At most 128 tasks per volume are retained; terminal records expire after 60 seconds or earlier capacity eviction. transfer-reset replaces the local view from a snapshot. This is not permanent history.

References: Apple NSProgress subscription and publishing progress.Read file contentsAdd fs.read alongside fs.watch; transfers is unnecessary:const { bytes, entry } = await watcher.readBytes(volumeId, mountId, 'Documents/report.csv', { signal: ctx.signal, }); const text = await watcher.readText(volumeId, mountId, 'Documents/report.csv'); const value = await watcher.readJson(volumeId, mountId, 'status.json');readBytes returns the complete Uint8Array and metadata in one call, without caller chunking. Internal transport chunks do not cap total file size. readText validates UTF-8 strictly. readJson additionally validates SDK JSON: depth 16, 8,192 nodes, no dangerous properties or nonfinite values; parse failure returns invalid_config. There is no SDK total-size cap. Optional maxBytes is a caller limit; exceeding it returns quota_exceeded rather than truncating. Convenience methods retain the whole result in memory. readBytes allocates the final buffer once from file size instead of retaining all chunks and copying again. For large files, process readChunks incrementally. Limits are two simultaneous chunks per subscription and eight across the App.// Run bounded background work; use the task signal and do not block file callbacks. let checkedBytes = 0; for await (const chunk of watcher.readChunks(volumeId, mountId, filePath, { signal: ctx.signal })) { // Update an incremental hash here; never retain all chunks or publish/store raw bytes. checkedBytes += chunk.bytes.length; } // Only normal iterator completion establishes EOF; discard partial verification on error. // Readability does not establish a matching source hash.readChunks yields {bytes,entry,offset,eof,version}. Omit chunkSize for automatic 4–16 MiB blocks: start at 4 MiB, double below 250 ms and halve above 1,000 ms. Explicit chunkSize fixes the size from one byte to 16 MiB; most plugins should omit it. This does not cap total size.Each iterator reuses its handle, avoiding repeated opens/ancestor traversal, while checking mount, identity/version and actual path before/after every chunk. The next chunk is read after consuming the previous one. There is no fixed per-chunk sleep: all plugin reads share 64 requests per rolling second and wait cancellably only when exhausted. Do not add sleeps. EOF/error/break/abort/stop/unmount release handles, which safe ejection waits for. Thirty idle seconds release the handle; continuing returns stale_event. Cancel/restart for longer pauses. contentRead.version:3, streaming:true and defaultMaxBytes/maxBytes:null describe the contract.Relative paths must match recursion and name/extension filters. Hidden paths, links/hardlinks, directories, cross-filesystem child mounts and absolute paths are forbidden. Identity, size or timestamp changes during reading return stale_event. Complete-file methods return no partial result; yielded chunks remain provisional and partial verification must be discarded on error. Every chunk checks the same inode, size and nanosecond timestamps. Stale mounts return stale_event; inaccessible paths return unavailable/permission_denied. Stop/revocation/abort interrupt reads. Each chunk check has a five-second budget, but the stream has no total deadline and underlying OS calls may return late. A successful read does not prove another application will not modify the file afterward.For a publisher-confirmed completed JSON transfer, a small read can be attempted as follows. Large validation belongs in background work:if (event.type === 'transfer' && event.transfer.state === 'completed' && event.transfer.path.endsWith('.json')) { try { const value = await watcher.readJson(event.volumeId, event.mountId, event.transfer.path); // Validate fields against the plugin state schema before publishing. ctx.log.info(JSON.stringify(value)); } catch (error) { ctx.log.warn(String(error)); } }With network permission, read content may be sent to the authorized service; permission review explicitly discloses this.Safe ejectionAdd fs.eject alongside fs.watch. fs.read/fs.transfers are unnecessary. Use IDs from the current mount event:const result = await watcher.eject(volumeId, mountId, { signal }); if (result.status === 'ejected') { // simulated=true means fixture success only. } else if (result.status === 'failed') { // busy / permission_denied / unsupported: show the reason and wait for user action. } else { // disconnected / timeout / backend_error: outcome unknown; never claim safe removal or retry. }Ejected means the OS safe-removal operation succeeded. Related volume/unmounted events follow and old mount IDs become invalid. Unmounted alone is insufficient because physical unplug emits it too. No force, path, drive letter or disk-number argument exists.The action affects the entire physical storage device and all its partitions. File filters/recursion do not narrow ejection scope. Permissions explain effects on other applications. Invoke from explicit touch, dial or user commands; declare write:true and permissions:["fs.eject"] on the corresponding event. Do not automatically eject after one copy completes; other applications may still be active.

Rediscover and validate external volume identity, mount and physical target before submitting. Stale mounts fail with stale_event.

macOS uses non-forced diskutil eject, resolving APFS to the physical disk. Ambiguous multi-disk containers return failed/unsupported.

Windows validates USB volume UniqueId, disk number and system/boot flags, then uses CM_Request_Device_EjectW. It never disables/offlines forcibly, kills processes or elevates. Unsupported drivers/grants fail explicitly; target Windows hardware still needs validation.

One real ejection across the App may run at a time; additional calls return overloaded without queue/retry. Pause this filesystem subsystem's reads/watchers on the physical disk across plugins and allow five seconds for handle release. Other image/video/application activity may still yield busy. Failure resumes watchers with reset.

OS wait is at most 30 seconds, excluding discovery/handle cleanup. Cancellation before submission prevents execution. After submission, cancellation/stop/revocation cannot undo the OS request. Unknown or stopped does not prove continued mounting; inspect actual state and never treat retry as idempotent.

Other filesystem grants do not imply fs.eject; adding it requires renewed authorization.

CLI/MCP/App previews remove only a virtual volume, returning {status:'ejected',simulated:true} and unmounted. Further operations on old IDs return stale_event. filesystemWatch.eject advertises platforms, physical-device scope, force:false and fixture behavior. No OS eject is called in preview.The macOS regression node apps/desktop/scripts/test-plugin-file-eject.mjs creates an isolated disk image, injects only that test volume, verifies removal/stale IDs and cleans up. Production discovery still excludes disk images.Platform boundaries and budgets

macOS uses diskutil external physical disks/APFS PhysicalStores, excluding images, internal disks and system-role volumes.

Windows supports USB fixed/removable storage with drive letters, excluding system, hidden and auxiliary partitions. Linux and non-USB Windows external storage are unsupported; uncertain classifications are excluded.

One recursive native fs.watch per volume triggers bounded metadata scans, with 60-second reconciliation. Discovery runs every five seconds, so hotplug has latency. File events wake pending RPC without plugin polling.

Hidden paths, symlinks/junctions, different-filesystem child mounts, trash and Windows system-volume directories are excluded. Scans/snapshots verify root realpath and identity and stop returning an index when invalid/inaccessible.

At most eight volumes per plugin and 64 collection subscriptions per App. Volume quota and unrecoverable discovery failures reject closed, never masquerading as empty results. Transient discovery errors keep the watcher open and recover automatically; see Discovery recovery.

Each scan allows 20,000 entries, 1,024 directories, depth 16 and eight seconds. Overflow reports status, never a falsely complete list. Estimated root index size is 8 MiB; App committed indexes/frozen views are bounded to 100,000 entries/32 MiB.

Batches hold 128 changes/48 KiB, with four events/second per collection. Pending queues hold 256 events/128 KiB; overflow resets epoch. Unavailable indexed roots retry every 60 seconds. OS permissions remain a user system-setting action.

Manually selected ctx.fs.watch roots, Dock-only scope, volume allowlists and directory selectors remain future design.

CLI, MCP and SDKAll authoring paths share this API. Previews expose Sample external storage with directory/JSON/CSV metadata and fs.read-protected sample content, never real disks. transfers:true emits source:fixture progress/completion, and status.json contains value:42. Continuous hotplug/writes belong to host integration tests, not preview simulation.PluginSession.read().context.filesystemWatch and Desktop plugin_context overview/sdk/all expose capabilities. Runtime status normalizes externalStorage. SDK/contract/skill ship with the standalone toolkit. Preview success is not real disk authorization or Windows hardware verification.node apps/desktop/scripts/test-plugin-file-progress.mjs checks cross-process completion, cancellation and unknown states. An optional external-volume path such as /Volumes/Lexar creates isolated test data, uses Finder to copy a 256 MiB file, verifies completion and cleans up.Live disk I/ODeclare fs.watch and subscribe per mount with watcher.subscribeIo(volumeId, mountId, listener, {signal?, onError?}), using IDs from volume/mounted events. The host samples OS counters about every two seconds and shares non-overlapping queries between listeners. A dashboard that only needs rates and volume lifecycle should disable file indexing:let watcher; watcher = await ctx.fs.watchExternal({ fileChanges: false, signal: ctx.signal }, event => { if (event.type !== 'volume' || event.action !== 'mounted') return; watcher.subscribeIo(event.volume.volumeId, event.mountId, sample => { // Update local state and use the shared coalescing publisher (at least 300 ms between starts). // Read both rates when sample.status === 'ready'; show an em dash for unavailable, never zero. updateIoState(sample); }, { signal: ctx.signal, onError: error => ctx.log.warn(String(error)) }); }); void watcher.closed.catch(error => ctx.log.warn(String(error))); // Await watcher.close() on shutdown, or call the returned function to unsubscribe individually.Keep scope (volume or physical device), sampledAt and intervalMs with each value. Unavailable/null is not idle/zero, and whole-device samples include other partitions, so never sum them per volume. Rates are actual window-averaged system I/O, not benchmark speed, SDK read speed or transfer progress. CLI and MCP previews return simulated fixed rates. Sample fields, platform counter sources, warm-up time, listener limits and cleanup are specified in the contract's Live Storage I/O section.Volume-only watchingfileChanges defaults to true. Pass fileChanges: false when a plugin needs only volume lifecycle, I/O, published transfers, reads of known paths or ejection: the host then skips directory traversal, file indexes and native file watchers, emits no volume-event, and snapshot() returns unsupported. Without indexing, macOS transfer observation subscribes only at mount roots, so some subdirectory-published tasks may be absent. Do not reintroduce scanning in plugin code. See Disable file-change indexing.Discovery recoveryAfter a transient OS error or timeout, the watcher stays open and retries discovery every five seconds without overlap, emitting {type:'discovery-status',status:'retrying',reason:'backend_error'} and then {type:'discovery-status',status:'available'}. While retrying, keep known volume names but treat reads, snapshots, I/O and ejection as unavailable: cancel active reads, clear stale rates, stop pending verification and mark unfinished transfers unknown. Recovery keeps IDs for unchanged mounts; replaced mounts arrive as unmounted/mounted events with a new mountId. Do not treat failure as an empty volume list, reopen the watcher or add polling. See Recover from volume discovery failures.