Molen

Appearance

@bendyline/molen-pack/cache ​

Molen content packs: build, open and read zip packs of models and documents from any source.

ts
import { /* … */ } from '@bendyline/molen-pack/cache';

Interfaces ​

ArchiveKey ​

What an archive is and where its bytes come from.

Properties ​

key? ​
ts
optional key?: string;

Identity of the bytes: a content hash or a URL whose content never changes.

keyBy? ​
ts
optional keyBy?: "etag";

Key by the server's strong ETag instead, for a stable URL whose bytes are replaced.

priority? ​
ts
optional priority?: ArchivePriority;

Default 'normal'.

size? ​
ts
optional size?: number;

Total size in bytes, when known. A read never asks for bytes past it.

url ​
ts
url: string;

Where the bytes come from; recorded with the archive (and the basis of keyBy: 'etag').

BlockCache ​

A shared cache of archive blocks; open each archive through it.

Properties ​

blockBytes ​
ts
readonly blockBytes: number;
store ​
ts
readonly store: ByteStore;

Methods ​

clear() ​
ts
clear(): Promise<void>;

Delete everything, in memory and in the store.

Returns ​

Promise&lt;void&gt;

dispose() ​
ts
dispose(): void;
Returns ​

void

drop() ​
ts
drop(key): Promise<void>;

Forget an archive by key.

Parameters ​
key ​

string

Returns ​

Promise&lt;void&gt;

flush() ​
ts
flush(): Promise<void>;

Write buffered blocks, evict down to the budget and refresh the store totals.

Returns ​

Promise&lt;void&gt;

open() ​
ts
open(archive, source): CachedArchive;
Parameters ​
archive ​

ArchiveKey

source ​

SpanReader

Returns ​

CachedArchive

setBudget() ​
ts
setBudget(bytes): void;
Parameters ​
bytes ​

number

Returns ​

void

stats() ​
ts
stats(): BlockCacheStats;

A snapshot; the store totals are as of the last flush().

Returns ​

BlockCacheStats

BlockCacheOptions ​

Properties ​

blockBytes? ​
ts
optional blockBytes?: number;

Block size in bytes (default 64 KiB).

evictHighWater? ​
ts
optional evictHighWater?: number;

Evict when the store holds more than this times the budget (default 1.05)…

evictLowWater? ​
ts
optional evictLowWater?: number;

…down to this times the budget (default 0.9).

flushDelayMs? ​
ts
optional flushDelayMs?: number;

Delay before buffered writes go to the store (default 50 ms).

maxSpanBytes? ​
ts
optional maxSpanBytes?: number;

Largest single network request when merging missing blocks (default 4 MiB).

memoryBytes? ​
ts
optional memoryBytes?: number;

In-memory tier (default 32 MiB).

now? ​
ts
optional now?: () => number;

Clock, for tests (default Date.now).

Returns ​

number

touchIntervalMs? ​
ts
optional touchIntervalMs?: number;

A block's last-used time is written at most this often (default 60 s).

BlockCacheStats ​

Counters and totals for a BlockCache. Block counts, not reads.

Properties ​

backend ​
ts
backend: "memory" | "indexeddb";
blockBytes ​
ts
blockBytes: number;
budgetBytes ​
ts
budgetBytes: number;
degraded ​
ts
degraded: boolean;
evictedBytes ​
ts
evictedBytes: number;
memoryBlocks ​
ts
memoryBlocks: number;
memoryBytes ​
ts
memoryBytes: number;
memoryHits ​
ts
memoryHits: number;
misses ​
ts
misses: number;
networkBytes ​
ts
networkBytes: number;
networkRequests ​
ts
networkRequests: number;
pendingBlocks ​
ts
pendingBlocks: number;
storeArchives ​
ts
storeArchives: number;
storeBlocks ​
ts
storeBlocks: number;
storeBytes ​
ts
storeBytes: number;
storeHits ​
ts
storeHits: number;
uncachedRequests ​
ts
uncachedRequests: number;

Reads passed straight through (no validator to key them, or a version mismatch).

ByteStore ​

Persistent block storage behind a BlockCache. Every method resolves, never rejects: a failure is a miss or a no-op, so a broken store only costs network.

Properties ​

budgetBytes ​
ts
readonly budgetBytes: number;

The budget in force (possibly clamped below the requested one by the origin's quota).

kind ​
ts
readonly kind: "memory" | "indexeddb";

Methods ​

archivesForUrl() ​
ts
archivesForUrl(url): Promise<StoredArchive[]>;

Every archive recorded with this URL.

Parameters ​
url ​

string

Returns ​

Promise&lt;StoredArchive[]&gt;

clear() ​
ts
clear(): Promise<void>;
Returns ​

Promise&lt;void&gt;

close()? ​
ts
optional close(): void;
Returns ​

void

deleteArchive() ​
ts
deleteArchive(key): Promise<void>;
Parameters ​
key ​

string

Returns ​

Promise&lt;void&gt;

evict() ​
ts
evict(targetBytes): Promise<number>;

Evict blocks in priority-biased LRU order until at most targetBytes remain; returns bytes freed.

Parameters ​
targetBytes ​

number

Returns ​

Promise&lt;number&gt;

getArchive() ​
ts
getArchive(key): Promise<StoredArchive | undefined>;
Parameters ​
key ​

string

Returns ​

Promise&lt;StoredArchive | undefined&gt;

getBlocks() ​
ts
getBlocks(key, indices): Promise<(Uint8Array<ArrayBufferLike> | undefined)[]>;
Parameters ​
key ​

string

indices ​

readonly number[]

Returns ​

Promise&lt;(Uint8Array&lt;ArrayBufferLike&gt; | undefined)[]&gt;

putBlocks() ​
ts
putBlocks(
   info, 
   blocks, 
   now
): Promise<void>;

Write blocks (replacing any with the same index) and the archive's record.

Parameters ​
info ​

StoredArchiveInfo

blocks ​

readonly object[]

now ​

number

Returns ​

Promise&lt;void&gt;

setBudget() ​
ts
setBudget(bytes): void;
Parameters ​
bytes ​

number

Returns ​

void

stats() ​
ts
stats(): Promise<ByteStoreStats>;
Returns ​

Promise&lt;ByteStoreStats&gt;

touch() ​
ts
touch(
   key, 
   indices, 
   now
): Promise<void>;

Mark blocks as used now (they move to the back of the eviction order).

Parameters ​
key ​

string

indices ​

readonly number[]

now ​

number

Returns ​

Promise&lt;void&gt;

updateArchive() ​
ts
updateArchive(info): Promise<void>;

Replace an archive's record without touching its blocks; creates an empty record if absent.

Parameters ​
info ​

StoredArchiveInfo

Returns ​

Promise&lt;void&gt;

ByteStoreStats ​

Totals reported by a store.

Properties ​

archives ​
ts
archives: number;
backend ​
ts
backend: "memory" | "indexeddb";
blocks ​
ts
blocks: number;
budgetBytes ​
ts
budgetBytes: number;
bytes ​
ts
bytes: number;
degraded ​
ts
degraded: boolean;

The store could not be opened or ran out of quota; nothing more is written.

CachedArchive ​

One archive opened through a BlockCache.

Methods ​

drop() ​
ts
drop(): Promise<void>;

Forget every cached block of this archive.

Returns ​

Promise&lt;void&gt;

etag() ​
ts
etag(): string | undefined;

The strong ETag of the cached bytes, when known.

Returns ​

string | undefined

read() ​
ts
read(
   offset, 
   length, 
   signal?, 
   etag?
): Promise<CachedRead>;

Read length bytes at offset; shorter only at the end of the archive. With etag, the caller expects that version: bytes cached for another version are never returned.

Parameters ​
offset ​

number

length ​

number

signal? ​

AbortSignal

etag? ​

string

Returns ​

Promise&lt;CachedRead&gt;

size() ​
ts
size(): number | undefined;

Total size, declared or learned from a short read at the end.

Returns ​

number | undefined

CachedRead ​

The result of a cached read.

Properties ​

bytes ​
ts
bytes: Uint8Array;
etag? ​
ts
optional etag?: string;

The validator these bytes belong to, when known.

CachingRangeReaderOptions ​

Properties ​

wholeBelow? ​
ts
optional wholeBelow?: number;

Archives at most this size are fetched whole on the first read, in one request (default 4 MiB), so a small pack costs one request on its first visit and none after.

DocumentCacheOptions ​

Properties ​

maxAgeMs? ​
ts
optional maxAgeMs?: number | ((url) => number);

Serve a stored copy without asking the server while it is younger than this, in ms (default 0: always revalidate). A function picks a policy per URL.

maxBytes? ​
ts
optional maxBytes?: number;

Largest document kept, in bytes (default 2 MiB).

now? ​
ts
optional now?: () => number;

Clock, for tests (default Date.now).

Returns ​

number

timeoutMs? ​
ts
optional timeoutMs?: number;

Serve the stored copy when the server has not answered within this many ms (default 4000).

IndexedDbByteStoreOptions ​

Properties ​

budgetBytes? ​
ts
optional budgetBytes?: number;

Bytes kept before eviction (default 512 MiB); clamped to half the origin's free quota.

indexedDB? ​
ts
optional indexedDB?: IDBFactory;

The IndexedDB factory (default globalThis.indexedDB).

name? ​
ts
optional name?: string;

Database name (default 'molen-bytes').

persist? ​
ts
optional persist?: boolean;

Ask the browser to make this origin's storage persistent (default false). Firefox shows a permission prompt for it, so only hosts that expect one (an installed app) should set it.

RangeSourceLike ​

The shape of a PMTiles Source (FetchSource, a retrying wrapper, a native reader).

Methods ​

getBytes() ​
ts
getBytes(
   offset, 
   length, 
   signal?, 
   etag?
): Promise<RangeSourceResponse>;
Parameters ​
offset ​

number

length ​

number

signal? ​

AbortSignal

etag? ​

string

Returns ​

Promise&lt;RangeSourceResponse&gt;

getKey() ​
ts
getKey(): string;
Returns ​

string

RangeSourceResponse ​

A PMTiles range response, typed structurally so this package never imports pmtiles.

Properties ​

cacheControl? ​
ts
optional cacheControl?: string;
data ​
ts
data: ArrayBuffer;
etag? ​
ts
optional etag?: string;
expires? ​
ts
optional expires?: string;

StoredArchive ​

A stored archive and the bytes it holds.

Extends ​

Properties ​

blocks ​
ts
blocks: number;
bytes ​
ts
bytes: number;
etag? ​
ts
optional etag?: string;

The strong ETag the bytes were read with.

Inherited from ​

StoredArchiveInfo.etag

key ​
ts
key: string;
Inherited from ​

StoredArchiveInfo.key

priority ​
ts
priority: ArchivePriority;
Inherited from ​

StoredArchiveInfo.priority

size? ​
ts
optional size?: number;
Inherited from ​

StoredArchiveInfo.size

type? ​
ts
optional type?: string;

Media type, for documents.

Inherited from ​

StoredArchiveInfo.type

url ​
ts
url: string;
Inherited from ​

StoredArchiveInfo.url

validated? ​
ts
optional validated?: number;

When a document was last confirmed current, in ms since the epoch.

Inherited from ​

StoredArchiveInfo.validated

StoredArchiveInfo ​

What a store records about one archive besides its blocks.

Extended by ​

Properties ​

etag? ​
ts
optional etag?: string;

The strong ETag the bytes were read with.

key ​
ts
key: string;
priority ​
ts
priority: ArchivePriority;
size? ​
ts
optional size?: number;
type? ​
ts
optional type?: string;

Media type, for documents.

url ​
ts
url: string;
validated? ​
ts
optional validated?: number;

When a document was last confirmed current, in ms since the epoch.

UrlRangeReaderOptions ​

Properties ​

fetch? ​
ts
optional fetch?: (input, init?) => Promise<Response>;

MDN Reference

Parameters ​
input ​

RequestInfo | URL

init? ​

RequestInit

Returns ​

Promise&lt;Response&gt;

retry? ​
ts
optional retry?: PackRetryOptions;
signal? ​
ts
optional signal?: AbortSignal;

Aborts every read.

size ​
ts
size: number;

Total size in bytes (a pack index lists it).

Type Aliases ​

ArchivePriority ​

ts
type ArchivePriority = "high" | "normal" | "low";

How long an archive's blocks are kept relative to others when the store is full.

SpanReader ​

ts
type SpanReader = (offset, length, signal, etag?) => Promise<{
  bytes: Uint8Array;
  etag?: string;
}>;

One read from the network (or wherever the archive lives) for the cache.

Parameters ​

offset ​

number

length ​

number

signal ​

AbortSignal

etag? ​

string

Returns ​

Promise&lt;{ bytes: Uint8Array; etag?: string; }&gt;

Variables ​

DOCUMENT_CACHE_HEADER ​

ts
const DOCUMENT_CACHE_HEADER: "x-molen-cache" = "x-molen-cache";

Response header naming where a cached document came from: hit, revalidated or stale.

PRIORITY_BIAS_MS ​

ts
const PRIORITY_BIAS_MS: Readonly<Record<ArchivePriority, number>>;

Recency bias per priority: added to a block's last-used time to order eviction.

Functions ​

cachingDocumentFetch() ​

ts
function cachingDocumentFetch(
   store, 
   fetchImpl?, 
   options?
): (input, init?) => Promise<Response>;

Wrap fetchImpl so GETs of small documents are kept in store.

Parameters ​

store ​

ByteStore

fetchImpl? ​

(input, init?) => Promise&lt;Response&gt;

options? ​

DocumentCacheOptions

Returns ​

ts
(input, init?): Promise<Response>;

MDN Reference

Parameters ​
input ​

RequestInfo | URL

init? ​

RequestInit

Returns ​

Promise&lt;Response&gt;

cachingRangeReader() ​

ts
function cachingRangeReader(
   reader, 
   cache, 
   archive, 
   options?
): RangeReader;

A pack RangeReader whose bytes come from cache when it has them. Key it by the pack's content hash (contentHash in a pack index) so a cached copy can never be another version.

Parameters ​

reader ​

RangeReader

cache ​

BlockCache

archive ​

ArchiveKey

options? ​

CachingRangeReaderOptions

Returns ​

RangeReader

cachingRangeSource() ​

ts
function cachingRangeSource(
   source, 
   cache, 
   archive
): RangeSourceLike;

A PMTiles source whose bytes come from cache when it has them. The inner source must honour Range requests. Its validator handling is kept: a request carrying the archive's ETag still reaches the inner source on a miss, so a replaced file fails with the inner source's own mismatch error (which also drops the cached bytes). Use keyBy: 'etag' for a stable URL whose contents are replaced in place; then the first read of a session (the header) always goes to the network and names the version the rest are cached under.

Parameters ​

source ​

RangeSourceLike

cache ​

BlockCache

archive ​

ArchiveKey

Returns ​

RangeSourceLike

createBlockCache() ​

ts
function createBlockCache(store?, options?): BlockCache;

Create a cache over store (default: an in-memory store with a 64 MiB budget).

Parameters ​

store? ​

ByteStore

options? ​

BlockCacheOptions

Returns ​

BlockCache

createIndexedDbByteStore() ​

ts
function createIndexedDbByteStore(options?): ByteStore | undefined;

Open (lazily) a block store in IndexedDB. Undefined where IndexedDB does not exist (Node, restricted embedded browsers); a database that fails to open behaves as an empty store.

Parameters ​

options? ​

IndexedDbByteStoreOptions

Returns ​

ByteStore | undefined

createMemoryByteStore() ​

ts
function createMemoryByteStore(options?): ByteStore;

A ByteStore in memory: for tests, private browsing, and hosts without IndexedDB.

Parameters ​

options? ​
budgetBytes? ​

number

Returns ​

ByteStore

evictionRank() ​

ts
function evictionRank(priority, touched): number;

A block's place in the eviction order: lower ranks go first.

Parameters ​

priority ​

ArchivePriority

touched ​

number

Returns ​

number

isChangedError() ​

ts
function isChangedError(error): boolean;

Whether an error says the server's bytes changed while they were being read.

Parameters ​

error ​

unknown

Returns ​

boolean

urlRangeReader() ​

ts
function urlRangeReader(url, options): RangeReader;

A RangeReader over a URL whose size is already known, with no opening request: the first response's strong ETag becomes the If-Range validator for every later read, so a file replaced mid-read fails with PackChangedError instead of mixing versions. Pair it with cachingRangeReader to open a pack from cached bytes.

Parameters ​

url ​

string

options ​

UrlRangeReaderOptions

Returns ​

RangeReader

References ​

RangeReader ​

Re-exports RangeReader

37 guides · 36 schema formats · 41 API entry points · 10 samples. API, CLI and schema pages are generated from the shipped build.

Molen / Bendyline