@thi.ng/vector-pools
Version:
Data structures for managing & working with strided, memory mapped vectors
309 lines (243 loc) • 11.6 kB
Markdown
<!-- This file is generated - DO NOT EDIT! -->
<!-- Please see: https://github.com/thi-ng/umbrella/blob/develop/CONTRIBUTING.md#changes-to-readme-files -->
# 
[](https://www.npmjs.com/package/@thi.ng/vector-pools)
[](https://mastodon.thi.ng/@toxi)
> [!NOTE]
> This is one of 210 standalone projects, maintained as part
> of the [.ng/umbrella](https://github.com/thi-ng/umbrella/) monorepo
> and anti-framework.
>
> 🚀 Please help me to work full-time on these projects by [sponsoring me on
> GitHub](https://github.com/sponsors/postspectacular). Thank you! ❤️
- [About](#about)
- [Status](#status)
- [Related packages](#related-packages)
- [Installation](#installation)
- [Dependencies](#dependencies)
- [Usage examples](#usage-examples)
- [API](#api)
- [WebGL geometry definition / manipulation](#webgl-geometry-definition--manipulation)
- [WASM interop](#wasm-interop)
- [Authors](#authors)
- [License](#license)
## About
Data structures for managing & working with strided, memory mapped vectors.
This still package provides several data structures for managing &
working with memory mapped vectors. Together with
[.ng/vectors](https://github.com/thi-ng/umbrella/tree/develop/packages/vectors),
these structures enable high-level, zero-copy<sup>*</sup> manipulation
of the underlying memory region and are largely intended for WebGL &
WASM use cases, e.g. to provide JS friendly views of a structured data
region of a WebGL or WASM memory buffer.
<sup>*</sup> The only copying taking place is to GPU memory
## Status
**ALPHA** - bleeding edge / work-in-progress
[Search or submit any issues for this package](https://github.com/thi-ng/umbrella/issues?q=%5Bvector-pools%5D+in%3Atitle)
This package might be merged with and/or superseded by
[.ng/ecs](https://github.com/thi-ng/umbrella/tree/develop/packages/ecs)
/
[.ng/soa](https://github.com/thi-ng/umbrella/tree/develop/packages/soa).
## Related packages
- [.ng/ecs](https://github.com/thi-ng/umbrella/tree/develop/packages/ecs) - Entity Component System based around typed arrays & sparse sets
- [.ng/malloc](https://github.com/thi-ng/umbrella/tree/develop/packages/malloc) - ArrayBuffer based malloc() impl for hybrid JS/WASM use cases, based on thi.ng/tinyalloc
- [.ng/soa](https://github.com/thi-ng/umbrella/tree/develop/packages/soa) - SOA & AOS memory mapped structured views with optional & extensible serialization
- [.ng/unionstruct](https://github.com/thi-ng/umbrella/tree/develop/packages/unionstruct) - C-style struct, union and bitfield read/write views of ArrayBuffers
- [.ng/vectors](https://github.com/thi-ng/umbrella/tree/develop/packages/vectors) - Optimized 2d/3d/4d and arbitrary length vector operations, support for memory mapping/layouts
- [.ng/webgl](https://github.com/thi-ng/umbrella/tree/develop/packages/webgl) - WebGL & GLSL abstraction layer
## Installation
```bash
yarn add .ng/vector-pools
```
ESM import:
```ts
import * as vp from "@thi.ng/vector-pools";
```
Browser ESM import:
```html
<script type="module" src="https://esm.run/@thi.ng/vector-pools"></script>
```
[JSDelivr documentation](https://www.jsdelivr.com/)
For Node.js REPL:
```js
const vp = await import("@thi.ng/vector-pools");
```
Package sizes (brotli'd, pre-treeshake): ESM: 3.02 KB
## Dependencies
- [.ng/api](https://github.com/thi-ng/umbrella/tree/develop/packages/api)
- [.ng/binary](https://github.com/thi-ng/umbrella/tree/develop/packages/binary)
- [.ng/checks](https://github.com/thi-ng/umbrella/tree/develop/packages/checks)
- [.ng/errors](https://github.com/thi-ng/umbrella/tree/develop/packages/errors)
- [.ng/logger](https://github.com/thi-ng/umbrella/tree/develop/packages/logger)
- [.ng/malloc](https://github.com/thi-ng/umbrella/tree/develop/packages/malloc)
- [.ng/transducers](https://github.com/thi-ng/umbrella/tree/develop/packages/transducers)
- [.ng/vectors](https://github.com/thi-ng/umbrella/tree/develop/packages/vectors)
Note: .ng/api is in _most_ cases a type-only import (not used at runtime)
## Usage examples
Three projects in this repo's
[/examples](https://github.com/thi-ng/umbrella/tree/develop/examples)
directory are using this package:
| Screenshot | Description | Live demo | Source |
|:------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------|:-------------------------------------------------------------|:------------------------------------------------------------------------------------------|
| <img src="https://raw.githubusercontent.com/thi-ng/umbrella/develop/assets/examples/geom-webgl-attrib-pool.jpg" width="240"/> | Augmenting thi.ng/geom shapes for WebGL, using instancing & attribute buffers | [Demo](https://demo.thi.ng/umbrella/geom-webgl-attrib-pool/) | [Source](https://github.com/thi-ng/umbrella/tree/develop/examples/geom-webgl-attrib-pool) |
| <img src="https://raw.githubusercontent.com/thi-ng/umbrella/develop/assets/examples/geom-webgl-basics.jpg" width="240"/> | Converting thi.ng/geom shape types for WebGL | [Demo](https://demo.thi.ng/umbrella/geom-webgl-basics/) | [Source](https://github.com/thi-ng/umbrella/tree/develop/examples/geom-webgl-basics) |
| <img src="https://raw.githubusercontent.com/thi-ng/umbrella/develop/assets/examples/webgl-msdf.jpg" width="240"/> | WebGL MSDF text rendering & particle system | [Demo](https://demo.thi.ng/umbrella/webgl-msdf/) | [Source](https://github.com/thi-ng/umbrella/tree/develop/examples/webgl-msdf) |
## API
[Generated API docs](https://docs.thi.ng/umbrella/vector-pools/)
### WebGL geometry definition / manipulation
```ts
import { AttribPool, GLType } from "@thi.ng/vector-pools";
import * as v from "@thi.ng/vectors";
import * as tx from "@thi.ng/transducers";
// create an interleaved (AOS layout) attribute buffer w/ default values
const geo = new AttribPool({
// initial size in bytes (or provide ArrayBuffer or @thi.ng/malloc/MemPool)
mem: { size: 0x200 },
// num elements
num: 4,
// attrib specs (data mapping layout)
attribs: {
pos: { type: GLType.F32, size: 3, byteOffset: 0 },
uv: { type: GLType.F32, size: 2, byteOffset: 12 },
col: { type: GLType.F32, size: 3, default: [1, 1, 1], byteOffset: 20 },
id: { type: GLType.U16, size: 1, byteOffset: 32 }
}
});
// computed overall stride length
geo.byteStride
// 36
// set attrib values
geo.setAttribs({
pos: { data: [[-5, 0, 0], [5, 0, 0], [5, 5, 0], [-5, 5, 0]]},
uv: { data: [[0, 0], [1, 0], [1, 1], [0, 1]] }
});
// ...or individually
geo.setAttribValues("id", [0, 1, 2, 3]);
// get view of individual attrib val
geo.attribValue("pos", 3)
// Float32Array [ -5, 5, 0 ]
// zero-copy direct manipulation of mapped attrib val
v.mulN(null, geo.attribValue("pos", 3), 2);
// Float32Array [ -10, 10, 0 ]
// get iterator of mapped attrib vals (e.g. for batch processing)
[...geo.attribValues("pos")]
// [ Float32Array [ -5, 0, 0 ],
// Float32Array [ 5, 0, 0 ],
// Float32Array [ 5, 5, 0 ],
// Float32Array [ -10, 10, 0 ] ]
// use with transducers, e.g. to map positions to colors
tx.run(
tx.map(([pos, col]) => v.maddN(col, [0.5, 0.5, 0.5], v.normalize(col, pos), 0.5)),
tx.zip(geo.attribValues("pos"), geo.attribValues("col"))
);
// updated colors
[...geo.attribValues("col")]
// [ Float32Array [ 0, 0.5, 0.5 ],
// Float32Array [ 1, 0.5, 0.5 ],
// Float32Array [ 0.8535534143447876, 0.8535534143447876, 0.5 ],
// Float32Array [ 0.1464466154575348, 0.8535534143447876, 0.5 ] ]
// dynamically add another attrib
// this will change the overall stride length and re-align all existing attribs
geo.addAttribs({
normal: { type: GLType.F32, size: 3, default: [0, 0, 1], byteOffset: 36 }
});
// updated overall stride length
geo.byteStride
// 48
// ...Webgl boilerplate omitted
const gl = ...
// only need to use & bind single (interleaved) buffer
// containing all attribs
buf = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, buf);
gl.bufferData(gl.ARRAY_BUFFER, geo.bytes(), gl.STATIC_DRAW);
// helper fn to bind a single shader attrib
const initAttrib = (gl, loc, attrib) => {
gl.enableVertexAttribArray(loc);
gl.vertexAttribPointer(
loc,
attrib.size,
attrib.type,
false,
attrib.byteStride, // computed by pool
attrib.byteOffset
);
};
initAttrib(gl, attribLocPosition, geo.specs.pos);
initAttrib(gl, attribLocNormal, geo.specs.normal);
initAttrib(gl, attribLocUV, geo.specs.uv);
```
### WASM interop
```c
// main.c
#include <emscripten.h>
#include <stdint.h>
typedef struct {
float pos[3];
float uv[2];
float col[3];
uint16_t id;
} Vertex;
Vertex vertices[] = {
{.pos = {-5, 0, 0}, .uv = {0, 0}, .col = {1, 0, 0}, .id = 0},
{.pos = {5, 0, 0}, .uv = {1, 0}, .col = {0, 1, 0}, .id = 1},
{.pos = {5, 5, 0}, .uv = {1, 1}, .col = {0, 0, 1}, .id = 2},
{.pos = {-5, 5, 0}, .uv = {0, 1}, .col = {1, 0, 1}, .id = 3},
};
int main() { return 0; }
EMSCRIPTEN_KEEPALIVE Vertex* getVertices() {
return vertices;
}
EMSCRIPTEN_KEEPALIVE int getNumVertices() {
return sizeof(vertices) / sizeof(Vertex);
}
```
```ts
import { Type } from "@thi.ng/api";
// ... WASM / Emscripten boilerplate omitted
const Module = ...
// initialize pool from mapped WASM memory
const geo = new vp.AttribPool(
// map WASM memory
Module.buffer,
// num elements (obtained from C function)
Module._getNumVertices(),
// attrib specs (data mapping layout)
// don't specify attrib defaults to avoid overriding
// values already initialized by WASM code
{
pos: { type: Type.F32, size: 3, byteOffset: 0 },
uv: { type: Type.F32, size: 2, byteOffset: 12 },
col: { type: Type.F32, size: 3, byteOffset: 20 },
id: { type: Type.U16, size: 1, byteOffset: 32 }
},
// pool options
{
// don't allow resizing (since we're mapping a fixed sized C array)
resizable: false,
// initialize mem pool to start @ C `vertices` array
mempool: {
start: Module._getVertices(),
}
}
);
[...geo.attribValues("pos")]
// [ Float32Array [ -5, 0, 0 ],
// Float32Array [ 5, 0, 0 ],
// Float32Array [ 5, 5, 0 ],
// Float32Array [ -5, 5, 0 ] ]
```
## Authors
- [Karsten Schmidt](https://thi.ng)
If this project contributes to an academic publication, please cite it as:
```bibtex
{thing-vector-pools,
title = "@thi.ng/vector-pools",
author = "Karsten Schmidt",
note = "https://thi.ng/vector-pools",
year = 2018
}
```
## License
© 2018 - 2025 Karsten Schmidt // Apache License 2.0