@needle-tools/facefilter
Version:
Needle Engine FaceFilter
156 lines (116 loc) • 6.87 kB
Markdown
Add face tracking to your Needle Engine projects with ease.
- Blendshape mesh face tracking
- Texture face tracking with google/mediapipe canonical or procreate texture layouts (Use the `FaceMeshTexture` class)
- Video face tracking: Play a video as a face texture (Use the `FaceMeshVideo` class)
- Custom shader face meshes: Use custom materials on your face mesh (Use the `FaceMeshCustomShader` class)
- Tracking for multiple faces at once (with smoothing)
- *Can be used with Unity to create filters, animations, materials...*
## Quickstart
Run `npm i @needle-tools/facefilter` in your web project
Then see the code or examples below:
### Face Filter with HTML only
See full examples in `/examples/html/`
#### Supported attributes
| | |
| -- | -- |
| `face-filter` | URL to either a image or model file. Supported formats: `glTF`, `GLB`, `FBX`, `OBJ` **or 2D** `jpeg`, `jpg`, `png`, `webp` |
| `face-filter-mask` | (optional, 2D only) URL to image file that will be used to mask out the face filter texture
| `face-filter-layout` | (optional, 2D only) Either `procreate`, `mediapipe` or `canonical`. Default `mediapipe`
| `face-filter-scale` | (optional, 3D only) Apply scale to the 3D face filter model (e.g. `face-filter-scale=".5"`)
| `face-filter-offset` | (optional, 3D only) Offset the 3D face filter model (e.g. `face-filter-offset="0.0, 0.1, 0.1"`)
| `face-filter-max-faces` | (optional) How many faces should be tracked automatically. Default: `1`
| `face-filter-show-video` | (optional) Should the camera videofeed be rendered in the background? Default: `true`. Can be set to `0` to hide the videofeed in the background.
| `face-filter-video-selector` | (optional) HTML selector for a HTMLVideoElement. Useful if you want to render provide your own video element on the website elsewhere. If none is provided a hidden video element will be created automatically.
#### HTML Example
```html
<!DOCTYPE html>
<html>
<head>
<script type="importmap">
{
"imports": {
"three": "https://cdn.jsdelivr.net/npm/@needle-tools/engine@4.4.0-alpha.5/dist/three.min.js",
"@needle-tools/engine": "https://cdn.jsdelivr.net/npm/@needle-tools/engine@4.4.0-alpha.5/dist/needle-engine.min.js",
"@needle-tools/facefilter": "https://cdn.jsdelivr.net/npm/@needle-tools/facefilter/dist/facefilter.min.js"
}
}
</script>
<script type="module" src="https://cdn.jsdelivr.net/npm/@needle-tools/facefilter/dist/facefilter.min.js"></script>
</head>
<body style="margin:0; padding:0;">
<needle-engine
background-color="#ffffdd"
face-filter="https://cdn.needle.tools/static/facefilter/facemask-template-procreate.webp"
face-filter-mask="https://cdn.needle.tools/static/facefilter/facemask-occlusion-procreate.webp"
face-filter-layout="procreate"
>
</needle-engine>
</body>
</html>
```
[](https://stackblitz.com/edit/needle-engine-facefilter-html-only?file=index.html) –
[](https://stackblitz.com/edit/needle-engine-facefilter-html-only-3d?file=index.html)
```ts
import { onStart } from '@needle-tools/engine';
import { FaceMeshTexture, NeedleFaceFilterTrackingManager } from '@needle-tools/facefilter';
onStart(context => {
const scene = context.scene;
// Create a face filter tracking manager and add it to the scene
const filtermanager = new NeedleFaceFilterTrackingManager();
filtermanager.createMenuButton = true;
scene.addComponent(filtermanager);
// Creating a filter
const filter = new FaceMeshTexture({
layout: 'procreate', // we support both the google/mediapipe canonical layout and procreate/arkit layouts
texture: {
url: './assets/crocodile.webp', // provide a URL to the texture
// texture: <your texture> // alternatively you can assign an existing texture directly
},
});
// Activate one of your filters
filtermanager.activateFilter(filter);
});
```
[](https://stackblitz.com/edit/needle-engine-facefilter)
```ts
import { onStart } from '@needle-tools/engine';
import { FaceMeshTexture, NeedleFaceFilterTrackingManager } from '@needle-tools/facefilter';
onStart(context => {
const scene = context.scene;
// Create a face filter tracking manager and add it to the scene
const filtermanager = new NeedleFaceFilterTrackingManager();
filtermanager.createMenuButton = false;
scene.addComponent(filtermanager);
// Creating a filter using a GLB/glTF URL model that has blendshapes
const filter = await FaceFilterRoot.create('https://cloud.needle.tools/-/assets/Z23hmXBZWllze-ZWllze/file', {
scale: 0.5,
offset: { x: 0, y: 0.01, z: 0 },
});
if (filter) filtermanager.activateFilter(filter);
});
```
[](https://stackblitz.com/edit/needle-engine-facefilter-blendshapes?file=src%2Fmain.ts)
- [Full project & source on Github](https://github.com/needle-engine/facefilter)
- [Example with Blendshapes on Stackblitz](https://stackblitz.com/edit/needle-engine-facefilter-blendshapes?file=src%2Fmain.ts)
- [Example with Sunglasses on Stackblitz](https://stackblitz.com/edit/needle-engine-facefilter-glasses?file=src%2Fmain.ts)
- [Example with Texture Mesh on Stackblitz](https://stackblitz.com/edit/needle-engine-facefilter)
- [Example with ShaderToy on Stackblitz](https://stackblitz.com/edit/needle-engine-shadertoy-facefilter)
- [Example without a bundler on Stackblitz](https://stackblitz.com/edit/needle-engine-facefilter-html?file=index.html)
- [Example with HTML only 2D filter on Stackblitz](https://stackblitz.com/edit/needle-engine-facefilter-html-only?file=index.html) ([or github](https://github.com/needle-engine/facefilter/blob/main/package/examples/html/index.html))
- [Example with HTML only 3D filter on Stackblitz](https://stackblitz.com/edit/needle-engine-facefilter-html-only-3d?file=index.html) ([or github](https://github.com/needle-engine/facefilter/blob/main/package/examples/html/model.html))
- [Demo Video](https://github.com/user-attachments/assets/51300430-6290-4672-b2aa-f1e870b9e99c)
See [Github](https://github.com/needle-engine/facefilter) for more information
Source files are 75 kB (gzip).
This package contains files for the Unity integration and are not included in web builds.
<b>[needle](https://needle.tools)</b> •
[](https://twitter.com/NeedleTools) •
[](https://forum.needle.tools) •
[](https://www.youtube.com/@needle-tools)