Layering Three.js on the camera feed — an ortho camera, root.scale = 17, and bloom resolution

How coordinates flow

MediaPipe landmarks are normalized 0–1 coordinates of the video. The on-screen canvas draws that video with object-fit cover / contain, so video coordinates and canvas coordinates differ — if cover crops the top and bottom, they are off by that much. So mapMediaPipePointToFitCanvas() first converts to a canvas position, which is then normalized back to 0–1 of the canvas before going to the 3D layer.

The 3D layer uses one orthographic camera shared by every layer (height 4.8), with width derived from the canvas aspect ratio. We avoid perspective because alignment with the hand is already resolved in 2D; size changing with depth would break the “stuck to the hand” look.

1const VIEW_HEIGHT = 4.8;            // 全 3D レイヤー共通2const RASENGAN_ROOT_SCALE = 17;     // 見かけの大きさはここだけで決める3const ANCHOR_DAMP_LAMBDA = 18;      // 手ブレの指数減衰4 5const viewW = VIEW_HEIGHT * (canvasW / canvasH);6targetPos.set((anchor.x - 0.5) * viewW, (0.5 - anchor.y) * VIEW_HEIGHT, 0);7vfx.root.position.x = THREE.MathUtils.damp(vfx.root.position.x, targetPos.x, ANCHOR_DAMP_LAMBDA, deltaSec);8vfx.root.position.y = THREE.MathUtils.damp(vfx.root.position.y, targetPos.y, ANCHOR_DAMP_LAMBDA, deltaSec);

damp is exponential decay over elapsed time, so it follows at the same speed at 30 fps or 120 fps. Call Studio can render above 120 fps (per a comment in useCallStudioHandSignTrigger); a per-frame lerp would make tracking speed vary by device.

Size only changes through root.scale

Rasengan’s VFX itself uses the same parameters as the standalone /dev/rasengan page; the AR layer just multiplies root.scale = 17. We do not rewrite geometry or particle layout for AR so that the look tuned on the standalone page — bloom response, particle density — carries over unchanged. When Call Studio wanted the effect 1.3× bigger (2026-09-23), only the scale changed.

Bloom is a screen-space pass, though, so changing scale also changes how the glow spreads on screen. “The VFX is untouched” does not mean the result looks identical.

Bloom resolution: a different policy per jutsu

Where we got stuck: the table’s values were never applied

On 2026-09-22 we added pixelRatio to BLOOM_BY_QUALITY to make Chidori’s bloom cheaper. But EffectComposer never received setPixelRatio, so bloom kept running at full resolution even on low / standard. The next commit wired it up: “the table already carried pixelRatio; wire it into EffectComposer so low/standard paths actually run cheaper bloom”.

1const BLOOM_BY_QUALITY = {2  low:      { strength: 0.42, radius: 0.26, threshold: 0.58, pixelRatio: 0.5 },3  standard: { strength: 0.55, radius: 0.3,  threshold: 0.55, pixelRatio: 0.65 },4  high:     { strength: 0.72, radius: 0.32, threshold: 0.55, pixelRatio: 0.85 },5  ultra:    { strength: 0.85, radius: 0.38, threshold: 0.48, pixelRatio: 1 }6};7 8const bloomConfig = BLOOM_BY_QUALITY[quality];9const composer = new EffectComposer(renderer, composerTarget);10composer.setPixelRatio(bloomConfig.pixelRatio); // ← これが抜けていた

The same day, the standalone Chidori test page also capped the renderer’s pixelRatio at 1.5 instead of 2 and ran the composer at 0.5.