When a sign actually “counts” — a 350 ms hold, a 1.5 s wind-up, and speech-recognition lag

The time constants

Sign-recognition constants live in handSignInferenceConfig.ts, Rasengan’s trigger rules in rasenganTriggerController.ts, and Call Studio’s hand handling in useCallStudioHandSignTrigger.ts.

• Drop under 0.45 confidence / stable after the same label holds 350 ms / 500 ms between accepted signs / no new trigger for 2,500 ms after a cast
• JUTSU_CAST_DELAY_MS = 1,500 ms from completed signs to cast (originally 1,000 ms)
• Rasengan: wait 5,000 ms after the Tiger sign for the spoken word / also accept speech from up to 900 ms before confirmation / ignore Tiger for 800 ms after it ends
• Call Studio: send to Hands every 70 ms / hold the anchor 500 ms after losing the hand / no sign inference for 2,000 ms while only one hand is visible

The 350 ms hold: ignoring poses in transit

While the hands move from one sign to the next, the CNN can briefly return a different sign’s label with high confidence. Advancing on a single matching frame lets those in-transit poses scramble the order. So a label only becomes stable after staying above threshold for 350 ms, and any label change restarts the clock.

1let stable = false;2if (raw.confidence >= HAND_SIGN_CONFIDENCE_THRESHOLD) {          // 0.453  const tracker = stableTrackerRef.current;4  if (!tracker || tracker.label !== raw.label) {5    stableTrackerRef.current = { label: raw.label, since: now };  // ラベルが変わった→計測やり直し6  } else if (now - tracker.since >= HAND_SIGN_STABLE_DURATION_MS) { // 350ms7    stable = true;8  }9}

Since inference runs every 125 ms (Vol.4), a 350 ms hold means roughly “3–4 consecutive inferences agreeing”.

Where we got stuck: signs complete, but no jutsu

In May 2026, the sign circuit would show “complete” but Rasengan or Chidori never started. The cause was a stale React closure. The camera loop keeps calling the callback it registered once, so the jutsuCircuitMode it captured (which technique’s circuit is active) never updated. When the 1 s wind-up ended and the loop asked “which jutsu do I cast”, it read the stale value and did nothing.

1// 修正: 毎フレーム呼ばれるループからは ref 経由で最新値を読む2const jutsuCircuitModeRef = useRef(jutsuCircuitMode);3jutsuCircuitModeRef.current = jutsuCircuitMode;4 5// カメラループ内6const activeJutsuMode = jutsuCircuitModeRef.current;

Since then, the rule in this codebase is that any state read from MediaPipe’s onResults or a rAF loop lives in a ref. That is why, in the hook shape from Vol.1, the circuit and classifier are always accessed as circuitRef.current / classifierRef.current.

Stretching the wind-up from 1 s to 1.5 s

The wind-up from completed signs to the cast (JUTSU_CAST_DELAY_MS) was originally 1,000 ms. When Call Studio gained sign- and voice-triggered casting on 2026-09-23, Chidori got a 1.5 s wait, and the constant itself is now 1,500 ms. During the wind-up inference stops and the HUD shows the countdown (“casts in N s — inference paused”).

Timing with voice: recognition results arrive late

Rasengan fires on an AND condition: Tiger sign confirmed, then “Rasengan” spoken within 5 seconds. Requiring both prevents false casts, and within the 5 s window the user may release the sign before speaking.

What implementation taught us is that Web Speech API results arrive after the utterance. Start speaking while forming the sign and the result lands after Tiger is confirmed; speak a bit early and the result is discarded because Tiger is not confirmed yet. So speech that arrives before confirmation is kept as pendingVoiceAt for up to 900 ms and consumed the moment Tiger confirms.

Call Studio had a different problem: hand detection drops easily during capture. If no hand is visible at the moment of speech, the utterance is still accepted when a hand was seen within the previous 1,500 ms; if spoken with no hand at all, it is held for 2,000 ms waiting for the hand to return. The anchor is held 500 ms after losing the hand, so the effect never blinks out instantly.