The camera is not just “open it and it works” — Chrome’s 300×150, Center Stage, and OBS virtual cameras

Baseline: the camera waits for a user gesture

useUserMediaCamera does not open the camera on mount. getUserMedia only runs once the caller invokes startCamera(). The hook’s own header comment says “Required for iPhone Safari permission UX” — the point is to keep the permission prompt tied to a user gesture on iPhone Safari. The video element always gets muted and playsInline.

Phase is one of five values: idle / requesting / live / denied / error. Only NotAllowedError (and the older PermissionDeniedError) maps to denied, which shows copy telling the user exactly what to do next: allow the camera in browser settings. Every other failure is error and surfaces the underlying message. Once live, camera_granted is tracked — only when the experience has an ID.

Chrome: the track starts at 300×150

On Chrome, the video track could get cropped and scaled to the <video> element’s layout box. Right after the element is inserted it has no real size yet, so the first sink is the default 300×150 placeholder — and the frames MediaPipe sees get dragged down to that size.

Two fixes. Add resizeMode: 'none' to the constraints to refuse browser-side scaling, and pin the video’s width / height attributes and CSS size to 1280×720 before attaching srcObject. Once the stream arrives, pinHandsVideoIntrinsicSize() re-pins to the real videoWidth × videoHeight.

1export const CAMERA_IDEAL_WIDTH = 1280;2export const CAMERA_IDEAL_HEIGHT = 720;3 4export function videoTrackConstraints(facingMode: 'user' | 'environment') {5  return {6    facingMode,7    width: { ideal: CAMERA_IDEAL_WIDTH },8    height: { ideal: CAMERA_IDEAL_HEIGHT },9    frameRate: { ideal: 30 },10    // Chrome otherwise crop-and-scales the track to the <video> CSS box.11    resizeMode: 'none'12  } as MediaTrackConstraints;13}14 15// useUserMediaCamera: attach の前に 1280×720 を固定する16video.width = CAMERA_IDEAL_WIDTH;17video.height = CAMERA_IDEAL_HEIGHT;18video.style.objectFit = 'fill';19const stream = await getPreferredUserMediaStream(facingMode);

An earlier version of this article said touch devices request 1280×720 and others 720×560. Current code requests ideal 1280×720 at 30 fps on every device.

Mac: Center Stage pushes the hand signs out of frame

Newer Mac FaceTime cameras enable Center Stage (PTZ auto-framing) by default. It zooms toward the face, so hand signs formed at chest height either leave the frame or shrink too much inside the crop we hand to the CNN. Correct behavior for a camera that films people — exactly wrong for an app that needs the hands.

disableCameraAutoFraming() checks getCapabilities() on the track; if zoom / pan / tilt exist, it pins zoom to its minimum (widest field of view) and pan / tilt to zero. Cameras and browsers without PTZ simply skip it.

Skipping the OBS virtual camera and reopening the real one

Call Studio is the screen OBS captures. But on a Mac with OBS Virtual Camera installed, Chrome sometimes prefers the previously selected device (including OBS) over facingMode, and we got OBS’s placeholder image instead of the camera.

Device labels are only readable after permission, so getPreferredUserMediaStream() calls getUserMedia once normally, then enumerateDevices(), and drops devices whose label matches obs / virtual cam / snap camera / mmhmm / droidcam and friends. For the front camera it prefers built-ins (facetime / built-in / continuity); if that differs from what is open, it reopens with deviceId: exact.