Sensors and calibration for an interactive experience

Depth: color, distance, and body are different data

A color image does not automatically provide depth or joints. Specify the required output: presence mask, distance, point cloud, or pose. Body tracking may need another library and resources beyond the camera.

The reviewed Orbbec TOP documentation identifies SDK v1 and warns newer cameras may require v2. Buying the same brand is insufficient. Kinect Azure uses a different route; evaluate Kinect generations against their specific hardware, SDK, and legacy support.

• Compatibility record: model/revision, firmware, SDK/runtime, node/plugin, build, OS, transport, and depth mode. Mark confirmed facts and assumptions.
• Zone: usable near/far limits, visible field, occlusions, lighting, and multiple people. Test with intended scenery and clothing.
• Operation: distinguish invalid data from absence, inform the operator, and return to idle on expiration.

Proposed rehearsal: one person, two crossing people, entry/exit, and a partly obstructed field. Record data loss positions and recovery. Confirm accuracy, body count, and useful range with the selected chain and its use conditions.

Depth, LiDAR, or radar: choose for the zone

Compare required data, not just advertised range. Depth cameras provide distance imagery by model; LiDAR scanners may measure in a plane; presence-radar results also depend on processing. Check whether the selected chain provides joint data that can drive the character.

Hokuyo publishes scanners such as UST-10/20LX; Infineon documents a BGT60TR13C presence solution. These are references for different technologies. Assess integration requirements and confirm equipment availability for the project.

• Draw detection field and placement: height, tilt, blind zones, and foreground objects. A scan plane does not cover an entire volume.
• Compare ambient light, reflections, materials, interference, and occlusions using the exact model manual. Do not extrapolate an indoor test outdoors.
• Request output format/rate, SDK/interface, OS support, and integration effort. Check whether output is presence, points, objects, or positions.

Proposed test: approach, dwell, two participants, stationary object, and exit. Record false triggers and missed detection. If the only need is choosing a message, a button may be clearer. This selection serves neither biometric identification nor surveillance.

Buttons, pressure, and contacts: clear input intent

A button communicates a choice; a contact mat may activate a zone; an analog pressure sensor also needs scaling and thresholds. Do not interpret an electrical value as body weight without appropriate measurement/calibration.

Phidgets distinguishes DigitalInput, VoltageInput, and VoltageRatioInput classes. Device and sensor must match type, supply, and limits. Similar connectors do not prove compatibility; the owner prepares integration using exact datasheets.

• Define press, release, hold, and repeat. Filter bounce to avoid several triggers from one action; tune the interval by testing.
• For pressure, use activation/release thresholds and review drift. Distinguish “not pressed” from disconnected cable/device.
• Place controls within reach, label their effect, and provide visual confirmation. Give the host an equivalent control.

Proposed fictional-data test: short activation, hold, bounce, disconnection, and restart. Then test the approved peripheral for intended use. Inputs control noncritical AV content; they do not modify mains electricity, structural loads, or emergency systems.

Hands, objects, and markers: three different controls

MediaPipe Hand Landmarker returns hand points; recognizing a useful gesture remains an application decision. OpenCV ArUco identifies dictionary markers. Markerless object tracking needs another method and validation; these inputs are not interchangeable.

• Hand: choose point/gesture, reachable area, confirmation time, and handling of another hand. A confidence score does not prove intent.
• Object: define target, lighting, background, and orientation. Test similar objects that should not trigger.
• Marker: fix dictionary, ID, physical size, placement, and readability. Pose needs geometry/calibration, not just an ID.

Proposed rehearsal: approach, rotate, partly cover, and remove hand/object/marker. Record false positives and recovery. Show a cursor or confirmation that explains detection without displaying audience imagery by default. Agree SDK/model, version, license, and integration before tool selection; offer equivalent manual selection.

Map sensor coordinates to the surface

1. Coordinate / unit
2. Defined transform
3. Boundary / valid zone
4. Canvas position

Draw axes, origin, units, direction, and valid area on both sides. Camera, 3D-world, and video-canvas coordinates are not interchangeable. A planar transform may suit a flat surface; a volume or irregular surface needs another model.

Fictional rectangle: for x from 0 to 2 m and a 1920-pixel canvas, u=x/2 and pixel_x=u×1919 for indices 0…1919. This illustrates scale only, not perspective, lens, or depth correction. Define the vertical axis and endpoints too.

• Use corner and interior reference points; check error across the zone, not only adjustment locations.
• Create a dead zone outside the useful area and constrain unexpected values. Missing detection must not become position zero.
• Save placement, parameters, version, and date; recalibrate if sensor, projector, or surface moves.

Proposed rehearsal: slow paths along edges/interior, intended height changes, and tracking loss. Compare cursor with authorized physical references and record deviation. Mathematical simulation does not calibrate a real space. The operator must be able to return to idle if mapping no longer aligns.