A multi-sensory pilot program enabling immersive engagement for residents with sensory deprivation.
Residents with mid-to-late stage dementia often face sensory deprivation and profound isolation. The San Francisco Campus for Jewish Living (SFCJL) sought to create an “Experience Station” to provide meaningful stimulation. While stakeholders initially explored Virtual Reality, our research indicated that headsets could cause confusion or fear in this demographic. We followed a “Tangible Immersion” strategy — creating a station that served as a window to the world, grounded in multisensory stimulation.
Prototyping Connection
Bridging the digital-physical gap with “hacked” hardware.
Existing content was static and passive. We hypothesized that adding a tactile dimension – giving residents something to hold – would increase emotional grounding and immersion.
I “hacked” three plush cats, embedding them with pressure sensors and haptic vibration motors wired to an Arduino. Petting the toy triggered a “purr” vibration and played a synchronized video of that cat on the screen.
The cat petting prototype — a plush toy embedded with pressure sensors and haptic vibration motors.
Prototype build — plush toy, Arduino wiring, internal sensors, and stereo hardware staged at home
Hover over the cat to activate the experience
Early testing revealed a strong emotional response; residents instinctively tried to pick up and hold the animals. This validated the need for “Tangible Companionship” and informed future iterations to be wireless and robust for daily facility use.
Adapting Haptics for Accessibility
Decoupling technology from furniture to ensure universal access.
Haptic feedback is critical for sensory stimulation. However, the existing prototype was a platform placed under a chair, which was inaccessible to the majority of our residents who use wheelchairs.
I iterated on the hardware at home, aiming to further mobilize the haptics. I used an existing foot rest since it could adapt to different people and be moved with ease, and attached a strong haptic emitter to the back of it, enabling the experience of haptics without compromising on accessibility and quality.
By changing the form factor from a floor panel to a footrest, we ensured 100% of residents could access the experience without leaving their wheelchairs.
We paired this hardware with a Driving Simulator. I thought it would be great to give the residents a sense of control, by having them play a realistic driving game paired with a Logitech steering wheel with force feedback.
The driving setup paired a Logitech force-feedback wheel with POV driving footage and the haptic footrest.
Initial tests with a video game (Assetto Corsa) failed because it was fundamentally a commercial title, not something purpose-built for our audience. It had no guardrails, which led to confusion, and it was cumbersome to set up.
I pivoted to “Simulated Agency.” We synced high-quality POV driving footage with the Logitech force-feedback wheel and my haptic footrest.
Residents got the tactile satisfaction of steering and “feeling” the road rumble through their feet, without the risk of failure.
The Caregiver Interface
Transforming medical metadata into a session tool.
The station is controlled by facility staff who are often stretched thin. If the digital interface was difficult to configure, the physical station would sit unused. Feedback indicated the original system felt like a static medical database, lacking the flexibility needed for improvised care sessions.
The interface prioritizes “Session Flow” over “Data Entry,” empowering staff to personalize experiences in seconds.
I removed extraneous widgets and filtered the Information Architecture to prioritize “Engagement Tips” and simple session controls.
I introduced “Quick Add” features for improvised content (e.g., specific YouTube requests) and recommended playlists based on resident history.
I replaced buried metadata with prominent staff notes, ensuring critical preferences (e.g., “responds to socially interactive experiences”) were visible at a glance.
Experience content page with guidance for caregivers on how to run a session
194+
Recorded sessions at SFCJL
95%
Positive sentiment (33% Very Positive, 62% Positive)
6+
Major award recognitions
“The Experience Station helps bring residents back online. It soothes the parasympathetic nervous system and helps mitigate distressing behaviors.”
Manager of Life Enrichment, SFCJL
“The Experience Station is a way to bridge connections, foster relationships for new staff to home in on residents. As well as facilitation for family members with their desire for connection.”
Life Enrichment Coordinator, SFCJL
Recognition
CABHI (Centre for Aging + Brain Health Innovation) — 2x Award Recipient
Finalist: Fast Company 2022 World Changing Ideas (Experimental Category)
SCAN Foundation Innovation Award
Leading Age Innovation Award
Sephardic Foundation Grant Recipient
Invited for journal publication
Side Note: The haptic footrest prototype was so engaging to use for media consumption that I actually built a second unit to keep for myself.
Reflection
I love to work in a hands-on capacity: screens, toys, games, physical form, all of it. This was a unique chance to stretch my skills and have the independence to create impactful experiences from scratch, as challenging as that was. This project came with a lot of special considerations for our users, and it changed my perspective as a designer to try to always consider the needs of all populations, not just the most common.
