“Your 👃 Knows” is a web app that lets users interact using their nose. It utilizes the webcam to track nose movements, and allows people who may not have easy use of their hands to perform basic web interactions such as click, hover, drag, and drop.

Technology

This app was made with the creatability.js library. With this technology, the web experience could be made more inclusive to those with mobility issues.

My Role

It was a team of 2 people, an engineer and me. We collaborated on early concepts, I provided creative direction, designed the UI and interaction, developed front-end prototypes.

How to use the demo

• In the sidebar, select ‘Body’ instead of ‘Custom Mouse/Keyboard’.
• Be sure that you’re sitting directly in front of your webcam so that it has a good view of your face.
• Allow the browser to use the webcam.
• Wait a few seconds.
• Click the “X” at the top right of the sidebar to collapse it.
• You should see a nose-shaped cursor on the screen which responds to the movements of your head. Follow the prompts to use the app!

Design & creaative front-end: Lauren Wang

Engineering: Michael Newton

Made for Amplify Hackathon 2019