Incorporating Incorporating gestures as a tool for interaction gestures as a tool for interaction using Sensecraft and TinyML

I recently had the honor of being invited by the MOME Robotics Studio in Budapest to give a lecture in Professor Renáta Dezső’s course focused on the use of (Italian) gestures inspired by Bruno Munari’s 1963 “Supplement to the Italian Dictionary.”

In fact, the goal of my lesson for the students was to train a model using Sensecraft with Xiao ESP32S3 Sense modules. Once this was done, the data would be sent to Touch Designer—a software widely used by students to create installations—via Node-RED. I made a YouTube video and saved everything in a GitHub repository so you can recreate the entire experience.

That same afternoon, I was invited to give a second lecture: this time at a major company focused on innovation and UX: Exalt Interactive. Here, I decided to change the final part of the workshop: instead of controlling Touch Designer via OSC, I would show Tamas Fogarasy’s team how to control the behavior of a Philips HUE light bulb. I recreated a demonstration of the experiment in 34 seconds, and here’s a short GIF

It was a wonderful experience that prompted me to revisit topics such as local computer vision, tiny machine learning, and, above all, the creation of sentient objects. Soon I’ll be adding more tutorials in this area using more powerful tools! Bye!