Building Multimodal Interfaces - My Time at the ELICIT Lab
From June 2022 to December 2023, I worked as an Undergraduate Research Assistant at the ELICIT Lab at the University of Illinois Chicago. It was here—tucked between lab hours, pilot studies, and long debugging sessions—that I discovered how much I love building experiences that feel natural to people.
Our project focused on multimodal user interactions—specifically how speech and gesture data could be used to train intelligent machine learning systems. I designed and built a custom mobile application using Flutter to log user speech and gesture input in real time. The app helped us correlate these two modes of interaction with precise timestamps and interaction labels.
To take things further, I integrated Google Cloud’s Speech-to-Text service into our workflow to ensure accurate transcriptions and sync points. Combining this with video logging and gesture tracking allowed us to gather rich datasets for our study participants.
I also led and conducted user studies—observing how participants interacted with the interface, iterating on the UI, and finding ways to make the experience feel less like testing and more like intuitive creative play.
🏆 3rd Place at the Undergraduate Research Forum
In April 2023, I presented my work at UIC’s Undergraduate Research Forum with a project titled
”Designing Speech and Gesture Interactions for Image Editing Mobile Applications.”
To my surprise and excitement, I was awarded 3rd place in the Engineering and Physical Science Category.
It was a moment I won’t forget—not just for the recognition, but for getting to stand beside so many other undergrads doing meaningful research across disciplines. I felt proud to represent the work we’d been doing behind the scenes.
I’m deeply grateful to my research advisor, Dr. Nikita Soni, for her mentorship throughout the project. Her guidance helped me approach research not just as a technical pursuit, but as a thoughtful, human-centered process.
This experience wasn’t just about learning how to build software. It was about listening—carefully—to how people want to interact. It taught me how to translate that into interfaces that feel natural, and data that can power smarter systems. And that mindset has shaped everything I’ve built since.