OSMO: Open-vocabulary Self-eMOtion Tracking
OSMO introduces egocentric self-emotion tracking with a large-scale dataset, a multi-task benchmark, and OSIRIS, a multimodal model for coherent emotion timelines.
Selected Projects
I enjoy making things. Here are a selection of projects that I have worked on over the years.
OSKAR: Omnimodal Self-supervised Knowledge Abstraction and Representation
OSKAR is a self-supervised multimodal foundation model that learns in the latent space by predicting masked multimodal features.
MaskCLR: Attention-Guided Contrastive Learning for Robust Action Representation Learning
MaskCLR improves the robustness of transformer-based action recognition methods against noisy and incomplete skeletons.
S-JEPA: Joint Embedding Predictive Architecture for Self-Supervised Skeletal Action Recognition
S-JEPA is an instantiation of JEPA for self-supervised skeletal action recognition.
ZeroWaste Dataset: Towards Deformable Object Segmentation in Cluttered Scenes
We take a step towards computer-aided waste detection and present the first in-the-wild industrial-grade waste detection and segmentation dataset, ZeroWaste.