sense pack · motion / imu

tremor

accelerometer and imu motion projected into the fusion embedding space, searchable in plain language.

family fusion embedding 2·pack unimts st-gcn·output 2048-d·license cc-by-nc

tremor brings body and device motion into the family. it reads accelerometer / imu signal and places it in the same text space fusion embedding 2 uses, so motion becomes something you can query with words.

only a small projector is trained; the motion encoder itself is frozen. it is a deliberately light attachment onto the shared space, and on held-out datasets it never trained on, it recognises activity at roughly three times chance.

how it works

a frozen unimts st-gcn accelerometer encoder produces motion features. a small trained projector maps those features into fe2's text space. the encoder does not move; only the projector learns.

a co-trained variant adds a motion↔video objective on synchronized egocentric data, teaching the projector to line motion up with what the camera saw at the same instant.

tremor architecture: an accelerometer window passes through a frozen unimts st-gcn encoder and a trained projector into the shared 2048-d space. — architecture: frozen components in grey, the trained / new path in terracotta, all reading out into one shared 2048-d space.

training data

only the projector is trained; the unimts encoder stays frozen, so the pack inherits its motion representation rather than learning one from scratch.

the co-trained variant adds a motion↔video objective on synchronized egocentric recordings. the encoder's unimts / amass lineage makes the pack cc-by-nc, non-commercial use only.

results

held-out zero-shot five-way activity recognition averages about 0.60 across datasets it never trained on (chance is 0.20). the co-trained variant lifts fine motion↔language matching from below chance to roughly 2.4× chance with no loss on the activity gate, and on a real unitree humanoid a "walking" query reaches average precision around 0.70. the strength is coarse motion (moving versus still, locomotion versus manipulation) while fine-grained activity semantics are still developing.

held-out zero-shot 5-way activity recognition (chance 0.20)

dataset	accuracy
realworld	0.74
usc-had	0.66
dsads	0.67
utd-mhad	0.32
mean	≈0.60
unitree humanoid "walking" (AP)	≈0.70

honest limitations

—coarse motion is solid; fine activity semantics are still weak, as the spread across datasets shows.
—the encoder is frozen, so the head inherits the motion representation it was given rather than learning its own.
—licensed CC-BY-NC through its unimts / amass lineage, non-commercial use only.

usage

import torch
from transformers import AutoModel

model = AutoModel.from_pretrained(
    "EximiusLabs/fusion-embedding-2-tremor",
    trust_remote_code=True,
).eval()

# an accelerometer / imu window lands in the same 2048-d text space
motion = model.embed_motion("imu_window.npy")   # (T, channels)
query  = model.embed_text("walking across the room")

similarity = (motion * query).sum()

citation

@software{eximius_tremor_2026,
  author = {Eximius Labs},
  title  = {Tremor: an IMU / motion sense pack for the
            Fusion Embedding 2 space},
  year   = {2026},
  url    = {https://huggingface.co/EximiusLabs/fusion-embedding-2-tremor},
}

how it fits the family

tremor is the motion sense pack on the fe2 base: same shared space, a small projector, no retraining of the anchor.

its co-trained head is the motion path engram uses, which is what lets "how it moved" be a real query alongside "what it saw".

← all models hugging face