sense pack · touch / tactile
tactus
pressure from tactile arrays embedded into the shared space, so a grasp can be recognised with a text query.
tactus gives the family a sense of touch. it reads 32x32 pressure arrays, the kind produced by fsr gloves, e-skins, and robot hands, and places each grasp in the same 2048-dimensional space fusion embedding 2 uses for text, image, video, and audio.
because the grasp lands in the text space, recognition is open-vocabulary: you ask "a mug" or "scissors" in plain language instead of picking from a fixed label set. the pack trains 13.5 million parameters against a frozen 2b base.
how it works
a grasp window of 32x32 taxel frames first passes through a sensor calibration affine that maps raw sensor counts into a common pressure range. a trained resnet-18-width cnn trunk encodes each frame, a fusion layer combines the frames of the window, and a projector maps the result into fe2's 2048-d space, where cosine against a text query does the recognition.
the trunk is mae-pretrained on 144k unlabeled pressure frames from the same sensor before any contrastive training. that same-sensor pretraining is worth about seven points of top-1 on its own.
training data
trained on stag (nature 2019), a 32x32 tactile glove dataset of 27 household objects. the wins came from the data path rather than the architecture: the sensor calibration affine, cluster-based sampling of training frames, and same-sensor mae pretraining each moved the score, while bigger modelling changes did not.
the negative result is part of the story: pooling additional tactile corpora from other sensors gave zero improvement, consistent with what htt and tacverse report. tactile data does not yet transfer across sensor types the way images transfer across cameras, so the pack is trained per sensor family. stag's non-commercial terms make the pack cc-by-nc-4.0, research preview.
results
on the stag benchmark (27 objects, held-out recordings) tactus reaches 0.771 ± 0.062 top-1 as the mean over four runs, with a best run of 0.829 and top-3 of 0.935, against 0.037 chance. the stag paper's own supervised cnn, a closed-set classifier trained for exactly these 27 labels, reports 0.76: tactus matches it while remaining open-vocabulary, answering free-form text queries rather than choosing among fixed classes.
stag 27-object recognition, held-out recordings
| measure | score |
|---|---|
| top-1, mean over four runs | 0.771 ± 0.062 |
| top-1, best run | 0.829 |
| top-3, mean | 0.935 |
| stag supervised cnn (closed-set) | 0.76 |
| chance | 0.037 |
limitations
- —evaluated on one benchmark and one sensor: stag's 32x32 glove. the scores say nothing yet about other tactile hardware.
- —cross-sensor pooling gave zero improvement, so a trunk is trained per sensor family rather than one trunk for all touch.
- —trained on stag, which is non-commercial: the pack ships as a cc-by-nc-4.0 research preview.
usage
import numpy as np
from inference import TactusEmbedder
ta = TactusEmbedder.from_pretrained(
"EximiusLabs/fusion-embedding-2-tactus",
revision="v0.1-preview",
)
# a grasp window: [F, 32, 32] pressure frames
window = np.load("grasp.npy")
for text, score in ta.rank(window, ["a mug", "scissors", "an empty hand"]):
print(f"{score:+.3f} {text}")
# or embed both sides into the shared 2048-d space directly
p = ta.embed_pressure(window)
t = ta.embed_text(["holding a mug"])citation
@software{eximius_tactus_2026,
author = {Eximius Labs},
title = {Tactus: a tactile sense pack for the
Fusion Embedding 2 space},
year = {2026},
url = {https://huggingface.co/EximiusLabs/fusion-embedding-2-tactus},
}how it fits the family
tactus is the touch sense pack on the fe2 base: a small trained trunk and projector, the base untouched, the same shared space.
pressure embeddings land beside fe2's text, image, video, and audio and tremor's motion, so a grasp becomes one more thing a cross-modal memory like engram can index and search.
you can try it directly in the playground: real held-out stag grasps, ranked live by the released model's embeddings.
related models