sense pack · second vision head

k3 vision

a second, semantics-forward vision tower projected into the fusion embedding space.

family fusion embedding 2·pack moonvit-v2·output 2048-d·license open weights

k3 vision adds a second way of seeing. a moonvit-v2 vision tower, from the kimi-k3 lineage, is projected into the fe2 space to sit alongside the base's own native vision path.

it is a semantics-forward vision head, a different lens on images that shares the same coordinate system as everything else in the family, so its results are interchangeable with the native path at query time.

how it works

the moonvit-v2 tower is projected into the fe2 space and runs as a second vision head next to the base's native vision. both land in the same shared space, so results from either are comparable and interchangeable at query time.

k3 vision architecture: an image passes through a frozen moonvit-v2 tower and a trained projector into the shared 2048-d space. — architecture: frozen components in grey, the trained / new path in terracotta, all reading out into one shared 2048-d space.

training data

the moonvit-v2 tower is frozen; only the projection into the fe2 space is trained. the second head therefore costs the base nothing and inherits the tower's semantics-forward representation.

results

under the family's canonical readout, flickr30k text→image retrieval reaches R@1 of about 0.346. it is a second head, a complement to the native vision path rather than a claim to beat it everywhere, and the figure is one retrieval metric under that readout.

image retrieval, canonical readout

benchmark	score
flickr30k text→image R@1	≈0.346

honest limitations

—it is a second head rather than a replacement, a complement to the native vision path, not a claim to beat it everywhere.
—the reported figure is one retrieval metric under the canonical readout.

usage

import torch
from transformers import AutoModel

model = AutoModel.from_pretrained(
    "EximiusLabs/fusion-embedding-2-k3-vision",
    trust_remote_code=True,
).eval()

# the second vision head writes the same 2048-d vectors as the base
image = model.embed_image("photo.jpg")
query = model.embed_text("a child on a red bicycle")

similarity = (image * query).sum()

citation

@software{eximius_k3_vision_2026,
  author = {Eximius Labs},
  title  = {K3 Vision: a second semantics-forward vision head
            for the Fusion Embedding 2 space},
  year   = {2026},
  url    = {https://huggingface.co/EximiusLabs/fusion-embedding-2-k3-vision},
}

how it fits the family

k3 vision shows the shared space can hold more than one head per modality: two vision towers, one coordinate system.

downstream, including engram, nothing changes to consume it, because it writes the same vectors.

← all models hugging face