marcuscedricridia/Fleet-Semantic-4B-v0.10

Fleet Semantic 4B v0.10

Fleet Semantic 4B v0.10 is an optimized FleetForCausalLM release built on Qwen/Qwen3-4B-Instruct-2507.

Fleet is a research Semantic Inference Language Model (SiLM) architecture: specialist LoRA transformations live inside one causal language model and are selected internally during inference.

What changed in v0.10

The learned model/expert tensors are inherited from noumenon-labs/Fleet-Semantic-4B. v0.10 changes the runtime architecture rather than retraining the checkpoint.

Always-on core optimizations:

• branchless dynamic k=0/1/2 routing;
• computationally sparse top-2 LoRA execution;
• batch-aware semantic routing;
• padding-aware batched semantic pooling;
• removal of repeated weight-prototype normalization in the generation hot path;
• dynamic .fleet install/uninstall/export;
• Accelerate CPU-offload compatibility.

Runtime modes

Exact

model.set_fleet_mode("exact")

Uses semantic routing plus Fleet's LoRA weight-space geometry and refreshes semantic routing during decoding. This is the default mode.

Balanced

model.set_fleet_mode("balanced")

Uses semantic-only expert selection while retaining semantic refresh during decoding. This removes per-projection weight-geometry scoring.

Fast

model.set_fleet_mode("fast")

Uses semantic-only routing and reuses the prompt semantic route throughout decoding.

Inspect the active behavior with:

print(model.fleet_speed_config())

Load

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

repo = "noumenon-labs/Fleet-Semantic-4B-v0.10"

tokenizer = AutoTokenizer.from_pretrained(repo)

model = AutoModelForCausalLM.from_pretrained(
    repo,
    trust_remote_code=True,
    dtype=torch.bfloat16 if torch.cuda.is_bf16_supported() else torch.float16,
    device_map="auto",
)

model.set_fleet_mode("exact")
print(model.experts())

Batched generation

v0.10 removes Fleet v0.9's batch-size-1 semantic-routing restriction.

tokenizer.padding_side = "left"
if tokenizer.pad_token_id is None:
    tokenizer.pad_token = tokenizer.eos_token

messages = [
    [{"role": "user", "content": "Solve 3x + 4 = 19."}],
    [{"role": "user", "content": "Explain how insulin lowers blood glucose."}],
]

texts = [
    tokenizer.apply_chat_template(
        m,
        tokenize=False,
        add_generation_prompt=True,
    )
    for m in messages
]

inputs = tokenizer(
    texts,
    padding=True,
    return_tensors="pt",
).to(model.get_input_embeddings().weight.device)

output = model.generate(
    **inputs,
    max_new_tokens=64,
    do_sample=False,
)

Installed experts

• math_reasoning
• medical_reasoning
• legal_ops
• intent
• function_calling
• structured

Dynamic experts

model.experts()
model.install("robotics.fleet")
model.uninstall("robotics")
model.export_adapter("structured", "structured.fleet")

Normal generation does not require an expert= argument. Routing stays internal to FleetForCausalLM.

Validation

The release notebook requires the following before upload:

• standalone AutoModelForCausalLM loading;
• FleetForCausalLM architecture;
• batch-size-2 generation;
• exact, balanced, and fast mode generation;
• .fleet export;
• expert uninstall;
• expert reinstall;
• restored expert registry.

See fleet_v010_validation.json in this repository after release.

Status

Fleet remains a research prototype. The v0.10 runtime optimizations should be benchmarked for both task quality and latency before treating balanced or fast as replacements for exact.

Large-scale fused kernels, vLLM/Punica integration, and continuous batching are intentionally outside this release. v0.10 focuses on the model architecture itself.

Architecture: FleetForCausalLM
Backbone: Qwen/Qwen3-4B-Instruct-2507
Fleet version: 0.10.0
Default runtime mode: exact