3dcodeverse/3dcodeverse-qwen3.6-27b-lora

3DCodeVerse · Qwen3.6-27B LoRA

A LoRA adapter for Qwen/Qwen3.6-27B, fine-tuned on the 3DCodeVerse corpus to turn a text description of a 3D object, scene, or shader into runnable 3D code: Blender Python (bpy), CadQuery, OpenSCAD, GLSL, and three.js.

This repository holds only the adapter (≈467 MB). The base model is downloaded from its own repository when you load the adapter; it is not redistributed here.

The sibling adapter for Qwen3.8-27B is 3dcodeverse/3dcodeverse-qwen3.8-27b-lora.

Model details

Base model	Qwen/Qwen3.6-27B
Method	LoRA, rank 16, alpha 32, dropout 0
Target modules	attention (q/k/v/o_proj), linear attention (in_proj_qkv, in_proj_a/b/z, out_proj), MLP (gate/up/down_proj)
Framework	LLaMA-Factory (unmodified), PEFT 0.18.1
Optimizer	AdamW, peak LR 1e-4, cosine schedule, 3% warmup
Training data	3DCodeVerse text-to-code SFT mix (3dcv_text_all, ShareGPT format)
Steps	5,515 (1 epoch)
Global batch size	48 (8 GPUs)
Final loss	train 0.239 · held-out eval 0.252

Files

file	contents
adapter_model.safetensors	LoRA weights
adapter_config.json	PEFT config; base_model_name_or_path points at Qwen/Qwen3.6-27B
trainer_log.jsonl	per-step training log (loss, learning rate, eval loss)

Usage

Prompt format

The adapter was trained on single-turn ShareGPT records with a system prompt that fixes the target dialect. Example (Blender):

system: You are an expert in procedural 3D modeling with Blender Python (bpy). Given a description of an
        object, write a complete, standalone Blender 5 Python script that builds it from scratch (clear the
        default scene first). Output ONLY the code in one ```python block.
user:   Create the 3D object of an agave succulent plant using Python Blender code, featuring a rosette
        arrangement of thick, fleshy, pointed leaves that emerge from a central base ...

The answer is one fenced code block. Thinking mode was disabled during training (enable_thinking: false).

Load with PEFT

import torch
from transformers import AutoModelForImageTextToText, AutoTokenizer
from peft import PeftModel

# the base checkpoint is Qwen3_5ForConditionalGeneration; the adapter only touches its language model
base = AutoModelForImageTextToText.from_pretrained("Qwen/Qwen3.6-27B", torch_dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(base, "3dcodeverse/3dcodeverse-qwen3.6-27b-lora")
tok = AutoTokenizer.from_pretrained("Qwen/Qwen3.6-27B")

messages = [
    {"role": "system", "content": "You are an expert in procedural 3D modeling with Blender Python (bpy). "
                                  "Given a description of an object, write a complete, standalone Blender 5 Python "
                                  "script that builds it from scratch (clear the default scene first). "
                                  "Output ONLY the code in one ```python block."},
    {"role": "user", "content": "Create the 3D object of a low wooden stool with three splayed legs using Python Blender code."},
]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, enable_thinking=False,
                                 return_tensors="pt").to(model.device)
out = model.generate(inputs, max_new_tokens=4096, do_sample=True, temperature=0.7, top_p=0.95)
print(tok.decode(out[0, inputs.shape[1]:], skip_special_tokens=True))

Merge, then serve with vLLM

This is how the 3DCodeVerse evaluation runs these models. Merge the adapter with LLaMA-Factory:

# export.yaml  →  llamafactory-cli export export.yaml
model_name_or_path: Qwen/Qwen3.6-27B
adapter_name_or_path: 3dcodeverse/3dcodeverse-qwen3.6-27b-lora
template: qwen3_5
enable_thinking: false
trust_remote_code: true
finetuning_type: lora
export_dir: ./3dcodeverse-qwen3.6-27b-merged
export_size: 5
export_device: cpu
export_legacy_format: false

then serve the merged directory with vLLM (e.g. vllm serve ./3dcodeverse-qwen3.6-27b-merged --tensor-parallel-size 2). The base model has linear-attention layers, so each concurrent sequence needs a Mamba cache block: if the engine fails to start, lower --max-num-seqs (e.g. 128).

Evaluation

Generated code should be judged by running it (Blender, CadQuery, OpenSCAD, glslangValidator, headless WebGL), not by text similarity. The evaluator lives in eval/llm/ of the 3DCodeVerse repository.

Limitations

• Greedy decoding can fall into repetition loops on boilerplate-heavy dialects (OpenSCAD, GLSL); sampling (temperature ≈ 0.7) is markedly more reliable.
• OpenSCAD and three.js are rare in the training corpus (about 1.2k and 340 distinct programs), so they are weaker than Blender, CadQuery, and GLSL.
• Generated code is not guaranteed to execute. Run it in a sandbox.

License

The adapter is released under CC BY-NC-SA 4.0, matching the 3DCodeVerse training data, and is for non-commercial research use. Use of the base model is also governed by the Qwen/Qwen3.6-27B license (Apache 2.0), and the per-source licenses of the training data apply.

Citation

@misc{gao2026_3dcodeverse,
  title  = {{3DCodeVerse}: Exploring {3D} Worlds as Code},
  author = {Gao, Yipeng and Zeng, Ziyao and Huang, Jinfa and Li, Yijiang and Xiong, Haolin and
            Qin, Wang and Yu, Xinlei and Yao, Youheng and Zhang, Jie and Lin, Kun-Yu and
            Liu, Rong and Zhao, Yinyuan and Zhao, Yajie and Guo, Meiqi and Luo, Jiebo and
            Shu, Lei and Ge, Yunhao and Hu, Xinyu and Itti, Laurent},
  year   = {2026},
  eprint = {2610.22341},
  archivePrefix = {arXiv},
  primaryClass = {cs.CV},
  doi    = {10.48550/arXiv.2610.22341},
  url    = {https://arxiv.org/abs/2610.22341}
}