maxwelljones14/refVFX-lora

RefVFX LoRA — Tuning-free Visual Effect Transfer across Videos

This is a LoRA adapter for Wan-AI/Wan2.1-FLF2V-14B-720P, trained for the RefVFX project on tuning-free visual effect transfer across videos.

Note: This is an unofficial reimplementation produced at CMU. All code and training data were created from scratch using the publicly available arXiv paper and AI coding tools as the only resources.

Links

• 📄 Paper: Tuning-free Visual Effect Transfer across Videos (arXiv:2601.07833)
• 🌐 Project page: https://snap-research.github.io/RefVFX/
• 💻 Code (GitHub): https://github.com/maxwelljones14/refVFX
• 🗂️ Training dataset: maxwelljones14/refVFX_dataset
• 🧱 Base model: Wan-AI/Wan2.1-FLF2V-14B-720P

Model Details

• Type: LoRA adapter (fine-tune on top of Wan2.1-FLF2V-14B-720P)
• Base model: Wan-AI/Wan2.1-FLF2V-14B-720P (first-last-frame-to-video, 14B parameters, 720P)
• Training data: maxwelljones14/refVFX_dataset
• Training steps: 40K steps
• Hardware: 8× NVIDIA Blackwell GPUs
• LoRA rank: 1024
• Precision: bf16
• Target modules: all 40 transformer blocks — self-attention (q, k, v, o), cross-attention (q, k, v, o), and feed-forward (ffn.0, ffn.2)
• Checkpoint file: step-10000.safetensors

What it does

RefVFX performs tuning-free visual effect transfer across videos: given a reference effect, the model transfers that effect onto new input content while preserving the underlying motion and structure. The adapter was trained on the multi-part RefVFX dataset, which combines:

• Code-based edits — deterministic spatial effects (posterize, pixelate, glitch, glow, halftone, …) composited with temporal transitions (wipes, circle/diamond reveals, checkerboard, dissolves, …).
• Neural V2V edits — diffusion-generated effect videos sharing motion with a base (no-effect) video.
• I2V LoRA effects — image-to-video effects generated with LoRA adapters.

Usage

This is a LoRA adapter and must be applied on top of the base Wan2.1-FLF2V-14B-720P weights. See the GitHub repo for inference scripts and the full pipeline. At a high level:

from huggingface_hub import hf_hub_download

lora_path = hf_hub_download(
    repo_id="maxwelljones14/refVFX-LoRA",
    filename="step-10000.safetensors",
)
# Load Wan2.1-FLF2V-14B-720P, then apply the LoRA weights from `lora_path`.
# Refer to https://github.com/maxwelljones14/refVFX for the exact loading code.

The checkpoint stores LoRA A/B matrices keyed as blocks.{i}.<module>.lora_A.default.weight / ...lora_B.default.weight.

Citation

@article{jones2026tuning,
  title={Tuning-free Visual Effect Transfer across Videos},
  author={Jones, Maxwell and Abdal, Rameen and Patashnik, Or and Salakhutdinov, Ruslan and Tulyakov, Sergey and Zhu, Jun-Yan and Wang, Kuan-Chieh Jackson},
  journal={arXiv preprint arXiv:2601.07833},
  year={2026}
}

License

Released under CC-BY-4.0. The adapter derives from Wan-AI/Wan2.1-FLF2V-14B-720P; please also review the base model's license terms before use.