SAE Interventions are Unreliable: Post-Intervention Recovery of Suppressed Behavior
Paper • 2606.18322 • Published • 17
AI & ML interests
None defined yet.
Recent Activity
View all activity
Papers
SAE Interventions are Unreliable: Post-Intervention Recovery of Suppressed Behavior
BadWorld: Adversarial Attacks on World Models
adamdad
submitted 2
papers to Daily Papers about 1 month ago
wyy-code
submitted a
paper to Daily Papers about 2 months ago
Simmonstt
authored a
paper 2 months ago
juezhi
submitted a
paper to Daily Papers 2 months ago
yanggangu
submitted a
paper to Daily Papers 2 months ago
wyy-code
submitted a
paper to Daily Papers 2 months ago
SeanLee97
submitted a
paper to Daily Papers 3 months ago
SeanLee97
authored a
paper 3 months ago
Post
8260
Our lab recently released a paper where we introduce ShadowPEFT, a new Parameter-Efficient Fine-Tuning (PEFT) paradigm tailored for edge computing scenarios.
Unlike traditional approaches such as LoRA and its variants, which inject trainable parameters directly into the weights of Transformer, requiring tight coupling with the backbone.
ShadowPEFT instead enhances the frozen large base model by adding a lightweight, centralized, pretrainable, and detachable Shadow network.
This shadow network operates in parallel with the base model, delivering learned corrections to each decoder layer. Because the shadow module is architecturally decoupled from the backbone, it can be independently trained, stored, and deployed, benefiting edge computing scenarios and edge-cloud collaboration computing.
- HF Paper: ShadowPEFT: Shadow Network for Parameter-Efficient Fine-Tuning (2604.19254)
- GitHub: https://github.com/ShadowLLM/shadow-peft
- HF Collection: https://huggingface.co/collections/shadow-llm/shadow-peft-models
Unlike traditional approaches such as LoRA and its variants, which inject trainable parameters directly into the weights of Transformer, requiring tight coupling with the backbone.
ShadowPEFT instead enhances the frozen large base model by adding a lightweight, centralized, pretrainable, and detachable Shadow network.
This shadow network operates in parallel with the base model, delivering learned corrections to each decoder layer. Because the shadow module is architecturally decoupled from the backbone, it can be independently trained, stored, and deployed, benefiting edge computing scenarios and edge-cloud collaboration computing.
- HF Paper: ShadowPEFT: Shadow Network for Parameter-Efficient Fine-Tuning (2604.19254)
- GitHub: https://github.com/ShadowLLM/shadow-peft
- HF Collection: https://huggingface.co/collections/shadow-llm/shadow-peft-models
- 7 replies
Ruixiang
submitted a
paper to Daily Papers 3 months ago
jinggqu
submitted a
paper to Daily Papers 4 months ago
tomhu
submitted a
paper to Daily Papers 5 months ago
Stephen-SMJ
submitted a
paper to Daily Papers 6 months ago
adamdad
submitted a
paper to Daily Papers 7 months ago
wegylexy
authored a
paper over 2 years ago