Instructions to use MCG-NJU/MotionRAG with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use MCG-NJU/MotionRAG with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline from diffusers.utils import load_image, export_to_video # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("MCG-NJU/MotionRAG", dtype=torch.bfloat16, device_map="cuda") pipe.to("cuda") prompt = "A man with short gray hair plays a red electric guitar." image = load_image( "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/guitar-man.png" ) output = pipe(image=image, prompt=prompt).frames[0] export_to_video(output, "output.mp4") - Notebooks
- Google Colab
- Kaggle
Upload folder using huggingface_hub
Browse files- .gitattributes +1 -0
- assets/architecture-1.jpg +3 -0
- checkpoints/CogVideoX/17_frames.ckpt +3 -0
- checkpoints/CogVideoX/Motion-Adapter.ckpt +3 -0
- checkpoints/CogVideoX/motion_proj.ckpt +3 -0
- checkpoints/CogVideoX/motion_transformer.ckpt +3 -0
- checkpoints/DynamiCrafter/Motion-Adapter.ckpt +3 -0
- checkpoints/DynamiCrafter/motion_proj.ckpt +3 -0
- checkpoints/DynamiCrafter/motion_transformer.ckpt +3 -0
- checkpoints/SVD/Motion-Adapter.ckpt +3 -0
- checkpoints/SVD/motion_proj.ckpt +3 -0
- checkpoints/SVD/motion_transformer.ckpt +3 -0
- datasets/OpenVid-1M/data/openvid-1k.parquet +3 -0
- datasets/OpenVid-1M/data/openvid-1m.parquet +3 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
assets/architecture-1.jpg filter=lfs diff=lfs merge=lfs -text
|
assets/architecture-1.jpg
ADDED
|
Git LFS Details
|
checkpoints/CogVideoX/17_frames.ckpt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:2888a023cfbcb629b1a23ac8ec2717e77d31e402e83b9f6d8ebf9058a98a9e58
|
| 3 |
+
size 2379170107
|
checkpoints/CogVideoX/Motion-Adapter.ckpt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ad349644a78389a444690c7184e0f9889696d93a98820097b884b90422379ca9
|
| 3 |
+
size 1417506086
|
checkpoints/CogVideoX/motion_proj.ckpt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:9ac9cf4525be66e0e76be0c0c0febcb21f0559929cbfec39c2a437720103ef8b
|
| 3 |
+
size 96069421
|
checkpoints/CogVideoX/motion_transformer.ckpt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:52fbe8a43e1134b41569fa1aef5cd7bd90980bd5a6d87e9112c53f6776bb017e
|
| 3 |
+
size 197442082
|
checkpoints/DynamiCrafter/Motion-Adapter.ckpt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:58313522280c02dd4f89c1baa815b8746e14102c5144ec5ad8df244245625598
|
| 3 |
+
size 147446498
|
checkpoints/DynamiCrafter/motion_proj.ckpt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:2fb2b79b0fc02596bc2b7d8c16ad51e15c51288ff48c792b083590c0886e2ae0
|
| 3 |
+
size 96069896
|
checkpoints/DynamiCrafter/motion_transformer.ckpt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:1b6a38d8c02d2ee6e39efa55da6832db38508cc030364dfd38e695fa920a60d2
|
| 3 |
+
size 197442082
|
checkpoints/SVD/Motion-Adapter.ckpt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:47820f48d58d7feeae86ec13012c2b49d51ac784f2075fab91583a863591de11
|
| 3 |
+
size 172203298
|
checkpoints/SVD/motion_proj.ckpt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:9e5068edfe261ccb15a2e03bb4d3d75e9de920be69f4938a08ac3c86123996da
|
| 3 |
+
size 96069027
|
checkpoints/SVD/motion_transformer.ckpt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:01107380e6fefda1f56db5a0d157ae95756e050eb9d552be385f59939510a4a6
|
| 3 |
+
size 197442146
|
datasets/OpenVid-1M/data/openvid-1k.parquet
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:cac66929a731c84bec4fb091dbfeca15abe12ed7cad6a5bc3caf3abd8863643d
|
| 3 |
+
size 376738
|
datasets/OpenVid-1M/data/openvid-1m.parquet
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:b28a4a93dfc4c01bfaf4bfc230957277a40209f3c97e34b8ee54eaf679e363fb
|
| 3 |
+
size 317642625
|