Instructions to use mradermacher/RomanEng2Nep-v2-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use mradermacher/RomanEng2Nep-v2-GGUF with Transformers:
# Use a pipeline as a high-level helper # Warning: Pipeline type "translation" is no longer supported in transformers v5. # You must load the model directly (see below) or downgrade to v4.x with: # pip install "transformers<5.0.0" from transformers import pipeline pipe = pipeline("translation", model="mradermacher/RomanEng2Nep-v2-GGUF")# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("mradermacher/RomanEng2Nep-v2-GGUF", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use mradermacher/RomanEng2Nep-v2-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf mradermacher/RomanEng2Nep-v2-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf mradermacher/RomanEng2Nep-v2-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf mradermacher/RomanEng2Nep-v2-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf mradermacher/RomanEng2Nep-v2-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf mradermacher/RomanEng2Nep-v2-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf mradermacher/RomanEng2Nep-v2-GGUF:Q4_K_M
Use Docker
docker model run hf.co/mradermacher/RomanEng2Nep-v2-GGUF:Q4_K_M
- Llama new
- LM Studio
- Jan
- Ollama
How to use mradermacher/RomanEng2Nep-v2-GGUF with Ollama:
ollama run hf.co/mradermacher/RomanEng2Nep-v2-GGUF:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use mradermacher/RomanEng2Nep-v2-GGUF with Docker Model Runner:
docker model run hf.co/mradermacher/RomanEng2Nep-v2-GGUF:Q4_K_M
- Lemonade
How to use mradermacher/RomanEng2Nep-v2-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull mradermacher/RomanEng2Nep-v2-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.RomanEng2Nep-v2-GGUF-Q4_K_M
List all available models
lemonade list
- Atomic Chat
Acknowledgement
#1
by syubraj - opened
The model might simply be broken, or not supported by llama.cpp
Which one do you suggest then to make the model running? or would you check from your side, if possible.
works fine for me in llama.cpp (to the extend I can test it, my nepali is notoriosuly bad :)
$ llama-cli -m RomanEng2Nep-v2.Q4_K_S.gguf -p muskuraudai
कुंजलिसहित [end of text]
mradermacher changed discussion status to closed
