Instructions to use Kabatubare/web-md-llama2-7b-3000 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Kabatubare/web-md-llama2-7b-3000 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Kabatubare/web-md-llama2-7b-3000")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Kabatubare/web-md-llama2-7b-3000")
model = AutoModelForCausalLM.from_pretrained("Kabatubare/web-md-llama2-7b-3000")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Kabatubare/web-md-llama2-7b-3000 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Kabatubare/web-md-llama2-7b-3000"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Kabatubare/web-md-llama2-7b-3000",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/Kabatubare/web-md-llama2-7b-3000

SGLang

How to use Kabatubare/web-md-llama2-7b-3000 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Kabatubare/web-md-llama2-7b-3000" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Kabatubare/web-md-llama2-7b-3000",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Kabatubare/web-md-llama2-7b-3000" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Kabatubare/web-md-llama2-7b-3000",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use Kabatubare/web-md-llama2-7b-3000 with Docker Model Runner:
```
docker model run hf.co/Kabatubare/web-md-llama2-7b-3000
```

Medical3000 Model Card

This is a model card for web-md-llama2-7b-3000 , a fine-tuned version of Llama-2-7B, specifically aimed at medical dialogues.

Covered areas:

General Medicine: Basic medical advice, symptoms, general treatments.

Cardiology: Questions related to heart diseases, blood circulation.

Neurology: Topics around brain health, neurological disorders.

Gastroenterology: Issues related to the digestive system.

Oncology: Questions about different types of cancers, treatments.

Endocrinology: Topics related to hormones, diabetes, thyroid.

Orthopedics: Bone health, joint issues.

Pediatrics: Child health, vaccinations, growth and development.

Mental Health: Depression, anxiety, stress, and other mental health issues.

Women's Health: Pregnancy, menstrual health, menopause.

Model Details

Base Model

Name: Llama-2-7B

Fine-tuned Model

Name: web-md-llama2-7b-3000
Fine-tuned on: Kabatubare/medical-guanaco-3000
Description: This model is fine-tuned to specialize in medical dialogues and healthcare applications.

Architecture and Training Parameters

Architecture

LoRA Attention Dimension: 64
LoRA Alpha Parameter: 16
LoRA Dropout: 0.1
Precision: 4-bit (bitsandbytes)
Quantization Type: nf4

Training Parameters

Epochs: 3
Batch Size: 4
Gradient Accumulation Steps: 1
Max Gradient Norm: 0.3
Learning Rate: 3e-4
Weight Decay: 0.001
Optimizer: paged_adamw_32bit
LR Scheduler: cosine
Warmup Ratio: 0.03
Logging Steps: 25

Datasets

Fine-tuning Dataset

Name: Kabatubare/medical-guanaco-3000
Description: This is a reduced and balanced dataset curated from a larger medical dialogue dataset using derived from 24,000 WebMD question and answer dialogue sessions . It aims to cover a broad range of medical topics and is suitable for training healthcare chatbots and conducting medical NLP research.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("Yo!Medical3000")
model = AutoModelForCausalLM.from_pretrained("Yo!Medical3000")

# Use the model for inference

Downloads last month: 5

Kabatubare
/

web-md-llama2-7b-3000