Instructions to use luzimu/WebGenAgent-LM-7B-SFT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use luzimu/WebGenAgent-LM-7B-SFT with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="luzimu/WebGenAgent-LM-7B-SFT")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("luzimu/WebGenAgent-LM-7B-SFT")
model = AutoModelForCausalLM.from_pretrained("luzimu/WebGenAgent-LM-7B-SFT", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use luzimu/WebGenAgent-LM-7B-SFT with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "luzimu/WebGenAgent-LM-7B-SFT"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "luzimu/WebGenAgent-LM-7B-SFT",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/luzimu/WebGenAgent-LM-7B-SFT

SGLang

How to use luzimu/WebGenAgent-LM-7B-SFT with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "luzimu/WebGenAgent-LM-7B-SFT" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "luzimu/WebGenAgent-LM-7B-SFT",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "luzimu/WebGenAgent-LM-7B-SFT" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "luzimu/WebGenAgent-LM-7B-SFT",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use luzimu/WebGenAgent-LM-7B-SFT with Docker Model Runner:
```
docker model run hf.co/luzimu/WebGenAgent-LM-7B-SFT
```

WebGenAgent-LM-7B-SFT

File size: 5,285 Bytes

---
base_model:
- Qwen/Qwen2.5-Coder-7B-Instruct
datasets:
- luzimu/webgen-agent_train_step-grpo
- luzimu/webgen-agent_train_sft
license: mit
pipeline_tag: image-text-to-text
library_name: transformers
---

# WebGen-Agent

WebGen-Agent is an advanced website generation agent designed to autonomously create websites from natural language instructions. It was introduced in the paper [WebGen-Agent: Enhancing Interactive Website Generation with Multi-Level Feedback and Step-Level Reinforcement Learning](https://arxiv.org/pdf/2509.22644v1).

Code: https://github.com/mnluzimu/WebGen-Agent

## Project Overview

WebGen-Agent combines state-of-the-art language models with specialized training techniques to create a powerful website generation tool. The agent can understand natural language instructions specifying appearance and functional requirements, iteratively generate website codebases, and refine them using visual and functional feedback.

## Resources

Links to the data and model parameters are as follows:

| **Data** | **HF Link** |
|----------|------|
| **webgen-agent_train_sft** | 🤗 [luzimu/webgen-agent_train_sft](https://huggingface.co/datasets/luzimu/webgen-agent_train_sft) |
| **webgen-agent_train_step-grpo** | 🤗 [luzimu/webgen-agent_train_step-grpo](https://huggingface.co/datasets/luzimu/webgen-agent_train_step-grpo) |

| **Model** | **HF Link** |
|----------|------|
| **WebGenAgent-LM-7B-SFT** | 🤗 [luzimu/WebGenAgent-LM-7B-SFT](https://huggingface.co/luzimu/WebGenAgent-LM-7B-SFT) |
| **WebGenAgent-LM-7B-Step-GRPO** | 🤗 [luzimu/WebGenAgent-LM-7B-Step-GRPO](https://huggingface.co/luzimu/WebGenAgent-LM-7B-Step-GRPO) |
| **WebGenAgent-LM-8B-SFT** | 🤗 [luzimu/WebGenAgent-LM-8B-SFT](https://huggingface.co/luzimu/WebGenAgent-LM-8B-SFT) |
| **WebGenAgent-LM-8B-Step-GRPO** | 🤗 [luzimu/WebGenAgent-LM-8B-Step-GRPO](https://huggingface.co/luzimu/WebGenAgent-LM-8B-Step-GRPO) |

## How WebGen-Agent Works

WebGen-Agent follows an iterative, multi-step paradigm for website generation:

1.  **Code Generation**: The agent generates code to create or edit website files based on natural language instructions
2.  **Code Execution**: Dependencies are installed and the website service is started
3.  **Feedback Gathering**: 
    -   A screenshot of the website is captured
    -   A Visual Language Model (VLM) provides appearance feedback and scores
    -   A GUI-agent tests the website functionality and provides functional feedback
4.  **Refinement**: Based on the feedback, the agent continues to improve the website until it meets requirements

![WebGen-Agent Workflow](fig/webgen-agent.png)

## Step-GRPO with Screenshot and GUI-agent Feedback

The Step-GRPO with Screenshot and GUI-agent Feedback approach uses the screenshot and GUI-agent scores inherently produced in the WebGen-Agent workflow as step-level rewards:
-   **Screenshot Score**: Quantifies the visual appeal and aesthetics of the website
-   **GUI-agent Score**: Measures how well the website meets functional requirements

These dual rewards provide dense, reliable process supervision that significantly improves the model's ability to generate high-quality websites.

![Step-GRPO with Screenshot and GUI-agent Feedback](fig/step-grpo.png)

## Sample Usage

Before running inference, you should rename `.env.template` to `.env` and set the base urls and api keys for the agent-engine LLM and feedback VLM. They can be obtained from any openai-compatible providers such as [openrouter](https://openrouter.ai/), [modelscope](https://www.modelscope.cn/my/overview), [bailian](https://bailian.console.aliyun.com/#/home), and [llmprovider](https://llmprovider.ai/).

You can also deploy open-source VLMs and LLMs by running `src/scripts/deploy_qwenvl_32b.sh` and `src/scripts/deploy.sh`. Scripts for single inference and batch inference can be found at `src/scripts/infer_single.sh` and `src/scripts/infer_batch.sh`.

### Single Inference

```bash
python src/infer_single.py \
    --model deepseek-chat \
    --vlm_model Qwen/Qwen2.5-VL-32B-Instruct \
    --instruction "Please implement a wheel of fortune website." \
    --workspace-dir workspaces_root/test \
    --log-dir service_logs/test \
    --max-iter 20 \
    --overwrite \
    --error-limit 5
```

## Citation

If you find our project useful, please cite:

```
@misc{lu2025webgenagentenhancinginteractivewebsite,
      title={WebGen-Agent: Enhancing Interactive Website Generation with Multi-Level Feedback and Step-Level Reinforcement Learning}, 
      author={Zimu Lu and Houxing Ren and Yunqiao Yang and Ke Wang and Zhuofan Zong and Junting Pan and Mingjie Zhan and Hongsheng Li},
      year={2025},
      eprint={2509.22644},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2509.22644}, 
}

@misc{lu2025webgenbenchevaluatingllmsgenerating,
      title={WebGen-Bench: Evaluating LLMs on Generating Interactive and Functional Websites from Scratch}, 
      author={Zimu Lu and Yunqiao Yang and Houxing Ren and Haotian Hou and Han Xiao and Ke Wang and Weikang Shi and Aojun Zhou and Mingjie Zhan and Hongsheng Li},
      year={2025},
      eprint={2505.03733},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2505.03733}, 
}
```