Image-Text-to-Text
Transformers
OpenVINO
English
idefics3
int4
quantization
edge-deployment
optimization
vision-language-model
multimodal
smolvlm
conversational
Instructions to use dev-bjoern/smolvlm-int4-ov with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use dev-bjoern/smolvlm-int4-ov with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="dev-bjoern/smolvlm-int4-ov") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("dev-bjoern/smolvlm-int4-ov") model = AutoModelForMultimodalLM.from_pretrained("dev-bjoern/smolvlm-int4-ov", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use dev-bjoern/smolvlm-int4-ov with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "dev-bjoern/smolvlm-int4-ov" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "dev-bjoern/smolvlm-int4-ov", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/dev-bjoern/smolvlm-int4-ov
- SGLang
How to use dev-bjoern/smolvlm-int4-ov with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "dev-bjoern/smolvlm-int4-ov" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "dev-bjoern/smolvlm-int4-ov", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "dev-bjoern/smolvlm-int4-ov" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "dev-bjoern/smolvlm-int4-ov", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use dev-bjoern/smolvlm-int4-ov with Docker Model Runner:
docker model run hf.co/dev-bjoern/smolvlm-int4-ov
Download openvino_tokenizer.xml from dev-bjoern/smolvlm-int4-ov: direct link, hf CLI and curl.
- Browser
- Download file 21.5 kB
-
https://huggingface.co/dev-bjoern/smolvlm-int4-ov/resolve/main/openvino_tokenizer.xml
- Command line
-
hf download hf://dev-bjoern/smolvlm-int4-ov/openvino_tokenizer.xml
-
curl -L -o openvino_tokenizer.xml https://huggingface.co/dev-bjoern/smolvlm-int4-ov/resolve/main/openvino_tokenizer.xml
21.5 kB
| <net name="tokenizer" version="11"> | |
| <layers> | |
| <layer id="0" name="Parameter_1531689" type="Parameter" version="opset1"> | |
| <data shape="?" element_type="string" /> | |
| <output> | |
| <port id="0" precision="STRING" names="Parameter_1531689"> | |
| <dim>-1</dim> | |
| </port> | |
| </output> | |
| </layer> | |
| <layer id="1" name="Constant_1531695" type="Const" version="opset1"> | |
| <data element_type="i64" shape="" offset="0" size="8" /> | |
| <output> | |
| <port id="0" precision="I64" /> | |
| </output> | |
| </layer> | |
| <layer id="2" name="StringTensorUnpack_1531690" type="StringTensorUnpack" version="opset15"> | |
| <input> | |
| <port id="0" precision="STRING"> | |
| <dim>-1</dim> | |
| </port> | |
| </input> | |
| <output> | |
| <port id="1" precision="I32"> | |
| <dim>-1</dim> | |
| </port> | |
| <port id="2" precision="I32"> | |
| <dim>-1</dim> | |
| </port> | |
| <port id="3" precision="U8"> | |
| <dim>-1</dim> | |
| </port> | |
| </output> | |
| </layer> | |
| <layer id="3" name="ShapeOf_1531691" type="ShapeOf" version="opset3"> | |
| <data output_type="i64" /> | |
| <input> | |
| <port id="0" precision="I32"> | |
| <dim>-1</dim> | |
| </port> | |
| </input> | |
| <output> | |
| <port id="1" precision="I64"> | |
| <dim>1</dim> | |
| </port> | |
| </output> | |
| </layer> | |
| <layer id="4" name="Constant_1531692" type="Const" version="opset1"> | |
| <data element_type="i64" shape="" offset="0" size="8" /> | |
| <output> | |
| <port id="0" precision="I64" /> | |
| </output> | |
| </layer> | |
| <layer id="5" name="Constant_1531693" type="Const" version="opset1"> | |
| <data element_type="i64" shape="" offset="0" size="8" /> | |
| <output> | |
| <port id="0" precision="I64" /> | |
| </output> | |
| </layer> | |
| <layer id="6" name="Gather_1531694" type="Gather" version="opset8"> | |
| <data batch_dims="0" /> | |
| <input> | |
| <port id="0" precision="I64"> | |
| <dim>1</dim> | |
| </port> | |
| <port id="1" precision="I64" /> | |
| <port id="2" precision="I64" /> | |
| </input> | |
| <output> | |
| <port id="3" precision="I64" /> | |
| </output> | |
| </layer> | |
| <layer id="7" name="Constant_1531696" type="Const" version="opset1"> | |
| <data element_type="i64" shape="" offset="8" size="8" /> | |
| <output> | |
| <port id="0" precision="I64" /> | |
| </output> | |
| </layer> | |
| <layer id="8" name="Range_1531697" type="Range" version="opset4"> | |
| <data output_type="i32" /> | |
| <input> | |
| <port id="0" precision="I64" /> | |
| <port id="1" precision="I64" /> | |
| <port id="2" precision="I64" /> | |
| </input> | |
| <output> | |
| <port id="3" precision="I32"> | |
| <dim>-1</dim> | |
| </port> | |
| </output> | |
| </layer> | |
| <layer id="9" name="Constant_1531698" type="Const" version="opset1"> | |
| <data element_type="i64" shape="" offset="8" size="8" /> | |
| <output> | |
| <port id="0" precision="I64" /> | |
| </output> | |
| </layer> | |
| <layer id="10" name="Constant_1531699" type="Const" version="opset1"> | |
| <data element_type="i64" shape="" offset="8" size="8" /> | |
| <output> | |
| <port id="0" precision="I64" /> | |
| </output> | |
| </layer> | |
| <layer id="11" name="Add_1531700" type="Add" version="opset1"> | |
| <data auto_broadcast="numpy" /> | |
| <input> | |
| <port id="0" precision="I64" /> | |
| <port id="1" precision="I64" /> | |
| </input> | |
| <output> | |
| <port id="2" precision="I64" /> | |
| </output> | |
| </layer> | |
| <layer id="12" name="Constant_1531701" type="Const" version="opset1"> | |
| <data element_type="i64" shape="" offset="8" size="8" /> | |
| <output> | |
| <port id="0" precision="I64" /> | |
| </output> | |
| </layer> | |
| <layer id="13" name="Range_1531702" type="Range" version="opset4"> | |
| <data output_type="i32" /> | |
| <input> | |
| <port id="0" precision="I64" /> | |
| <port id="1" precision="I64" /> | |
| <port id="2" precision="I64" /> | |
| </input> | |
| <output> | |
| <port id="3" precision="I32"> | |
| <dim>-1</dim> | |
| </port> | |
| </output> | |
| </layer> | |
| <layer id="14" name="Constant_1531764" type="Const" version="opset1"> | |
| <data element_type="u8" shape="372" offset="16" size="372" /> | |
| <output> | |
| <port id="0" precision="U8"> | |
| <dim>372</dim> | |
| </port> | |
| </output> | |
| </layer> | |
| <layer id="15" name="SpecialTokensSplit_1531765" type="SpecialTokensSplit" version="extension"> | |
| <input> | |
| <port id="0" precision="I32"> | |
| <dim>-1</dim> | |
| </port> | |
| <port id="1" precision="I32"> | |
| <dim>-1</dim> | |
| </port> | |
| <port id="2" precision="I32"> | |
| <dim>-1</dim> | |
| </port> | |
| <port id="3" precision="I32"> | |
| <dim>-1</dim> | |
| </port> | |
| <port id="4" precision="U8"> | |
| <dim>-1</dim> | |
| </port> | |
| <port id="5" precision="U8"> | |
| <dim>372</dim> | |
| </port> | |
| </input> | |
| <output> | |
| <port id="6" precision="I32"> | |
| <dim>-1</dim> | |
| </port> | |
| <port id="7" precision="I32"> | |
| <dim>-1</dim> | |
| </port> | |
| <port id="8" precision="I32"> | |
| <dim>-1</dim> | |
| </port> | |
| <port id="9" precision="I32"> | |
| <dim>-1</dim> | |
| </port> | |
| <port id="10" precision="U8"> | |
| <dim>-1</dim> | |
| </port> | |
| <port id="11" precision="BOOL"> | |
| <dim>-1</dim> | |
| </port> | |
| </output> | |
| </layer> | |
| <layer id="16" name="Constant_1531767" type="Const" version="opset1"> | |
| <data element_type="u8" shape="95" offset="388" size="95" /> | |
| <output> | |
| <port id="0" precision="U8"> | |
| <dim>95</dim> | |
| </port> | |
| </output> | |
| </layer> | |
| <layer id="17" name="RegexSplit_1531768" type="RegexSplit" version="extension"> | |
| <data behaviour="isolate" invert="false" max_splits="-1" /> | |
| <input> | |
| <port id="0" precision="I32"> | |
| <dim>-1</dim> | |
| </port> | |
| <port id="1" precision="I32"> | |
| <dim>-1</dim> | |
| </port> | |
| <port id="2" precision="I32"> | |
| <dim>-1</dim> | |
| </port> | |
| <port id="3" precision="I32"> | |
| <dim>-1</dim> | |
| </port> | |
| <port id="4" precision="U8"> | |
| <dim>-1</dim> | |
| </port> | |
| <port id="5" precision="BOOL"> | |
| <dim>-1</dim> | |
| </port> | |
| <port id="6" precision="U8"> | |
| <dim>95</dim> | |
| </port> | |
| </input> | |
| <output> | |
| <port id="7" precision="I32"> | |
| <dim>-1</dim> | |
| </port> | |
| <port id="8" precision="I32"> | |
| <dim>-1</dim> | |
| </port> | |
| <port id="9" precision="I32"> | |
| <dim>-1</dim> | |
| </port> | |
| <port id="10" precision="I32"> | |
| <dim>-1</dim> | |
| </port> | |
| <port id="11" precision="U8"> | |
| <dim>-1</dim> | |
| </port> | |
| <port id="12" precision="BOOL"> | |
| <dim>-1</dim> | |
| </port> | |
| </output> | |
| </layer> | |
| <layer id="18" name="Constant_1531770" type="Const" version="opset1"> | |
| <data element_type="i32" shape="49155" offset="483" size="196620" /> | |
| <output> | |
| <port id="0" precision="I32"> | |
| <dim>49155</dim> | |
| </port> | |
| </output> | |
| </layer> | |
| <layer id="19" name="Constant_1531772" type="Const" version="opset1"> | |
| <data element_type="i32" shape="49155" offset="197103" size="196620" /> | |
| <output> | |
| <port id="0" precision="I32"> | |
| <dim>49155</dim> | |
| </port> | |
| </output> | |
| </layer> | |
| <layer id="20" name="Constant_1531774" type="Const" version="opset1"> | |
| <data element_type="u8" shape="332417" offset="393723" size="332417" /> | |
| <output> | |
| <port id="0" precision="U8"> | |
| <dim>332417</dim> | |
| </port> | |
| </output> | |
| </layer> | |
| <layer id="21" name="Constant_1531782" type="Const" version="opset1"> | |
| <data element_type="i32" shape="48900" offset="726140" size="195600" /> | |
| <output> | |
| <port id="0" precision="I32"> | |
| <dim>48900</dim> | |
| </port> | |
| </output> | |
| </layer> | |
| <layer id="22" name="Constant_1531784" type="Const" version="opset1"> | |
| <data element_type="i32" shape="48900" offset="921740" size="195600" /> | |
| <output> | |
| <port id="0" precision="I32"> | |
| <dim>48900</dim> | |
| </port> | |
| </output> | |
| </layer> | |
| <layer id="23" name="Constant_1531786" type="Const" version="opset1"> | |
| <data element_type="u8" shape="173505" offset="1117340" size="173505" /> | |
| <output> | |
| <port id="0" precision="U8"> | |
| <dim>173505</dim> | |
| </port> | |
| </output> | |
| </layer> | |
| <layer id="24" name="Constant_1531788" type="Const" version="opset1"> | |
| <data element_type="i32" shape="48900" offset="1290845" size="195600" /> | |
| <output> | |
| <port id="0" precision="I32"> | |
| <dim>48900</dim> | |
| </port> | |
| </output> | |
| </layer> | |
| <layer id="25" name="Constant_1531790" type="Const" version="opset1"> | |
| <data element_type="i32" shape="48900" offset="1486445" size="195600" /> | |
| <output> | |
| <port id="0" precision="I32"> | |
| <dim>48900</dim> | |
| </port> | |
| </output> | |
| </layer> | |
| <layer id="26" name="Constant_1531792" type="Const" version="opset1"> | |
| <data element_type="u8" shape="158410" offset="1682045" size="158410" /> | |
| <output> | |
| <port id="0" precision="U8"> | |
| <dim>158410</dim> | |
| </port> | |
| </output> | |
| </layer> | |
| <layer id="27" name="Constant_1531776" type="Const" version="opset1"> | |
| <data element_type="i32" shape="19" offset="1840455" size="76" /> | |
| <output> | |
| <port id="0" precision="I32"> | |
| <dim>19</dim> | |
| </port> | |
| </output> | |
| </layer> | |
| <layer id="28" name="Constant_1531778" type="Const" version="opset1"> | |
| <data element_type="i32" shape="19" offset="1840531" size="76" /> | |
| <output> | |
| <port id="0" precision="I32"> | |
| <dim>19</dim> | |
| </port> | |
| </output> | |
| </layer> | |
| <layer id="29" name="Constant_1531780" type="Const" version="opset1"> | |
| <data element_type="u8" shape="254" offset="1840607" size="254" /> | |
| <output> | |
| <port id="0" precision="U8"> | |
| <dim>254</dim> | |
| </port> | |
| </output> | |
| </layer> | |
| <layer id="30" name="Constant_1531793" type="Const" version="opset1"> | |
| <data element_type="i32" shape="19" offset="1840861" size="76" /> | |
| <output> | |
| <port id="0" precision="I32"> | |
| <dim>19</dim> | |
| </port> | |
| </output> | |
| </layer> | |
| <layer id="31" name="BPETokenizer_1531794" type="BPETokenizer" version="extension"> | |
| <data unk_token="" fuse_unk="false" suffix_indicator="" end_suffix="" byte_fallback="false" cache_capacity="20000" /> | |
| <input> | |
| <port id="0" precision="I32"> | |
| <dim>-1</dim> | |
| </port> | |
| <port id="1" precision="I32"> | |
| <dim>-1</dim> | |
| </port> | |
| <port id="2" precision="I32"> | |
| <dim>-1</dim> | |
| </port> | |
| <port id="3" precision="I32"> | |
| <dim>-1</dim> | |
| </port> | |
| <port id="4" precision="U8"> | |
| <dim>-1</dim> | |
| </port> | |
| <port id="5" precision="I32"> | |
| <dim>49155</dim> | |
| </port> | |
| <port id="6" precision="I32"> | |
| <dim>49155</dim> | |
| </port> | |
| <port id="7" precision="U8"> | |
| <dim>332417</dim> | |
| </port> | |
| <port id="8" precision="I32"> | |
| <dim>48900</dim> | |
| </port> | |
| <port id="9" precision="I32"> | |
| <dim>48900</dim> | |
| </port> | |
| <port id="10" precision="U8"> | |
| <dim>173505</dim> | |
| </port> | |
| <port id="11" precision="I32"> | |
| <dim>48900</dim> | |
| </port> | |
| <port id="12" precision="I32"> | |
| <dim>48900</dim> | |
| </port> | |
| <port id="13" precision="U8"> | |
| <dim>158410</dim> | |
| </port> | |
| <port id="14" precision="I32"> | |
| <dim>19</dim> | |
| </port> | |
| <port id="15" precision="I32"> | |
| <dim>19</dim> | |
| </port> | |
| <port id="16" precision="U8"> | |
| <dim>254</dim> | |
| </port> | |
| <port id="17" precision="I32"> | |
| <dim>19</dim> | |
| </port> | |
| </input> | |
| <output> | |
| <port id="18" precision="I32"> | |
| <dim>-1</dim> | |
| </port> | |
| <port id="19" precision="I32"> | |
| <dim>-1</dim> | |
| </port> | |
| <port id="20" precision="I32"> | |
| <dim>-1</dim> | |
| </port> | |
| </output> | |
| </layer> | |
| <layer id="32" name="Constant_1531795" type="Const" version="opset1"> | |
| <data element_type="i32" shape="" offset="1840937" size="4" /> | |
| <output> | |
| <port id="0" precision="I32" /> | |
| </output> | |
| </layer> | |
| <layer id="33" name="Constant_1531797" type="Const" version="opset1"> | |
| <data element_type="u8" shape="4" offset="1840941" size="4" /> | |
| <output> | |
| <port id="0" precision="U8"> | |
| <dim>4</dim> | |
| </port> | |
| </output> | |
| </layer> | |
| <layer id="34" name="Constant_1531799" type="Const" version="opset1"> | |
| <data element_type="u8" shape="13" offset="1840945" size="13" /> | |
| <output> | |
| <port id="0" precision="U8"> | |
| <dim>13</dim> | |
| </port> | |
| </output> | |
| </layer> | |
| <layer id="35" name="Truncate_1531800" type="Truncate" version="extension"> | |
| <data m_num_inputs="1" /> | |
| <input> | |
| <port id="0" precision="I32"> | |
| <dim>-1</dim> | |
| </port> | |
| <port id="1" precision="I32"> | |
| <dim>-1</dim> | |
| </port> | |
| <port id="2" precision="I32"> | |
| <dim>-1</dim> | |
| </port> | |
| <port id="3" precision="I32" /> | |
| <port id="4" precision="U8"> | |
| <dim>4</dim> | |
| </port> | |
| <port id="5" precision="U8"> | |
| <dim>13</dim> | |
| </port> | |
| </input> | |
| <output> | |
| <port id="6" precision="I32"> | |
| <dim>-1</dim> | |
| </port> | |
| <port id="7" precision="I32"> | |
| <dim>-1</dim> | |
| </port> | |
| <port id="8" precision="I32"> | |
| <dim>-1</dim> | |
| </port> | |
| </output> | |
| </layer> | |
| <layer id="36" name="Subtract_1531801" type="Subtract" version="opset1"> | |
| <data auto_broadcast="numpy" /> | |
| <input> | |
| <port id="0" precision="I32"> | |
| <dim>-1</dim> | |
| </port> | |
| <port id="1" precision="I32"> | |
| <dim>-1</dim> | |
| </port> | |
| </input> | |
| <output> | |
| <port id="2" precision="I32"> | |
| <dim>-1</dim> | |
| </port> | |
| </output> | |
| </layer> | |
| <layer id="37" name="Constant_1531802" type="Const" version="opset1"> | |
| <data element_type="i32" shape="" offset="1840958" size="4" /> | |
| <output> | |
| <port id="0" precision="I32" /> | |
| </output> | |
| </layer> | |
| <layer id="38" name="ReduceMax_1531803" type="ReduceMax" version="opset1"> | |
| <data keep_dims="false" /> | |
| <input> | |
| <port id="0" precision="I32"> | |
| <dim>-1</dim> | |
| </port> | |
| <port id="1" precision="I32" /> | |
| </input> | |
| <output> | |
| <port id="2" precision="I32" /> | |
| </output> | |
| </layer> | |
| <layer id="39" name="Constant_1531804" type="Const" version="opset1"> | |
| <data element_type="i32" shape="" offset="1840962" size="4" /> | |
| <output> | |
| <port id="0" precision="I32" /> | |
| </output> | |
| </layer> | |
| <layer id="40" name="RaggedToDense_1531805" type="RaggedToDense" version="extension"> | |
| <data pad_right="false" m_pad_max_length="false" /> | |
| <input> | |
| <port id="0" precision="I32"> | |
| <dim>-1</dim> | |
| </port> | |
| <port id="1" precision="I32"> | |
| <dim>-1</dim> | |
| </port> | |
| <port id="2" precision="I32"> | |
| <dim>-1</dim> | |
| </port> | |
| <port id="3" precision="I32" /> | |
| <port id="4" precision="I32" /> | |
| </input> | |
| <output> | |
| <port id="5" precision="I32"> | |
| <dim>-1</dim> | |
| <dim>-1</dim> | |
| </port> | |
| <port id="6" precision="BOOL"> | |
| <dim>-1</dim> | |
| <dim>-1</dim> | |
| </port> | |
| </output> | |
| </layer> | |
| <layer id="41" name="Convert_1531806" type="Convert" version="opset1"> | |
| <data destination_type="i32" /> | |
| <input> | |
| <port id="0" precision="BOOL"> | |
| <dim>-1</dim> | |
| <dim>-1</dim> | |
| </port> | |
| </input> | |
| <output> | |
| <port id="1" precision="I32"> | |
| <dim>-1</dim> | |
| <dim>-1</dim> | |
| </port> | |
| </output> | |
| </layer> | |
| <layer id="42" name="Convert_1531806.0" type="Convert" version="opset1"> | |
| <data destination_type="i64" /> | |
| <input> | |
| <port id="0" precision="I32"> | |
| <dim>-1</dim> | |
| <dim>-1</dim> | |
| </port> | |
| </input> | |
| <output> | |
| <port id="1" precision="I64" names="attention_mask"> | |
| <dim>-1</dim> | |
| <dim>-1</dim> | |
| </port> | |
| </output> | |
| </layer> | |
| <layer id="44" name="RaggedToDense_1531805.0" type="Convert" version="opset1"> | |
| <data destination_type="i64" /> | |
| <input> | |
| <port id="0" precision="I32"> | |
| <dim>-1</dim> | |
| <dim>-1</dim> | |
| </port> | |
| </input> | |
| <output> | |
| <port id="1" precision="I64" names="input_ids"> | |
| <dim>-1</dim> | |
| <dim>-1</dim> | |
| </port> | |
| </output> | |
| </layer> | |
| <layer id="45" name="Result_1531807" type="Result" version="opset1" output_names="input_ids"> | |
| <input> | |
| <port id="0" precision="I64"> | |
| <dim>-1</dim> | |
| <dim>-1</dim> | |
| </port> | |
| </input> | |
| </layer> | |
| <layer id="43" name="Result_1531808" type="Result" version="opset1" output_names="attention_mask"> | |
| <input> | |
| <port id="0" precision="I64"> | |
| <dim>-1</dim> | |
| <dim>-1</dim> | |
| </port> | |
| </input> | |
| </layer> | |
| </layers> | |
| <edges> | |
| <edge from-layer="0" from-port="0" to-layer="2" to-port="0" /> | |
| <edge from-layer="1" from-port="0" to-layer="8" to-port="0" /> | |
| <edge from-layer="2" from-port="2" to-layer="15" to-port="3" /> | |
| <edge from-layer="2" from-port="3" to-layer="15" to-port="4" /> | |
| <edge from-layer="2" from-port="1" to-layer="15" to-port="2" /> | |
| <edge from-layer="2" from-port="1" to-layer="3" to-port="0" /> | |
| <edge from-layer="3" from-port="1" to-layer="6" to-port="0" /> | |
| <edge from-layer="4" from-port="0" to-layer="6" to-port="1" /> | |
| <edge from-layer="5" from-port="0" to-layer="6" to-port="2" /> | |
| <edge from-layer="6" from-port="3" to-layer="8" to-port="1" /> | |
| <edge from-layer="6" from-port="3" to-layer="11" to-port="0" /> | |
| <edge from-layer="7" from-port="0" to-layer="8" to-port="2" /> | |
| <edge from-layer="8" from-port="3" to-layer="15" to-port="0" /> | |
| <edge from-layer="9" from-port="0" to-layer="13" to-port="0" /> | |
| <edge from-layer="10" from-port="0" to-layer="11" to-port="1" /> | |
| <edge from-layer="11" from-port="2" to-layer="13" to-port="1" /> | |
| <edge from-layer="12" from-port="0" to-layer="13" to-port="2" /> | |
| <edge from-layer="13" from-port="3" to-layer="15" to-port="1" /> | |
| <edge from-layer="14" from-port="0" to-layer="15" to-port="5" /> | |
| <edge from-layer="15" from-port="6" to-layer="17" to-port="0" /> | |
| <edge from-layer="15" from-port="7" to-layer="17" to-port="1" /> | |
| <edge from-layer="15" from-port="8" to-layer="17" to-port="2" /> | |
| <edge from-layer="15" from-port="9" to-layer="17" to-port="3" /> | |
| <edge from-layer="15" from-port="10" to-layer="17" to-port="4" /> | |
| <edge from-layer="15" from-port="11" to-layer="17" to-port="5" /> | |
| <edge from-layer="16" from-port="0" to-layer="17" to-port="6" /> | |
| <edge from-layer="17" from-port="7" to-layer="31" to-port="0" /> | |
| <edge from-layer="17" from-port="8" to-layer="31" to-port="1" /> | |
| <edge from-layer="17" from-port="9" to-layer="31" to-port="2" /> | |
| <edge from-layer="17" from-port="10" to-layer="31" to-port="3" /> | |
| <edge from-layer="17" from-port="11" to-layer="31" to-port="4" /> | |
| <edge from-layer="18" from-port="0" to-layer="31" to-port="5" /> | |
| <edge from-layer="19" from-port="0" to-layer="31" to-port="6" /> | |
| <edge from-layer="20" from-port="0" to-layer="31" to-port="7" /> | |
| <edge from-layer="21" from-port="0" to-layer="31" to-port="8" /> | |
| <edge from-layer="22" from-port="0" to-layer="31" to-port="9" /> | |
| <edge from-layer="23" from-port="0" to-layer="31" to-port="10" /> | |
| <edge from-layer="24" from-port="0" to-layer="31" to-port="11" /> | |
| <edge from-layer="25" from-port="0" to-layer="31" to-port="12" /> | |
| <edge from-layer="26" from-port="0" to-layer="31" to-port="13" /> | |
| <edge from-layer="27" from-port="0" to-layer="31" to-port="14" /> | |
| <edge from-layer="28" from-port="0" to-layer="31" to-port="15" /> | |
| <edge from-layer="29" from-port="0" to-layer="31" to-port="16" /> | |
| <edge from-layer="30" from-port="0" to-layer="31" to-port="17" /> | |
| <edge from-layer="31" from-port="18" to-layer="35" to-port="0" /> | |
| <edge from-layer="31" from-port="19" to-layer="35" to-port="1" /> | |
| <edge from-layer="31" from-port="20" to-layer="35" to-port="2" /> | |
| <edge from-layer="32" from-port="0" to-layer="35" to-port="3" /> | |
| <edge from-layer="33" from-port="0" to-layer="35" to-port="4" /> | |
| <edge from-layer="34" from-port="0" to-layer="35" to-port="5" /> | |
| <edge from-layer="35" from-port="7" to-layer="36" to-port="0" /> | |
| <edge from-layer="35" from-port="6" to-layer="36" to-port="1" /> | |
| <edge from-layer="35" from-port="6" to-layer="40" to-port="0" /> | |
| <edge from-layer="35" from-port="7" to-layer="40" to-port="1" /> | |
| <edge from-layer="35" from-port="8" to-layer="40" to-port="2" /> | |
| <edge from-layer="36" from-port="2" to-layer="38" to-port="0" /> | |
| <edge from-layer="37" from-port="0" to-layer="38" to-port="1" /> | |
| <edge from-layer="38" from-port="2" to-layer="40" to-port="3" /> | |
| <edge from-layer="39" from-port="0" to-layer="40" to-port="4" /> | |
| <edge from-layer="40" from-port="6" to-layer="41" to-port="0" /> | |
| <edge from-layer="40" from-port="5" to-layer="44" to-port="0" /> | |
| <edge from-layer="41" from-port="1" to-layer="42" to-port="0" /> | |
| <edge from-layer="42" from-port="1" to-layer="43" to-port="0" /> | |
| <edge from-layer="44" from-port="1" to-layer="45" to-port="0" /> | |
| </edges> | |
| <rt_info> | |
| <add_attention_mask value="True" /> | |
| <add_prefix_space /> | |
| <add_special_tokens value="True" /> | |
| <bos_token_id value="1" /> | |
| <chat_template value="<|im_start|>{% for message in messages %}{{message['role'] | capitalize}}{% if message['content'][0]['type'] == 'image' %}{{':'}}{% else %}{{': '}}{% endif %}{% for line in message['content'] %}{% if line['type'] == 'text' %}{{line['text']}}{% elif line['type'] == 'image' %}{{ '<image>' }}{% endif %}{% endfor %}<end_of_utterance> {% endfor %}{% if add_generation_prompt %}{{ 'Assistant:' }}{% endif %}" /> | |
| <clean_up_tokenization_spaces /> | |
| <detokenizer_input_type value="i64" /> | |
| <eos_token_id value="49154" /> | |
| <handle_special_tokens_with_re /> | |
| <max_length /> | |
| <number_of_inputs value="1" /> | |
| <openvino_tokenizers_version value="2025.2.0.1-567-7885335c24b" /> | |
| <openvino_version value="2025.2.0-19140-c01cd93e24d-releases/2025/2" /> | |
| <original_tokenizer_class value="<class 'transformers.models.gpt2.tokenization_gpt2_fast.GPT2TokenizerFast'>" /> | |
| <pad_token_id value="2" /> | |
| <sentencepiece_version value="0.2.0" /> | |
| <skip_special_tokens value="True" /> | |
| <streaming_detokenizer value="False" /> | |
| <tiktoken_version value="0.9.0" /> | |
| <tokenizer_output_type value="i64" /> | |
| <tokenizers_version value="0.21.4.dev0" /> | |
| <transformers_version value="4.52.4" /> | |
| <use_max_padding value="False" /> | |
| <use_sentencepiece_backend value="False" /> | |
| <utf8_replace_mode value="replace" /> | |
| <with_detokenizer value="True" /> | |
| </rt_info> | |
| </net> | |