# Qwen2.5 VL AWQ 3B

**Qwen2.5 VL AWQ 3B** is a vision-language model that offers advanced capabilities such as visual analysis, agentic reasoning, long video comprehension, visual localization, and structured output generation. It supports 3 bilion parameters.

## Model details

| Category | Details |
|----------|----------|
| **Model Name** | Qwen2.5 VL |
| **Version** | AWQ 3B |
| **Model Category** | VLM |
| **Size** | 3B params |
| **HuggingFace Model** | [Qwen/Qwen2.5-VL-3B-Instruct-AWQ](https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct-AWQ) |
| **OpenAI Compatible endpoint** | [Chat API Overview](https://developers.openai.com/api/reference/overview) |
| **License** | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/resolve/main/markdown/apache-2.0.md) |

## Capabilities 

| Feature | Details |
|---------|--------|
| Tool Calling | ✅ |
| Context Length | 32k tokens |
| Supports LoRA | ✅ |
| Input data | Text + Image |

## Usage 

### Basic chat completion

This is a basic chat completion request example using this model:

```ts
const modelResponse = await Azion.AI.run("qwen-qwen25-vl-3b-instruct-awq", {
  "stream": true,
  "messages": [
    {
      "role": "system",
      "content": "You are a helpful assistant."
    },
    {
      "role": "user",
      "content": "Name the european capitals"
    }
  ]
})
```

| Property      | Type | Description                                                    |
| --------------------- | -------- | ------------------------------------------------------------------ |
| `stream`              | boolean | Indicates whether to stream the response.   |
| `messages[]`          | array   | Array of message objects, containing the system and user messages. |
| `messages[].role`    | string  | The role of the message sender.           |
| `messages[].content` | string  | The content of the message.               |

Response example:

```json
{
  "id": "chatcmpl-e27716424abf4b3f891ff4850470cb09",
  "object": "chat.completion",
  "created": 1746821581,
  "model": "qwen-qwen25-vl-3b-instruct-awq",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "reasoning_content": null,
        "content": "Sure! Here is a list of some European capitals...",
        "tool_calls": []
      },
      "logprobs": null,
      "finish_reason": "stop",
      "stop_reason": null
    }
  ],
  "usage": {
    "prompt_tokens": 9,
    "total_tokens": 527,
    "completion_tokens": 518,
    "prompt_tokens_details": null
  },
  "prompt_logprobs": null
}
```

| Property                       | Type | Description                                                            |
| -------------------------------------- | -------- | -------------------------------------------------------------------------- |
| `id`                                   | string   | Unique identifier for the chat completion.                             |
| `object`                               | string   | Type of object.                      |
| `created`                              | number   | Timestamp when the chat completion was created.                        |
| `model`                                | string   | Model used for chat completion. |
| `choices[]`                            | array    | Array containing choices made by the model.                            |
| `choices[].index`                     | number   | Index of the choice.      |
| `choices[].message`                   | object   | Message returned by the model in the choice.                     |
| `choices[].message.role`              | string   | Role of the message sender.                               |
| `choices[].message.reasoning_content` | string   | Reasoning behind the model's response.             |
| `choices[].message.content`           | string   | Content of message sent by the model.                              |
| `choices[].message.tool_calls[]`      | array    | Array of tool calls.                                  |
| `choices[].logprobs`                  | number   | Log probabilities.                                     |
| `choices[].finish_reason`             | string   | Reason for finishing the response.                            |
| `usage`                                | object   | Contains information about token usage.                                    |
| `usage.prompt_tokens`                  | number   | Number of tokens in the prompt.                                        |
| `usage.total_tokens`                   | number   | Total number of tokens used.                     |
| `usage.completion_tokens`              | number   | Number of tokens used in the completion.                               |
| `usage.prompt_tokens_details`          | string   | Detailed information about prompt tokens.              |
| `prompt_logprobs`                      | number   | Log probabilities for the prompt.                      |

### Tool Calling Example

This is a tool calling request example using this model:

```ts
const modelResponse = await Azion.AI.run("qwen-qwen25-vl-3b-instruct-awq", {
  "stream": true,
  "messages": [
    {
      "role": "system",
      "content": "You are a helpful assistant with access to tools."
    },
    {
      "role": "user",
      "content": "What is the weather in London?"
    }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get the current weather for a location",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "The city and state"
            }
          },
          "required": [
            "location"
          ]
        }
      }
    }
  ]
})
```

| Property                                               | Type | Description                                                           |
| -------------------------------------------------------------- | --------- | ------------------------------------------------------------------------- |
| `messages[]`                                                   | array     | Array containing the messages exchanged between the system and the user.  |
| `messages[].role`                                             | string    | The role of the message sender.                   |
| `messages[].content`                                          | string    | The content of the message.                      |
| `tools[]`                                                      | array     | Array containing the tools used in the query.                             |
| `tools[].type`                                                | string    | The type of the tool.                    |
| `tools[].function`                                            | object   | The function being called by the tool.                                    |
| `tools[].function.name`                                       | string    | The name of the function.                                 |
| `tools[].function.description`                                | string    | A description of the function. |
| `tools[].function.parameters`                                 | object   | The parameters required by the function.                                  |

Response example:

```json
{
  "id": "chatcmpl-88affc4730cf4219a06d2b15aad9ad44",
  "object": "chat.completion",
  "created": 1746821866,
  "model": "qwen-qwen25-vl-3b-instruct-awq",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "reasoning_content": null,
        "content": null,
        "tool_calls": [
          {
            "id": "chatcmpl-tool-fd3311e75aed4cbfbeb7244ced77379f",
            "type": "function",
            "function": {
              "name": "get_weather",
              "arguments": "{\"location\": \"London\"}"
            }
          }
        ]
      },
      "logprobs": null,
      "finish_reason": "tool_calls",
      "stop_reason": null
    }
  ],
  "usage": {
    "prompt_tokens": 293,
    "total_tokens": 313,
    "completion_tokens": 20,
    "prompt_tokens_details": null
  },
  "prompt_logprobs": null
}
```

| Property                                      | Type | Description                                                                  |
| ----------------------------------------------------- | -------- | -------------------------------------------------------------------------------- |
| `choices[]`                                           | array    | Array containing the choices made by the assistant.                              |
| `choices[].index`                                    | integer  | The index of the first choice.                                  |
| `choices[].message`                                  | object   | The message sent by the assistant in the first choice.                           |
| `choices[].message.role`                             | string   | The role of the assistant in the message.                          |
| `choices[].message.reasoning_content`                | null     | The reasoning content.                                       |
| `choices[].message.content`                          | null     | The content of the message.                                  |
| `choices[].message.tool_calls[]`                     | array    | Array of tool calls invoked by the assistant in the message.                     |
| `choices[].message.tool_calls[].id`                 | string   | The ID of the tool call.                                                   |
| `choices[].message.tool_calls[].type`               | string   | The type of the tool call.                      |
| `choices[].message.tool_calls[].function`           | object   | The function called by the tool.                                           |
| `choices[].message.tool_calls[].function.name`      | string   | The name of the function called.                                 |
| `choices[].message.tool_calls[].function.arguments` | string   | The arguments passed to the function in JSON format. |
| `logprobs`                                            | null     | The log probabilities value.                                          |
| `finish_reason`                                       | string   | The reason why the completion finished.              |
| `stop_reason`                                         | null     | The reason why the completion was stopped.                                     |
| `usage.prompt_tokens`                                 | integer  | The number of prompt tokens used.                                                |
| `usage.total_tokens`                                  | integer  | The total number of tokens used.                           |
| `usage.completion_tokens`                             | integer  | The number of completion tokens used.                                            |
| `usage.prompt_tokens_details`                         | null     | The details for prompt tokens.                               |
| `prompt_logprobs`                                     | null     | Log probabilities for the prompt.                                          |

### Multimodal (text + image) example

This is a multimodal request example using this model:

```ts
const modelResponse = await Azion.AI.run("qwen-qwen25-vl-3b-instruct-awq", {
  "stream": true,
  "messages": [
    {
      "role": "system",
      "content": "You are a helpful assistant."
    },
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "What is in this image?"
        },
        {
          "type": "image_url",
          "image_url": {
            "url": "https://example.com/image.jpg"
          }
        }
      ]
    }
  ]
})
```

| Property                       | Type | Description                                                     |
| -------------------------------------- | --------- | ------------------------------------------------------------------- |
| `messages[]`                           | array    | Array containing the messages sent by the system and the user.      |
| `messages[].role`                     | string   | The role of the message sender.                            |
| `messages[].content[]`                | array    | Array containing the content objects in the message.         |
| `messages[].content[]`               | object   | The content object in the message.                     |
| `messages[].content[].type`          | string   | The type of the content object.                      |
| `messages[].content[].image_url`     | object   | The image URL object in the content object.                  |
| `messages[].content[].image_url.url` | string   | The URL of the image.             |
| `stream`                               | boolean  | Indicates whether the response should be streamed. |

The response will be similar to the one in the basic chat completion example.

## JSON schema

```json
{
    "$schema": "http://json-schema.org/draft-07/schema#",
    "type": "object",
    "required": ["messages"],
    "properties": {
        "messages": {"type": "array", "items": {"$ref": "#/components/schemas/Message"}},
        "temperature": {"type": "number", "minimum": 0, "maximum": 2},
        "top_p": {"type": "number", "minimum": 0, "maximum": 1, "default": 1},
        "n": {"type": "integer", "minimum": 1, "default": 1},
        "stream": {"type": "boolean", "default": false},
        "max_tokens": {"type": "integer", "minimum": 1},
        "presence_penalty": {"type": "number", "minimum": -2, "maximum": 2, "default": 0},
        "frequency_penalty": {"type": "number", "minimum": -2, "maximum": 2, "default": 0},
        "tools": {"type": "array", "items": {"$ref": "#/components/schemas/ToolDefinition"}}
    },
    "components": {
        "schemas": {
            "Message": {"oneOf": [{"$ref": "#/components/schemas/SystemMessage"},{"$ref": "#/components/schemas/UserMessage"},{"$ref": "#/components/schemas/AssistantMessage"},{"$ref": "#/components/schemas/ToolMessage"}]},
            "SystemMessage": {"type": "object", "required": ["role","content"], "properties": {"role": {"type": "string", "enum": ["system"]}, "content": {"$ref": "#/components/schemas/TextContent"}}},
            "UserMessage": {"type": "object", "required": ["role","content"], "properties": {"role": {"type": "string", "enum": ["user"]}, "content": {"oneOf": [{"type": "string"},{"type": "array", "items": {"oneOf": [{"$ref": "#/components/schemas/TextContentItem"},{"$ref": "#/components/schemas/ImageContentItem"}]}}]}}},
            "AssistantMessage": {"oneOf": [{"$ref": "#/components/schemas/AssistantMessageWithoutToolCalls"},{"$ref": "#/components/schemas/AssistantMessageWithToolCalls"}]},
            "ToolMessage": {"type": "object", "required": ["role","content","tool_call_id"], "properties": {"role": {"enum": ["tool"]}, "content": {"type": "string"}, "tool_call_id": {"type": "string"}}},
            "AssistantMessageWithoutToolCalls": {"type": "object", "required": ["role","content"], "properties": {"role": {"type": "string", "enum": ["assistant"]}, "content": {"$ref": "#/components/schemas/TextContent"}}, "not": {"required": ["tool_calls"]}},
            "AssistantMessageWithToolCalls": {"type": "object", "required": ["role","tool_calls"], "properties": {"role": {"type": "string", "enum": ["assistant"]}, "tool_calls": {"type": "array", "items": {"$ref": "#/components/schemas/ToolCalls"}}}},
            "TextContent": {"oneOf": [{"type": "string"},{"type": "array", "items": {"$ref": "#/components/schemas/TextContentItem"}}], "description": "Text content that can be provided either as a simple string or as an array of TextContentItem objects"},
            "ImageContent": {"type": "array", "items": {"$ref": "#/components/schemas/ImageContentItem"}},
            "TextContentItem": {"type": "object", "required": ["type","text"], "properties": {"type": {"type": "string", "enum": ["text"]}, "text": {"type": "string"}}},
            "ImageContentItem": {"type": "object", "required": ["type","image_url"], "properties": {"type": {"type": "string", "enum": ["image_url"]}, "image_url": {"type": "object", "required": ["url"], "properties": {"url": {"type": "string", "format": "uri"}}}}},
            "ToolCalls": {"type": "object", "required": ["function","id","type"], "properties": {"function": {"type": "object", "required": ["name","arguments"], "properties": {"name": {"type": "string"}, "arguments": {"type": "string"}}}, "id": {"type": "string"}, "type": {"enum": ["function"]}}, "description": "The name and arguments of a function that should be called, as generated by the model."},
            "ToolDefinition": {"type": "object", "required": ["type","function"], "properties": {"type": {"type": "string", "enum": ["function"]}, "function": {"type": "object", "required": ["name"], "properties": {"name": {"type": "string"}, "description": {"type": "string"}, "parameters": {"type": "object", "additionalProperties": true}, "strict": {"type": "boolean", "default": false}}}}, "description": "Definition of a tool that can be used by the model"}
        }
    }
}
```