花崗巖3.0

Granite 3.0 是一個(gè)開源、輕量級(jí)的生成語言模型系列，專為一系列企業(yè)級(jí)任務(wù)而設(shè)計(jì)。它原生支援多語言功能、編碼、推理和工具使用，適合企業(yè)環(huán)境。

我測(cè)試了運(yùn)行這個(gè)模型，看看它可以處理哪些任務(wù)。

環(huán)境設(shè)定

我在 Google Colab 中設(shè)定了 Granite 3.0 環(huán)境，並使用以下指令安裝了必要的函式庫：

!pip install torch torchvision torchaudio
!pip install accelerate
!pip install -U transformers

執(zhí)行

我測(cè)試了Granite 3.0的2B和8B型號(hào)的性能。

2B型號(hào)

我運(yùn)行了 2B 模型。這是 2B 模型的程式碼範(fàn)例：

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

device = "auto"
model_path = "ibm-granite/granite-3.0-2b-instruct"
tokenizer = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForCausalLM.from_pretrained(model_path, device_map=device)
model.eval()

chat = [
    { "role": "user", "content": "Please list one IBM Research laboratory located in the United States. You should only output its name and location." },
]
chat = tokenizer.apply_chat_template(chat, tokenize=False, add_generation_prompt=True)
input_tokens = tokenizer(chat, return_tensors="pt").to("cuda")
output = model.generate(**input_tokens, max_new_tokens=100)
output = tokenizer.batch_decode(output)
print(output[0])

輸出

<|start_of_role|>user<|end_of_role|>Please list one IBM Research laboratory located in the United States. You should only output its name and location.<|end_of_text|>
<|start_of_role|>assistant<|end_of_role|>1. IBM Research - Austin, Texas<|end_of_text|>

8B型號(hào)

將2b替換為8b即可使用8B模型。以下是 8B 模型的沒有角色和使用者輸入欄位的程式碼範(fàn)例：

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

device = "auto"
model_path = "ibm-granite/granite-3.0-8b-instruct"
tokenizer = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForCausalLM.from_pretrained(model_path, device_map=device)
model.eval()

chat = [
    { "content": "Please list one IBM Research laboratory located in the United States. You should only output its name and location." },
]
chat = tokenizer.apply_chat_template(chat, tokenize=False, add_generation_prompt=True)

input_tokens = tokenizer(chat, add_special_tokens=False, return_tensors="pt").to("cuda")
output = model.generate(**input_tokens, max_new_tokens=100)
generated_text = tokenizer.decode(output[0][input_tokens["input_ids"].shape[1]:], skip_special_tokens=True)
print(generated_text)

輸出

1. IBM Almaden Research Center - San Jose, California

函數(shù)呼叫

我探索了函數(shù)呼叫功能，並使用虛擬函數(shù)對(duì)其進(jìn)行了測(cè)試。這裡，get_current_weather 被定義為傳回模擬天氣資料。

虛擬函數(shù)

import json

def get_current_weather(location: str) -> dict:
    """
    Retrieves current weather information for the specified location (default: San Francisco).
    Args:
        location (str): Name of the city to retrieve weather data for.
    Returns:
        dict: Dictionary containing weather information (temperature, description, humidity).
    """
    print(f"Getting current weather for {location}")

    try:
        weather_description = "sample"
        temperature = "20.0"
        humidity = "80.0"

        return {
            "description": weather_description,
            "temperature": temperature,
            "humidity": humidity
        }
    except Exception as e:
        print(f"Error fetching weather data: {e}")
        return {"weather": "NA"}

即時(shí)創(chuàng)作

我建立了一個(gè)呼叫函數(shù)的提示：

functions = [
    {
        "name": "get_current_weather",
        "description": "Get the current weather",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "The city and country code, e.g. San Francisco, US",
                }
            },
            "required": ["location"],
        },
    },
]
query = "What's the weather like in Boston?"
payload = {
    "functions_str": [json.dumps(x) for x in functions]
}
chat = [
    {"role":"system","content": f"You are a helpful assistant with access to the following function calls. Your task is to produce a sequence of function calls necessary to generate response to the user utterance. Use the following function calls as required.{payload}"},
    {"role": "user", "content": query }
]

響應(yīng)生成

使用以下程式碼，我產(chǎn)生了一個(gè)回應(yīng)：

instruction_1 = tokenizer.apply_chat_template(chat, tokenize=False, add_generation_prompt=True)
input_tokens = tokenizer(instruction_1, return_tensors="pt").to("cuda")
output = model.generate(**input_tokens, max_new_tokens=1024)
generated_text = tokenizer.decode(output[0][input_tokens["input_ids"].shape[1]:], skip_special_tokens=True)
print(generated_text)

輸出

{'name': 'get_current_weather', 'arguments': {'location': 'Boston'}}

這證實(shí)了模型能夠根據(jù)指定城市產(chǎn)生正確的函數(shù)呼叫。

增強(qiáng)互動(dòng)流程的格式規(guī)範(fàn)

Granite 3.0 允許格式規(guī)格以促進(jìn)結(jié)構(gòu)化格式的回應(yīng)。本節(jié)說明如何使用 [UTTERANCE] 進(jìn)行回應(yīng)，並使用 [THINK] 進(jìn)行內(nèi)心想法。

另一方面，由於函數(shù)呼叫以純文字形式輸出，因此可能需要實(shí)作單獨(dú)的機(jī)制來區(qū)分函數(shù)呼叫和常規(guī)文字回應(yīng)。

指定輸出格式

以下是指導(dǎo) AI 輸出的範(fàn)例提示：

prompt = """You are a conversational AI assistant that deepens interactions by alternating between responses and inner thoughts.
<Constraints>
* Record spoken responses after the [UTTERANCE] tag and inner thoughts after the [THINK] tag.
* Use [UTTERANCE] as a start marker to begin outputting an utterance.
* After [THINK], describe your internal reasoning or strategy for the next response. This may include insights on the user's reaction, adjustments to improve interaction, or further goals to deepen the conversation.
* Important: **Use [UTTERANCE] and [THINK] as a start signal without needing a closing tag.**
</Constraints>

Follow these instructions, alternating between [UTTERANCE] and [THINK] formats for responses.
<output example>
example1:
  [UTTERANCE]Hello! How can I assist you today?[THINK]I’ll start with a neutral tone to understand their needs. Preparing to offer specific suggestions based on their response.[UTTERANCE]Thank you! In that case, I have a few methods I can suggest![THINK]Since I now know what they’re looking for, I'll move on to specific suggestions, maintaining a friendly and approachable tone.
...
</output example>

Please respond to the following user_input.
<user_input>
Hello! What can you do?
</user_input>
"""

執(zhí)行程式碼範(fàn)例

產(chǎn)生回應(yīng)的程式碼：

chat = [
    { "role": "user", "content": prompt },
]
chat = tokenizer.apply_chat_template(chat, tokenize=False, add_generation_prompt=True)

input_tokens = tokenizer(chat, return_tensors="pt").to("cuda")
output = model.generate(**input_tokens, max_new_tokens=1024)
generated_text = tokenizer.decode(output[0][input_tokens["input_ids"].shape[1]:], skip_special_tokens=True)
print(generated_text)

範(fàn)例輸出

輸出如下：

[UTTERANCE]Hello! I'm here to provide information, answer questions, and assist with various tasks. I can help with a wide range of topics, from general knowledge to specific queries. How can I assist you today?
[THINK]I've introduced my capabilities and offered assistance, setting the stage for the user to share their needs or ask questions.

[UTTERANCE] 和 [THINK] 標(biāo)籤已成功使用，允許有效的回應(yīng)格式。

根據(jù)提示的不同，輸出中有時(shí)可能會(huì)出現(xiàn)結(jié)束標(biāo)籤（例如[/UTTERANCE]或[/THINK]），但總的來說，一般都可以成功指定輸出格式。

串流程式碼範(fàn)例

讓我們看看如何輸出流響應(yīng)。

以下程式碼使用 asyncio 和線程庫來非同步傳輸來自 Granite 3.0 的回應(yīng)。

!pip install torch torchvision torchaudio
!pip install accelerate
!pip install -U transformers

範(fàn)例輸出

執(zhí)行上述程式碼將產(chǎn)生以下格式的非同步回應(yīng)：

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

device = "auto"
model_path = "ibm-granite/granite-3.0-2b-instruct"
tokenizer = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForCausalLM.from_pretrained(model_path, device_map=device)
model.eval()

chat = [
    { "role": "user", "content": "Please list one IBM Research laboratory located in the United States. You should only output its name and location." },
]
chat = tokenizer.apply_chat_template(chat, tokenize=False, add_generation_prompt=True)
input_tokens = tokenizer(chat, return_tensors="pt").to("cuda")
output = model.generate(**input_tokens, max_new_tokens=100)
output = tokenizer.batch_decode(output)
print(output[0])

此範(fàn)例示範(fàn)了成功的串流。每個(gè)token都是非同步生成並順序顯示，讓使用者可以即時(shí)查看生成過程。

概括

Granite 3.0 即使使用 8B 機(jī)型也能提供相當(dāng)強(qiáng)的反應(yīng)。函數(shù)呼叫和格式規(guī)範(fàn)功能也運(yùn)作良好，顯示其具有廣泛的應(yīng)用潛力。

以上是我嘗試過花崗巖。的詳細(xì)內(nèi)容。更多資訊請(qǐng)關(guān)注PHP中文網(wǎng)其他相關(guān)文章！