Streaming Response Messages

Last updated on October 12, 2023

GigaChat enables you to send tokens as a stream to process a reply while generating it.

In streaming mode, GigaChat sends tokens as a stream of short messages that correspond to the server-sent events protocol. The last event in a stream is data: [DONE].

To enable streaming mode, send the "stream": true parameter in the POST /chat/completionsrequest.

Request example:

curl https://gigachat.devices.sberbank.ru/api/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer Token ***' \
  -d '{
  "model": "GigaChat:latest",
  "messages": [{"role": "user", "content": "Say this is a test!"}],
  "temperature": 0.7,
  "stream": true
}'

Event stream example:

data: 
{
    "choices":
    [
        {
            "delta":
            {
                "content": "GigaСhat",
                "role": "assistant"
            },
            "index": 0
        }
    ],
    "created": 1683034756,
    "model": "GigaChat:latest",
    "object": "chat.completion"
}

data:
{
    "choices":
    [
        {
            "delta":
            {
                "content": " спешит"
            },
            "index": 0
        }
    ],
    "created": 1683034756,
    "model": "GigaChat:latest",
    "object": "chat.completion"
}

data:
{
    "choices":
    [
        {
            "delta":
            {
                "content": " на"
            },
            "index": 0
        }
    ],
    "created": 1683034756,
    "model": "GigaChat:latest",
    "object": "chat.completion"
}

data:
{
    "choices":
    [
        {
            "delta":
            {
                "content": " помощь"
            },
            "index": 0
        }
    ],
    "created": 1683034756,
    "model": "GigaChat:latest",
    "object": "chat.completion"
}

data: [DONE]