How to stream LLM responses

Turn on streaming

Set stream to true and read the chunks as they arrive. Users see text immediately instead of waiting for the full reply.

Example

from openai import OpenAI

client = OpenAI(api_key="YOUR_TOKENREX_KEY", base_url="https://tokenrex.ai/api/public")

resp = client.chat.completions.create(
    model="gpt-6-astra",
    messages=[{"role": "user", "content": "Write a haiku"}],
    stream=True,
)
for chunk in resp:
    print(chunk.choices[0].delta.content or "", end="")

Questions

Do I need a separate key for each provider?

No. One TokenRex API key works for every enabled model; change only the model ID.