BlazeInferDocs
Integrations

LlamaIndex

Use BlazeInfer through LlamaIndex's OpenAI-compatible LLM.

SupportedTested with: llama-index-llms-openai-like 0.8.1 · Last verified: 2026-11-02

LlamaIndex's OpenAILike LLM accepts a custom API base, so it points at BlazeInfer.

pip install llama-index-llms-openai-like
export BLAZE_API_KEY="blz_..."
import os
from llama_index.llms.openai_like import OpenAILike

llm = OpenAILike(
    model="google/gemma-4-31b-it",
    api_base="https://blaze.e2enetworks.net/v1",
    api_key=os.environ["BLAZE_API_KEY"],
    is_chat_model=True,
    is_function_calling_model=True,
    context_window=81920,
)

print(llm.complete("Say hello in one sentence."))

is_chat_model=True is required. Without it, requests go to /v1/completions and return junk. is_function_calling_model=True enables agents and structured output. context_window defaults to 3,900 tokens, so set it from the console.