Connect to a vLLM Managed Inference endpoint
Send a chat-completion request to a Managed Inference endpoint from the CLI or an OpenAI-compatible client.
Send a chat-completion request to a Managed Inference endpoint from the CLI or any OpenAI-compatible client.
Prerequisites
You need the following before you start:
- A running Managed Inference Job with a serving endpoint. See Create a Managed Inference Job.
- Your Managed Inference API key, if the endpoint requires an authorization header. See Create an API key.
- The CosmicAC CLI installed and configured, for the CLI method. See Install the CLI.
Steps
Find your endpoint name
List your endpoints:
cosmicac models statsEach endpoint appears as Endpoint: <endpoint-name>. Copy the one you want to call.
Send a request
Use the CLI or any OpenAI-compatible client.
If you enabled Require Authorization header when you created the job, you need an API key. If you didn't, answer no to the CLI's API key prompt and omit the Authorization header from the curl request.
Start an interactive chat session:
cosmicac inference chatAnswer the prompts to start the session.
The cosmicac inference chat session asks for these values:
- Use an API key: answer yes to authenticate, then enter the key at the masked prompt.
- Endpoint ID: the endpoint name from the previous step.
The session accepts these commands:
| Command | What it does |
|---|---|
/video <url> | Attaches a video to your next message. The URL must point to a video file the model can download in one request. A streaming URL that delivers in chunks, such as a YouTube link, doesn't work. |
/clear-video | Removes the attachment. |
/think and /no-think | Show or hide the model's reasoning in later replies. |
/exit or /quit | Ends the session, as does Ctrl+C. |
/no-think hides the reasoning and asks the model to stop producing it. Some models, such as MiniMax, ignore that request and keep spending tokens on thinking you no longer see.