Skip to main content

Usage

Once you have nilAI API access, you can start using LLMs on nilAI nodes with any OpenAI-compatible library.

Getting Started with Private LLMs​

pnpm install @nillion/nilai-ts

You can either use:

  • API Key flow as the sole developer / organization or
  • Delegation Flow to provide permissions to another user / organization.

API Key Flow​

  1. Use https://nilai-a779.nillion.network/v1 as the BASE URL
  2. Check available models or query the /v1/models endpoint or
  3. Select an available model and use it with the /v1/chat/completions nilAI node endpoint

With OpenAI compatibility, you can use any OpenAI library. Here's an example for querying the Llama-3.1-8B model:

import "dotenv/config";
import { NilaiOpenAIClient, NilAuthInstance } from "@nillion/nilai-ts";

// To obtain an API key, navigate to https://subscription.nillion.com
// and create a new subscription.
// The API key will be displayed in the subscription details.
// The NilaiOpenAIClient class automatically handles the NUC token creation and management.

const API_KEY = process.env.NILLION_API_KEY;

async function main() {
// Initialize the client in API key mode
// For sandbox, use the following:
const client = new NilaiOpenAIClient({
    baseURL: "https://nilai-a779.nillion.network/v1/",
    apiKey: API_KEY,
    nilauthInstance: NilAuthInstance.SANDBOX,
    // For production, use the following:
    // nilauthInstance: NilAuthInstance.PRODUCTION,
});

// Make a request to the Nilai API
const response = await client.chat.completions.create({
    model: "google/gemma-3-27b-it",
    messages: [
    { role: "user", content: "Hello! Can you help me with something?" }
    ],
});

console.log(`Response: ${response.choices[0].message.content}`);
}

// Run the example
main().catch(console.error);

Delegation flow​

To use the delegation flow, you need to create a delegation token server.

The server then creates the delegation tokens and managing their expiration and usage. Then the delegation token allows you to make requests to the nilAI API.

examples/1-delegation-token.ts
loading...