Skip to main content

Usage

Once you have nilAI API access, you can start using LLMs on nilAI nodes with any OpenAI-compatible library.

info

The Old Flow will be deprecated by the 1st of August 2025. Please use the NUCS flow to create your API key.

Getting Started with Private LLMs

pip install nilai-py

You can either use:

  • API Key flow as the sole developer / organization or
  • Delegation Flow to provide permissions to another user / organization.

API Key Flow

  1. Use https://nilai-a779.nillion.network/nuc/v1 as the BASE URL
  2. Check available models or query the /v1/models endpoint or
  3. Select an available model and use it with the /v1/chat/completions nilAI node endpoint

With OpenAI compatibility, you can use any OpenAI library. Here's an example for querying the Llama-3.1-8B model:

examples/0-api_key_mode.py
loading...

Delegation flow

To use the delegation flow, you need to create a delegation token server.

The server then creates the delegation tokens and managing their expiration and usage. Then the delegation token allows you to make requests to the nilAI API.

examples/1-delegation_token_mode.py
loading...