Skip to main content

Usage

Once you have a nilAI API key, you can start using LLMs on nilAI nodes with any OpenAI-compatible library.

info

The Old Flow will be deprecated by the 1st of August 2025. Please use the NUCS flow to create your API key.

Getting Started with Private LLMs​

Getting started is straightforward

pip install nilai-py

You can either use:

  • API Key flow as the sole developer / organization or
  • Delegation Flow to provide permissions to another user / organization.

API Key Flow​

  1. Use https://nilai-a779.nillion.network/nuc/v1 as the BASE URL
  2. Check available models or query the /v1/models endpoint or
  3. Select an available model and use it with the /v1/chat/completions nilAI node endpoint

With OpenAI compatibility, you can use any OpenAI library. Here's an example for querying the Llama-3.1-8B model:

from nilai_py import Client

from config import API_KEY

def main():
    client = Client(
        base_url="https://nilai-a779.nillion.network/nuc/v1",
        api_key=API_KEY,
    )

    # Make a request to the Nilai API
    response = client.chat.completions.create(
        model="meta-llama/Llama-3.2-3B-Instruct",
        messages=[
            {"role": "user", "content": "Hello! Can you help me with something?"}
        ],
    )

    print(f"Response: {response.choices[0].message.content}")

if __name__ == "__main__":
    main()

Delegation flow​

To use the delegation flow, you need to create a delegation token server.

The server then creates the delegation tokens and managing their expiration and usage. Then the delegation token allows you to make requests to the nilAI API.


from nilai_py import (
    Client,
    DelegationTokenServer,
    AuthType,
    DelegationServerConfig,
    DelegationTokenRequest,
    DelegationTokenResponse,

)

from config import API_KEY

def main(): # >>> Server initializes a delegation token server # The server is responsible for creating delegation tokens # and managing their expiration and usage.
    server = DelegationTokenServer(
    private_key=API_KEY,
    config=DelegationServerConfig(
        expiration_time=10, # 10 seconds validity of delegation tokens
        token_max_uses=1, # 1 use of a delegation token
        ),
    )

    # >>> Client initializes a client
    # The client is responsible for making requests to the Nilai API.
    # We do not provide an API key but we set the auth type to DELEGATION_TOKEN
    client = Client(
        base_url="https://nilai-a779.nillion.network/nuc/v1/",
        auth_type=AuthType.DELEGATION_TOKEN,
    )

    # >>> Client produces a delegation request
    delegation_request: DelegationTokenRequest = client.get_delegation_request()

    # <<< Server creates a delegation token
    delegation_token: DelegationTokenResponse = server.create_delegation_token(
        delegation_request
    )

    # >>> Client sets internally the delegation token
    client.update_delegation(delegation_token)

    # >>> Client uses the delegation token to make a request
    response = client.chat.completions.create(
        model="meta-llama/Llama-3.2-3B-Instruct",
        messages=[
            {"role": "user", "content": "Hello! Can you help me with something?"}
        ],
    )

    print(f"Response: {response.choices[0].message.content}")

if  __name__ == "__main__":
    main()