Usage
Once you have nilAI API access, you can start using LLMs on nilAI nodes with any OpenAI-compatible library.
The Old Flow
will be deprecated by the 1st of August 2025. Please use the NUCS
flow to create your API key.
- NUCS
- Old Flow
Getting Started with Private LLMs
- Python
- TypeScript
- pip
- uv
pip install nilai-py
uv pip install nilai-py
pnpm install @nillion/nilai-ts
You can either use:
API Key
flow as the sole developer / organization orDelegation Flow
to provide permissions to another user / organization.
API Key Flow
- Use
https://nilai-a779.nillion.network/nuc/v1
as the BASE URL - Check available models or query the
/v1/models
endpoint or - Select an available model and use it with the
/v1/chat/completions
nilAI node endpoint
With OpenAI compatibility, you can use any OpenAI library. Here's an example for querying the Llama-3.1-8B
model:
- Python
- Typescript
loading...
import "dotenv/config";
import { NilaiOpenAIClient, NilAuthInstance } from "@nillion/nilai-ts";
// To obtain an API key, navigate to https://nilpay.vercel.app/
// and create a new subscription.
// The API key will be displayed in the subscription details.
// The NilaiOpenAIClient class automatically handles the NUC token creation and management.
const API_KEY = process.env.NILLION_API_KEY;
async function main() {
// Initialize the client in API key mode
// For sandbox, use the following:
const client = new NilaiOpenAIClient({
baseURL: "https://nilai-a779.nillion.network/nuc/v1/",
apiKey: API_KEY,
nilauthInstance: NilAuthInstance.SANDBOX,
// For production, use the following:
// nilauthInstance: NilAuthInstance.PRODUCTION,
});
// Make a request to the Nilai API
const response = await client.chat.completions.create({
model: "meta-llama/Llama-3.2-3B-Instruct",
messages: [
{ role: "user", content: "Hello! Can you help me with something?" }
],
});
console.log(`Response: ${response.choices[0].message.content}`);
}
// Run the example
main().catch(console.error);
Delegation flow
To use the delegation flow, you need to create a delegation token server.
The server then creates the delegation tokens and managing their expiration and usage. Then the delegation token allows you to make requests to the nilAI API.
- Python
- Typescript
loading...
loading...
Getting Started with SecretLLM
Getting started with SecretLLM is straightforward:
- Select a nilAI node url
- Query the
/v1/models
endpoint or check available models - Select an available model and use it with the
/v1/chat/completions
nilAI node endpoint
Since SecretLLM is OpenAI-compatible, you can use any OpenAI library. Here's an example for querying the Llama-3.1-8B
model:
- Node.js
- Python
- Next JS (API routes)
loading...
Example response from SecretLLM
Signature and Response
Signature: MEYCIQCfafKvn1nRmBdFYyxp+mJrRVaTuuN1ME40TRDC4Wg/HAIhAIpm4H2QCqlPmgnOd8/aJXuvWvIAEzKl3ObbbyFMz6iI
Response: As a fitness coach, I'd say it's not necessarily about which one is better, but rather about balance and moderation. Both salads and pizza can be part of a healthy diet, but it depends on how they're prepared and consumed.
A salad, when made with nutrient-dense ingredients like leafy greens, vegetables, lean proteins, and healthy fats, can be an excellent choice for a healthy meal. It's high in fiber, vitamins, and minerals, and can help support weight loss, improve digestion, and boost energy levels.
On the other hand, pizza, when made with refined flour, processed meats, and high amounts of cheese, can be a less-than-ideal choice. It's high in calories, saturated fat, sodium, and refined carbohydrates, which can lead to weight gain, inflammation, and other health problems.
However, if you're craving pizza, there are ways to make it healthier. Look for options that use:
1. Whole-wheat or cauliflower crust
2. Lean protein sources like chicken, turkey, or plant-based options
3. Fresh vegetables like bell peppers, onions, and mushrooms
4. Low-fat cheese or dairy-free alternatives
5. Herbs and spices for flavor instead of salt and sugar
In contrast, a salad can be made healthier by:
1. Using a variety of colorful vegetables
2. Adding lean protein sources like grilled chicken, salmon, or tofu
3. Incorporating healthy fats like avocado, nuts, or seeds
4. Using a homemade vinaigrette dressing instead of store-bought
5. Limiting or avoiding added sugars and refined grains
Ultimately, the better choice between salad and pizza depends on your individual needs and preferences. If you're looking for a quick, easy, and nutritious meal, a salad might be the way to go. But if you're craving something more indulgent, a healthier pizza option can be a better choice.
As a fitness coach, I always recommend balance and variety in your diet. Aim to include a mix of whole, unprocessed foods, and indulge in moderation. Listen to your body and make choices that nourish your mind, body, and soul.
from openai import OpenAI
# Initialize the OpenAI client
client = OpenAI(
base_url="https://nilai-<node>.nillion.network/v1/",
api_key="YOUR_API_KEY"
)
# Send a chat completion request
response = client.chat.completions.create(
model="meta-llama/Llama-3.1-8B-Instruct",
messages=[
{
"role": "system",
"content": "You are a helpful assistant."
},
{
"role": "user",
"content": "What is your name?"
}
],
stream=False
)
loading...