LLM

자동 라우팅을 지원하는 여러 LLM 프로바이더를 위한 통합 인터페이스

Overview

LLM 추상화 계층은 여러 AI 프로바이더와의 원활한 통합을 제공하여, 코드를 변경하지 않고도 OpenAI, Claude, Gemini, Ollama 간에 전환할 수 있게 합니다. 프로바이더별 구현, 메시지 포맷팅, 스트리밍을 처리하면서 모든 프로바이더에 걸쳐 일관된 API를 제공합니다.

Supported Providers

Astreus는 자동 모델 라우팅을 지원하는 4개의 주요 LLM 프로바이더를 지원합니다.

OpenAI

지원되는 14개 모델 전체:

• Latest: gpt-4.5, gpt-4.1, gpt-4.1-mini, gpt-4.1-nano, o4-mini, o4-mini-high, o3
• Stable: gpt-4o, gpt-4o-mini, gpt-4-turbo, gpt-4, gpt-3.5-turbo, gpt-3.5-turbo-16k, gpt-3.5-turbo-instruct
• API Key: OPENAI_API_KEY 환경 변수를 설정하세요

Anthropic Claude

지원되는 9개 모델 전체:

• Latest: claude-sonnet-4-20250514, claude-opus-4-20250514, claude-3.7-sonnet-20250224
• Stable: claude-3-5-sonnet-20241022, claude-3-5-sonnet-20240620, claude-3-5-haiku-20241022, claude-3-opus-20240229, claude-3-sonnet-20240229, claude-3-haiku-20240307
• API Key: ANTHROPIC_API_KEY 환경 변수를 설정하세요

Google Gemini

지원되는 12개 모델 전체:

• Latest: gemini-2.5-pro, gemini-2.5-pro-deep-think, gemini-2.5-flash, gemini-2.5-flash-lite
• Stable: gemini-2.0-flash, gemini-2.0-flash-thinking, gemini-2.0-flash-lite, gemini-2.0-pro-experimental, gemini-1.5-pro, gemini-1.5-flash, gemini-1.5-flash-8b, gemini-pro
• API Key: GEMINI_API_KEY 환경 변수를 설정하세요

Ollama (Local)

지원되는 31개 모델 전체:

• Latest: deepseek-r1, deepseek-v3, deepseek-v2.5, deepseek-coder, deepseek-coder-v2, qwen3, qwen2.5-coder, llama3.3, gemma3, phi4
• Popular: mistral-small, codellama, llama3.2, llama3.1, qwen2.5, gemma2, phi3, mistral, codegemma, wizardlm2
• Additional: dolphin-mistral, openhermes, deepcoder, stable-code, wizardcoder, magicoder, solar, yi, zephyr, orca-mini, vicuna
• Configuration: OLLAMA_BASE_URL을 설정하세요 (기본값: http://localhost:11434)

Configuration

# OpenAI
export OPENAI_API_KEY="your-openai-key"
export OPENAI_BASE_URL="https://api.openai.com/v1"  # Optional

# Anthropic Claude
export ANTHROPIC_API_KEY="your-anthropic-key"
export ANTHROPIC_BASE_URL="https://api.anthropic.com"  # Optional

# Google Gemini
export GEMINI_API_KEY="your-gemini-key"

# Ollama (Local)
export OLLAMA_BASE_URL="http://localhost:11434"  # Optional

import { Agent } from '@astreus-ai/astreus';

const agent = await Agent.create({
  name: 'MyAgent',
  model: 'gpt-4.5',  // Model automatically routes to correct provider
  temperature: 0.7,
  maxTokens: 2000
});

Usage Examples

Basic LLM Usage

import { getLLM } from '@astreus-ai/astreus';

const llm = getLLM();

// Generate response
const response = await llm.generateResponse({
  model: 'claude-sonnet-4-20250514',
  messages: [{ role: 'user', content: 'Explain quantum computing' }],
  temperature: 0.7,
  maxTokens: 1000
});

console.log(response.content);

Streaming Responses

// Stream response in real-time
for await (const chunk of llm.generateStreamResponse({
  model: 'gpt-4.5',
  messages: [{ role: 'user', content: 'Write a story about AI' }],
  stream: true
})) {
  if (!chunk.done) {
    process.stdout.write(chunk.content);
  }
}

Function Calling

const response = await llm.generateResponse({
  model: 'gpt-4o',
  messages: [{ role: 'user', content: 'What\'s the weather in Tokyo?' }],
  tools: [{
    type: 'function',
    function: {
      name: 'get_weather',
      description: 'Get current weather information',
      parameters: {
        type: 'object',
        properties: {
          location: { 
            type: 'string',
            description: 'City name'
          }
        },
        required: ['location']
      }
    }
  }]
});

// Handle tool calls
if (response.toolCalls) {
  response.toolCalls.forEach(call => {
    console.log(`Tool: ${call.function.name}`);
    console.log(`Args: ${call.function.arguments}`);
  });
}

LLM Options

다음 옵션으로 LLM 동작을 설정하세요.

interface LLMRequestOptions {
  model: string;              // Required: Model identifier
  messages: LLMMessage[];     // Required: Conversation history
  temperature?: number;       // Creativity level (0.0-1.0, default: 0.7)
  maxTokens?: number;         // Max output tokens (default: 4096)
  stream?: boolean;           // Enable streaming responses
  systemPrompt?: string;      // System instructions
  tools?: Tool[];             // Function calling tools
}

Parameter Details

• temperature: 무작위성을 제어합니다 (0.0 = 결정적, 1.0 = 매우 창의적)
• maxTokens: 응답의 최대 토큰 수입니다 (모델별로 다름)
• stream: 긴 응답에 대해 실시간 스트리밍을 활성화합니다
• systemPrompt: 모델의 동작과 컨텍스트를 설정합니다
• tools: 함수 호출 기능을 활성화합니다

Provider Features

Vision Models

각 프로바이더는 이미지 분석을 위한 특정 모델을 지원합니다.

OpenAI Vision

• gpt-4o, gpt-4o-mini, gpt-4-turbo, gpt-4-vision-preview
• gpt-4o-2024-08-06, gpt-4o-2024-05-13

Claude Vision

• claude-3-5-sonnet-20241022, claude-3-5-sonnet-20240620
• claude-3-opus-20240229, claude-3-sonnet-20240229, claude-3-haiku-20240307

Gemini Vision

• gemini-1.5-pro, gemini-1.5-flash
• gemini-1.0-pro-vision-latest, gemini-pro-vision

Ollama Vision

• llava, llava:7b, llava:13b, llava:34b
• llava-llama3, llava-phi3, moondream

Embedding Models

지식 베이스와 시맨틱 검색을 위한 모델입니다.

OpenAI Embeddings

• text-embedding-3-large (3072 dimensions)
• text-embedding-3-small (1536 dimensions)
• text-embedding-ada-002 (1536 dimensions)

Gemini Embeddings

• text-embedding-004
• embedding-001

Ollama Embeddings

• nomic-embed-text, mxbai-embed-large
• all-minilm, snowflake-arctic-embed

참고: Claude/Anthropic은 임베딩을 지원하지 않습니다. 임베딩 생성에는 OpenAI 또는 Gemini를 사용하세요.

Additional Environment Variables

# Dedicated embedding API keys (optional)
OPENAI_EMBEDDING_API_KEY="your-embedding-key"
OPENAI_EMBEDDING_BASE_URL="https://api.openai.com/v1"
GEMINI_EMBEDDING_API_KEY="your-gemini-embedding-key"

# Dedicated vision API keys (optional)
OPENAI_VISION_API_KEY="your-vision-key"
OPENAI_VISION_BASE_URL="https://api.openai.com/v1"
ANTHROPIC_VISION_API_KEY="your-anthropic-vision-key"
ANTHROPIC_VISION_BASE_URL="https://api.anthropic.com"
GEMINI_VISION_API_KEY="your-gemini-vision-key"

Model Selection Guide

For Code Generation

• Best: gpt-4o, claude-3-5-sonnet-20241022, deepseek-coder
• Fast: gpt-4o-mini, claude-3-5-haiku-20241022

For Reasoning Tasks

• Best: claude-opus-4-20250514, gpt-4.5, o3
• Balanced: claude-sonnet-4-20250514, gpt-4o

For Creative Writing

• Best: gpt-4.5, claude-3-opus-20240229
• Fast: gemini-2.5-pro, gpt-4o-mini

For Privacy/Local Use

• Best: deepseek-r1, llama3.3, qwen3
• Code: deepseek-coder, codellama