meloqui

meloqui logo and tagline banner

Melodic speech for LLMs

Enterprise-ready multi-provider LLM chat SDK with a unified interface for OpenAI, Anthropic, Google, and Ollama (local models).

Features

Stage 1 - Foundation ✅

Stage 2 - Core Features ✅

Stage 3 - Enterprise Features ✅

Stage 4 - Advanced Features ✅

Stage 5 - Extensibility ✅

Coming Soon

Stage 6 - Future Enhancements

Documentation

Installation

npm install meloqui

Quick Start

Basic Chat

import { ChatClient } from 'meloqui';

// Create a client (API key from OPENAI_API_KEY env variable)
const client = new ChatClient({
  provider: 'openai',
  model: 'gpt-4'
});

// Send a message
const response = await client.chat('Hello! What is TypeScript?', {
  temperature: 0.7,
  maxTokens: 150
});

console.log(response.content);
// Output: TypeScript is a strongly typed programming language...

Streaming Responses

import { ChatClient } from 'meloqui';

const client = new ChatClient({
  provider: 'openai',
  model: 'gpt-4'
});

// Stream responses in real-time
for await (const chunk of client.stream('Tell me a story')) {
  process.stdout.write(chunk.content);
}

Conversation History

import { ChatClient } from 'meloqui';

const client = new ChatClient({
  provider: 'openai',
  model: 'gpt-4',
  conversationId: 'user-123' // Enable automatic history tracking
});

await client.chat('My name is Alice');
await client.chat('What is my name?'); // Remembers context

// Retrieve full history
const history = await client.getHistory();
console.log(history); // [{ role: 'user', content: 'My name is Alice' }, ...]

Persistent Storage

import { ChatClient, FileStorage } from 'meloqui';

const storage = new FileStorage('./conversations');

const client = new ChatClient({
  provider: 'openai',
  model: 'gpt-4',
  conversationId: 'user-123',
  storage // Conversations persist across sessions
});

await client.chat('Remember: my favorite color is blue');

// Later, in a new session...
const newClient = new ChatClient({
  provider: 'openai',
  model: 'gpt-4',
  conversationId: 'user-123',
  storage // Loads previous conversation
});

await newClient.chat('What is my favorite color?'); // Remembers!

Ollama (Local Models)

Run models locally with Ollama for privacy and offline use.

import { ChatClient } from 'meloqui';

// Use Ollama with local models (no API key needed)
const client = new ChatClient({
  provider: 'ollama',
  model: 'llama3.2'  // or mistral, codellama, phi, etc.
});

const response = await client.chat('Explain quantum computing');
console.log(response.content);

Remote Ollama Server:

const client = new ChatClient({
  provider: 'ollama',
  model: 'llama3.2',
  baseUrl: 'http://my-server:11434'
});

Requirements: Ollama must be running locally (default: http://localhost:11434) or accessible via network. Install with brew install ollama (macOS) or see ollama.ai for other platforms.

Enterprise Features

Retry Logic

Automatic retry with exponential backoff for resilience against transient failures.

import { ChatClient } from 'meloqui';

const client = new ChatClient({
  provider: 'openai',
  model: 'gpt-4',
  retryConfig: {
    maxAttempts: 3,           // Maximum retry attempts
    initialBackoffMs: 1000,   // Initial backoff delay
    maxBackoffMs: 10000,      // Maximum backoff delay
    backoffMultiplier: 2      // Exponential multiplier
  }
});

// Automatically retries on network errors, rate limits, and transient failures
const response = await client.chat('Your message');

Features:

Example: See examples/with-retry.ts

Rate Limiting

Token bucket rate limiting to prevent hitting API limits.

import { ChatClient } from 'meloqui';

const client = new ChatClient({
  provider: 'openai',
  model: 'gpt-4',
  rateLimitConfig: {
    requestsPerMinute: 60,    // Max requests per minute
    tokensPerMinute: 90000    // Max tokens per minute
  }
});

// Requests automatically queued to stay within limits
const response = await client.chat('Your message');

Features:

Default Limits:

Example: See examples/with-rate-limiting.ts

Logging

Structured logging with configurable log levels for observability.

import { ChatClient, ConsoleLogger } from 'meloqui';

// Create logger with desired level
const logger = new ConsoleLogger({
  level: 'debug'  // 'debug' | 'info' | 'warn' | 'error'
});

const client = new ChatClient({
  provider: 'openai',
  model: 'gpt-4',
  logger
});

// All operations are logged with context
const response = await client.chat('Your message');

Log Levels:

Logged Events:

Custom Loggers:

Implement the ILogger interface for custom logging:

import { ILogger } from 'meloqui';

class CustomLogger implements ILogger {
  debug(message: string, context?: Record<string, unknown>): void {
    // Your implementation
  }
  info(message: string, context?: Record<string, unknown>): void {
    // Your implementation
  }
  warn(message: string, context?: Record<string, unknown>): void {
    // Your implementation
  }
  error(message: string, error?: Error, context?: Record<string, unknown>): void {
    // Your implementation
  }
}

const client = new ChatClient({
  provider: 'openai',
  model: 'gpt-4',
  logger: new CustomLogger()
});

Example: See examples/with-logging.ts

Tool System

Register and execute tools/functions for LLM integration.

import { ToolRegistry } from 'meloqui';

// Create registry
const registry = new ToolRegistry();

// Register a tool
registry.registerTool(
  'getWeather',
  async (location: string) => {
    // Your implementation
    return { temperature: 72, condition: 'sunny' };
  },
  {
    description: 'Get current weather for a location'
  }
);

// Execute directly
const result = await registry.executeTool('getWeather', 'Paris');
console.log(result); // { temperature: 72, condition: 'sunny' }

// Get all tools for LLM context
const tools = registry.getTools();

// Use with ChatClient for automatic tool calling
const client = new ChatClient({
  provider: 'openai',
  model: 'gpt-4',
  tools: registry
});

Features:

Tool Interface:

interface Tool {
  name: string;
  handler: ToolHandler;
  metadata: {
    description?: string;
    [key: string]: unknown;
  };
}

Example: See examples/with-tools.ts for both simple and agentic tool calling demonstrations.

Custom Provider Plugins

Create your own provider to use any LLM:

import { ChatClient, ProviderPlugin } from 'meloqui';

class MyCustomProvider implements ProviderPlugin {
  readonly name = 'my-provider';
  readonly capabilities = {
    chat: true,
    streaming: true,
    toolCalling: false,
    vision: false,
    audio: false
  };

  async chat(messages, options) {
    // Your implementation here
    return { content: 'Response', role: 'assistant' };
  }

  async *stream(messages, options) {
    // Your streaming implementation
    yield { content: 'Chunk', role: 'assistant' };
  }

  supportsTools() { return this.capabilities.toolCalling; }
  supportsStreaming() { return this.capabilities.streaming; }

  async chatWithTools(messages, tools, options) {
    // Tool calling implementation (if supported)
    return { content: 'Response', role: 'assistant' };
  }

  async *streamWithTools(messages, tools, options) {
    // Streaming with tools (if supported)
    yield { content: 'Chunk', role: 'assistant' };
  }
}

// Use your custom provider
const provider = new MyCustomProvider();
const client = new ChatClient({ provider, model: 'my-model' });

const response = await client.chat('Hello!');

Features:

Capability Validation:

The client validates capabilities before operations:

// If your provider has streaming: false
const client = new ChatClient({ provider, model: 'model' });
await client.stream('Hello'); // Throws CapabilityError

Configuration

Basic Configuration

import { ChatClient } from 'meloqui';

const client = new ChatClient({
  provider: 'openai',
  model: 'gpt-4',
  apiKey: 'your-api-key' // Optional: defaults to OPENAI_API_KEY env var
});

Advanced Configuration

const client = new ChatClient({
  provider: 'openai',
  model: 'gpt-4',

  // Authentication (recommended over apiKey)
  auth: {
    type: 'api-key',
    apiKey: 'your-api-key'
  },

  // Custom base URL (for proxies or local models)
  baseUrl: 'https://custom-endpoint.com/v1',

  // Conversation tracking
  conversationId: 'user-123-session-456'
});

Chat Options

const response = await client.chat('Your message', {
  temperature: 0.7,      // Sampling temperature (0-2)
  maxTokens: 1000,       // Maximum tokens to generate
  topP: 0.9,             // Nucleus sampling parameter
  model: 'gpt-3.5-turbo' // Override default model
});

API Reference

ChatClient

Main client for interacting with LLM providers.

Constructor

new ChatClient(config: ChatConfig)

Parameters:

Methods

chat(message: string, options?: ChatOptions): Promise<ChatResponse>

Send a chat message and get a response.

Parameters:

Returns:

Example:

const response = await client.chat('Hello!', { temperature: 0.7 });
console.log(response.content);
stream(message: string, options?: ChatOptions): AsyncIterator<StreamChunk>

Stream a chat response for real-time output.

Parameters:

Returns:

Example:

for await (const chunk of client.stream('Tell me a story')) {
  process.stdout.write(chunk.content);
}
getHistory(): Promise<Message[]>

Retrieve the full conversation history for the current conversation.

Returns:

Example:

const history = await client.getHistory();
history.forEach(msg => {
  console.log(`${msg.role}: ${msg.content}`);
});
clearHistory(): Promise<void>

Clear the conversation history for the current conversation.

Example:

await client.clearHistory();

Types

ChatResponse

interface ChatResponse {
  content: string;           // The generated response
  role: 'assistant';         // Always 'assistant'
  metadata?: {
    model?: string;          // Model that generated response
    tokensUsed?: number;     // Total tokens consumed
    finishReason?: string;   // Why generation stopped
  };
}

ChatOptions

interface ChatOptions {
  temperature?: number;      // 0-2, higher = more random
  maxTokens?: number;        // Max tokens to generate
  topP?: number;             // Nucleus sampling (0-1)
  model?: string;            // Override default model
}

Message

interface Message {
  role: 'user' | 'assistant' | 'system' | 'tool';
  content: string;
  name?: string;             // For tool messages
  toolCallId?: string;       // For tool responses
}

Examples

See the examples directory for complete working examples:

Core Features

Enterprise Features

To run examples:

# Set your API key
export OPENAI_API_KEY=your-key-here

# Core examples
npm run example:basic      # Basic chat
npm run example:streaming  # Streaming responses
npm run example:history    # Conversation history
npm run example:storage    # Persistent storage

# Enterprise examples
npm run example:retry      # Retry logic
npm run example:rate-limit # Rate limiting
npm run example:logging    # Structured logging
npm run example:tools      # Tool system (no API key needed)

Development

Prerequisites

Setup

# Install dependencies
npm install

# Run tests
npm test

# Run tests in watch mode
npm run test:watch

# Type checking
npm run typecheck

# Linting
npm run lint

# Build
npm run build

Project Structure

agent-chat/
├── src/
│   ├── client/          # ChatClient implementation
│   ├── providers/       # Provider implementations (OpenAI, etc.)
│   ├── types/           # TypeScript type definitions
│   └── index.ts         # Public API exports
├── examples/            # Usage examples
├── dist/                # Compiled output
└── tests/               # Test files (colocated with source)

Running Tests

# All tests
npm test

# Watch mode
npm run test:watch

# Coverage
npm test -- --coverage

Architecture

The SDK follows a layered architecture:

ChatClient (Public API)
    ↓
Provider Interface (IProvider)
    ↓
Provider Implementations (OpenAI, Anthropic, etc.)
    ↓
LangChain Integration
    ↓
LLM APIs

Design Principles

Roadmap

Stage 1: Foundation ✅ (Complete)

Stage 2: Core Features ✅ (Complete)

Stage 3: Enterprise Features ✅ (Complete)

Stage 4: Advanced Features ✅ (Complete)

Stage 5: Extensibility ✅ (Complete)

Stage 6: Local Models ✅ (Complete)

Stage 7: Future Enhancements

Contributing

Contributions are welcome! We value all contributions, from bug reports to new features.

Before contributing, please read:

Quick Start for Contributors

  1. Fork and clone the repository
  2. Install dependencies: npm install
  3. Create a feature branch
  4. Write tests first (TDD)
  5. Implement functionality
  6. Ensure all tests pass
  7. Submit a Pull Request

See CONTRIBUTING.md for detailed guidelines.

License

MIT License - see LICENSE for details.

Support

This site is open source. Improve this page.