Quick Start

Get up and running with InferenceIQ in 5 minutes. Start routing your inference requests to save 73% on costs automatically.

Installation

Install InferenceIQ using your package manager:

Copy

                        pip install inferenceiq
                    
                        npm install inferenceiq

Basic Usage

Initialize the client and make your first inference request:

Copy

                        from inferenceiq import InferenceIQ

iq = InferenceIQ(api_key="your-api-key")

response = iq.chat.completions.create(
    model="gpt-4",
    messages=[
        {"role": "user", "content": "What is AI?"}
    ]
)

print(response.choices[0].message.content)
                    

                        const { InferenceIQ } = require('inferenceiq');

const iq = new InferenceIQ({
    apiKey: 'your-api-key'
});

const response = await iq.chat.completions.create({
    model: 'gpt-4',
    messages: [
        { role: 'user', content: 'What is AI?' }
    ]
});

console.log(response.choices[0].message.content);
                    

Configuration

Configure routing preferences and provider settings:

Option	Type	Default	Description
`api_key`	string	Required	Your InferenceIQ API key
`strategy`	string	"cost"	Routing strategy: "cost", "latency", or "balanced"
`fallback`	string	null	Provider to fallback to if primary fails
`timeout`	number	30000	Request timeout in milliseconds
`cache`	boolean	true	Enable response caching

API Endpoints

Available REST endpoints for inference requests:

Method	Endpoint	Status
POST	`/v1/chat/completions`	Stable
POST	`/v1/completions`	Stable
POST	`/v1/embeddings`	Stable
GET	`/v1/models`	Stable
GET	`/v1/usage`	Beta

FAQ

How does InferenceIQ save money?

InferenceIQ routes each request to the cheapest provider that meets your quality requirements. With intelligent routing, you save 73% on average without sacrificing performance or quality.

Is my data secure?

Yes. We never store your prompts or completions. All data is encrypted in transit (TLS 1.3) and we comply with SOC 2 Type II, GDPR, and HIPAA.

What providers are supported?

We support OpenAI, Anthropic, Google Cloud, Together, Hugging Face, AWS Bedrock, and 15+ other providers. New providers are added regularly.

Can I customize routing behavior?

Yes! You can set custom routing rules based on model, latency requirements, cost thresholds, and more. See our Routing Rules guide for details.