Weighting

You can influence the model selection by providing a weighting object in the extra_body parameter. The weighting object contains three parameters: quality, costs, and latency. The values of these parameters should be between 0.0 and 5.0. The default value for all three parameters is 1.0. Adjust them to your preferences to prioritize what is important to you.

Using OpenAI:

...

client.chat.completions.create(
    model="auto",
    messages=[<omitted>],
    extra_body: {
		'weighting': {
			'quality': 1.5,
			'costs': 1.0,
			'latency': 0.8,
		}
    }
)

Using langchain:

llm = ChatOpenAI(
        model="auto",
        base_url='https://api.airouter.io',
        model_kwargs={
            'extra_body': {
                'weighting': {
                    'quality': 1.5,
                    'costs': 1.0,
                    'latency': 0.8,
                }
            }
        }
    )

Using langchain.js:

import { ChatOpenAI } from '@langchain/openai';
import { PromptTemplate } from '@langchain/core/prompts';

const llm = new ChatOpenAI(
  {
    model: 'auto',
    apiKey: '<THE-API-KEY-YOU-GENERATED>'
    configuration: { baseURL: "https://api.airouter.io" },
    modelKwargs: {
      weighting: {
        quality: 1.5,
        costs: 1.0,
        latency: 0.8,
      }
    },
  },
);
const prompt = PromptTemplate.fromTemplate(
    'What is the capital of France?',
);
const chain = prompt.pipe(llm);
const result = await chain.invoke({});