OpenAI

Integrate OpenAI with Portkey to get production metrics for your requests and make chat completions, audio, image generation, structured outputs, function calling, fine-tuning, batch, and more requests.

Provider Slug: openai

Overview

Portkey integrates with OpenAI’s APIs to help you create production-grade AI sppd with enhanced reliability, observability, and governance features.

Getting Started

1. Obtain your OpenAI API Key

   Visit the OpenAI dashboard to generate your API key.

2. Create a Virtual Key in Portkey

   Portkey’s virtual key vault simplifies your interaction with OpenAI. Virtual keys act as secure aliases for your actual API keys, offering enhanced security and easier management through budget limits to control your API usage.

   Use the Portkey app to create a virtual key associated with your OpenAI API key.

3. Initialize the Portkey Client

   Now that you have your virtual key, set up the Portkey client:

   Portkey Hosted App

   Use the Portkey API key and the OpenAI virtual key to initialize the client in your preferred programming language.

   Python

   from portkey_ai import Portkey
   
   portkey = Portkey(
       api_key="PORTKEY_API_KEY",  # Replace with your Portkey API key
       provider="@PROVIDER"   # Replace with your virtual key for OpenAI
   )
   

   Open Source Use

   Alternatively, use Portkey’s Open Source AI Gateway to enhance your app’s reliability with minimal code:

   Python

   from portkey_ai import Portkey, PORTKEY_GATEWAY_URL
   
   portkey = Portkey(
       api_key="dummy",  # Replace with your Portkey API key
       base_url=PORTKEY_GATEWAY_URL,
       Authorization="OPENAI_API_KEY", # Replace with your OpenAI API Key
       provider="openai"
   )
   

🔥 That’s it! You’ve integrated Portkey into your application with just a few lines of code. Now let’s explore making requests using the Portkey client.

Supported Models

Supported OpenAI Models

• GPT-4o
• GPT-4o mini
• o1-preview
• o1-mini
• GPT-4 Turbo
• GPT-4
• GPT-3.5 Turbo
• DALL·E
• TTS (Text-to-Speech)
• Whisper
• Embeddings
• Moderation
• GPT base

OpenAI Supported Features

Chat Completions

Generate chat completions using OpenAI models through Portkey:

Python

completion = portkey.chat.completions.create(
    messages=[{"role": "user", "content": "Say this is a test"}],
    model="gpt-4o"
)

print(completion.choices[0].message.content)

Streaming

Stream responses for real-time output in your applications:

Python

chat_complete = portkey.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Say this is a test"}],
    stream=True
)

for chunk in chat_complete:
    print(chunk.choices[0].delta.content or "", end="", flush=True)

Function Calling

Leverage OpenAI’s function calling capabilities through Portkey:

Node.js

let tools = [{
    type: "function",
    function: {
        name: "getWeather",
        description: "Get the current weather",
        parameters: {
            type: "object",
            properties: {
                location: { type: "string", description: "City and state" },
                unit: { type: "string", enum: ["celsius", "fahrenheit"] }
            },
            required: ["location"]
        }
    }
}];

let response = await portkey.chat.completions.create({
    model: "gpt-4o",
    messages: [
        { role: "system", content: "You are a helpful assistant." },
        { role: "user", content: "What's the weather like in Delhi - respond in JSON" }
    ],
    tools,
    tool_choice: "auto",
});

console.log(response.choices[0].finish_reason);

Vision

Process images alongside text using OpenAI’s vision capabilities:

Python

response = portkey.chat.completions.create(
    model="gpt-4-vision-preview",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "What's in this image?"},
                {
                    "type": "image_url",
                    "image_url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg",
                },
            ],
        }
    ],
    max_tokens=300,
)

print(response)

Embeddings

Generate embeddings for text using OpenAI’s embedding models:

Python

response = portkey.embeddings.create(
    input="Your text string goes here",
    model="text-embedding-3-small"
)

print(response.data[0].embedding)

Transcription and Translation

Portkey supports both Transcription and Translation methods for STT models:

Python

audio_file= open("/path/to/file.mp3", "rb")

# Transcription
transcription = portkey.audio.transcriptions.create(
  model="whisper-1",
  file=audio_file
)
print(transcription.text)

# Translation
translation = portkey.audio.translations.create(
  model="whisper-1",
  file=audio_file
)
print(translation.text)

Text to Speech

Convert text to speech using OpenAI’s TTS models:

Python

from pathlib import Path

speech_file_path = Path(__file__).parent / "speech.mp3"
response = portkey.audio.speech.create(
  model="tts-1",
  voice="alloy",
  input="Today is a wonderful day to build something people love!"
)

with open(speech_file_path, "wb") as f:
    f.write(response.content)

Prompt Caching

Implement prompt caching to improve performance and reduce costs:

Prompt Caching Guide

Learn how to implement prompt caching for OpenAI models with Portkey.

Structured Output

Use structured outputs for more consistent and parseable responses:

Structured Outputs Guide

Discover how to use structured outputs with OpenAI models in Portkey.

Supported Endpoints and Parameters

Endpoint	Supported Parameters
complete	model, prompt, max_tokens, temperature, top_p, n, stream, logprobs, echo, stop, presence_penalty, frequency_penalty, best_of, logit_bias, user, seed, suffix
embed	model, input, encoding_format, dimensions, user
chatComplete	model, messages, functions, function_call, max_tokens, temperature, top_p, n, stream, stop, presence_penalty, frequency_penalty, logit_bias, user, seed, tools, tool_choice, response_format, logprobs, top_logprobs, stream_options, service_tier, parallel_tool_calls, max_completion_tokens
imageGenerate	prompt, model, n, quality, response_format, size, style, user
createSpeech	model, input, voice, response_format, speed
createTranscription	All parameters supported
createTranslation	All parameters supported

---

Portkey’s Advanced Features

Track End-User IDs

Portkey allows you to track user IDs passed with the user parameter in OpenAI requests, enabling you to monitor user-level costs, requests, and more:

Python

response = portkey.chat.completions.create(
  model="gpt-4o",
  messages=[{"role": "user", "content": "Say this is a test"}],
  user="user_123456"
)

When you include the user parameter in your requests, Portkey logs will display the associated user ID, as shown in the image below:

In addition to the user parameter, Portkey allows you to send arbitrary custom metadata with your requests. This powerful feature enables you to associate additional context or information with each request, which can be useful for analysis, debugging, or other custom use cases.

Learn More About Metadata

Explore how to use custom metadata to enhance your request tracking and analysis.

Using The Gateway Config

Here’s a simplified version of how to use Portkey’s Gateway Configuration:

1. Create a Gateway Configuration

   You can create a Gateway configuration using the Portkey Config Dashboard or by writing a JSON configuration in your code. In this example, requests are routed based on the user’s subscription plan (paid or free).

   config = {
     "strategy": {
       "mode": "conditional",
       "conditions": [
         {
           "query": { "metadata.user_plan": { "$eq": "paid" } },
           "then": "gpt4o"
         },
         {
           "query": { "metadata.user_plan": { "$eq": "free" } },
           "then": "gpt-3.5"
         }
       ],
       "default": "base-gpt4"
     },
     "targets": [
       {
         "name": "gpt4o",
         "provider":"@xx"
       },
       {
         "name": "gpt-3.5",
         "provider":"@yy"
       }
     ]
   }
   

2. Process Requests

   When a user makes a request, it will pass through Portkey’s AI Gateway. Based on the configuration, the Gateway routes the request according to the user’s metadata.

   
3. Set Up the Portkey Client

   Pass the Gateway configuration to your Portkey client. You can either use the config object or the Config ID from Portkey’s hosted version.

   Python

   from portkey_ai import Portkey
   
   portkey = Portkey(
       api_key="PORTKEY_API_KEY",
       provider="@PROVIDER",
       config=portkey_config
   )
   

That’s it! Portkey seamlessly allows you to make your AI app more robust using built-in gateway features. Learn more about advanced gateway features:

Load Balancing

Distribute requests across multiple targets based on defined weights.

Fallbacks

Automatically switch to backup targets if the primary target fails.

Conditional Routing

Route requests to different targets based on specified conditions.

Caching

Enable caching of responses to improve performance and reduce costs.

Guardrails

Portkey’s AI gateway enables you to enforce input/output checks on requests by applying custom hooks before and after processing. Protect your user’s/company’s data by using PII guardrails and many more available on Portkey Guardrails:

{
	"provider:"@openai-xxx",
	"before_request_hooks": [{
		"id": "input-guardrail-id-xx"
	}],
	"after_request_hooks": [{
		"id": "output-guardrail-id-xx"
	}]
}

Learn More About Guardrails

Explore Portkey’s guardrail features to enhance the security and reliability of your AI applications.

Next Steps

The complete list of features supported in the SDK are available in our comprehensive documentation:

Portkey SDK Documentation

Explore the full capabilities of the Portkey SDK and how to leverage them in your projects.

---

Limitations

Portkey does not support the following OpenAI features:

• Streaming for audio endpoints
• Chat completions feedback API
• File management endpoints

For the most up-to-date information on supported features and endpoints, please refer to our API Reference.