LangChain with Oracle Cloud Infrastructure (OCI) Embeddings

Oracle Cloud Infrastructure (OCI) Generative AI provides state-of-the-art language models through a managed service. In this guide, we'll look at how to use OCI's embedding models with LangChain through the OCIGenAIEmbeddings class.

What is OCIGenAIEmbeddings?

OCIGenAIEmbeddings is a LangChain integration that allows you to generate embeddings (vector representations) of text using Oracle Cloud's Generative AI models. These embeddings can be used for semantic search, text similarity, and other NLP tasks.

Reference

Key parameters:

• model_id: ID of the OCI embedding model to use
• service_endpoint: OCI GenAI service endpoint URL
• compartment_id: OCID of your OCI compartment
• auth_type: Authentication method ('API_KEY', 'SECURITY_TOKEN', 'INSTANCE_PRINCIPLE', 'RESOURCE_PRINCIPLE')
• auth_profile: Profile name from OCI config file (default: 'DEFAULT')
• batch_size: Number of texts to process per request (max 96)
• truncate: How to handle long texts ('NONE', 'START', 'END')

How to Use OCIGenAIEmbeddings

Installation

First, install the required packages:

pip install -U oci langchain-community

Authentication Setup

The OCIGenAIEmbeddings class supports multiple authentication methods. The most common is using an API key:

from langchain_community.embeddings import OCIGenAIEmbeddings

embeddings = OCIGenAIEmbeddings(
    model_id="your_model_id",
    service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com",
    compartment_id="your_compartment_ocid",
    auth_type="API_KEY",  # Default auth method
    auth_profile="DEFAULT"  # Uses ~/.oci/config default profile
)

You can also use session token authentication:

embeddings = OCIGenAIEmbeddings(
    model_id="your_model_id",
    service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com",
    compartment_id="your_compartment_ocid",
    auth_type="SECURITY_TOKEN",
    auth_profile="my_profile"  # Specify custom profile
)

Generating Embeddings

Single Text Embedding

To generate an embedding for a single piece of text:

query = "What is machine learning?"
embedding = embeddings.embed_query(query)
# Returns a list of floats representing the text embedding

Batch Text Embeddings

To generate embeddings for multiple texts at once:

documents = [
    "Machine learning is a subset of AI",
    "Neural networks are inspired by the human brain",
    "Deep learning uses multiple layers"
]
doc_embeddings = embeddings.embed_documents(documents)
# Returns a list of embedding vectors, one for each input text

Async Usage

For better performance in async applications, you can use the async methods:

import asyncio

async def generate_embeddings():
    query = "What is AI?"
    embedding = await embeddings.aembed_query(query)
    
    documents = ["Text 1", "Text 2", "Text 3"]
    doc_embeddings = await embeddings.aembed_documents(documents)
    
    return embedding, doc_embeddings

# Run the async function
embedding, doc_embeddings = asyncio.run(generate_embeddings())

Advanced Configuration

You can customize the embedding generation with additional parameters:

embeddings = OCIGenAIEmbeddings(
    model_id="your_model_id",
    service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com",
    compartment_id="your_compartment_ocid",
    batch_size=50,  # Process 50 texts per request
    truncate="END",  # Truncate long texts from the end
    model_kwargs={
        # Additional model-specific parameters
    }
)

Remember to handle credentials securely and follow Oracle's best practices for authentication when deploying to production environments.