AI Engineeringlangchain

Demystifying LangChain Models: From LLMs to Embeddings in Plain English

P&C

Prompt & Circuit Editorial

May 14, 2026 · 12 Min Read

Demystifying LangChain Models: From LLMs to Embeddings in Plain English

Why this guide exists

LangChain tutorials throw a lot of vocabulary at newcomers: LLMs, chat models, embeddings, vector stores. The concepts are simpler than the terminology suggests, and the model layer is the right place to start because everything else in LangChain is built on top of it. This guide explains that layer in plain English and shows code that works with the current LangChain 1.x packages.

What the model layer is

In plain English, LangChain's model layer is a universal adapter. Each provider (OpenAI, Anthropic, Google, a model running locally through Ollama, and many more) has its own SDK and request format. LangChain wraps them behind one consistent interface, so switching providers is mostly a matter of changing a model name and installing the matching integration package.

Two families of models

Language models: text in, text out

InputOutput
"What is the capital of India?""New Delhi."

Typical uses: question answering, drafting, summarisation, code generation, and the reasoning step inside agents.

Embedding models: text in, numbers out

InputOutput
"Delhi is the capital of India."[0.023, -0.091, 0.044, ...]

An embedding is a list of numbers (a vector) that represents the meaning of a piece of text. Texts with similar meanings produce vectors that are close together, which lets software compare meaning rather than exact words.

Typical uses: semantic search, recommendations, clustering, de-duplication, and the retrieval step in retrieval-augmented generation (RAG).

Chat models versus legacy text-completion LLMs

LangChain historically had two kinds of language model wrapper:

FeatureText-completion LLM (legacy)Chat model
InputOne plain stringA list of messages with roles (system, user, assistant, tool)
OutputA plain stringA message object (AIMessage)
ExtrasFewTool calling, structured output, multimodal content
Use todayOlder code and a few completion-only modelsAlmost everything

Providers have largely moved to chat-style APIs. OpenAI's text-davinci-003, for example, was shut down on 4 January 2024. In current LangChain code you will almost always use a chat model.

One clarification: a chat model does not remember earlier turns on its own. It accepts a list of messages, so your application (or a framework such as LangGraph) is responsible for storing the history and sending it back each time. What chat models add is a structured way to express roles: the system message sets behaviour, user messages carry requests, and assistant and tool messages record what happened.

Closed versus open-weight models

CategoryExamplesProsCons
Hosted, proprietaryOpenAI, Anthropic and Google model APIsStrong quality, no infrastructure, managed scalingPer-token cost, data leaves your network, less control
Open-weightgpt-oss, DeepSeek and many others on Hugging FaceRun anywhere, full control, data can stay localYou supply GPUs and operations, or pay a hosting provider

A sensible path is to prototype with a hosted API, then evaluate open-weight models once you know your quality bar, volumes and data-handling requirements.

Getting and running open-weight models

The Hugging Face Hub is the main catalogue of open-weight models. There are two ways to run them:

  1. Hosted inference. Hugging Face Inference Providers routes requests to partner providers (Groq, Together, Fireworks and others) through one API and token, including an OpenAI-compatible endpoint. It is pay-as-you-go, with monthly included credits for PRO users and Team and Enterprise organisations.
  2. Local or self-hosted. Tools such as Ollama run models on your own machine, and LangChain integrates with them through packages like langchain-ollama.

For a rough memory estimate, multiply the parameter count by the bytes per parameter: a 7-billion-parameter model needs about 14 GB for its weights at 16-bit precision and roughly a quarter of that at 4-bit quantisation, plus headroom for activations and context.

The invoke() method

invoke() is the standard way to call anything in LangChain: a model, a chain or an agent. You pass an input, you get a result.

pip install -U "langchain[openai]"
export OPENAI_API_KEY="sk-..."
from langchain.chat_models import init_chat_model

model = init_chat_model("gpt-5.5")
response = model.invoke("Explain gravity in one sentence.")
print(response.text)

init_chat_model picks the right provider integration from the model name, so switching to another provider is a one-line change once its package is installed. invoke() returns an AIMessage, and .text gives you its text.

To use roles, pass a list of messages:

from langchain.messages import HumanMessage, SystemMessage

conversation = [
    SystemMessage("You are a concise physics tutor."),
    HumanMessage("Why do astronauts float on the ISS?"),
]
print(model.invoke(conversation).text)

A local model works the same way:

from langchain_ollama import ChatOllama  # pip install -U langchain-ollama

local = ChatOllama(model="gpt-oss:20b", temperature=0)  # after: ollama pull gpt-oss:20b
print(local.invoke("Explain gravity in one sentence.").text)

Older tutorials import OpenAI from langchain.llms. That path belongs to pre-1.0 LangChain and the legacy completion interface; use the chat model integrations instead.

Tuning model behaviour

ParameterWhat it doesPractical guidance
temperatureControls randomness of the output.Low for extraction, code and facts; higher for brainstorming.
max_tokensLimits the length of the response.Set it to cap cost and latency.
timeoutSeconds to wait before cancelling the request.Always set one in production.

Pass these to init_chat_model or the provider class. Not every model supports every parameter, so check the provider's documentation for the model you choose.

Embeddings without the maths

  • Input: any text, from a sentence to a document chunk.
  • Output: a fixed-length vector that captures its meaning.
  • Comparison: similarity between two texts is computed from their vectors, most often with cosine similarity.
pip install -U langchain-openai
from langchain_core.vectorstores import InMemoryVectorStore
from langchain_openai import OpenAIEmbeddings

embeddings = OpenAIEmbeddings(model="text-embedding-3-large")

vector = embeddings.embed_query("Delhi is the capital of India.")
print(len(vector))  # the vector's dimensionality

store = InMemoryVectorStore.from_texts(
    [
        "Delhi is the capital of India.",
        "The team captain announced the squad for the tour.",
        "Photosynthesis converts light into chemical energy.",
    ],
    embedding=embeddings,
)
for doc in store.similarity_search("Who leads the national cricket side?", k=1):
    print(doc.page_content)

The query shares no keywords with the captain sentence, yet it should come back first, because the two sentences are close in meaning. That is semantic search in miniature.

How semantic search works

  1. Index: split your documents into chunks, embed each chunk, and store the vectors.
  2. Query: embed the user's question with the same embedding model.
  3. Match: return the stored chunks whose vectors are nearest to the query vector.

In RAG, those matched chunks are then placed in the chat model's context so it can answer from your data.

Where the vectors live

InMemoryVectorStore is fine for experiments. For anything persistent, LangChain integrates with dedicated vector databases and libraries, such as Chroma (simple local development), FAISS (an in-process similarity-search library), and managed services such as Pinecone and Weaviate. Storing vectors means you embed each document once instead of on every request.

Applications built on embeddings

  • Search over documentation, tickets or knowledge bases
  • "Chat with your documents" assistants
  • Recommendations for articles, products or media
  • Duplicate and near-duplicate detection
  • Routing requests to the right team or tool

Key takeaways

  1. The model layer is a universal adapter across providers.
  2. There are two families: language models (text out) and embedding models (vectors out).
  3. Use chat models; text-completion LLMs are legacy.
  4. Chat models take message lists. Memory is something your application manages.
  5. Embeddings turn meaning into numbers, which is what makes semantic search and RAG work.

Next step: a small FAQ bot

  1. Embed your FAQ entries with OpenAIEmbeddings (or another embedding model) and load them into a vector store.
  2. For each question, retrieve the top matches with similarity_search.
  3. Put the matches in the prompt and call a chat model with invoke().
  4. Adjust chunking and the number of results, and measure answer quality on real questions.

Older tutorials use RetrievalQA for step 3. That is a pre-1.0 chain; LangChain's current RAG tutorial instead gives an agent a search tool that queries the vector store, which is the pattern to learn today.

Sources

Tags:langchainllmembeddingsragpython
Share Article

Responses (0)

Sign in to join the conversation and share your thoughts.

P&C

Prompt & Circuit Editorial

Contributing Editor

Articles on AI infrastructure for cloud architects, written from official vendor sources and checked claim by claim before publishing.