Street Learner

Author

12 min read

Last Updated: a year ago

Building Intelligent Chatbots with LangChain and OpenAI: A Complete Guide

In the age of AI, chatbots have evolved from simple rule-based systems to sophisticated conversational agents powered by Large Language Models (LLMs). With OpenAI’s GPT models and the LangChain framework, building chatbots that can handle nuanced conversations, remember context, and output structured responses has never been easier. In this blog, we’ll explore step-by-step how to leverage LangChain and OpenAI to create advanced chatbots.

1. Introduction to OpenAI Chat Completion

OpenAI’s ChatCompletion API allows developers to interact with LLMs using a conversational paradigm. A chat consists of messages with roles: system, user, and assistant.

System: Provides instructions to the AI about its behavior.
User: Represents input from a human user.
Assistant: Represents the AI’s responses.

Here’s an example of creating a sarcastic chatbot:

import os
from dotenv import load_dotenv
import openai

load_dotenv()
openai.api_key = os.getenv('OPENAI_API_KEY')
client = openai.OpenAI()

completion = client.chat.completions.create(
    model='gpt-4',
    messages=[
        {'role':'system', 'content':'You are Marv, a chatbot that reluctantly answers questions with sarcastic responses.'},
        {'role':'user', 'content':'I’ve recently adopted a dog. Could you suggest some dog names?'}
    ]
)

print(completion.choices[0].message.content)

Explanation: The system message instructs the chatbot to be sarcastic, while the user message is the query. The response is stored in completion.choices[0].message.content.

2. Controlling Responses with Temperature, Max Tokens, and Seed

OpenAI models provide parameters to influence response behavior:

max_tokens: Limits the length of the response.
temperature: Controls randomness (0 = deterministic, higher values = more creative).
seed: Ensures reproducibility (optional).

completion = client.chat.completions.create(
    model='gpt-4',
    messages=[{'role':'user', 'content':'Explain briefly what a black hole is.'}],
    max_tokens=250,
    temperature=0,
    seed=365
)
print(completion.choices[0].message.content)

Reasoning: Setting temperature=0 ensures informative, deterministic responses suitable for educational or factual chatbots. Limiting max_tokens prevents unnecessarily long outputs.

3. Streaming Responses

Sometimes, you want the chatbot to stream responses in real-time instead of waiting for the full answer:

completion = client.chat.completions.create(
    model='gpt-4',
    messages=[{'role':'user', 'content':'Explain briefly what a black hole is.'}],
    max_tokens=250,
    temperature=0,
    stream=True
)

for chunk in completion:
    print(chunk.choices[0].delta.content, end="")

Explanation: Each ChatCompletionChunk is a partial response. Streaming improves user experience in chat applications.

4. LangChain: ChatOpenAI

LangChain simplifies working with OpenAI models. The ChatOpenAI class wraps GPT models for easier integration:

from langchain_openai.chat_models import ChatOpenAI

chat = ChatOpenAI(model_name='gpt-4', model_kwargs={'seed':365}, temperature=0, max_tokens=100)
response = chat.invoke("I've recently adopted a dog. Could you suggest some dog names?")
print(response.content)

Reasoning: LangChain provides an abstraction over OpenAI APIs, enabling chaining, memory, and prompt management.

5. System and Human Messages in LangChain

LangChain defines structured messages with roles:

from langchain_core.messages import SystemMessage, HumanMessage

message_s = SystemMessage(content="You are Marv, a sarcastic chatbot.")
message_h = HumanMessage(content="Can you suggest dog names?")
response = chat.invoke([message_s, message_h])
print(response.content)

Explanation: This is similar to OpenAI’s roles but wrapped in LangChain objects, improving code readability and maintainability.

6. Prompt Templates

Prompt templates allow dynamic construction of prompts:

from langchain_core.prompts import PromptTemplate

TEMPLATE = '''
System:
{description}

Human:
I've recently adopted a {pet}. Could you suggest some {pet} names?
'''
prompt_template = PromptTemplate.from_template(TEMPLATE)
prompt_value = prompt_template.invoke({'description':'Sarcastic bot', 'pet':'dog'})
print(prompt_value.text)

Reasoning: Templates separate logic from content, making prompts reusable for multiple scenarios.

7. Chat Prompt Templates

LangChain also supports chat-oriented templates:

from langchain_core.prompts.chat import SystemMessagePromptTemplate, HumanMessagePromptTemplate, ChatPromptTemplate

message_template_s = SystemMessagePromptTemplate.from_template("{description}")
message_template_h = HumanMessagePromptTemplate.from_template("I've adopted a {pet}. Suggest names.")
chat_template = ChatPromptTemplate.from_messages([message_template_s, message_template_h])
chat_value = chat_template.invoke({'description':'Sarcastic bot', 'pet':'dog'})
response = chat.invoke(chat_value)
print(response.content)

Reasoning: Chat-specific templates help organize multi-turn conversations.

8. Few-Shot Learning with LangChain

Few-shot templates allow the bot to learn patterns from examples:

from langchain_core.prompts import FewShotChatMessagePromptTemplate, AIMessagePromptTemplate, HumanMessagePromptTemplate, ChatPromptTemplate

# Example templates
examples = [{'pet':'dog','response':'How about "Bark Twain"?'},
            {'pet':'cat','response':'How about "Sir Meowsalot"?'}]

message_template_h = HumanMessagePromptTemplate.from_template("I've adopted a {pet}. Suggest names.")
message_template_ai = AIMessagePromptTemplate.from_template("{response}")
example_template = ChatPromptTemplate.from_messages([message_template_h, message_template_ai])

few_shot_prompt = FewShotChatMessagePromptTemplate(examples=examples, example_prompt=example_template, input_variables=['pet'])
chat_template = ChatPromptTemplate.from_messages([few_shot_prompt, message_template_h])
chat_value = chat_template.invoke({'pet':'rabbit'})
response = chat.invoke(chat_value)
print(response.content)

Reasoning: Few-shot prompts guide the model to respond in a specific style without retraining.

9. LLMChain

LLMChain connects a prompt template with an LLM for direct execution:

from langchain.chains.llm import LLMChain

chain = LLMChain(llm=chat, prompt=chat_template)
response = chain.invoke({'pet':'fish'})
print(response)

Reasoning: LLMChain simplifies running prompts with variables and managing responses.

10. Chat Message History

Tracking previous messages allows context-aware responses:

from langchain_community.chat_message_histories import ChatMessageHistory

history = ChatMessageHistory()
history.add_user_message("Give me a fun fact.")
history.add_ai_message("Longest place name is 85 letters long.")
print(history.messages)

Reasoning: Maintaining history improves multi-turn conversation consistency.

11. Conversation Buffer Memory

LangChain memory allows automatic context retention:

from langchain.memory import ConversationBufferMemory

chat_memory = ConversationBufferMemory(memory_key='message_log', chat_memory=history, return_messages=True)
chain = LLMChain(llm=chat, prompt=chat_template, memory=chat_memory)
response = chain.invoke({'question':'Can you give another interesting fact?'})
print(response['text'])

Reasoning: Buffer memory stores past messages for coherent follow-ups.

12. Conversation Buffer Window Memory

This variant remembers only the last k messages, controlling memory size:

from langchain.memory import ConversationBufferWindowMemory

chat_memory = ConversationBufferWindowMemory(memory_key='message_log', chat_memory=history, return_messages=True, k=2)

13. Conversation Summary Memory

Summarizes past conversation to reduce token usage:

from langchain.memory import ConversationSummaryMemory

chat_memory = ConversationSummaryMemory(llm=chat, memory_key='message_log', return_messages=False)

14. Combined Memory

Combines buffer and summary memories for efficient and contextual responses:

from langchain.memory import CombinedMemory

chat_combined_memory = CombinedMemory(memories=[chat_buffer_memory, chat_summary_memory])

15. Output Parsers

LangChain provides structured output parsers:

String Parser:

from langchain_core.output_parsers import StrOutputParser
str_parser = StrOutputParser()
parsed = str_parser.invoke(response)

Comma-Separated List Parser:

from langchain_core.output_parsers import CommaSeparatedListOutputParser
list_parser = CommaSeparatedListOutputParser()
parsed_list = list_parser.invoke(response)

Datetime Parser:

from langchain.output_parsers import DatetimeOutputParser
date_parser = DatetimeOutputParser()
parsed_date = date_parser.invoke(response)

Reasoning: Output parsers convert unstructured text into structured, machine-readable formats.

16. End-to-End Chatbot Example: Using Best Practices

In this section, we’ll build a complete chatbot using LangChain and OpenAI. This bot will:

Use a system message to define behavior.
Accept human messages dynamically.
Use Prompt Templates for structured questions.
Remember conversation context using memory.
Use an LLMChain for execution.
Output responses in a structured format using an output parser.

Step 1: Import Required Modules

%load_ext dotenv
%dotenv

import os
from langchain_openai.chat_models import ChatOpenAI
from langchain_core.messages import SystemMessage, HumanMessage
from langchain_core.prompts import PromptTemplate, ChatPromptTemplate, HumanMessagePromptTemplate, MessagesPlaceholder
from langchain.chains.llm import LLMChain
from langchain.memory import ConversationBufferMemory
from langchain_core.output_parsers import CommaSeparatedListOutputParser

Step 2: Define System and Human Messages

The system message instructs the bot to be helpful, witty, and sarcastic:

system_message = SystemMessage(
    content="You are Marv, a chatbot that answers questions wittily and sarcastically."
)

human_template = HumanMessagePromptTemplate.from_template(
    "I've recently adopted a {pet}. Can you suggest some {pet} names?"
)

Why this step is necessary: System messages define the chatbot’s personality, while Human messages capture dynamic user input.

Step 3: Create a Chat Prompt Template

We combine system instructions and user prompts:

chat_template = ChatPromptTemplate.from_messages([system_message, human_template])

Why: Templates make prompts reusable, structured, and clear, which is critical for maintaining complex chatbots.

Step 4: Set Up Memory

We want the chatbot to remember previous suggestions and respond contextually:

chat_memory = ConversationBufferMemory(
    memory_key='message_log',
    return_messages=True
)

Why: Memory allows follow-up questions to be answered intelligently, giving the bot “context awareness”.

Step 5: Initialize the Chat Model

chat_model = ChatOpenAI(
    model_name='gpt-4',
    model_kwargs={'seed': 365},
    temperature=0.5,  # Slight creativity for witty responses
    max_tokens=150
)

Why: Temperature is set to slightly higher than zero for humorous but still relevant answers.

Step 6: Combine Everything in an LLMChain

chain = LLMChain(
    llm=chat_model,
    prompt=chat_template,
    memory=chat_memory
)

Why: LLMChain integrates prompt templates, memory, and the chat model for a single point of execution.

Step 7: Invoke the Bot with User Input

user_input = {'pet': 'dog'}
response = chain.invoke(user_input)
print("Bot Response:\n", response['text'])

Step 8: Parse the Response

Suppose we want the bot to return names in a comma-separated format:

parser = CommaSeparatedListOutputParser()
parsed_response = parser.invoke(HumanMessage(content=response['text']))
print("Parsed Names:", parsed_response)

Why: Output parsers ensure structured responses for downstream applications (like storing pet names in a database).

Step 9: Follow-Up Interaction

The bot remembers previous conversation:

follow_up_input = {'pet': 'cat'}
follow_up_response = chain.invoke(follow_up_input)
print("Bot Response to Follow-Up:\n", follow_up_response['text'])
parsed_follow_up = parser.invoke(HumanMessage(content=follow_up_response['text']))
print("Parsed Names Follow-Up:", parsed_follow_up)

Why: Memory ensures that the follow-up question is answered contextually, maintaining continuity.

Conclusion

LangChain combined with OpenAI GPT models empowers developers to build context-aware, memory-enabled, and style-controlled chatbots. From basic chat completions to complex memory and output parsing, LangChain provides a robust framework for efficient and maintainable AI applications.

By mastering templates, LLM chains, memory management, and output parsing, you can create chatbots that are not only interactive but also intelligent, engaging, and reliable.