GitHub logo

Build Your Own AI QA Assistant โ€” Automate Test Understanding with LangChain & OpenAI

Stop digging through test cases, requirement docs, and logs manually. Build your own QA Assistant that can answer your questions using natural language โ€” powered by OpenAI and LangChain.

๐Ÿ‘‹ Why This?

As QA engineers, we often deal with:

  • Test cases in JSON or Jira/Xray
  • BDD specs in .feature files
  • Markdown requirements
  • Logs in .txt and .html
  • Allure reports and PDFs

…scattered across tools and folders.

The problem? Searching across all these files is slow, repetitive, and error-prone.

The solution? An AI-powered bot that understands all your test artifacts and answers your questions โ€” like a real assistant.

๐Ÿง  What This Bot Does

It allows you to:

โœ… Ask things like:

  • “What are the test steps in TC-002?”
  • “Summarize login test cases.”
  • “What does the PDF say about edge cases?”

๐Ÿ“ It reads from:

  • .json โ†’ structured test cases
  • .feature โ†’ BDD scenarios
  • .md, .txt, .html โ†’ requirements, logs, reports
  • .pdf, .docx โ†’ spec files

๐Ÿ”Ž Then it:

  • Breaks the content into searchable chunks
  • Creates semantic embeddings
  • Stores it in a FAISS vector DB
  • Uses OpenAI to generate responses in natural language

๐Ÿ› ๏ธ Tech Stack

  • LangChain โ€” to connect everything
  • OpenAI โ€” to generate QA-friendly responses
  • FAISS โ€” to index content for fast semantic search
  • Python โ€” because we like speed and simplicity

๐Ÿš€ How to Use It

1. ๐Ÿ”„ Clone the Repo

git clone https://github.com/aiqualitylab/AI-QA-Assistant-Bot.git
cd AI-QA-Assistant-Bot

2. ๐Ÿ“ฆ Install Dependencies

pip install -r requirements.txt

3. ๐Ÿ” Set Your OpenAI Key

Create a .env file:

OPENAI_API_KEY=sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

๐Ÿ” You can get your API key from: https://platform.openai.com/account/api-keys

4. ๐Ÿš€ Run the Assistant

python bot.py

If it’s your first run, it will index the documents. After that, it reuses the saved index.

You’ll see:

๐Ÿค– QA Assistant Ready! Ask anything (type 'exit' to quit')

Start asking questions โ€” and get instant, AI-powered answers.

๐Ÿ’ฌ Example Use Cases

  • ๐Ÿ” “What’s in TC-003?”
  • ๐Ÿ“ “Summarize all test cases about checkout”
  • ๐Ÿž “What do the logs say about payment errors?”
  • ๐Ÿ“‹ “Give me test steps for login with invalid credentials”

๐Ÿ› ๏ธ Behind the Scenes

Under the hood:

  • Document loaders handle multiple formats using langchain_community.loaders
  • Recursive chunking splits large text into overlapping pieces
  • OpenAIEmbeddings converts chunks into vectors
  • FAISS indexes and searches them semantically
  • RetrievalQA runs your question against the best-matching documents

๐Ÿ’ก Why It Matters

QA engineers shouldn’t have to:

  • Ctrl+F through hundreds of lines
  • Switch between Jira, Confluence, Allure, Git
  • Manually correlate logs and test cases

This assistant lets you work smarter, not harder.

๐Ÿงช Demo Available

Try it out locally on your own test artifacts. Clone the repo ๐Ÿ‘‰ https://github.com/aiqualitylab/AI-QA-Assistant-Bot

Have questions or suggestions? Drop a comment below or open an issue on GitHub!

GitHub logo

aiqualitylab
/
AI-QA-Assistant-Bot

QA Assistant that can answer your questions using natural language โ€” powered by OpenAI and LangChain.

๐Ÿค– QA Assistant Bot

AI-powered assistant for QA engineers
Uses OpenAI + LangChain to answer questions from test cases, feature files, logs, specs, and more โ€” all from natural language.

๐Ÿง  Purpose

This bot helps QA engineers:

  • โœ… Ask natural questions like:
    “What are the steps in TC-001?”
    “List test cases for login.”

  • ๐Ÿ“„ Understand test specs and logs quickly

  • ๐Ÿ” Search across multiple formats (JSON, Markdown, PDF, etc.)

  • ๐Ÿงช Automate documentation understanding and reduce manual effort

๐Ÿ“ Supported File Types

It can read and index:

  • .json โ€” Test cases
  • .feature โ€” BDD specs
  • .md โ€” Requirements
  • .txt โ€” Logs
  • .html โ€” Reports
  • .pdf โ€” Test documents
  • .docx โ€” Word specs

Place files inside the data/ folder.

โš™๏ธ Installation

โœ… 1. Clone the repo

git clone https://github.com/aiqualitylab/AI-QA-Assistant-Bot
cd qa-bot

โœ… 2. Set up Python environment

Make sure Python 3.10+ is installed. Then:

pip install -r requirements.txt

โœ… 3. Create .env

โ€ฆ

Similar Posts