UIUC AI Teaching Assistant Runs 11 Models in Parallel for Sub-2s Responses

The Center for AI Innovation at UIUC has released an open-source AI teaching assistant for Electrical Engineering courses, live on HuggingFace Spaces. The system orchestrates 11 separate models in parallel for text/image retrieval, generation, moderation, and ranking, achieving a median 2-second response time.

Architecture

The assistant uses retrieval-augmented generation (RAG) with a Pinecone vector database. Data sources include textbooks, lecture videos (transcribed via Whisper), and student QA forums. The codebase is MIT-licensed and ready to plug into your own Pinecone database.

RLHF Dataset

The team hired five Electrical Engineering students to produce a comparison dataset for RLHF, covering UIUC's ECE 120 course. The dataset is freely available on HuggingFace: kastan/rlhf-qa-comparisons.

Evaluation

Every new feature triggers an automated evaluation. The evaluation dataset, written by expert EE engineers, is used to generate answers from each model. GPT-3 then judges whether the generated answers are "better" or "worse" than human-written ground truth. Known limitation: GPT-3 evaluates itself and tends to favor GPT-3 outputs.

Quick Start

pip install -r requirements.txt
# Set API keys in run_ta_gradio.sh, then:
bash run_ta_gradio.sh

Build your own Pinecone index using the provided scripts: textbooks from PDF, video transcripts from Whisper, or lecture slides from .jpg images.

Key Files

• main.py — model aggregation
• TA_gradio_ux.py — Gradio UI
• prompting.py — prompt templates
• evaluation.py — GPT-3 evaluation loop
• feedback.json — collected user feedback

Repo contains 146 commits, 57 stars, and a system diagram showing the parallel model pipeline.