Back to Hanger

Chat Agent

Conversational AI that guides users through progressive style refinement, building detailed queries from natural dialogue to deliver highly personalized product recommendations.

Real-time
Context-Aware
Sub-second

Problem

Traditional search requires users to translate their fashion intent into keywords. Natural conversation is how people actually think about style.

Solution

Built a conversational agent that understands context, maintains dialogue history, and translates natural language into semantic search queries.

Impact

Users discover products 3x faster through conversation. Chat-driven sessions have 2.4x higher conversion than traditional search.

Why We Built This

Shopping for fashion is inherently exploratory—users rarely know exactly what they want upfront. They think in vibes, occasions, and context. Traditional search forces them to commit to specific terms too early. We built progressive refinement to mirror how people actually discover style: starting broad, then narrowing through guided questions that surface options they didn't know to ask for.

Architecture

Architecture
Natural Language Query
Gemini Intent Classification
Style Descriptor Extraction
CLIP Query Enhancement
Pinecone Vector Search
Progressive Refinement Tools
Streaming Response

Technical Deep Dive

Intent Understanding

Gemini-powered intent classification determines whether user wants general search, product comparison, or styling suggestions. Extracts structured filters (price, category, brand) and visual descriptors (neckline, sleeve length, material) from natural language.

  • Three intent types: general search, compare_with (find similar products), pair_with (styling suggestions)
  • Automatic filter extraction: "Nike shoes under $150" → brand_filter: Nike, max_price: 150, taxonomy: Shoes
  • Style descriptor accumulation: "professional tops" → "v-neck" → "silk" builds enhanced query over conversation
  • Product context support: users can click "Compare Similar" or "Pair With" on any product card

Progressive Refinement System

Category-specific refinement tools guide users through style discovery. Clothing triggers neckline/sleeve/fit questions. Shoes prompt heel height and style. Each response accumulates descriptors that enhance subsequent searches.

  • Two refinement types: multiple_choice (structured options like neckline styles) and follow_up_question (open-ended)
  • Example flow: "professional tops" → "What neckline?" → "v-neck" → "What sleeve length?" → "long sleeves"
  • Query evolution: "professional tops" → "professional tops v-neck" → "professional tops v-neck long sleeves silk"
  • Visual descriptor focus: emphasizes terms that improve CLIP embedding matches (materials, cuts, patterns)

Session Management & History

Google Chat Module integration maintains full conversation history with proper role attribution. Sessions persist accumulated style descriptors, active filters, and chat history for context-aware responses across multiple turns.

  • Memory-based session store (Redis recommended for production) with automatic cleanup
  • Session data includes: chat_history, current_filters, style_descriptors, last_search_results, previously_seen_ids
  • Guest user support: auto-generates user_id if not provided, enabling anonymous shopping
  • Endpoints: /chat (main), /chat/show-more-products, /chat/session-info, /chat/clear-session, /chat/available-filters

Search Integration & Product Actions

Enhanced queries feed into Pinecone vector search with CLIP embeddings. Product-specific actions enable users to compare similar items or get styling suggestions directly from product cards, maintaining conversation context throughout.

  • CLIP-optimized query building: accumulated descriptors improve semantic search relevance over conversation turns
  • Compare Similar: finds products matching brand and category with similar visual attributes
  • Pair With: generates styling suggestions by category (bottoms, outerwear, accessories, shoes)
  • Show More: pagination support with previously_seen_ids tracking to avoid duplicates

Key Learnings

  • Progressive refinement works: Starting broad and narrowing through guided questions produces better results than forcing users to specify everything upfront
  • Visual descriptors matter: Terms like "v-neck", "silk", "midi length" significantly improve CLIP embedding matches compared to generic style terms
  • Product actions reduce friction: Compare and Pair With buttons let users explore without breaking conversation flow or starting new searches
  • Category-specific refinements: Different product types need different questions—clothing focuses on fit and fabric, shoes on heel height and style
Back to Hanger Overview