2026-11-17 12:20
Introduction: Direct context injection via product catalog eliminates 6–16 seconds of voice round-trip overhead. What is FashionMind? FashionMind is Google’s open-source multimodal real-time voice fashion assistant, built on Google ADK and Gemini Live 2.5 Flash.
FashionMind is Google’s open-source multimodal real-time voice fashion assistant, powered by Google ADK and Gemini Live 2.5 Flash. It enables users to view outfits through a camera, engage in natural voice conversations, retain user preferences across sessions, store precise factual data in relational databases, preserve style preferences in semantic memory, and extract only conclusions from raw audio-video inputs. The core design of FashionMind leverages direct product catalog injection into context and read-write separation, enabling seamless full-duplex conversation without any additional voice round-trips.

runner.run_live() events and forwards them to the frontend. Queues are managed via send_realtime() for continuous audio (detected via VAD speech boundaries) and send_content() for discrete image, text, and context injection (capable of triggering inference or interruption instantly), decoupling WebSocket handlers from ADK runner logic.send_content(). Catalog becomes available in milliseconds, while memory arrives in ~1.5 seconds. Since injection does not interrupt the real-time audio stream, the agent already holds complete context by the time the user finishes their first utterance—enabling an immediate response like “Do you have suits?” without requiring tool calls.recommend_products() is designed as a pure signal tool—no database lookup occurs; it generates detectable function_call events within the ADK event stream. The downstream task uses these signals to push highlighting instructions to the frontend, synchronizing voice narration with card highlighting.
WeChat follow and reply “open source” to join the AI Open Source Project Community
./setup_gcp.sh YOUR_PROJECT_ID us-central1; the script automatically creates a Cloud SQL PostgreSQL instance and Vertex AI Agent Engine, and generates the backend/.env configuration file.alembic upgrade head (database migration), python seed_products.py (import product catalog), followed by uvicorn main:app --reload --port 8080 to start the service.USE_MEMORY_BANK=true to enable cross-session memory.npm install and npm run dev, then open http://localhost:5173.| Dimension | FashionMind | Pinterest Assistant |
| Product Positioning | Open-source reference architecture for developers—reusable vertical voice application engineering template | Consumer-ready feature embedded within Pinterest platform—currently in beta phase |
| Voice Interaction Mode | Full-duplex continuous dialogue: 16kHz continuous audio stream, wake-word-free, fully interruptible (barge-in) | Button-based voice input: hold mic to speak (similar to sending voice messages), non-streaming, audio playback cannot be disabled |
| Visual Capabilities | 1fps real-time camera feed continuously analyzes outfits, delivering comments within the same voice turn | Multimodal "visual-first" model capable of generating outfit suggestions from posters or screen images—but not real-time video streaming |
| Memory Architecture | Three-tier separation: precise facts in relational DB, style preferences asynchronously distilled into semantic memory, raw audio-video only retains conclusions | Relies on Pinterest user behavior signals (saved Pins, browsing history) for personalization—no publicly documented memory tiering mechanism |
| Product Integration | Self-hosted product catalog fully injected into context—product mention triggers immediate frontend card highlighting | Recommends internal Pins—voice describes pairing and displays visual result |
| Latency Engineering | Deeply optimized: asynchronous catalog/memory injection, signal tools eliminating round-trips, local barge-in, local session state | No public details on latency optimization; voice output intentionally kept brief |
Source: AI Tools Collection
Disclaimer: Contains third-party opinions, does not constitute financial advice
Step Code – The Open-Source Terminal Programming Agent by Jieyue Star
09-24
Hy4 Preview Lightweight Version – Tencent Hunyuan's Open-Source Hy4 Quantized Compression Model
09-02
Claude Fable 5.1 – Anthropic's Latest Flagship Large Model
09-02
Open-source Retool is here, drag-and-drop backend with 60+ components
08-19
Claude Opus 5 is Here: Half the Price Approaching Fable 5
07-25
Tencent Open-Sources 260 Real-World Tasks to Test AI Workforce Capabilities
07-25
Meituan Open-Sources AI Exam Platform: 813 Minecraft Tasks
07-25






