Shimmi AI — Multimodal Chatbot for WhatsApp
Voice-and-text AI agent that listens, transcribes, reasons, and replies over WhatsApp — multimodal conversations that actually get work done.
Customer
Confidential
Role
Developer
Type
Chatbot • Multimodal AI • Conversational Agent • WhatsApp • Voice
Timeframe
Weeks
Tech stack
LangChainOpenAI GPT-4Whisper Speech-to-TextOllamaWhatsApp APIs (Maytapi / Twilio)Python Agents
Challenges
Real users don’t type perfect prompts — they send voice notes on WhatsApp. Needed an agent that can ingest audio, transcribe, decide tools, and continue a natural conversation toward a goal (e.g. sourcing and messaging babysitters) without a custom app install.
Solution & details
Simulation mode for safe testing, webhook message intake, LLM tool routing, audio download/transcribe pipeline, and WhatsApp send integrations. Positioned as conversational AI meets operations — not a toy chatbot, a channel-native agent.