Shimmi AI — Multimodal Chatbot for WhatsApp

Voice-and-text AI agent that listens, transcribes, reasons, and replies over WhatsApp — multimodal conversations that actually get work done.

Project type

ChatbotMultimodal AIConversational AIWhatsApp BotVoice AI

Tech stack

LangChainOpenAIWhisperOllamaWhatsApp APIMaytapiTwilioPython

Challenges

Real users don’t type perfect prompts — they send voice notes on WhatsApp. Needed an agent that can ingest audio, transcribe, decide tools, and continue a natural conversation toward a goal (e.g. sourcing and messaging babysitters) without a custom app install.

Solution & details

Simulation mode for safe testing, webhook message intake, LLM tool routing, audio download/transcribe pipeline, and WhatsApp send integrations. Positioned as conversational AI meets operations — not a toy chatbot, a channel-native agent.

Interested in a similar project?

Let's talk about how we can help.

Start a project