--- title: DungeonMaster AI emoji: "\U0001F3B2" colorFrom: yellow colorTo: red sdk: gradio sdk_version: "6.0.0" app_file: app.py pinned: true tags: - agent-demo-track - mcp - dnd - llama-index - gemini - elevenlabs - gaming - voice license: mit --- # DungeonMaster AI **Your AI-Powered D&D 5e Game Master with Voice Narration** *Roll for initiative! An immersive tabletop experience powered by cutting-edge AI technology.* --- ## Demo Video [![DungeonMaster AI Demo](https://img.shields.io/badge/Watch-Demo%20Video-red?style=for-the-badge&logo=youtube)](https://youtube.com/your-video-link) --- ## Features ### AI Dungeon Master - **Intelligent Storytelling**: Dynamic narrative generation that adapts to your choices - **Authentic D&D Experience**: Full D&D 5e rules implementation via TTRPG Toolkit MCP - **Multi-Modal Agents**: Specialized agents for storytelling, rules lookup, and voice narration ### Voice Narration - **Immersive Audio**: Professional voice narration powered by ElevenLabs - **Character Voices**: Different voice profiles for the DM, NPCs, and monsters - **Real-Time Streaming**: Low-latency voice synthesis for smooth gameplay ### Complete Game System - **Full Character Management**: Create, track, and level up D&D 5e characters - **Combat Engine**: Turn-based combat with initiative tracking and condition management - **Dice Rolling**: All standard D&D dice with advantage/disadvantage and modifiers - **Rules Lookup**: RAG-powered rules search for instant rule clarification ### Beautiful Fantasy UI - **Themed Interface**: Dark fantasy aesthetic with parchment textures and golden accents - **Responsive Design**: Works on desktop, tablet, and mobile - **Real-Time Updates**: Live character sheet, combat tracker, and dice roll history --- ## How to Play ### Starting a New Game 1. **Launch the App**: Visit the HuggingFace Space or run locally 2. **Create Your Character**: Use the character creation wizard or load a pre-made template 3. **Choose Your Adventure**: Start with "The Rusty Dragon Tavern" or "The Goblin Cave" 4. **Start Playing**: Type your actions and watch the story unfold! ### Basic Commands | Action | What to Type | |--------|-------------| | **Attack** | "I attack the goblin with my sword" | | **Investigate** | "I search the room for hidden doors" | | **Talk** | "I ask the bartender about recent rumors" | | **Cast a Spell** | "I cast Magic Missile at the enemy" | | **Move** | "I sneak down the dark corridor" | ### Special Commands - `/roll [notation]` - Roll dice directly (e.g., `/roll 2d6+3`) - `/character` - View your character sheet - `/inventory` - Check your items - `/rest` - Take a short or long rest ### Combat Tips - The DM will call for initiative when combat begins - Describe your intended action - the AI will handle the mechanics - Use the combat tracker to see turn order and HP - Strategic positioning and creative solutions are rewarded! --- ## Technical Architecture ``` ┌─────────────────────────────────────────────────────────────────────────┐ │ DUNGEONMASTER AI │ ├─────────────────────────────────────────────────────────────────────────┤ │ │ │ LAYER 1: GRADIO 6 UI │ │ ├── Chat Panel (with voice playback) │ │ ├── Character Sheet Panel │ │ ├── Dice & Roll History Panel │ │ ├── Combat Tracker Panel │ │ └── Game Controls Panel │ │ │ │ LAYER 2: AGENT ORCHESTRATION (LlamaIndex) │ │ ├── Dungeon Master Agent (storytelling, game flow) │ │ ├── Rules Arbiter Agent (rules lookup, adjudication) │ │ └── Voice Narrator Agent (ElevenLabs integration) │ │ │ │ LAYER 3: GAME STATE MANAGEMENT │ │ ├── Game State Manager (session, party, location, combat) │ │ ├── Story Context Builder (LLM context preparation) │ │ └── Event Logger (action history tracking) │ │ │ │ LAYER 4: MCP INTEGRATION │ │ ├── TTRPG Toolkit Client (connection to MCP server) │ │ ├── Tool Wrappers (game-aware tool enhancements) │ │ └── Connection Manager (health checks, reconnection) │ │ │ └─────────────────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────────────────┐ │ TTRPG TOOLKIT MCP │ │ (Separate MCP Server - Blaxel) │ │ ├── Dice Roller Server │ │ ├── Character Manager Server │ │ ├── Rules Lookup Server (LlamaIndex RAG) │ │ ├── Generators Server (NPCs, encounters, loot) │ │ ├── Combat Engine Server │ │ └── Session Manager Server │ └─────────────────────────────────────────────────────────────────────────┘ ``` --- ## Built With ### AI & Agents - **[Google Gemini](https://ai.google.dev/)** - Primary LLM for storytelling and game mastering - **[OpenAI GPT-4](https://openai.com/)** - Fallback LLM for reliability - **[LlamaIndex](https://www.llamaindex.ai/)** - Agent framework and RAG for rules lookup ### Voice - **[ElevenLabs](https://elevenlabs.io/)** - Voice synthesis for immersive narration ### MCP & Backend - **[TTRPG Toolkit MCP](https://github.com/ttrpg-toolkit/ttrpg-toolkit-mcp)** - Game mechanics server - **[Blaxel](https://blaxel.com/)** - MCP server deployment ### UI & Deployment - **[Gradio 6](https://gradio.app/)** - Modern UI framework - **[HuggingFace Spaces](https://huggingface.co/spaces)** - Deployment platform --- ## Local Development ### Prerequisites - Python 3.10+ - UV or pip - API keys for Gemini, OpenAI, and ElevenLabs ### Installation ```bash # Clone the repository git clone https://github.com/dungeonmaster-ai/dungeonmaster-ai.git cd dungeonmaster-ai # Create virtual environment python -m venv .venv source .venv/bin/activate # On Windows: .venv\Scripts\activate # Install dependencies pip install -r requirements.txt # Copy environment file and add your API keys cp .env.example .env # Edit .env with your actual API keys # Run the application python -m ui.app ``` ### Running with TTRPG Toolkit (Local) ```bash # In one terminal, run the MCP server cd ../TTRPG-Toolkit python -m src.main # In another terminal, run DungeonMaster AI cd ../DungeonMaster-AI python -m ui.app ``` --- ## Project Structure ``` dungeonmaster-ai/ ├── src/ │ ├── agents/ # LlamaIndex agents │ ├── voice/ # ElevenLabs integration │ ├── mcp_integration/ # TTRPG Toolkit MCP client │ ├── game/ # Game state management │ ├── config/ # Settings and prompts │ └── utils/ # Helpers and formatters ├── ui/ │ ├── app.py # Main Gradio application │ ├── components/ # UI components │ ├── handlers/ # Event handlers │ ├── themes/ # Fantasy theme │ └── assets/ # Images and CSS ├── adventures/ # Pre-made adventure content │ ├── tutorial_goblin_cave.json │ ├── tavern_start.json │ └── sample_characters/ ├── tests/ # Test suite └── docs/ # Documentation ``` --- ## Adventures Included ### The Rusty Dragon Tavern A classic sandbox start. Meet colorful NPCs, hear rumors, and find your first adventure hook. Perfect for open-ended exploration. ### The Goblin Cave A beginner-friendly adventure where heroes investigate goblin raids. Learn combat, exploration, and puzzle-solving in a focused 30-45 minute experience. --- ## Sample Characters Get started immediately with pre-made characters: - **Valric the Bold** - Human Fighter, Level 1 - **Elara Starweaver** - High Elf Wizard, Level 1 - **Shade Quickfingers** - Halfling Rogue, Level 1 --- ## Hackathon Submission This project was built for the **MCP Hackathon 2025** on HuggingFace. ### Track **Agentic Demo Track** - Showcasing the power of AI agents in gaming ### Sponsor Technologies Used | Sponsor | Usage | |---------|-------| | **Gemini** | Primary LLM for DM narration and storytelling | | **OpenAI** | Fallback LLM for reliability | | **ElevenLabs** | Voice synthesis for immersive narration | | **LlamaIndex** | Agent framework and RAG-powered rules lookup | | **Blaxel** | TTRPG Toolkit MCP server hosting | | **Gradio** | UI framework for the demo | | **HuggingFace** | Application deployment | --- ## Contributing Contributions are welcome! Please read our contributing guidelines before submitting PRs. --- ## License MIT License - see [LICENSE](LICENSE) for details. --- ## Acknowledgments - The D&D 5e SRD for game rules and content - The MCP protocol for enabling modular game systems - All the amazing open-source libraries that made this possible - The hackathon organizers and sponsors --- **May your rolls be ever in your favor!** *Built with passion for tabletop gaming and AI technology.*