As developers, integrating AI tools like ChatGPT, Claude, and Gemini into our daily workflows has become a necessity. However, when trying to feed an entire codebase into an LLM for debugging or architectural review, we instantly hit a massive wall. Consequently, we face strict token limits and severe context fragmentation.
The Problem with Standard Project-to-Text Tools
Existing tools often load entire file contents into memory, which frequently causes crashes on massive codebases. Worse yet, they dump thousands of lines of useless temporary files, caches, and build artifacts straight into your context window. Therefore, this quickly explodes your token count, spikes your API costs, and completely confuses the AI.
Meet Project2Markdown: Your AI Context Generator
To address this exact issue, I engineered an open-source solution called nematollahshojaei/project2markdown. Built from the ground up using Go (Golang), it serves as a high-performance, ultra-lightweight CLI tool. Specifically, it compresses your entire project structure and relevant source code into a single, beautifully organized Markdown file optimized for AI consumption.
Core Features and Architecture
This tool is designed to bypass traditional bottlenecks. In addition to its speed, it ensures your data remains structured. As a result, the generated context becomes immediately understandable for any large language model.
⚡ Stream-Based Pipeline
The tool utilizes a high-efficiency bufio.Scanner architecture. Because it reads line-by-line rather than buffering entire files into RAM, it processes massive repositories with near-zero memory footprint.
🌳 Directory Tree Generation
It automatically injects a clean ASCII directory tree at the top of the output file. Consequently, this gives the LLM an immediate birds-eye view of your architecture before it analyzes the code blocks.
🛑 Smart Exclusion Filters
You get out-of-the-box support for ignoring heavy build directories and symlinks. Furthermore, it automatically avoids self-inclusion by skipping its own executable and output files.
🛡️ Markdown Safety Grid
The tool detects existing triple backticks inside your codebase and sanitizes them. Therefore, it successfully prevents formatting breaks when uploading your document to Claude or ChatGPT.
Open Source and Licensing
This utility is fully open-source and released under the permissive MIT License. Hence, you are completely free to clone, modify, and integrate it into your automated CI/CD pipelines or custom developer environments.
Get the latest compiled binary or check out the full source code on GitHub:
View on GitHub🎮 Are you an Unreal Engine Developer?
While Project2Markdown works perfectly with any programming language, it features native pipelines tailored specifically for Unreal Engine 5. Therefore, we highly recommend checking out our comprehensive step-by-step tutorial on how to extract text-based metadata, bulk-export blueprints, and successfully pass your entire UE5 project context to AI.