MiroFish

A Swarm Intelligence Engine That Predicts the Future

37.8k
GitHub Stars
5.1k
Forks
220
Commits

github.com/666ghj/MiroFish · Python 57.8% · Vue 41.1% · AGPL-3.0

ELI5

Explain Like I'm Five

What if you could build a tiny world inside a computer, fill it with thousands of little people who think for themselves, and then watch what happens next?

Imagine a Magic Sandbox

You have a giant sandbox. But instead of sand, you fill it with little people — each one has their own personality, memories, and opinions.

You drop in a newspaper article or a new rule, and then you sit back and watch what happens.

The little people read the news, argue about it, share posts, change their minds, and form groups — all on their own.

📰
Drop in news
Seed the world
🧑‍💻
Agents wake up
Thousands of them
💬
They interact
Post, like, argue
🔮
See the future
Prediction report

What Can You Predict?

Serious Stuff

  • Will people be angry about a new policy?
  • How will financial markets react to a rumor?
  • What happens if a company launches a controversial product?
  • How does public opinion shift after a crisis?

Fun Stuff

  • What if characters from a novel had social media?
  • How would historical figures tweet about modern events?
  • What ending would readers vote for in a choose-your-own-adventure?
  • Could you simulate a fictional election?
The big idea: Instead of guessing what might happen, you simulate it with AI agents that behave like real people on social media.

Five Steps, One Prediction

1
Upload your stuff
News articles, policy documents, story chapters — anything text-based
↓
2
Build a knowledge map
AI reads it all and figures out who/what/where matters
↓
3
Create thousands of agents
Each gets a personality, memory, and opinions based on the map
↓
4
Run the simulation
Agents post, like, comment, and argue on fake Twitter & Reddit
↓
5
Read the prediction report
AI writes a detailed report on what happened and what it means

Why This Is Wild

"Preview the future in a digital sandbox"
  • No real people harmed — test controversial ideas safely
  • Replay with changes — tweak one variable, run it again
  • Speed up time — simulate months of social dynamics in minutes
  • God-mode view — watch every agent's thoughts and actions
  • Interview agents — ask any simulated person why they did what they did
  • Open source — 37.8k stars, fully hackable

Technical Deep Dive

Architecture, code patterns, and implementation details

System Architecture

Frontend — Vue 3 + Vite

Single-page app on :3000. Five wizard steps track the full pipeline. Real-time polling at 2-3s intervals updates a dual-platform timeline feed.

Backend — Flask (Python 3.11+)

REST API on :5001. Orchestrates the full lifecycle: ontology generation, graph building, profile synthesis, simulation execution, and report writing.

External Services

Zep Cloud for GraphRAG memory. LLM API (OpenAI-compatible) for all generation. OASIS engine for agent simulation.

Key Numbers

ComponentDetails
Backend services12 Python modules
API endpoints30+ REST routes
Vue components8 views + 5 step components
Simulation platformsTwitter + Reddit parallel
IPC mechanismFile-based JSON polling
Agent profiles16 MBTI types, 11 countries

The Five-Stage Pipeline

Each stage maps to a Vue step component and a set of backend services:

# Stage Frontend Backend Service What Happens
1 Graph Build Step1GraphBuild graph_builder Upload docs, generate ontology, build knowledge graph in Zep
2 Env Setup Step2EnvSetup simulation_manager Read entities from graph, generate agent profiles & config
3 Simulation Step3Simulation simulation_runner Launch OASIS, run rounds on Twitter & Reddit in parallel
4 Report Step4Report report_agent ReACT-based agent writes multi-section prediction report
5 Interaction Step5Interaction simulation_ipc Interview agents, explore the simulated world

Stage 1: Ontology Generation

What the LLM does

Given uploaded documents and a natural-language simulation requirement, the OntologyGenerator asks the LLM to produce a structured schema:

  • Up to 10 entity types (people, orgs, policies, etc.)
  • Up to 10 edge types (relationships between entities)
  • Each with typed attributes and descriptions
  • Auto-adds Person and Organization fallbacks
Zep constraint: Max 10 entity types and 10 edge types per graph. Reserved field names (uuid, name, summary, etc.) are blocked.

Output format

{
  "entity_types": [
    {
      "name": "Politician",
      "description": "A public official",
      "attributes": [
        {"name": "party",
         "description": "Political party"}
      ]
    }
  ],
  "edge_types": [
    {
      "name": "opposes",
      "source_targets": [
        {"source": "Politician",
         "target": "Policy"}
      ]
    }
  ]
}

Stage 1: Graph Building with Zep

GraphBuilderService flow

A
Create graph
mirofish_{uuid} via Zep API
B
Set ontology
Dynamic Pydantic class generation from ontology JSON
C
Chunk text
500-char chunks, 50-char overlap
D
Batch ingest
3 chunks per batch to Zep as EpisodeData
E
Poll completion
3s intervals, 600s timeout

Dynamic ontology mapping

Entity types from the LLM become actual Python classes at runtime:

# Dynamic class creation
entity_class = type(
    name,           # "Politician"
    (EntityModel,), # Zep base
    attrs           # Fields from ontology
)

# Edge classes with source/target pairs
edge_class = type(
    class_name,   # "Opposes"
    (EdgeModel,),
    attrs
)
# Bound to EntityEdgeSourceTarget
Why this matters: The schema is fully determined by the LLM based on your input documents — no manual schema design needed.

Stage 2: Agent Profile Generation

From graph entities to agents

The ZepEntityReader reads all nodes from the knowledge graph, filtering out system labels (Entity, Node). Each entity becomes a candidate agent.

The OasisProfileGenerator then enriches each entity with:

  • Dual Zep search — edges (facts) + nodes (summaries) via ThreadPoolExecutor
  • Context builder — attributes, relationships, up to 3000 chars of Zep retrieval
  • LLM persona synthesis — separate prompts for individuals vs. institutions

Profile structure

{
  "user_id": 1,
  "name": "Wang Xiaoming",
  "age": 34,
  "gender": "male",
  "mbti": "ENTJ",
  "profession": "Policy Analyst",
  "country": "China",
  "interested_topics": [...],
  "persona": "2000-char biography
  with memories, opinions,
  behavioral tendencies...",
  "bio": "Short public bio"
}

Retry logic: 3 attempts with decreasing temperature (0.7 → 0.5 → 0.3). JSON repair for truncated LLM output.

Stage 2: Smart Config Generation

LLM-driven simulation parameters

Instead of manual tuning, the SimulationConfigGenerator asks the LLM to reason about optimal parameters given:

  • The simulation requirement (natural language)
  • The document text (context)
  • Entity list and types
  • Platform selection (Twitter, Reddit, or both)

The LLM outputs a SimulationParameters object with reasoning for its choices.

Generated parameters include

ParameterPurpose
Num roundsSimulation duration
Time per roundSimulated time granularity
Activity levelHow often agents act
Post frequencyContent creation rate
Interaction rateLikes, follows, comments
Platform splitTwitter vs Reddit balance
Design choice: Fully automated config means zero-expertise barrier to entry.

Stage 3: OASIS Simulation Runner

Process management

The SimulationRunner launches OASIS as a subprocess using Python's subprocess module. It supports three modes:

  • run_twitter_simulation.py
  • run_reddit_simulation.py
  • run_parallel_simulation.py (both)

State machine

class RunnerStatus(Enum):
    IDLE      STARTING
    RUNNING   PAUSED
    STOPPING  STOPPED
    COMPLETED FAILED

Real-time monitoring

Agent actions are logged to JSONL files. The runner parses these in real time:

# Each action is an AgentAction
{
  "round": 3,
  "platform": "twitter",
  "agent_id": 42,
  "action": "CREATE_POST",
  "success": true,
  "timestamp": "..."
}

Completion detected via "simulation_end" event. Cross-platform cleanup uses killpg (Unix) or taskkill (Windows). Signal handlers + atexit callbacks ensure orphan prevention.

What Is OASIS?

The simulation backbone

OASIS (Open Agent Social Interaction Simulations) is an external framework by CAMEL-AI that MiroFish wraps.

It provides:

  • Social platform simulation (Twitter-like, Reddit-like)
  • Agent behavior loops — each agent has an LLM "brain"
  • Action types per platform:
    • CREATE_POST, LIKE_POST, REPOST, FOLLOW
    • CREATE_COMMENT, SEARCH_POSTS, VOTE
  • Round-based execution with configurable tick rates

MiroFish adds

  • GraphRAG knowledge injection — agents start with structured world knowledge from Zep
  • Automated profile synthesis — entities from your documents become OASIS agents
  • Dual-platform parallel simulation — Twitter and Reddit simultaneously
  • Real-time IPC — interview agents mid-simulation
  • ReACT report generation — post-simulation analysis
  • Graph memory updates — Zep integration for entity relationship extraction during simulation
MiroFish is the orchestration and intelligence layer on top of OASIS's raw simulation engine.

Stage 4: ReACT Report Agent

How reports are generated

The ReportAgent uses a ReACT loop (Reason + Act) to write each section of the prediction report.

T
Thought
LLM reasons about what information is needed
A
Action
Calls one of four tools to gather evidence
O
Observation
Processes tool output, decides to continue or finalize

Min 3 tool calls, max 5 per section. Conflict resolution with 3-tier retry.

The four tools

ToolPurpose
insight_forgeDeep multi-dimensional analysis with auto sub-question decomposition
panorama_searchFull-scope event timeline and evolution tracking
quick_searchFast targeted fact verification
interview_agentsReal OASIS API calls — asks simulated agents questions

Output structure

reports/{report_id}/
  meta.json
  outline.json
  progress.json
  section_01.md ... section_N.md
  full_report.md
  agent_log.jsonl
  console_log.txt

Stage 5: Agent Interviews via IPC

File-based IPC protocol

MiroFish uses a simple but effective file-based inter-process communication system to talk to running OASIS simulations:

simulation_dir/
  ipc_commands/
    {uuid}.json   ← server writes
  ipc_responses/
    {uuid}.json   ← OASIS writes back
  env_status.json ← alive/stopped

The SimulationIPCClient writes a command JSON, then polls the responses directory at 0.5s intervals with a 60s timeout.

Command types

CommandWhat it does
INTERVIEWAsk one agent a question
BATCH_INTERVIEWAsk multiple agents at once
CLOSE_ENVGracefully shut down simulation

Why file-based?

  • OASIS runs as a separate Python process
  • No shared memory or socket setup needed
  • Works cross-platform (Windows + Unix)
  • Commands are naturally persistent and debuggable
  • Cleanup is just deleting JSON files

Zep: The Memory Layer

GraphRAG for agent knowledge

Zep Cloud provides the knowledge graph infrastructure that gives agents their "world knowledge".

  • Graph creation — each simulation gets a unique Zep graph
  • Episode ingestion — text chunks become graph episodes
  • Entity extraction — Zep automatically identifies entities and relationships
  • Temporal metadata — edges have valid_at, invalid_at, expired_at timestamps
Key insight: Zep handles the hard NLP — entity resolution, relationship extraction, deduplication. MiroFish just sends text chunks and gets a structured graph back.

Three Zep integration points

ModuleRole
graph_builderCreates graphs, sets ontology, ingests text
zep_entity_readerReads nodes/edges with retry + pagination
zep_graph_memory_updaterUpdates graph during simulation

Pagination helper

# zep_paging.py
def fetch_all_nodes(client, graph_id):
    # Handles Zep's paginated API
    # Returns complete node list

def fetch_all_edges(client, graph_id):
    # Same for edges

Frontend: Vue 3 Architecture

View structure

ViewPurpose
Home.vueLanding page
MainView.vue5-step wizard container
Process.vuePipeline progress tracker
SimulationView.vueSimulation config & launch
SimulationRunView.vueLive simulation monitor
ReportView.vueGenerated report display
InteractionView.vueAgent interview UI

Step components

  • Step1GraphBuild — upload, ontology preview, graph visualization
  • Step2EnvSetup — entity review, profile generation progress
  • Step3Simulation — dual-platform timeline feed
  • Step4Report — section-by-section report output
  • Step5Interaction — chat with agents

Real-time patterns

  • Status polling every 2 seconds
  • Detail polling every 3 seconds
  • Action deduplication on client side
  • Phase transitions: starting → running → completed

REST API Design

The backend exposes 30+ endpoints across three Flask Blueprints:

Simulation API (largest)

EndpointPurpose
POST /createNew simulation
POST /prepareAsync entity + profile + config
POST /startLaunch OASIS subprocess
POST /stopKill simulation
GET /run-statusRound progress, action counts
GET /actionsPaginated action history
GET /timelinePer-round summaries
POST /interviewTalk to an agent
POST /interview/batchTalk to many agents

Graph API

EndpointPurpose
POST /ontologyGenerate ontology from text
POST /buildStart graph construction
GET /dataFull graph nodes + edges

Report API

EndpointPurpose
POST /generateStart report generation
GET /statusSection-level progress
GET /contentFull/partial report MD

Pattern: Async via Threading

The task pattern

Long-running operations use a consistent pattern across the codebase:

def build_graph_async(self, text, ...):
    # 1. Create a task ID
    task_id = self.task_manager.create_task(
        task_type="graph_build",
        metadata={...}
    )

    # 2. Spawn daemon thread
    thread = threading.Thread(
        target=self._build_graph_worker,
        args=(task_id, text, ...)
    )
    thread.daemon = True
    thread.start()

    # 3. Return task ID immediately
    return task_id

Progress tracking

Workers update task progress via the TaskManager:

self.task_manager.update_task(
    task_id,
    status=TaskStatus.PROCESSING,
    progress=60,
    message="Waiting for Zep..."
)

The frontend polls /prepare/status to get updates. This avoids WebSockets while still providing real-time feedback.

Trade-off: Threading + polling is simpler to implement than WebSockets/SSE, at the cost of slightly higher latency (2-3s). Good enough for a simulation that runs for minutes.

Pattern: LLM Robustness

MiroFish relies heavily on LLM-generated JSON. Here is how it handles the messiness of real-world LLM output:

Multi-layer JSON repair

# Layer 1: Fix truncated JSON
def _fix_truncated_json(text):
    # Count open/close braces
    # Append missing closers

# Layer 2: Regex extraction
def _try_fix_json(text):
    # Strip markdown code fences
    # Remove control characters
    # Extract JSON via regex
    # Fallback to basic structure

# Layer 3: Temperature descent
# Retry: 0.7 → 0.5 → 0.3
# Lower temp = more deterministic

ReACT conflict handling

When the LLM simultaneously outputs a tool call AND a final answer (a common failure mode):

  • Tier 1: Strip the final answer, keep the tool call
  • Tier 2: Re-prompt with explicit instruction
  • Tier 3: Force content generation at iteration limit

Prompt engineering guards

  • Chinese-only enforcement with translation
  • Minimum tool call requirements (3 per section)
  • Single tool call per LLM response enforced
  • Previous sections capped at 4000 chars to prevent token bloat

Pattern: Retry & Resilience

Exponential backoff everywhere

def _call_with_retry(
    self, func,
    operation_name,
    max_retries=3,
    initial_delay=1.0
):
    for attempt in range(max_retries):
        try:
            return func()
        except Exception as e:
            delay = initial_delay * (2 ** attempt)
            time.sleep(delay)
    raise Exception(
        f"Failed after {max_retries}"
    )

Resilience strategies across the codebase

ComponentStrategy
Zep API callsExponential backoff, 3 retries
LLM profile genTemperature descent, JSON repair
IPC commands60s timeout, 0.5s poll interval
Episode processing600s timeout, 3s poll, partial success
Report sectionsSave partial on failure, section isolation
Process cleanupSignal handlers + atexit + killpg
Philosophy: Every external call can fail. The system should degrade gracefully, save partial progress, and never leave orphan processes.

Deployment Options

Source code

# Prerequisites
Node.js 18+ / Python 3.11-3.12 / uv

# One-click setup
$ npm run setup:all

# Launch both services
$ npm run dev
# Frontend → :3000
# Backend  → :5001

Environment variables

# .env (from .env.example)
LLM_API_KEY=sk-...
LLM_BASE_URL=https://...
LLM_MODEL_NAME=qwen-plus
ZEP_API_KEY=z_...

Docker

# Single command
$ docker compose up -d

The docker-compose.yml bundles both services. GitHub Actions CI builds the image on push.

Required external services

ServiceNotes
LLM APIAny OpenAI-compatible endpoint. Recommended: Aliyun qwen-plus
Zep CloudFree tier available (monthly limits)
Cost note: The biggest expense is LLM API calls. Profile generation, config generation, and report writing all hit the LLM multiple times per simulation.

End-to-End Data Flow

IN
User uploads documents + describes prediction need
PDF, Markdown, plain text — up to 50MB
↓
1
TextProcessor chunks → OntologyGenerator (LLM) → GraphBuilder (Zep)
500-char chunks → entity/edge schema → knowledge graph with temporal edges
↓
2
ZepEntityReader → OasisProfileGenerator (LLM) → SimConfigGenerator (LLM)
Entities with context → agent personas with MBTI/memory → simulation parameters
↓
3
SimulationRunner → OASIS subprocess → JSONL action logs
Agents post, like, comment on Twitter + Reddit — all actions logged per round
↓
4
ReportAgent (ReACT loop) → 4 tools → full_report.md
Thought-Action-Observation cycles, 3-5 tool calls per section, agent interviews as evidence
↓
OUT
Prediction report + interactive agent world
User can read the report, interview agents, replay with modified variables

Strengths & Trade-offs

Strengths

  • Zero-config barrier — LLM generates everything: ontology, profiles, simulation config, reports
  • End-to-end pipeline — documents in, prediction reports out
  • GraphRAG integration — agents have structured world knowledge, not just prompts
  • Dual-platform simulation — Twitter + Reddit capture different social dynamics
  • Agent interviews — interrogate the simulation for qualitative insights
  • Streaming results — sections save incrementally, partial reports on failure

Trade-offs

  • LLM cost — many LLM calls per simulation (ontology + profiles + config + report)
  • Polling over push — 2-3s latency from threading + HTTP polling vs WebSockets
  • File-based IPC — simple but higher latency than sockets/shared memory
  • Zep dependency — requires external cloud service for graph storage
  • No validation layer — predictions are only as good as the agent model quality
  • Sequential pipeline — each stage must complete before the next starts

Key Takeaways

What MiroFish really is

An orchestration layer that chains together:

  • LLMs for schema/profile/config/report generation
  • Zep for GraphRAG knowledge storage
  • OASIS for multi-agent social simulation
  • Vue frontend for a guided wizard experience

The key innovation is making this accessible: upload documents, describe what you want to predict, get a report.

Numbers worth remembering

5
Pipeline stages
12
Backend services
30+
API endpoints
4
LLM generation points
The big bet: That swarm intelligence simulations of social media agents can produce useful predictions about real-world outcomes.

Learn More

  • GitHub: github.com/666ghj/MiroFish
  • OASIS (CAMEL-AI): the underlying simulation engine
  • Zep Cloud: GraphRAG memory layer
  • License: AGPL-3.0
  • Stack: Python + Flask + Vue 3 + Vite

agent37.club