Development Process Guide

This guide explains our infrastructure, development workflow, and security practices for the Zhiheng project.

Architecture Overview

Our production system runs on a single cloud VM with Docker Compose managing 7 services:

Service	Container	Port	Purpose
Nginx	zhiheng-nginx-1	:8888	Reverse proxy, security headers, SPA hosting
Frontend	zhiheng-frontend-1	internal :80	Vue 3.5+Vite SPA
Backend	zhiheng-backend-1	internal :8000	FastAPI (auth, lessons, chat, RAG)
LightRAG	zhiheng-lightrag-1	:9621	Graph-based knowledge base (PostgreSQL+AGE)
DeerFlow	zhiheng-deerflow	:8001	AI agent platform (tools, skills, PPT generation)
PostgreSQL	zhiheng-db	internal :5432	pgvector + AGE graph extension
Redis	zhiheng-redis-1	:6379	Chat thread sessions

External dependencies (not running on our VM):

• vLLM — Qwen3.6-35B-A3B-FP8 at vllm-fast.cxr-insight.com (via Cloudflare tunnel)
• Embedding service — via OpenRouter (qwen/qwen3-embedding-8b, proxied through LiteLLM)
• S3 storage — PVC-backed zhiheng-user-data (no external S3 needed)
• Cloudflare — DNS + TLS termination + tunnel for zhiheng.app

Git Worktree Setup

Each developer works in a git worktree — a lightweight branch workspace that shares the same git objects as the main repo.

Root repo (production): /home/ubuntu/dev/GXNU/zhiheng/ (branch main)

Your worktree: /home/ubuntu/dev/GXNU/zhiheng-{your-name}/

Worktree Benefits

• Disk efficient — ~few MB vs ~100MB+ per clone
• Instant branch sharing — commits visible to all worktrees immediately
• Parallel work — multiple branches simultaneously without switching

Creating Your Environment

Set up your worktree:

Run the setup commands in AGENTS.md

Setup will:

1. Create the worktree under /home/ubuntu/dev/GXNU/zhiheng-{name}/
2. Shift ports (nginx, redis, lightrag, deerflow) to avoid conflicts
3. Create isolated Docker network + named volumes
4. Fix LightRAG .env (masked *** values)
5. Install Apache AGE extension in your database
6. Set up Cloudflare tunnel routes + DNS

Your Instance Access

What	URL / Port
Web UI	https://{name}.zhiheng.app/
Local	http://localhost:{nginx-port}/
Nginx port	Each worktree gets a unique port
LightRAG port	127.0.0.1:{lightrag-port}

HTTP Basic Auth: admin / *(check .env or ask team)

Docker Compose Workflow

All Docker commands should be run inside your worktree directory:

cd /home/ubuntu/dev/GXNU/zhiheng-{your-name}/

Common Commands

# Full rebuild + restart
docker compose up -d --build

# Frontend only (after code changes)
docker compose build frontend && docker compose up -d --force-recreate frontend

# Backend only (after Python changes)
docker compose build backend && docker compose up -d --force-recreate backend

# Check status
docker compose ps

# View logs
docker compose logs -f backend
docker compose logs -f lightrag

# Stop everything
docker compose down

⚠️ Critical Docker Pitfalls

1. docker compose restart does NOT pick up new images — After docker compose build, you must stop + rm + create or use up -d --force-recreate.

2. docker compose restart does NOT reload .env — The container must be recreated to pick up new env vars. Use up -d instead.

3. Port merging in docker-compose.override.yml — Override files MERGE port arrays rather than replacing them. Edit docker-compose.yml directly.

4. Nginx stale DNS — After backend restart, nginx may cache the old IP. Restart nginx: docker compose restart nginx.

5. LightRAG .env masking — New worktrees inherit literal *** values from Docker secret masking. Fixed automatically during setup.

6. Apache AGE missing — Fresh DB containers don't have AGE. Installed during setup.

Cloudflare Tunnel

Our production domain zhiheng.app is served through a Cloudflare Tunnel (cloudflared):

• Tunnel: zhiheng-main (2c79c700)
• Systemd service: cloudflared-zhiheng-app.service
• Config: /etc/cloudflared/zhiheng-app-tunnel.yml

Each teammate's subdomain ({name}.zhiheng.app) is routed through the same tunnel to their unique nginx port.

Error Codes

Error	Meaning	Fix
1033	Tunnel connector not running	sudo systemctl restart cloudflared-zhiheng-app.service
504	Backend not responding (100s timeout)	Check backend logs: docker compose logs backend
530	Origin SSL/TLS issue	Check Cloudflare tunnel config

Security Practices ⚠️

We are developing on production infrastructure. Every change affects real users. Follow these practices:

1. Never Expose Services Publicly

• All services bind to 127.0.0.1 or Docker internal networks
• Only nginx (:8888) and teammate nginx ports are exposed
• Redis and LightRAG ports are localhost-only
• Never publish database ports to the host

2. Protect Credentials

Our .env files contain sensitive data:

• Database passwords
• JWT secrets
• API keys (vLLM, embedding, internal)
• Cloudflare tokens

Rules:

• ✅ .env is in .gitignore — never commit
• ❌ Never share .env contents in chat or docs
• ❌ Never hardcode secrets in source code
• ✅ Use environment variables

3. Test Before Deploying

When you make changes to code:

1. Build locally in your worktree first
2. Test your subdomain ({name}.zhiheng.app)
3. Only then merge to main for production

4. Database Safety

• Each worktree has its own isolated database — you won't accidentally affect others
• Never connect to the production DB (zhiheng_dev or zhiheng main) unless explicitly authorized
• Back up before destructive operations

5. Cloudflare Workers

Our auth is handled by a Cloudflare Worker (session-based, not Basic Auth headers). Don't modify the Worker without coordinating with the team.

6. PVC Storage

All worktrees share the same PVC (zhiheng-user-data-pvc). Files are mounted via per-user subPath — no cross-user data leakage.

Working with the AI Agent

DeerFlow is our AI agent platform — accessible via the chat endpoint or through the frontend chat interface.

How the AI Agent Works

• DeerFlow runs as a Kubernetes deployment
• It has custom tools for LightRAG queries, lesson management, and PPT generation
• It uses skills (reusable procedures) for complex workflows
• Chat is stateless per run — conversation history is maintained by the backend

Example Prompts

Generate a PPT:

Please generate a PPT for Lesson 10: "Understanding the Constitution" 
Include 5 slides with key points about China's constitutional system.

Query the knowledge base:

What does the textbook say about "法治建设" (rule of law construction) 
in the context of middle school education?

Create a lesson plan:

Create a lesson plan for Grade 8 Unit 4 Lesson 2, focusing on 
"公民的基本权利和义务" (basic rights and duties of citizens).
Include learning objectives, key points, and classroom activities.

Debug an issue:

The knowledge page returns empty results when I search for "宪法" (constitution).
Can you help me diagnose the issue?

AI Agent via Feishu (Lark)

You can also interact with the AI agent through our Feishu group. Use the chat interface to get help.

Tips for Better Results

1. Be specific — Include lesson numbers, grades, and specific topics
2. Provide context — "For Grade 8, Unit 4, the textbook mentions..."
3. Ask for structure — "Organize into 3 sections with bullet points"
4. Iterate — If the first result isn't right, refine your prompt

Troubleshooting

Common Issues

Problem	Diagnosis	Fix
Frontend shows old code	Vite hash determinism	docker compose build --no-cache frontend && docker compose up -d --force-recreate frontend
Backend 502 after rebuild	Nginx stale upstream IP	docker compose restart nginx
LightRAG returns 0 results	Workspace not set or empty	Check docker-compose.yml has command: --workspace {name}
Chat hangs (504)	Agent timeout or ClientDisconnect	Check backend logs for ClientDisconnect errors
Can't login	Wrong Basic Auth password	admin / (check .env)
Error 1033	Tunnel not running	sudo systemctl restart cloudflared-zhiheng-app.service
LightRAG crash-looping	*** in .env or missing AGE	Fix .env or reinstall AGE

Getting Help

1. Check your container logs: docker compose logs -f {service}
2. Check Docker status: docker compose ps
3. Use the chat interface or Feishu
4. Check this doc for common issues