NSGIA Memory System

TITLE: NSG Coding Episodic Memory Index # NSG Coding Episodic Memory Index **Purpose:** Compact reference injected at conversation start. Prevents repeating failures. **Format:** Episodes with dates, outcomes, and lessons. ~15KB target for context efficiency. **Update:** Add new episodes as they occur. --- ## CRITICAL FAILURES (Memorize These) ### Episode: Jinja Tags in HTML Comments **Date:** 2025-12-30 | **Project:** Eagles Flock | **Hours Lost:** 3+ **What Happened:** Put `{% include %}` and `{{ variable }}` inside HTML comments. Jinja parsed them anyway, causing infinite recursion. **The Fix:** Write plain text in comments. Never put Jinja syntax anywhere, even commented out. ```html <!-- BAD: {% include 'partials/_navbar.html' %} --> <!-- GOOD: includes partials/_navbar.html --> ``` ### Episode: GPS Calibration - 3 Hours for One Line **Date:** 2025-10-04 | **Project:** Flood Mapper | **Hours Lost:** 3 **What Happened:** Wrote new GPS code from scratch. Introduced bugs. Chased symptoms for hours. **The Truth:** `locateUser()` already existed and worked. One function call fixed everything. **The Fix:** ALWAYS ask "What already works?" before writing new code. ### Episode: Credentials Page - 10+ Hours **Date:** 2025-12 | **Project:** NSG Website | **Hours Lost:** 10+ **What Happened:** Jumped into coding without reading standards. Wrote salesy American copy. Multiple rewrites. **The Fix:** Fetch ALL standards URLs at session start. Read them BEFORE any coding. ### Episode: RAG /rag Endpoint - 7 Versions **Date:** 2026-01-31 | **Project:** API | **Hours Lost:** 2+ **What Happened:** Made 7 versions (v1.1→v1.7) patching blind instead of researching. The working code was in index_db.py the whole time. **The Truth:** Should have: (1) searched RAG for existing code, (2) compared working vs broken, (3) copied working pattern. **The Fix:** Stop at failure 3. Research what already works. Copy it exactly. ### Episode: CityPlan Ghana - Spaghetti Deployment **Date:** 2025-12 | **Project:** CityPlan | **Hours Lost:** 6+ **What Happened:** Created parallel folder structures (`cityplan_deployment/`, `cityplan_additions/`, `cityplan_updates/`). Different CSS names. Multiple zip files. **The Fix:** Map directory structure BEFORE coding. One folder. Files named exactly as on server. --- ## SUCCESSFUL PATTERNS (Copy These) ### Pattern: SQLite FTS5 for Search **Date:** 2026-01-25 | **Project:** RAG Index **What Worked:** After USB freeze failures, switched to SQLite FTS5. Indexed 6M files. Searches return in milliseconds. **Key Code:** ```sql CREATE VIRTUAL TABLE files USING fts5(filepath, filename, content, tokenize="porter unicode61"); SELECT filepath, snippet(files, 2, '>>>', '<<<', '...', 25) FROM files WHERE files MATCH ? LIMIT 20; ``` ### Pattern: Magic Key Authentication **Date:** 2026-01-25 | **Project:** RAG Index **What Worked:** 64-char hex keys in MySQL. Per-user, per-index permissions. Deactivatable. **Structure:** `/index/k/{magic_key}/plain/search/{index}/{query}/{limit}` ### Pattern: Microsoft Project XML via MPXJ **Date:** 2026-01 | **Project:** Project Management **What Worked:** Used MPXJ library in Java to generate valid MS Project XML. Python libraries failed. **Lesson:** Sometimes the right tool is in a different language. ### Pattern: Accordion Structures for Content **Date:** 2026-01 | **Project:** Multiple websites **What Worked:** Reduced text walls. Users see headers, expand what interests them. **Key:** Don't dump everything at once. Structure for scanning. ### Pattern: str_replace for Small Edits **Date:** Ongoing **What Works:** For changing 3 words, use surgical str_replace, not file regeneration. **Anti-pattern:** Rewriting 3000 lines to change 3 words. --- ## ARCHITECTURE DECISIONS (Reference) ### Server Layout - **PHR Server:** 104.191.236.122 (production) - **Workstation:** 104.191.236.121 (same network) - **All sites:** /var/www/html/{subdomain}.nsgia.com - **API:** /var/www/api.nsgia.com (port 40003 via gunicorn) - **SCP:** Use `mh@`, not `root@` ### Flask App Structure - Main app imports blueprints - Blueprints handle routes - Gunicorn serves, nginx proxies - Restart: `pkill -f "gunicorn.*{port}" && gunicorn --bind 127.0.0.1:{port} --workers 1 --timeout 90 {app}:app --daemon` ### Database Patterns - **FTS5:** For keyword search (files, content) - **sqlite-vss:** For vector/semantic search (384 dim embeddings) - **MySQL (arubi):** For user auth, permissions - **Registry pattern:** registry.db tracks all indexes with metadata ### File Limits - Python files: 300 lines max - Mandatory error handling - Change logs required (but minimal - no code examples in changelog) --- ## THE RULES (Non-Negotiable) 1. **Ready-Aim-Fire, NOT Ready-Fire-Aim** - Research before coding - Fetch standards at session start - Ask "what already works?" 2. **5-Failure Rule** - After 3 failed attempts, STOP - Research the architecture - Find what already works - Copy the working pattern 3. **Faithful Replication** - Copy working pages exactly - Then modify content only - Don't invent new patterns 4. **Deployment Simplicity** - Ask: "Where exactly do you want these files?" - Simplest solution wins - One folder, correct names 5. **Communication Standards** - Peer-to-peer tone (no teaching, no cheerleading) - State facts and capacity - Positive framing per Kahneman - No speculation - say "to be determined" --- ## QUICK DIAGNOSTICS **If stuck on a bug:** 1. Search RAG for similar code 2. Find what already works 3. Compare working vs broken 4. Copy the working pattern **If deployment fails:** 1. Check live site structure first 2. Verify template extends base.html 3. Check CSS class names match 4. Clear cache / version bust static files **If tone is wrong:** 1. Re-read NSG Communication Standards 2. Remove directives, teaching, cheerleading 3. State facts only 4. Get outline approval before HTML **If file is too complex:** 1. Split at 300 lines 2. Use partials/includes 3. Map structure before coding 4. One responsibility per file --- ## EPISODE LOG (Recent) | Date | Project | Outcome | Key Lesson | |------|---------|---------|------------| | 2026-01-31 | RAG /rag endpoint | Success after 7 versions | Research first, copy working code | | 2026-01-31 | Document Search UI | Success | Conditional rendering for index types | | 2026-01-25 | SQLite FTS5 index | Major success | FTS5 solved USB freeze problem | | 2026-01-25 | Magic key auth | Success | Simple hex keys, MySQL permissions | | 2025-12-30 | Eagles Flock | C+ grade | Jinja in comments, coded before research | | 2025-12 | Credentials Page | C+ grade | 10+ hours, didn't read standards | | 2025-10-04 | GPS Calibration | C- grade | 3 hours for one function call | --- *Last Updated: 2026-01-31* *Add new episodes as they occur* *Target: Read this at session start, prevent repeated failures*