LOUST
LOUST-PRO/LLMmempipe
Period: Aug 2025 —
The challenge
My chat history exports from Claude, ChatGPT, and Gemini each had their own schema, their own stale PII, and their own broken tool calls. When I fed them into long-running agents as long-term memory, the contamination propagated downstream and surfaced weeks later as confused behavior.
My role
I built a Rust-based cleaner that normalizes heterogeneous chat exports into a single canonical schema with deterministic, rule-based PII scrubbing and a diff-friendly output format.
What I did
01 Single canonical schema for all vendors
Why: One schema means one audit chain. Per-vendor schemas mean per-vendor failures, and the agent's memory picks up whichever vendor's quirks leaked in.
Trade-off: Some vendor-specific features get flattened during normalization. For a memory substrate the consistency wins; for a feature showcase it would not.
02 Deterministic, rule-based PII scrubbing
Why: Regex guessing leaks, and a reviewer cannot trace a redaction back to the rule that fired. A small, auditable rule book makes every redaction explainable.
Trade-off: New PII patterns require explicit rule updates. The rule book stays small because the same dozen patterns keep appearing across vendors.
03 Diff-friendly output format
Why: Memory file drift has to be reviewable per line. Per-line diffs catch what file-level diffs miss — a single bad export no longer hides inside a hundred clean ones.
Trade-off: Output files are larger than compact formats. The reviewability earns the extra bytes.
What changed
Schema drift across vendors
Before: 3 vendors, 3 schemas, 3 failure modes
After: 1 canonical schema, 1 failure surface
PII scrub coverage
Before: About 70% (regex heuristics)
After: About 98% (rule-based + audit trail)
Evidence: estimate, not measured
Memory file reviewability
Before: File-level diff (single line buried in hundreds)
After: Per-line diff with stable IDs
Trade-offs
I gave up some per-vendor feature richness for the canonical substrate. For long-running agent memory the substrate is the product; the vendor-specific extras would be a tax.
What I learned
Memory substrate quality is the limiting factor on long-running agents, not the agent's reasoning. A clean substrate lets a small model perform like a large one.
Stack
- Rust
- LLM tooling
- normalization