Glas Intelligence
Multi-agent simulation that turns uploaded documents into stakeholder agents and runs policy scenarios.
Why there is no measurement
No accuracy number yet — and that is the honest answer
No measurement
Why not
There is no accuracy number here that I can stand behind. The repository contains proper scoring code — Brier score, log score, CRPS, Murphy decomposition, calibration curves — and it is unit-tested against textbook identities. But it has never been run against resolved real-world outcomes, so it currently scores nothing. Until a set of these forecasts has actually resolved and been graded, any accuracy claim would be exactly the kind of unbacked number this site exists to avoid.
Problem
Policy and market analysis usually stops at a single narrative: one analyst's read of how stakeholders will react. That read is unfalsifiable and it hides its own assumptions. I wanted a system where the assumptions are explicit objects — who the actors are, what they want, what they know — so a scenario can be re-run with one of them changed and the difference inspected.
Approach
Uploaded documents are chunked and pushed into a Zep Cloud temporal knowledge graph, which extracts entities and relationships. Those nodes and edges are read back to synthesise agent personas, which are written to a config and run as an OASIS (camel-ai) social simulation in an isolated subprocess. The simulated interactions are aggregated into stance distributions and LLM-generated probability triplets, which pass through a calibration guardrail layer before a ReACT-style report agent writes them up. Celery orchestrates the chain; results land in Supabase.
What it actually does
A user uploads source documents — consultations, financial settlements, trade
press. TextProcessor chunks them and GraphBuilder pushes each chunk into
Zep Cloud as an episode, where Zep performs the entity and relationship
extraction. That graph is the shared world model.
OasisProfileGenerator then reads nodes and edges back out and synthesises
agent personas from them: a community pharmacist, a chain operator, a
commissioner, a patient advocate. The personas are written to a config file and
an OASIS simulation is launched as a subprocess, deliberately isolated from the
Flask process, for a fixed number of rounds.
What comes back is a corpus of simulated posts and reactions.
QuantitativeAnalysisService aggregates those into stance distributions and
asks an LLM for probability triplets — low, central, high — for each outcome of
interest. A report agent then queries the graph and those statistics to produce
the written output.
The part I would defend in a review
The calibration guardrail layer. LLMs emit probability estimates that are individually plausible and collectively incoherent: intervals ordered wrongly, confidence pinned at 0 or 100, ranges narrower than the model could possibly justify, likelihood ratios that imply more evidence than exists.
So the triplets do not reach the report unchecked. They are rescaled to a common basis, sorted, clamped away from the certainty rails, widened when the stated interval is implausibly tight, and their likelihood ratios bounded. Thirty-six tests hold that behaviour in place.
This is a claim about output hygiene, not about correctness. The guardrails stop the system from stating things it has no basis for. They do not make the underlying forecast right.
What is not finished
The forecast scoring module is written but not wired into the live pipeline — its own docstring says so. The Monte Carlo engine has no test file. Stripe is in test mode. A correlation discount constant in the aggregation is a heuristic with no formal derivation behind it, which is noted in the repository’s own task list rather than quietly left in the code.
I would rather say all of that here than have someone find it by reading the repository I linked.