Bhargav Urs Sumantharaj

AboutWorkResumeReader

ConvergeAI

unknown

A document-grounded platform where three independent language models answer, critique each other against the source text, revise, and converge on one cited answer.

Liveconverge-ai-nine.vercel.appRepositorygithub.com

Architecture

  • React 18 + Vite client (Vercel), client. Uploads documents, streams the debate phase by phase as it happens, and renders the final cited answer. Connected to Apache Tika parse over REST. Connected to Orchestrator state machine over STOMP over SockJS.
  • Apache Tika parse (ingestion), service. Parses PDF, Word, HTML, and plain text uploads into one normalised text stream before chunking. Connected from React 18 + Vite client over REST. Connected to Recursive overlapping splitter.
  • Recursive overlapping splitter, service. Cuts parsed text into overlapping chunks sized for the embedder, preserving continuity across chunk boundaries. Connected from Apache Tika parse. Connected to ONNX MiniLM embedder.
  • ONNX MiniLM embedder (all-MiniLM-L6-v2, 384-d), service. Turns every chunk and every query into a 384-dimension vector, in-process, on the CPU. Connected from Recursive overlapping splitter. Connected to PostgreSQL + pgvector over vector(384).
  • Orchestrator state machine (5 phases, virtual threads), service. Drives each debate through five phases: retrieval, independent answers, cross-critique, revision, consensus. Connected from React 18 + Vite client over STOMP over SockJS. Connected to Provider router. Connected to Consensus engine.
  • Provider router (retry + failover), service. Sends each model call to its provider, honours Retry-After on rate limits, fails over across providers, and parses every response defensively. Connected from Orchestrator state machine. Connected from Consensus engine over same router. Connected to Groq. Connected to Cerebras. Connected to Google. Connected to OpenRouter.
  • Consensus engine (Qwen3 27B via Groq), service. Takes all three revised answers plus the retrieved chunks and synthesises a single verdict as strict JSON: the final answer, the points where the agents agreed, the points where they diverged, and a 0 to 100 confidence score. Connected from Orchestrator state machine. Connected to Provider router over same router. Connected to PostgreSQL + pgvector over JDBC.
  • Groq (The Reviewer, GPT-OSS 120B), external. Serves GPT-OSS 120B as The Reviewer, one of the three independent debaters, and Qwen3 27B for the consensus pass. Connected from Provider router.
  • Cerebras (The Engineer, GLM-4.7), external. Serves GLM-4.7 as The Engineer, the second independent debater. Connected from Provider router.
  • Google (The Analyst, Gemini 2.5 Flash), external. Serves Gemini 2.5 Flash as The Analyst, the third independent debater. Connected from Provider router.
  • OpenRouter (fallback pool), external. The fallback pool: when a primary provider rate-limits or fails, the call reroutes here. Connected from Provider router.
  • PostgreSQL + pgvector (HNSW cosine, Flyway), datastore. One database holds chunks, vectors, debates, and phase events; pgvector's HNSW index serves cosine retrieval. Connected from ONNX MiniLM embedder over vector(384). Connected from Consensus engine over JDBC.
  • Boundary: Spring Boot 3.5 service, Java 21, Amazon EC2.
    The five phases:
  1. Retrieval
  2. Independent answers
  3. Cross-critique
  4. Revision
  5. Consensus

Trace

live
Fires one real read-only request at the Spring Boot health endpoint on the EC2 instance and times each hop.

Decisions

  • Embeddings stay in-process

    The decision
    Embeddings run in-process on a quantised ONNX build of all-MiniLM-L6-v2 rather than through an embedding API.
    The constraint that forced it
    Indexing has to cost nothing, and document text must not leave the process.
    Rejected
    A hosted embedding API, which meters every index rebuild and ships user documents out.
    Consequence accepted
    The 384-dimension output is the schema contract for the vector(384) column, and embedding quality is capped at what a small local model gives.

  • Concurrency on virtual threads

    The decision
    The three agents run concurrently on Java 21 virtual threads, so a debate round costs the slowest model rather than the sum of all three.
    The constraint that forced it
    Sequential rounds put end-to-end latency at roughly 4.5 minutes, which is unusable.
    Rejected
    A reactive rewrite on WebFlux. Same concurrency, at the price of infecting every signature with reactive types.
    Consequence accepted
    End-to-end latency fell to under a minute, and to about 5 seconds in fast mode, while the code stays in plain blocking style.

  • Assume providers misbehave

    The decision
    Multi-provider routing with retry that honours Retry-After, cross-provider failover, and defensive JSON parsing.
    The constraint that forced it
    Free-tier model endpoints return malformed output and rate limits as a matter of routine rather than as an exception.
    Rejected
    Treating provider errors as exceptional and surfacing them straight to the user.
    Consequence accepted
    Every response passes through a parser that assumes it is broken, which costs code and a little latency on the happy path.

  • One database, not two

    The decision
    Vectors and relational rows live in one PostgreSQL database with Flyway migrations, rather than a separate vector store.
    The constraint that forced it
    A debate writes chunks, vectors, and events together, and they need to stay consistent.
    Rejected
    A dedicated vector database running beside Postgres.
    Consequence accepted
    One backup, one transaction boundary, one connection pool. The ceiling is Postgres's vector performance, accepted knowingly.

  • The consensus step is a model, not a merge

    The decision
    The consensus engine sends all three revised answers plus the retrieved chunks to Qwen3 27B on Groq and gets back strict JSON: a final answer, the points of agreement, the points of divergence, and a 0 to 100 confidence score.
    The constraint that forced it
    After three debate rounds each agent holds its own revised answer, and three answers is not an answer.
    Rejected
    A deterministic merge. Concatenating, voting, and picking one all collapse into fragments on a genuinely contested question, and none of them can recognise that two differently worded claims mean the same thing.
    Consequence accepted
    One more provider round trip on the critical path. It runs through the same router as the agents, Groq first and OpenRouter as fallback, so a rate limit on the last call degrades to the Reviewer's answer instead of failing the whole debate.

  • Late subscribers see the whole story

    The decision
    WebSocket subscribers receive a hydration payload on subscribe, so a client that connects mid-pipeline sees the events it missed instead of an empty panel.
    The constraint that forced it
    Debates outlive page loads. Refreshing during phase three must not blank the screen.
    Rejected
    Making clients poll a REST endpoint for history alongside the socket.
    Consequence accepted
    The server keeps an ordered event log per debate and pays a replay cost on every subscribe.

Known limitations

  • Free-tier providers dominate the latency profile and the quality ceiling.
  • There is no evaluation harness, so "the debate improves the answer" is a design argument rather than a measured result.
  • Confidence scores are model-reported and not calibrated against ground truth.