Deplyra Kesvara

How it works, exactly

Every rule below is the rule the code runs. Where a number appears here it is the number in the source, not a rounded description of it.

1 · Intake is a webhook or forwarded email

Each workspace gets one address at in.kesvara.deplyra.com, shown on the workspace page. It needs no integration, no OAuth app and no access to your infrastructure — most alerting tools can send to an arbitrary address.

On this deployment nothing is receiving mail at in.kesvara.deplyra.com: the endpoint exists and is tested, and no mail routing points at it, so a forwarded alert is discarded before Kesvara sees it. The webhook is the door that works today.

The address carries random bytes rather than being w7@…. It is a capability: anyone who knows it can write into that workspace's history, and the entire value of the product is that the history is real.

A workspace can also have a webhook URL, created from its settings page and not before — a live URL that can write into your incident history is not something to hold by default. POST to it from Prometheus Alertmanager or Grafana with no adapting, or from anything else that can send JSON with a summary, title or message field.

One Alertmanager POST usually carries a whole batch. Every alert in it becomes its own row and the grouping rule below decides whether they are one incident — the same rule, reading the same alert text, whichever door the alert came through. Alerts from your webhook and your email address land on the same card during the same outage.

Two things it will not do with a payload. A resolved or OK notification is never stored as an alert, because a cleared condition stored as one would open an incident whose only content is the news that something is fine — and would then be matchable text, so a future suggestion could cite a recovery. It is handled separately; see step 3. And a payload it cannot find alert text in is refused with a 422 naming the fields it looked for, rather than stored under a placeholder title. Nothing invented gets into a history whose whole value is that it is real.

The webhook URL is itself the credential — there is no separate password, so it will appear in the logs of whatever proxy sits in front of your alerting tool. It is one click to revoke and one to rotate. It is deliberately not also in our logs: the access log redacts it.

2 · The grouping rule, in full

No model decides that two alerts are the same incident. That decision is the one that silently destroys information — two incidents merged means the second is on nobody's screen — so it is made by a rule you can read and argue with, and the rule that fired is printed on every alert.

  1. Same fingerprint, within two hours. The subject is normalised — firing counts, timestamps, UUIDs, IP addresses, percentages and bare numbers are replaced with placeholders — and hashed with the service name. The identical check firing again is the same incident.
  2. Same service, within twenty minutes. This is what collapses a real storm: "connection pool exhausted", "api 5xx rate" and "checkout timing out" are three checks and one outage. The window is much shorter than the first rule on purpose — this one is an inference about different symptoms, and a long window would swallow a genuinely separate incident an hour later.
  3. Otherwise, a new incident.

Where a service name cannot be found in the alert, it is recorded as unknown and stays unknown. Rule 2 then cannot fire, so two unattributable alerts are never merged on the strength of a guess.

3 · When your tools say it stopped

Alertmanager, Grafana and most other tools send a notification when a condition clears. Kesvara can close the card on it, under a rule as explicit as the grouping one — because a rule that closes an incident is a rule you are entitled to read before it does.

  1. A clear is never stored as an alert and never opens an incident. It is recorded in its own table, so it can never become text a future incident is matched against.
  2. It is matched to an open incident by identical fingerprint — the same check — and not by service. A storm card holds several different checks, and the database alert clearing tells you nothing about whether checkout is still timing out.
  3. The card closes only when every distinct check on it has cleared, each after its own most recent alert. Until then it stays open and says how many checks are still firing. Closing a six-alert card because one check recovered would take a live outage off the screen of the person handling it.
  4. A check that fires again after clearing has an alert newer than its clear, so the clear stops counting. Flapping needs no separate rule.
  5. A clear that matches nothing is recorded and does nothing — including whether the incident it named was already resolved, because "your clears are not reaching us" and "they reached us and matched nothing" have different fixes and look identical otherwise.

Over email the rule is narrower still: only a bracketed prefix — [RESOLVED], [OK], (Resolved) — counts as a clear. "DNS lookup could not be resolved" is a real alert and treating it as a recovery would drop it from your history and close a live card. The cost is that a tool writing RESOLVED: with no brackets is not recognised, and its clear is stored as an ordinary alert.

4 · The action item is what writes the library

Closing an incident asks one question: what actually fixed it. That sentence is exactly what a future incident will be shown, so it is typed by a person and never generated. A generated action item would make tomorrow's suggestion a suggestion sourced from yesterday's suggestion, with no evidence at the bottom of the chain.

So an incident your alerting tool closed is closed and not in the library. Nobody wrote down what fixed it, so there is nothing to show anyone — and the workspace page counts these separately, because otherwise a team whose tools close their own cards watches resolved incidents pile up while the library stays at zero and concludes the matching is broken.

5 · Matching, and the two ways in

Matching is Postgres full-text search over incidents in your workspace that are resolved and have an action item written on them. It is lexical, not semantic, and not a trained model. A candidate is shown only when it clears one of these, and the card always says which:

The shared-term count exists because rank alone rewards a single rare word. One shared hostname would otherwise link two unrelated outages and put a confident number next to the link. Below both bars, the screen says nothing is similar — which is a real answer and the one a new workspace gets for weeks.

The second way in is the subject of 5a, and it is deliberately not a loosening of the first. A vocabulary match is not Kesvara deciding two incidents are alike on thinner evidence; it is Kesvara applying a rule somebody on your team wrote down, and the suggestion says so in those words, names the entry, and shows which words hit on each side.

5a · Your words for your failures

Shared-word matching has one obvious hole, and it is the expensive one: your tool says PostgresConnectionsHigh in August and somebody types DB maxed out in November. Those two sentences share nothing, so the same outage is never connected to itself — and a failure that keeps coming back is exactly the one worth citing.

A workspace can close that hole by writing it down. A vocabulary entry is a name and the words your tools and your people use for one failure. Two incidents that hit the same entry are linked, and the card says so in those terms: the entry's name, the words that matched on each side, and who wrote the entry. It is labelled differently from an ordinary match and it does not show a score, because it is a different claim — not "Kesvara found these words in both" but "somebody here said these words mean the same failure".

Two rules keep it honest. A vocabulary entry widens what is found and never changes what is claimed: the shared-term count on a card only ever counts words genuinely present in both incidents, so an entry can never quietly push a pair over the lexical bar and have the result reported as shared wording. And an entry matching more than half of your resolved incidents is switched off — it cannot tell two incidents apart, so it would link almost everything to almost everything. The settings page shows the fraction next to every entry.

5b · Why this is not semantic search

The obvious fix for the hole above is embeddings — match on meaning rather than words. It was measured against this product's bar before anything was built, and it missed by a wide margin. Roughly half of what it would have put in front of an on-call engineer was a citation to an unrelated outage, and no threshold, ranking or hybrid rule tried moved that ceiling.

The failures were not marginal. The two highest-scoring pairs in the test corpus were both wrong: "disk space low" against "disk space recovered", and "payments checkout failing" against "card payments declining". The second is the one that settled it. Both are true descriptions of the same visible symptom — payments not working. One was our own database out of connections; the other was the card processor having a bad afternoon. The action item from either, shown against the other, sends somebody at the wrong system at four in the morning.

No threshold separates that pair, because the similarity is real. An embedding measures how alike two descriptions are; this product's claim is about what fixed the thing described. Those are different questions, and the gap between them is the difference between a symptom and a cause — the one line this product does not cross. So it was not built. The script is in the repository, along with the bar for revisiting the decision with evidence rather than enthusiasm.

5b′ · Finding out you have two names for one failure

A vocabulary entry only helps a team who has already noticed the drift and typed it in. Nobody notices: the two incidents are four months apart and the person who saw both has left. So there is a page that shows you words two or more of your own action items have in common — where the same word turns up in how two incidents were fixed and the alerts themselves shared nothing, it is often one failure your tools gave two names.

It does not decide that any of them are the same failure, and that is a measured position rather than caution. The version that scored the pairs and proposed the entry was built and measured first. It fails on the generic fix: “restarted the pods and it came good” is a true and complete action item for three unrelated outages, and a rule reading action items cannot tell it from “raised max_connections and recycled pgbouncer”. Tuned tight enough to exclude the first, it also excludes the PostgresConnectionsHigh / DB maxed out pair the feature exists for — the best-scoring wrong pair in the corpus outscores it, so there is no threshold between them.

A person makes that distinction in about a second. So the page shows the shared word, the incidents, and each action item as written, and you tick the words that mean the same failure. What comes out is an ordinary vocabulary entry with your name on it, because ticking the words is the assertion. Nothing on the page is scored or ranked by likelihood; the groups are ordered by how many action items share the word, smallest first, which is a fact about the group and not an opinion about it.

What it misses is worth saying plainly: it only finds repeats where two people happened to write the fix down using a word in common. A team who fixed one failure twice and described it two different ways is invisible to it. There is nothing in that history saying those two are one failure, so there is nothing to show — it surfaces what you wrote, and it is only as good as your writing.

5c · Everything shown is sourced

Everything shown is sourced. A suggestion carries the prior incident's date, its title, the exact overlapping terms, and the action item as it was written. There is no path in the code by which a suggestion appears without a stored prior incident behind it: the database column that points at it cannot be null.

6 · Having the card come to you

Everything above makes a card and waits to be visited. At 03:40 nobody visits. A workspace can set one destination URL on its settings page — a Slack, Mattermost or Discord incoming webhook, or any endpoint of your own that accepts a JSON POST — and Kesvara posts one message there when an incident opens. The message carries the citation if the same search described above returns one, and says there is nothing similar when it does not.

Only the alert that opens a card is announced. Collapsing a storm into one incident and then sending forty messages would put the problem back, one room over. Alerts that join an existing card send nothing, and a card that opens and closes inside the same payload sends nothing at all.

No model writes any part of that message. The one sentence a model is allowed to write about a match is checked before it is shown, and a sentence already posted into your channel cannot be taken back.

The limits, because they matter at 03:40. Delivery is three attempts over a few seconds and there is no queue: if your endpoint is down for all three, that message is lost and is not re-sent — the incident is a card here either way. At most twenty messages an hour per workspace; over that they are dropped and counted rather than dropped quietly. Destinations must be https and must resolve to a public address, checked when you save one and again before every send, and redirects are not followed. The request is then made to the address that check just validated rather than to a second lookup that could answer differently, with the certificate still verified against your hostname.

Every message is signed, so an endpoint of your own can tell a Kesvara POST from anyone else's: an HMAC-SHA256 of the body, and a signed timestamp, under a key on your settings page. Being straight about what that is worth — Slack, Discord and Mattermost ignore it, and it protects nothing unless your receiver checks the signature and rejects the message when it does not match. That is code at your end, and we can neither write it for you nor tell whether you have.

Where a language model is, and is not, involved

A model never chooses a match and never sees your library. Once search has already linked two incidents, a model may be asked for one sentence about what the two records have in common. That sentence is then checked, outside the code that calls the model:

A dropped sentence costs a little polish. The match is still shown, with its shared terms and no prose. Every model call is recorded with its token counts and cost in tenths of a penny, visible on your settings page.