Almost every tracker in this category starts from the same assumption: the user will log each application manually, keep the spreadsheet tidy, and update statuses as replies come in. That assumption collapses fast. The people who need a tracker most are applying under time pressure, juggling portals, and already context-switching across dozens of roles. The system falls apart exactly when the user needs it most.

Atriveo exists because I wanted the tracking step to disappear.

#The real problem was adherence

The surface problem looks like organization: people want a better way to see applications, interviews, rejections, and response rates. The real problem underneath it is adherence. If capture is manual, the data goes stale in a week. Once the data is stale, every chart on top of it becomes decorative.

That drove the entire product direction. I was less interested in making a prettier dashboard than in making sure the dashboard had a chance to stay true.

#Why the browser extension came first

The key move was pushing collection to the point where the action actually happens: the job board itself.

Atriveo's Chrome extension watches for submission events in the page, extracts the structured payload, and sends it to the backend without asking the user to do anything else. The point was not novelty for its own sake. It was to turn tracking into a byproduct of applying.

That decision made the rest of the system possible:

• analytics could assume fresh data instead of reminding users to update it
• the backend could deduplicate and normalize centrally instead of trusting local notes
• the product could compete on signal quality rather than on how disciplined the user felt that week

It was also the part that created the most product leverage. A dashboard can be copied. Reliable passive capture is much harder.

#Generic parsing was the wrong abstraction

One lesson from the early builds was that a "universal" parser sounds elegant and performs badly.

Job boards look similar at a glance and wildly different in the DOM. The long tail is especially messy: custom components, inconsistent field names, and submission flows that change without warning. A one-size-fits-all parser degraded exactly where users were counting on it.

I moved to explicit per-board heuristics for the highest-traffic sites instead. That raised maintenance cost, but the trade was worth it. Precision matters more than architectural neatness when a false positive means someone's pipeline history is wrong.

That same trade showed up elsewhere too. I kept as much logic as possible server-side because a fix in the API can ship immediately, while a fix in the extension waits on Chrome Web Store review and then waits again for users to update.

#The backend exists to make the analytics honest

Once passive capture worked, the rest of the product could focus on what the data meant.

The API validates incoming events, deduplicates repeated applications, and persists them into a normalized PostgreSQL schema. That structure mattered because the questions users actually ask are relational:

• which sources convert best
• which companies respond fastest
• which stages stall
• whether application volume is turning into interviews at all

Those queries become simple once companies, roles, sources, and stages are modeled cleanly. They become painful if everything is stored as free-form documents and fixed later in application code.

The result is a system that now serves 100+ active customers, handles 2K+ daily queries, and stays up at 99.9% uptime. Those numbers matter, but they are downstream of the more important decision: make the input automatic.

#What I would keep and what I would change

I would absolutely keep the principle that capture comes before analytics. It was the right product call and the right technical one.

I would also keep the discipline of responding closely to users early. The 5.0 rating on the Chrome Web Store did not come from feature count. It came from closing feedback loops quickly and treating small product friction as serious.

What I would change is the tooling around detector health. The most dangerous failures in Atriveo are silent ones: a board changes its DOM, capture rate drops, and the user assumes nothing is wrong. Health checks around board-specific detection should have existed earlier and should probably be even more aggressive now.

The main thing Atriveo taught me is simple: in products like this, the moat is not the chart. The moat is the boring, reliable system that makes the chart worth believing.

They answer quickly, sound fluent, and make the model feel confident. In legal work, that is not enough. A legal answer without a source somebody can inspect is not useful just because it reads well. It is often worse than no answer, because it invites trust without giving the reader any way to verify the claim.

That was the starting constraint for this project: the answer is not the product unless the citation holds up.

#Why I inverted the usual RAG contract

The default RAG pattern is straightforward: retrieve some chunks, hand them to the model, and ask for a summary. The failure mode is straightforward too. If retrieval is weak, the model smooths over the gaps and produces something that sounds coherent anyway.

That is tolerable in some domains. It is not tolerable here.

So I flipped the contract. Instead of "answer if you can," the system behaves more like "retrieve first, answer only from what retrieval can support, and refuse when the evidence is weak." That is a less magical user experience on the surface, but it is a much better trust contract.

The goal was never to make the model feel smart. The goal was to make the system auditable.

#Structural chunking mattered more than model choice

One of the earliest design choices was to chunk legal documents on structural boundaries instead of fixed token windows.

Fixed windows are easy to implement and usually wrong for this kind of material. Clauses get split in the middle, headings detach from the text they govern, and the citation lands on a fragment that cannot stand on its own. The retrieval score may look acceptable while the evidence is still unusable.

Chunking by sections, clauses, and other document boundaries costs more preprocessing work, but it pays back in the only place that matters: when a reader opens the cited span, it reads like a real unit of meaning instead of a clipped excerpt.

That single choice improved answer quality more than swapping models did.

#Retrieval quality is the whole product

This project reinforced something I keep running into with LLM systems: many "generation" problems are retrieval problems in disguise.

If the wrong chunk comes back, the answer is doomed before the prompt even starts. If the right chunks come back but in the wrong order, the answer drifts. If two relevant passages disagree and the system hides the disagreement, the model looks smoother while the product gets less trustworthy.

So most of the real work ended up in the layers before generation:

• better chunk boundaries
• ranking that respected document structure
• metadata filters for narrowing by document and jurisdiction
• a response contract that forbids unsupported claims

Once those pieces were right, the model had a fair chance to behave. Before that, it was just papering over retrieval mistakes.

#Refusal is a feature, not a fallback

One decision I feel strongly about is refusal under weak retrieval.

People sometimes see refusal as a sign that the system is failing. In legal contexts it is often the system behaving correctly. If the evidence is thin, approximate confidence is the dangerous outcome. A refusal that says, in effect, "I do not have enough support for this answer" is much closer to what a careful human would do.

That makes the product less theatrical and more dependable.

It also changes how the interface should feel. A good legal assistant is not trying to hide uncertainty. It is trying to surface the boundary between what the source supports and what the model is tempted to infer.

#What this project changed in how I think about RAG

The main shift for me was seeing retrieval not as plumbing around the model, but as the system's actual core logic.

Improving chunking, ranking, and refusal behavior moved output quality far more than changing the model did. The answer contract only works if the retrieval layer deserves that power.

If I extended this system further, I would keep pushing in the same direction: stronger handling of contradictory passages, better provenance on every claim, and more explicit UI around what came from source text versus what came from synthesis.

That is the version of RAG I trust most now. Not the version that tries hardest to answer, but the one that makes it hardest to answer casually.

In medical imaging, people tend to focus first on raw accuracy. That matters, of course. But in practice a workflow can still be slow, awkward, and hard to trust even with a strong model if it adds friction in the wrong place. What delayed value here was not only classification quality. It was time-to-first-read.

That pushed me toward a less common deployment target: the browser.

#The queue matters more than the headline metric

Reviewing scans is a throughput problem as much as an ML problem.

If clinicians wait on uploads, remote inference, or tool-switching before they can even inspect a result, the model may be technically accurate and operationally unhelpful. A smaller improvement that fits the workflow cleanly can matter more than another benchmark win that arrives late.

That is why I treated the viewer as part of the model, not as a shell around it. The product needed to shorten the path from opening a scan to seeing something useful.

#Why the browser was the right deployment target

Running inference client-side solved two problems at once.

First, it reduced privacy exposure. MRI data is not the kind of thing you casually move around just because a hosted endpoint is easier to maintain. Keeping pixels on the reviewer's machine removes an entire category of risk and compliance overhead.

Second, it cut latency in the workflow. There is no upload wait and no server round-trip before the reviewer can start interacting with the scan. That matters when the alternative is sending large volumes to a service first and only then rendering the result.

The core path became:

• train offline on GCP
• export and quantize for TensorFlow.js
• run inference in the browser
• render slice-by-slice overlays the reviewer can inspect immediately

That design choice is what turned the system into a practical tool instead of only a research artifact.

#Quantization was a product decision

Getting the model into the browser was not a mechanical export step. It was a trade study.

A model that is too large to load or too slow to run interactively is functionally unavailable, even if it is more accurate on paper. Quantization was therefore not just an optimization task. It was part of product design: decide how much model complexity the real workflow can afford.

This is where a lot of ML projects quietly fall apart. They treat deployment as a packaging problem after the "real work" is done. In reality, the deployment target changes what counts as a good model.

For this viewer, a slightly leaner model that loads reliably and responds quickly is often the better clinical experience than a heavier model that wins the metric table but loses the room.

#Accuracy needed the right framing

The reported classification accuracy is useful, but it is not enough on its own.

Tumor voxels are a small fraction of any given volume, which means naive aggregate accuracy can flatter a model that mostly predicts background correctly. That is why the evaluation had to care about recall and segmentation behavior, not just a broad percentage.

The viewer design also reflects that mindset. The segmentation appears as an overlay, not a replacement for the original scan. The scan stays authoritative. The model assists interpretation; it does not pretend to become the image.

That is a small interface detail with a big implication: the system is built to support clinical judgment, not to obscure it.

#What this project changed for me

This was one of the clearest examples I have seen of workflow placement beating raw model glamour.

The throughput gain came from meeting the clinician where the work actually happens. Reducing the path from scan to first useful visual mattered more than squeezing for an extra talking point in evaluation.

If I pushed this further, I would keep working on browser-side performance and richer interaction around overlays, but I would protect the same core principle: the most valuable model is the one that arrives early enough, privately enough, and clearly enough to fit the real workflow.

That is the version that turns ML from a result into a tool.

A producer changes a field. Nothing breaks where the change happened. Hours later a downstream consumer fails in a different service, with a message that points nowhere near the real cause. By the time somebody traces it back, the team has already spent time debugging the wrong layer.

PolicyFabric started from wanting that failure mode to be visible earlier and in the right shape.

#Contract failures are topology problems

The usual tooling around schema changes tends to flatten the problem into logs, reports, or lists of violations. That is useful up to a point, but it loses the most important piece of context: who depends on whom.

Contract failures are inherently topological. The core question is never just "is this field change breaking." The real question is "which downstream edges does this break, and how far does the blast radius travel?"

Once I framed the problem that way, the product direction became clearer. The graph itself had to stop being a hidden implementation detail. It had to become the surface.

#Why I chose a graph over a report

PolicyFabric parses services, schemas, and producer-consumer relationships into a directed graph, then evaluates changes against that graph before they ship.

That sounds like an implementation choice, but it was really a UI choice too. I did not want the output to be a block of text somebody needed to interpret after the fact. I wanted the system to answer the downstream question visually:

• what changed
• what depends on it
• where the violation sits
• which path through the system is affected

The visual layer is the difference between "something somewhere might break" and "this edge here is the problem."

#The full graph is often the wrong view

One challenge showed up quickly: graph interfaces become unreadable faster than people expect.

Rendering every service and every contract at once sounds comprehensive and often produces a diagram nobody can parse. Past a certain size, the graph stops explaining the system and starts advertising its complexity.

That is why the more useful view is usually the affected subgraph, not the whole topology. When a change is evaluated, the interesting question is the local consequence set. Collapse everything else and let the reader focus on the edges that matter right now.

That sounds like a visual convenience. It is actually a correctness feature. A readable warning is more enforceable than a perfect but overwhelming one.

#Enforcement only works when it is cheap to obey

I keep coming back to this idea in developer tools: enforcement is only effective when the cost of compliance stays low.

If the system catches a breaking contract only after deployment, it is already too late. If it catches the issue earlier but explains it poorly, people work around it instead of trusting it. If it explains the problem clearly at change time, the right fix is usually the easiest fix.

That is the real value PolicyFabric aims for. Not just detecting breakage, but detecting it early enough and clearly enough that teams can act before the incident exists.

#What I would push next

If I kept extending this system, I would invest in richer rule authoring and deeper change previews: not just "this breaks," but "this breaks because this consumer expects this field contract, and here is the exact edge path."

I would also keep protecting the same product instinct that shaped the first version: make the architecture legible. Most systems already contain the information you need to reason about contract risk. The problem is that the information is scattered across definitions and repositories and logs.

PolicyFabric is my attempt to pull that structure into one place and make it visible enough to act on.

When a tool like this works, the best outcome is boring. The breaking change never ships, the incident never starts, and nobody has to figure out at 2 a.m. which service quietly depended on the field you just renamed.

There was no bug in the model. There was a bug in time.

#The shape of the problem

The pipeline ingests Federal Reserve communications — statements, minutes, speeches — extracts a stance signal from each, and aligns that signal against market data to see whether the language moves anything. Standard applied NLP, structurally.

The trouble is that every one of those inputs has two timestamps, and it is very easy to use the wrong one:

• Event time — when the thing happened. A meeting concluded on March 19.
• Publication time — when you could actually have known about it. Minutes from that meeting are released three weeks later.

Join on event time and you have built a machine that trades on documents from the future. It will look extraordinary.

#Where the leaks actually hide

Look-ahead bias is not one bug you fix once. It is a category, and it reappears anywhere the pipeline touches time.

Revised data. Macroeconomic series get restated after initial publication. Pull GDP for Q1 2024 today and you get the revised figure, not the one that was on the screen in Q1 2024. Your model trains on numbers nobody had.

Survivorship in the document set. Scraping the Fed's current site gives you documents that still exist. Anything moved, superseded, or quietly re-filed is gone. The archive you get is the one that survived, which is not the one a contemporaneous reader saw.

Normalization across the whole set. This is the subtle one. Fitting a scaler, computing a mean, or building a vocabulary across the entire corpus bakes global statistics into every individual sample. Standardizing sentiment scores against the full-period mean tells each training row something about the distribution of a period that had not happened yet. No single row looks wrong. The aggregate is poisoned.

Model artifacts. Embeddings from a model trained on text through 2025, applied to documents from 2021, means the representation itself encodes four years of subsequent language. The leak is inside the weights.

#The fix is architectural, not analytical

You cannot test your way out of this reliably, because a leak produces results that look better, and nobody debugs a good number. The constraint has to be structural — the pipeline should make the wrong join awkward to express.

Three things did most of the work:

1. Every record carries available_at.

Not date. Not timestamp. An explicit field meaning: the earliest wall-clock moment this row could have been known. Ingestion computes it once, at the boundary, and it is never derived later.

@dataclass(frozen=True)
class Document:
    content: str
    event_time: datetime      # when it happened
    available_at: datetime    # when it was knowable — the only one joins may use
    source: str
    revision: int = 0

Making it frozen matters more than it looks. A mutable timestamp is a timestamp somebody will quietly overwrite three transformations downstream.

2. Feature lookups take an as_of argument, and it is required.

def features_as_of(self, as_of: datetime) -> pd.DataFrame:
    """Every feature knowable at `as_of`. Nothing else, by construction."""
    visible = self._store[self._store.available_at <= as_of]
    return (
        visible
        .sort_values("available_at")
        .groupby("feature_key")
        .last()
    )

No default value. A caller who has not thought about what time it is cannot get data out. That single design choice caught more mistakes than any test I wrote.

3. Append-only storage with revisions.

Restatements insert a new row with a later available_at rather than updating the old one. The original stays queryable, because reconstructing what was known on a given date is the entire job. Updating in place destroys exactly the history you need.

#What happened to the 27%

It got much smaller, and it became real.

The signal that survived was narrower than the one I started with: not "the Fed sounded hawkish" but "the Fed sounded more hawkish than in its previous statement, and the change was concentrated in the forward-guidance paragraph." Absolute sentiment on central bank prose is almost constant — it is engineered to be. Nearly all the information is in the delta.

That is a less exciting finding. It is also one I believe, which the first number was not.

#What I would tell myself at the start

Treat any backtest result that surprises you as a bug report against your data pipeline, not as a discovery. The prior should be overwhelming: you have made a mistake with time. Financial data is adversarial toward the careless in a way that most ML data is not, because the errors flatter you.

The modeling was maybe a fifth of this project. Correct time handling was the rest, and it was where all the risk lived.

The FOMC Intelligence pipeline is research work at Stony Brook University. Reported figures describe a simulation, not a trading record.