mikias\abera

WRITING

Build logs and failure reports

From production · receipts included

New here? Start with these three

01My research agent fabricated 4 of 5 citations, so I built a verification gateThe failure that started everything02You don't need an ML engineer. You need a product engineer who doesn't trust the model.How I work, and the question I ask every AI system03Ontario changes its laws about 500 times a year. I built the tool that reads every one.Proof it ships in a regulated domain
verification2026-07-01

Same words, opposite claim

I built a checker that verifies AI claims against the real papers they cite, then paid two frontier models to break it. They did, twice. Both breaks were the same lesson: grounding checkers fail in the direction of agreement, and the fix is to accept less, not to get smarter.

fundamentals2026-07-01

Red-teaming my own pipelines: what a regulated buyer asks first

I walked my own systems the way a hostile reviewer would, from the prompt injection surface to the PII boundaries a regulated buyer asks about first, and wrote down the findings, including the one I can't fix.

fundamentals2026-06-29

Suggest, don't act: approval queues and the table that can't forget

Why high-stakes AI should propose rather than act, and what a trustworthy approval queue needs underneath it. The build is a Postgres audit table where UPDATE and DELETE fail by trigger, walked through line by line.

fundamentals2026-06-26

Give the agent a room, not a building

Agency helps when the path can't be scripted in advance, and it stays safe when the action space is designed so the worst possible run is affordable. The tiered mutation layer and the growth agent that have been teaching me this in production.

fundamentals2026-06-24

Context is a budget, not a backpack

Context engineering: million-token windows make stuffing everything in feel free, and it isn't. Attention degrades in the middle, and tokens cost the same whether they changed the answer or not. Three context strategies for the same task, with the real cost levers, batch and cache, priced out.

evals2026-06-21

The newest model lost, and only the eval knew

I benchmarked a range of vision models on the same real task and the newest, biggest one lost to an older, cheaper one. Without a small set of real outcomes to grade against, I would've shipped the wrong choice and never known. A field note on evals and golden sets.

ai-product2026-06-18

You don't need an ML engineer. You need a product engineer who doesn't trust the model.

Most companies shipping AI features aren't doing machine learning, they're doing product engineering around a probabilistic component. Hiring as if those are the same job is why so many AI features stall before production. On what the work actually is.

fundamentals2026-06-14

Why models fabricate, mechanically

Why language models fabricate, mechanically: the distribution has to put its mass somewhere, and 'I don't know' is rarely the most probable continuation. A temperature demo you can drag, what the research says about incentives, and the autopsy of my own four fabricated citations.

verification2026-06-14

The average answer is a trap

In a regulated domain the generic rule is usually close and occasionally, expensively, wrong. A system that answers from the average case is a confident-wrong machine. On why the unit of correctness is the specific instance, not the category.

fundamentals2026-06-10

Structured extraction: the validator is the product

Why output validation belongs outside the model, and what schema-constrained extraction looks like when the input is thousands of messy PDFs. The schema is a contract, the validator enforces it, and fields the model can't fill stay empty instead of guessed.

verification2026-06-07

The most useful thing my tool does is refuse to answer

A readiness checker that confidently flags the wrong thing is worse than no tool at all. The feature that makes mine trustworthy is the one that says 'I can't confirm this, check it yourself.' On designing abstention in from the start.

fundamentals2026-06-06

The queries vector search can't see

Identifier queries like 'O. Reg. 176/26' are where pure vector search goes blind, because rare exact strings have no semantic neighborhood. A live BM25 demo with per-term scores, why my pipeline keeps a full-text index next to its vector index, and where a reranker would earn its cost.

product2026-06-03

Stop predicting, start verifying

PermitCheck started as a tool to predict whether a permit would be approved. That promise was unkeepable, and no amount of data could fix it, because the outcome didn't live in the document. The fix was to change the question. On the difference between a prediction you can't keep and a verification you can.

fundamentals2026-06-02

Chunking decides what your AI can know

Chunk boundaries dominate retrieval quality more than model choice. A live visualizer shows a height limit getting separated from the exception that overrides it, and what that failure looks like in legal text.

verification2026-05-31

Ontario changes its laws about 500 times a year. I built the tool that reads every one.

Enacted tracks all 3,055 Ontario regulations and statutes, computes the exact diff when one changes, and explains it in plain English. The interesting part is what the AI isn't allowed to do.

fundamentals2026-05-29

What cosine similarity actually measures

Cosine similarity is an angle between two points in a space built to capture relatedness, and relatedness is not assertion. A draggable demo of the geometry, the real pgvector setup behind my research pipeline, and the case where I chose no embeddings at all.

verification2026-05-28

Never let the LLM author the numbers

A hallucinated macro total is a lie someone eats. Every number in my systems comes from plain code, and the model only chooses between validated options.

fundamentals2026-05-25

Where the arithmetic actually lives

Week 2 of going deeper on production AI fundamentals: which numbers the model should never author. The primary sources say models predict tokens, not sums, and the field's own answer is to hand the math to a tool. A walk through the four-layer resolver behind my meal engine, and why the LLM sits at the bottom of it, flagged.

pipelines2026-05-24

Extracting structured claims from 11,000 papers on a desk, not a datacenter

The architecture of a research extraction pipeline that has triaged 90,111 papers across its two big corpora and fully extracted 11,490 of them: acquisition, schema-constrained extraction, entity resolution, and the optimizations that mattered more than model choice.

verification2026-05-21

My research agent fabricated 4 of 5 citations, so I built a verification gate

I spot-checked the citations my extraction pipeline produced and most of them were wrong. Here's the gate I built so it can't happen again, and why I now wrap deterministic checks around every probabilistic system I ship.