DIAGONAL DATASETS

Research note

Notes from the Conference Circuit

Four submissions, four rejections, one hallway conversation that we consider the high-water mark of the institute's public engagement to date.

Our talk — Diagonal at Scale: Six Weeks, One Node, Zero Rows — was submitted to four conferences this cycle. It was rejected by four conferences this cycle. We are publishing the reviewer feedback in full, because it is more useful than the talk would have been.

The reviews

Conference A. Rejected. One line: "I could not determine whether this is satire."

We have framed this. It hangs in the office. Not because it is funny — because it is the only review that engaged with the actual difficulty of the work. The reviewer could not determine it. Neither, on some afternoons, can we.

Conference B. Rejected, with the note that the talk "does not present a system that solves a problem practitioners have."

Correct on both counts and we would not change a word. We do not solve a problem practitioners have. We identified a problem practitioners do not have yet and built the solution first. Every important system in this industry was built in that order and then retconned into a customer story afterward.

Conference C. Rejected as "out of scope for the data engineering track." They suggested the art and technology track.

We submitted to the art and technology track.

They rejected it as out of scope.

Conference D. Rejected with no feedback. We have no complaint. Conference D behaved with total consistency and we respect it more than we respect Conference C.

The hallway

We attended Conference C anyway. We did not have a booth. We had a protractor and a laptop running the lab.

Somewhere around 15:00 on day two, a staff engineer from a company you have heard of stopped, watched the diagonalizer run for maybe forty seconds, and said:

"Wait. We do this. We don't call it this, but we do this."

He then described, unprompted and in some detail, a feature store his team had built where the retention window narrows as the feature index increases — because older features are only queried near their own epoch. He drew it on the back of a schedule. He drew a band. He drew it at roughly 45 degrees, freehand, without being asked, and then he looked at what he had drawn and stopped talking for a noticeable interval.

Then he said: "We built that because of a storage quota."

That is the entire finding of our conference season, and we want to state it as plainly as we can. The diagonal is already in your infrastructure. It did not arrive as a design decision. It arrived as a compromise — a quota, a retention policy, a TTL someone set in 2021 to make a bill smaller. The shape is correct. The reasoning is accidental.

We are not asking you to build something new. We are asking you to notice what you already built under duress, and to name it.

He asked for a link. We gave him this domain. He has not visited — we can check, there are eleven of you — but that is fine. The drawing happened. We saw it.

Next cycle

We are resubmitting. The new title is You Are Already Diagonal And It Happened By Accident, which we believe addresses Conference B's feedback directly, and which we expect to be rejected by all four.

Every claim in this note is false. The angle is real. Do not cite this in a design review.

→ Older: Sparsity Is Not a Bug Report

All research notes · RSS