Most interfaces are laid out once, for an average person, and then shrunk to fit a phone. The person using it is never average. They might be on a bus with one thumb free, at a desk with a screen reader, or in a hurry to find one thing. I built a lab to ask what it would take to render the interface for that person, in that moment, and to measure whether it worked.

The lab grew out of earlier work on keyboard focus order. That work took a finished page and asked one question: in what order should a keyboard user reach the controls? It treated the order as a set of continuous ranks, wrote reading order and landmark grouping as constraints, and verified the result. The page itself was taken as given. Rendering for one person removes that given, and three questions come first.

Three questions before focus order

1. What to render. Which components earn a place on this screen, for this person, within the space there is.
2. How to group the controls. Which controls belong together, so that people see them as one thing and a screen reader meets them as one unit.
3. Where to put them. A position for every component on the grid, so that the task is quick to finish and nothing important falls out of reach.

Each question has several reasonable answers, from a designer's hand-written rules to methods that learn. The lab lets you swap the algorithm at every stage and scores every combination on the same yardstick.

One yardstick for every algorithm

The context describes the person and the moment: device, input method, what they came to do, expertise, hurry, low vision, reduced motor control, and which hand holds the phone. The catalogue has 19 components, from a logo and a search box to a results list, a form, and a chat button. Each component reflows for the device, and touch or motor needs raise every interactive target to 44 pixels.

Behind the catalogue sits a simulated world. Each component has hidden weights that give the probability someone uses it in a given context, and each belongs to one of five task flows: the page shell, browsing, finding, completing a task, and getting help. No learner sees the weights. A learner sees only what a real system would see: the context, and whether each component it showed was used.

The world is a simulator, so every number on this page is a number about the simulator, not about people. Its value is that the same yardstick applies to every algorithm, so a difference in the score is a difference in what the algorithm did.

Stage	What it is scored on
What to render	The value of what was rendered as a fraction of the best possible set under the same space budget, which the lab computes exactly from the hidden weights
How to group	Agreement with the hidden task flows, as an adjusted Rand index: 1 is perfect, 0 is what chance would give
Where to put them	Exact counts on the final layout: overlaps, components off the screen, split groups, priority pairs out of order, top priorities below the fold, and actions out of thumb reach
The whole page	Simulated task time: Fitts' law for each move between the steps of the person's task, 0.2 seconds for every component scanned before the target, 0.35 seconds per Tab stop for keyboard and screen-reader users, and 2.5 seconds for each step that was not rendered

What to render: the log decides more than the model

Every selector produces a score per component and hands it to the same exact solver, which picks the best set that fits the space budget and a cap on the number of components. The only difference between selectors is where the scores come from. I compared hand-written rules, random choice, a logistic model per component, and two bandit methods that learn while they run: LinUCB, which adds a bonus for what it is unsure about, and Thompson sampling, which acts on a random draw from what it believes.

After 600 rounds of random contexts, each learner was scored on 60 new ones:

Selector	Share of the best possible value
Random	0.37
Hand-written rules	0.77
Thompson sampling	0.88
Logistic model, learning online	0.89
LinUCB	0.92

The more useful result came from training the same logistic model offline on different logs. Trained on 400 sessions of a page that used the hand-written rules, it scored 0.68, below the rules themselves at 0.77. Trained on 400 sessions where the page showed components at random, it scored 0.94. A page that never shows a component never learns what that component is worth, and a model trained on its log inherits the blind spot. If you want a learner to improve on your rules, the log it learns from has to include what the rules left out.

Grouping the controls

Two signals say which controls belong together. One is meaning: each component has a position on six axes such as navigation, search, and transaction. The other is behavior: how often two components are used in the same visit, measured from the simulated sessions. The lab blends the two, and a slider sets the mix. Four methods turn the blend into groups: grouping by landmark role, which is what the focus-order work assumed; agglomerative clustering; spectral clustering, which cuts the graph of affinities where it is weakest; and a soft assignment that a gradient can move.

Grouping method	Agreement with the hidden task flows
Landmark role	0.41
Agglomerative clustering	0.56
Spectral clustering	0.58
Soft assignment	0.51

Grouping by role, the familiar header, navigation, main, and footer, recovered the task flows worst. People use controls in flows, and a flow often crosses landmarks. The search box and the results list belong together for someone looking for something, even though one sits in the header and the other in the main region.

Grouping also matters for access. The lab requires every group to be contiguous in reading order, and checks it with the same verifier the focus-order work used. A screen-reader user meets a split group as two unrelated pieces, and so does a browser agent reading the same accessibility tree.

Placing them: the objective is not the yardstick

Three methods place the components on a 12-column grid. The flow layout is what a style sheet would give: groups in priority order, each wrapping across the width. Simulated annealing tries random moves and keeps the ones that lower the cost. The third is a placement objective written so a gradient can flow through it. It penalizes overlap, leaving the screen, high priorities below the fold, lower priorities read before higher ones, groups pulled apart, and actions out of thumb reach. Every result is then rounded to the grid, pushed apart until nothing overlaps, and checked.

Over 216 runs, covering six contexts, three selectors, and four grouping methods:

Placer	Task time, seconds	Split groups	Actions out of reach
Flow layout	3.26	0.06	0.74
Annealing	3.54	0.25	0.36
Placement objective	3.32	0.14	0.40

The flow layout was hard to beat. The optimized placement came within 2 percent of it on task time and cut the actions out of thumb reach by almost half. Annealing was 9 percent slower, because its random moves scramble priority order and the reader pays for that in scanning. The step that makes every layout valid, rounding to the grid and pushing overlaps apart, also undoes much of what the objective asked for. In a separate run, raising the weight on priority order from 0.05 to as much as 1.0 changed the mean task time by 0.02 seconds or less. Tuning the objective is not the same as improving the measure you care about, and the lab makes that gap visible.

Real time means holding still

A context does not stay put. The person gets in a hurry, changes what they came for, or puts down the laptop and picks up a phone. Each change re-runs the whole pipeline in a few hundred milliseconds. A layout that rearranges itself every time is worse than a slightly slower one that stays put, because people lose their place.

The placement objective can start from the previous frame. Components that stay on screen begin where they were and are anchored there, and new ones start from the flow layout. Over an eight-frame sequence of changing contexts, the components that stayed moved 6,550 pixels in total with the warm start and 26,789 from a cold start, about a quarter as far. On single frames the ratio ran from a twenty-fourth to six tenths. Stability has a cost. On three of the seven frames the preserved layout was slower than a fresh one, and worst when the person switched to getting help: 2.78 seconds against 1.49, because the old arrangement no longer fit the new intent.

Two more controls help. Hysteresis keeps a component on screen until its score drops clearly below the threshold, and admits a new one only when it clears the threshold by the same margin, so components do not flicker in and out. A move cap limits how far a kept component may travel in one frame. Together they cut the total movement by about a third. The cap is applied before rounding and repair, so it limits what the optimizer asks for rather than what the person sees: the largest single move still reached 226 pixels against a 120-pixel cap.

What a designer needs in order to steer it

An algorithm that decides the layout is only useful if a designer can steer it, understand it, and disagree with it. The second half of the lab is about that.

• People, not sliders. Six people with a one-line story each. One click sets the whole context, and a side-by-side view shows the same settings rendered for someone else.
• Constraints that hold. A designer can mark a component always or never shown, keep it in the top quarter or bottom third of the first screen, keep it above the fold, or drag it to pin it. Each constraint is part of the objective and is enforced exactly before the final check, and the checklist reports whether it held.
• An answer to "why is it here?" Click any component to see its score against the threshold, whether the best possible set would include it, what it is grouped with and how strongly, its reading position and Tab stop, and every penalty acting on it.
• Taste without weights. Nobody can set eight objective weights by hand. Instead, the lab shows two layouts and asks which you would ship. A Bradley-Terry model, the standard model of paired comparisons, fits your choices to seven things you can see, and the best-ranked setting can be applied in one click.
• A handoff. The layout exports as a CSS grid with one rule per component, and the whole setup exports as a code someone else can paste.

With a demo set of constraints on the desktop browse context, every constraint held under all three placers. The cost appeared elsewhere: the flow layout went from no split groups to two, and the optimized placement from none to one.

The preference fit has a limit worth knowing. In the Python run, a synthetic designer who disliked priority violations, slow task times, and tall pages made 40 choices. The fit put that designer's first two candidates first and the third fourth. Page height varied by less than a tenth of a screen across the candidates, and the fit got its sign wrong. A preference on something the choices barely exercise cannot be learned reliably, however many choices you make.

Learning the principles by playing

Each lab ends with a short card on the principle underneath it, with a small demo of the idea on its own. There is a knapsack you can drag where choosing by value per pixel loses to the exact answer, dots where position and color disagree about grouping, Fitts' law with sliders for distance and size, a noisy score crossing one threshold against two, the paired-comparison curve, and a two-by-two experiment that separates main effects from interactions.

The last lab is a game. You get the same components, the same person, and the same yardstick as the placement objective, and you drag the components onto the screen yourself. Then the lab scores you against the flow layout and the machine and tells you what you missed. My first attempt, stacking everything from the top of a phone screen, lost by 49 points. It placed every component without an overlap, but nine priority pairs read out of order and the scanning cost alone was 1.6 seconds more than the machine's.

How the lab is built

The reference implementation is a Python module that needs only numpy. The objectives are written as circuits of linear maps and the ReLU function, so every comparison, minimum, and threshold is exact and still passes a gradient; only the squared terms are smooth. The page is a single HTML file that re-implements the pipeline in JavaScript, so changing the context re-solves everything in the browser. On load, the page runs the three circuits exported from Python and checks that its own losses and gradients match them. The rendering module has 28 tests, including exact checks of the solver against brute force and numerical checks of every gradient.

What I would try next

1. Anchor relative positions when the device changes. Today the warm start keeps a component's old coordinates, so moving from a 1280-pixel desktop to a 390-pixel phone anchors components to places that no longer exist.
2. Put the rounding and repair inside the objective, or fit the weights to the task time directly, so the weights reach the final layout.
3. Fit the objective weights themselves from a designer's choices, instead of ranking a fixed set of candidates.
4. Replace the simulator with real sessions. The learners need only the context and whether each component that was shown was used.
5. Hand the grouped and placed layout to the focus-order solver and test the rendered page with a screen reader, which closes the loop the focus-order work opened.

Limits

The world is simulated, the task-time model is a simple model of pointing, scanning, and tabbing, and nothing here has been tested with people. The catalogue has 19 components and five flows, far fewer than a real site. The figures and screenshots come from the lab's build of October 18, 2026. Small differences in the yardstick, such as the 2 percent between the flow layout and the optimized placement, are within what a different simulated world could reverse. What the lab shows reliably is the mechanism: where an algorithm's choices come from, what each choice costs on a shared measure, and where the objective and the measure part ways.

Sources

• Paul M. Fitts, "The information capacity of the human motor system in controlling the amplitude of movement," Journal of Experimental Psychology 47(6), 1954
• Ralph A. Bradley and Milton E. Terry, "Rank analysis of incomplete block designs: I. The method of paired comparisons," Biometrika 39, 1952
• Lihong Li, Wei Chu, John Langford, and Robert E. Schapire, "A contextual-bandit approach to personalized news article recommendation," WWW 2010
• William R. Thompson, "On the likelihood that one unknown probability exceeds another in view of the evidence of two samples," Biometrika 25, 1933
• Andrew Y. Ng, Michael I. Jordan, and Yair Weiss, "On spectral clustering: analysis and an algorithm," NIPS 2001
• Lawrence Hubert and Phipps Arabie, "Comparing partitions," Journal of Classification 2, 1985