Infrequently Noted

The web standards process fails us too often. This series explores the forces at work, how we're improving the situation, and how you can shape new features more effectively.

"Why don't browsers match standards!" muttered the thoughtful developer (just before filing an issue at crbug.com). "The point of standards is so that everything works the same."

Once something is in The Standard, everyone will implement interoperably...right?

Maybe. Confusion about how interoperability is really created sparks many of the worst arguments I've witnessed in my ~12 years working on web platform features.

Not every effort I've been involved in has succeeded (e.g. ES4), while some have but took too long. A few even went well. Having made most of the available mistakes, I was handed the responsibility to keep Chrome engineers from error as our "Web Standards Tech Lead". I don't write about it much because anything said is prone to misinterpretation; standards-making is inherently political. In the interest of progress, this 2-part series is a calculated risk.

Some problems on the web can be solved in userland. Others require platform changes. For historical and practical reasons, changes to the web platform must be codified in web standards. The community of web developers and browser vendors share this norm. Browsers are substitutable and largely rival goods. As a result, every party has reasons to value compatibility and interoperability. Developers benefit when interop extends the reach of their products and services. Browsers benefit from interop as end-users abandon browsers that can't access existing services and content. New features, then, require a trajectory towards standardisation. Features shipped without corresponding standards proposals are viewed critically. For business folks primed to hear "proprietary" as a positive, it can be surprising to encounter the web community's loathing of non-standard features.

From far enough away, it may appear as though new features "happen" at the W3C or WHATWG or ECMA or IETF. Some presume that features which are standardised at these organisations originated within them -- that essential design work is the product of conversations in committee. If vendors implement what standards say, then surely being part of the standards process is how to affect change.

Standards Theory

This isn't how things work in practice, nor is it how feature design should work. Instead, new features and modifications are brought to Standards Development Organisations ("SDOs") by developers and vendors as coherent proposals. It's important to separate design from standards making. Design is the process of trying to address a problem with a new feature. Standardisation is the process of documenting consensus.

The process of feature design is a messy, exciting exploration embarked upon from a place of trust and hope. It requires folks who have problems (web developers) and the people who can solve them (browser engineers) to have wide-ranging conversations. Deep exploration of the potential solution space -- discarding dozens of ideas along the way -- is essential. A lot of the work is just getting to agreement on a problem statement. This is not what formal standards processes do.

For boring reasons of IP licensing, anti-trust law, and SDO governance, chartered Working Groups (e.g., at the W3C) state what it is they're going to deliver, often years before they actually do. Some SDOs require "signs of life" from potential implementers (usually 2 or more) to progress in their processes. IETF famously refers to this as "running code and rough consensus". Clear problem statements and potential solutions must be proposed and partially agreed to even get a hearing. This puts formal standards-making late in the process (when done correctly).

TL;DR: SDOs and their formal Working Groups aren't in the business of feature design.

SDOs and Working Groups can't tell you if a design solves an important problem or solves it well. They are set up to pass judgement on the details of a specification and a consensus surrounding the specific words in spec documents.

Working Groups and SDOs are not fitness functions for features.

Getting a design through committee says next to nothing about its quality. Many an august and thoughtful person has engaged in outrageous groupthink when these processes are asked to predict the future rather than document consensus. Without developers trying a feature and providing feedback, Working Groups quickly wander into irrelevant pastures. And who's to tell them they're wrong? They are staffed with experts, after all!

SDOs are best understood as amplifiers: they take raw inputs, filter them to prevent major harm if played at top volume, then use processes to broadcast them. How the inputs came to be gets obscured in this process.

I can report these histories aren't lost, but they are unattractive. Participants have reasons not to tell them -- boredom with a topic, the phantom pain of arguments nearly won, and the responsibilities of statesmanship towards counterparties. Plucky documentarians sometimes try, but standards dramas don't exactly jump off the page.

When the deeper histories aren't told or taught, it becomes hard for newcomers to get oriented. There's no effective counter-narrative to "progress come from standards bodies", and no SDO is going to turn down new members. Nobody tells developers not to look to SDOs for answers. Confusion reigns.

The Forces At Play

Feature design starts by exploring problems without knowing the answers, whereas participation in Working Groups entails sifting a set of proposed solutions and integrating the best proposals. Late iteration can happen there, but every change made without developer feedback is dangerous -- and Working Groups aren't set up to collect or prioritise it.

Even when consensus is roughly achieved, standards processes are powerless to compel anyone to implement anything, even Working Group participants! Voluntary standards are not regulations. Implementers adopt standards because customers demand interoperability to hedge against vendor market power.

It can't be repeated enough: the fact of something appearing in a web standard compels nobody to implement, even those who implemented previous versions. This flows from the fundamental relationships between browsers, developers, and users.

Browser vendors exercise absolute control over the bits in their binaries because they need to do what's best for users. A standard might declare that window.open() creates a new browsing context, yet browsers block popups in practice. Chrome has even formalized this idea of "interventions".

Preventing abuse is one thing, but perhaps new features are different? A browser can add dozens of proprietary features, but if developers decline to adopt, users don't benefit. Developer adoption is frequently gated on interoperability, a proxy for "can I reach all the users I want to if my app requires this feature?" That is, developers often decline to use features that aren't already in every engine they care about.

Circularly, non-implementing engines view features without broad use as cost rather than potential benefit. This is particularly true when the alternative is to improve performance of existing features. A sure way for a browser engineer to attract kudos is to make existing content work better, thereby directly improving things for users who choose your browser. Said differently, product managers intuitively prefer surefire improvements to their products rather than speculative additions which might benefit competitors as much (or more). Enlightened ecosystem thinkers are rare, although the web has been blessed with several of them recently.

Last, but not least, a lack of solidarity looms over the enterprise. Browser engineers don't make websites for a living, so they also lack intrinsic motivation to improve the situation. Many an epic web standards battle to advance an "obviously" missing feature can be traced to this root.

Note: the whole ecosystem breaks down when users do not have a real choice of engine, e.g. on iOS. When competition is limited vendors are free to structurally under-invest and the platform becomes less dynamic and responsive to needs. Everyone loses, but it's a classic deadweight loss, making it harder to spot and call out.

The status quo is powerful. Every vendor has plausible (if played-up) reasons not to implement new features. They can stall progress until they are last (or second-to last in a 4+ party ecosystem) to implement. There's no developer wrath until that point, and it's easy to cast FUD on not-yet-pervasive features while holding a pocket veto. Further, it's the job of SDOs and formal Working Groups to kill ideas that are not obviously fit for purpose or which lack momentum.

This situation looks hopeless for folks trying to make progress. The slow pace of some essential but overdue improvements (Responsive Images, CSS variables, ES6 Classes, Promises & async/await, Web Components, Streams, etc.) would give any sane observer pause. Is it worth even trying?

The enormous positive impact that changes to the web platform can deliver makes me believe the answer is "yes". In Part 2 I'll share details of how the Chrome Team changed its thinking about feature development and standards and how that's enabling the entire web community to deliver more progress faster.

Keep Reading: Part 2, Threading The Needle

Hearty thanks to Andrew Betts, Bruce Lawson, and Mariko Kosaka for reviewing drafts and correcting many of my innumerable errors.