I really vehemently disagree with the 'feedforward, tolerance, feedback' pattern.
Protocols and standards like HTML built around "be liberal with what you accept" have turned out to be a real nightmare. Best-guessing the intent of your caller is a path to subtle bugs and behavior that's difficult to reason about.
If the LLM isn't doing a good job calling your api, then make the LLM get smarter or rebuild the api, don't make the API looser.
I'm not sure it's possible to have a technology that's user-facing with multiple competing implementations, and not also, in some way, "liberal in what it accepts".
Back when XHTML was somewhat hype and there were sites which actually used it, I recall being met with a big fat "XML parse error" page on occasion. If XHTML really took off (as in a significant majority of web pages were XHTML), those XML parse error pages would become way more common, simply because developers sometimes write bugs and many websites are server-generated with dynamic content. I'm 100% convinced that some browser would decide to implement special rules in their XML parser to try to recover from errors. And then, that browser would have a significant advantage in the market; users would start to notice, "sites which give me an XML Parse Error in Firefox work well in Chrome, so I'll switch to Chrome". And there you have the exact same problem as HTML, even though the standard itself is strict.
The magical thing of HTML is that they managed to make a standard, HTML 5, which incorporates most of the special case rules as implemented by browsers. As such, all browsers would be lenient, but they'd all be lenient in the same way. A strict standard which mandates e.g "the document MUST be valid XML" results in implementations which are lenient, but they're lenient in different ways.
HTML should arguably have been specified to be lenient from the start. Making a lenient standard from scratch is probably easier than trying to standardize commonalities between many differently-lenient implementations of a strict standard like what HTML had to do.
Are you aware of HTML 5? Fun fact about it: there's zero leniency in it. Instead, it specifies a precise semantics (in terms of parse tree) for every byte sequence. Your parser either produces correct output or is wrong. This is the logical end point of being lenient in what you accept - eventually you just standardize everything so there is no room for an implementation to differ on.
The only difference between that and not being lenient in the first place is a whole lot more complex logic in the specification.
> Are you aware of HTML 5? Fun fact about it: there's zero leniency in it.
I think you understand what I mean. Every byte sequence has a
> The only difference between that and not being lenient in the first place is a whole lot more complex logic in the specification.
Not being lenient is how HTML started out.
History has gone the way it went & we have HTML now, there's not much point harking back, but I still find it very odd that people today - with the wisdom of foresight - believe that the world opting for HTML & abandoning XHTML was the sensible choice. It seems odd to me that it's not seen as one of those "worse winning out" stories in the history of technology, like betamax.
The main argument about XHTML not being "lenient" always centred around client UX of error display - Chrome even went on to actually implement a user-friendly partial-parse/partial-render handling of XHTML files that literally solved everyone's complaints via UI design without any spec changes but by this stage it was already too late.
The whole story of why we went with HTML is somewhat hilarious: 1 guy wrote an ill informed blog post bitching about XHTML, generated a lot of hype, made zero concrete proposals to solve its problems, & then somehow convinced major browser makers (his current & former employers) to form an undemocratic rival group to the W3C, in which he was appointed dictator. An absolutely bizarre story for the ages, I do wish it was documented better but alas most of the resources around it were random dev blogs that link rotted.
>The whole story of
Is that really the story? I think it was more like "backward compatible solution soon about more pure, theoretically better solution"
There's enormous non-xhtml legacy than nobody wanted to port. And tooling back in the day didn't make it easy to write correct xhtml.
Also like it or not, HTML is still written by humans sometimes, and they don't like parser blowing up because of a minor problem. Especially since such problems are often detected late, and a page which displays slightly wrong is much better outcome than the page blowing up.
More or less? https://en.m.wikipedia.org/wiki/WHATWG is fairly neutral. As someone in userland at the time on the other side of it, it was all a bit nuts.
IE we got new standards invented out of thin air - https://developer.mozilla.org/en-US/docs/Web/HTML/Guides/Mic... - which ignored what hundreds had worked on before, which seemed to be driven by one person controlling the "standard" making it up as they went along.
Microformats and RDFa were the more widely adopted solutions at the time, had a lot of design and thought put into them, worked with HTML4 (but thrived if used with xhtml), etc etc.
JSON-LD/schema.org has now filled the niche and arguably it's a lot better for devs, but imagine how much better the "AI web UX" would be now if we'd just standardised earlier on one and stuck with it for those years?
This is the main area where I saw the behaviour on display, where I interacted most. So the original comment feels absolutely in line with my recollections.
I love bits of HTML5, but the way it congealed into reality isn't one of them.
> There's enormous non-xhtml legacy than nobody wanted to port.
This is a fair argument if content types were being enforced but XML parsing was opt-in (for precisely this reason).
> And tooling back in the day didn't make it easy to write correct xhtml.
True. And instead of developing such tooling, we decided to boil the ocean to get to the point where tooling today doesn't make it any easier to lint / verify / validate your HTML. Mainly because writing such tooling to a non-strict target like HTML is a million times harder than to a target with strict syntax.
A nice ideal would've been IDEs & CI with strict XHTML parsers & clients with fallbacks (e.g. what Chromium eventually implemented)
> If XHTML really took off (as in a significant majority of web pages were XHTML), those XML parse error pages would become way more common
This is not true because you are imagining a world with strict parsing but where people are still acting as though they have lax parsing. In reality, strict parsing changes the incentives and thus people’s behaviour.
This is really easy to demonstrate: we already have a world with strict parsing for everything else. If you make syntax error with JSON, it stops dead. How often is it that you run into a website that fails to load because there is a syntax error in JSON? It’s super rare, right? Why is that? It’s because syntax errors are fatal errors. This means that when developing the site, if the developer makes a syntax error in JSON, they are confronted with it immediately. It won’t even load in their development environment. They can’t run the code and the new change can’t be worked on until the syntax error is resolved, so they do that.
In your hypothetical world, they are making that syntax error… and just deploying it anyway. This makes no sense. You changed the initial condition, but you failed to account for everything that changes downstream of that. If syntax errors are fatal errors, you would expect to see far, far fewer syntax errors because it would be way more difficult for a bug like that to be put into production.
We have strict syntax almost everywhere. How often do you see a Python syntax error in the backend code? How often do you run across an SVG that fails to load because of a syntax error? HTML is the odd one out here, and it’s very clear that Postel was wrong:
> This is not true because you are imagining a world with strict parsing but where people are still acting as though they have lax parsing. In reality, strict parsing changes the incentives and thus people’s behaviour.
Dude I lived in that world. A fair amount of developers explicitly opted into strict parsing rules by choosing to serve XHTML. And yet, those developers who opted into strict parsing messed up their XML generation frequently enough that I, as an end user, was presented with that "XML Parse Error" page on occasion. I don't understand why you'd think all developers would stop messing up if only strict parsing was hoisted upon everyone rather than only those who explicitly opt in.
> In your hypothetical world, they are making that syntax error… and just deploying it anyway.
No, they're not. In my (non-hypothetical, actually experienced in real life) world of somewhat wide-spread XHTML, I'm assuming that developers would make sites which appeared to work with their test content, but would produce invalid XML in certain situations with some combination of dynamic content or other conditions. Forgetting to escape user content is the obvious case, but there are many ways to screw up HTML/XHTML generation in ways which appear to work during testing.
> We have strict syntax almost everywhere. How often do you see a Python syntax error in the backend code?
Never, but people don't dynamically generate their Python back-end code based on user content.
> How often do you run across an SVG that fails to load because of a syntax error?
Never, but people don't typically dynamically generate their SVGs based on user content. Almost all SVGs out there are served as static assets.
> Dude I lived in that world. A fair amount of developers explicitly opted into strict parsing rules by choosing to serve XHTML.
No they didn’t, unless you and I have wildly different definitions of “a fair amount”. The developers who did that were an extreme minority because Internet Explorer, which had >90% market share, didn’t support application/xhtml+xml. It was a curiosity, not something people actually did in non-negligible numbers.
And you’re repeating the mistake I explicitly called out. Opting into XHTML parsing does not transport you to a world in which the rest of the world is acting as if you are in a strict parsing world. If you are writing, say, PHP, then that language was still designed for a world with lax HTML parsing no matter how you serve your XHTML. There is far more to the world than just your code and the browser. A world designed for lax parsing is going to be very different to a world designed for strict parsing up and down the stack, not just your code and the browser.
> I'm assuming that developers would make sites which appeared to work with their test content, but would produce invalid XML in certain situations with some combination of dynamic content or other conditions. Forgetting to escape user content is the obvious case, but there are many ways to screw up HTML/XHTML generation in ways which appear to work during testing.
Again, you are still making the same mistake of forgetting to consider the second-order effects.
In a world where parsing is strict, a toolchain that produces malformed syntax has a show-stopping bug and would not be considered reliable enough to use. The only reason those kinds of bugs are tolerated is because parsing is lax. Where is all the JSON-generating code that fails to escape values properly? It is super rare because those kinds of problems aren’t tolerated because JSON has strict parsing.
> No they didn’t, unless you and I have wildly different definitions of “a fair amount”. The developers who did that were an extreme minority because Internet Explorer, which had >90% market share, didn’t support application/xhtml+xml. It was a curiosity, not something people actually did in non-negligible numbers.
Despite being an extreme minority of strict parsing enthusiasts who decided to explicitly opt into strict parsing, they still messed up enough for me to occasionally have encountered "XML Parse Error" pages. You'd think that if anyone managed to correctly generate strict XHTML, it'd be those people.
> You'd think that if anyone managed to correctly generate strict XHTML, it'd be those people.
Once more, they were operating in a world designed for lax parsing. Even if their direct choices were for strict parsing, everything surrounding them was lax.
Somebody making the choice to enable strict parsing in a world designed for lax parsing is a fundamentally different scenario than “If XHTML really took off (as in a significant majority of web pages were XHTML)”, where the entire technology stack from top to bottom would be built assuming strict parsing.
> Never, but people don't dynamically generate their Python back-end code based on user content.
Perhaps not much in the past - but I suspect with Agentic systems a lot more in the future - are you suggesting relaxing the Python syntax to make it easier for auto-generated code to 'run'?
There's a difference between static chatbot-generated code that you commit to your VCS, and dynamic code generated based on user content. I'm not talking about the former case.
> I'm not talking about chatbot generated code that's committed to your VCS.
I'm talking about code that's dynamically generated in response to user input in order to perform the task the user specified. This is happening today in agentic systems.
Should we relax python syntax checking to work around the poorly generated code?
> I recall being met with a big fat "XML parse error" page on occasion. If XHTML really took off (as in a significant majority of web pages were XHTML), those XML parse error pages would become way more common
Except JSX is being used now all over the place and JSX is basically the return of XHTML! JSX is an XML schema with inline JavaScript.
The difference now days is all in the tooling. It is either precompiled (so the devs see the error) or generated on the backend by a proper library and not someone YOLOing PHP to super glue strings together, as per how dynamic pages were generated in the glory days of XHTML.
We basically got full circle back to XHTML, but with a lot more complications and a worse user experience!
Speaking of PHP, XHP predates JSX I believe:
https://en.wikipedia.org/wiki/XHP
Same idea: an XML-like syntax for creating object trees typically used to model HTML.
Nobody is generating JSX dynamically.
Not directly as strings of course, but a for loop that outputs a bunch of JSX components based on the array return values from a DB fetch is dynamically generated JSX.
No, it's not. The JSX, as in the text in the source file, is static. You can't accidentally forget to escape a string from the database and therefore end up with invalid JSX syntax, like you can when dynamically generating HTML. You're dynamically generating shadow DOM nodes, but the JSX is static.
You are saying the exact same thing I am, just with different words.
JSX makes it impossible to crap out invalid HTML because it is a library + toolchain (+ entire ecosystem) that keeps it from happening. JSX is always checked for validity before it gets close to the user, so the most irritating failure case of XHTML just doesn't happen.
XHTML never had they benefit. My point is that if libraries like React or Vue had been popular at the time of XHTML, then XHTML's strictness wouldn't have been an issue because JSX always generated valid outputs (well ignoring compiler bugs which I far too damn many of early on in React's life)
I remember "HTML 5 W3C Valid" buttons, proudly displayed by web pages. This was considered cool, so why doing the same for XHTML wouldn't be same?
lol CVE-2020-26870
> Protocols and standards like HTML built around "be liberal with what you accept" have turned out to be a real nightmare.
This feels a bit like the setup to the “But you have heard of me” joke in Pirates of the Caribbean [2003].
Or "There are only two kinds of languages: the ones people complain about and the ones nobody uses."
Sure, this holds for any government “current state” but it shouldn’t be used as an excuse to _keep doing dumb things_.
oh yes so true, but I would generalize it to "to flexible"
- content type sniffing spawned a whole class of attacks, and should have been unnecessary
- a ton of historic security issues where related to html parsing being too flexible, or some JS parts being to flexible (e.g. Array prototype override)
- or login flows being too flexible creating a easy to overlook way to bypass (part of) login checks
- or look at the mess OAuth2/OIDC had been for years because they insisted to over-enginer it and how especially it being liberal about quite many parts lead to more then one or two big security incidents
- (more then strictly needed) cipher flexibility is by now widely accepted to have been an anti pattern
- or how so much theoretically okay but "old" security tech is such a pain to use because it was made to be supper tolerant to everything, like every use case imaginable, every combination of parameters, every kind of partial uninterpretable parts (I'm looking at you ASN.1, X509 certs and many old CA software, theoretically really not bad designed, practically such a pain).
And sure you also can be too strict, high cipher flexibility being an anti-pattern was incorporated into TLS 1.3. But TLS still needs some cipher flexibility, so they fund a compromise of (oversimplified) you can choose 1 of 5 cipher suites but can't change any parameter of that suites.
Just today I read an article (at work, I don't have the link at hand) about some so hypothetical but practically probably doable (with a bunch of more work) scenarios to trick very flexible multi step agents into leaking your secrets. The core approach was that they found a way to have a relative small snippet of text which if it end up in the context has a high chance to basically override the whole context with just your instruction (quite a bit oversimplified). In turn if you can sneak it into someones queries (e.g. you GTP model is allowed to read you mails and it's in a mail send to you) you can then trick the multi step model to grab a secret from your computer (because the agents often run with user permissions) and send it to you (by e.g. instrumenting the agent to scan a website under an url which happens to now contain the secret).
Its a bit hypothetical, its hard to pull of, but it's very well in the realm of possibility due to how content and instructions are on a very fundamental level not cleanly separated (I mean AI vendors do try, but so far that never worked reliable it's in the end all the same input).
HTML being lenient is what made progressive enhancement possible -- right down the original <img> tag. The web would not have existed at all if HTML was strict right from the start.
> progressive enhancement possible
no not at all extensible isn't the same as lenient
having a Content-Type header where you can put in new media types (e.g. for images) once browsers support it is extensibility
sniffing the media type instead of strictly relying on the Content-Type header is leniency and had been the source of a lot of security vulnerabilities over the years
or having new top level JS object exposing new APIs is extensibility but allowing overriding the prototypes of fundamental JS objects (i.e. Array.prototype) turned out to be a terrible idea associated with multiple security issues (like idk. ~10 years ago, hence why it now is read only)
same for SAML, its use of XML made it extensible, but they way it leniently encoded XML for signing happened to be a security nightmare
or OAuth2 which is very extensible, but it being too lenient in what you can combine how was the source of many early security incidents and is still source of incompatibilities today (but OAuth2 is anyway a mess)
> no not at all extensible isn't the same as lenient
I never said it was. But lenient provides for extensibility that isn't planned for. The entire evolution of the web is based on that. Standards that were too strict or too inflexible have been long forgotten by history.
That's not to say that isn't the source of security vulnerabilities and bugs but that doesn't negate the point.
that is just not true
web is mostly based on standards which always have been supper lenient, messy und had massive gaps of unclearity mainly because it was more "lets somehow standarize what browsers already do" then anything else
through I guess you could say that the degree to which you can polyfill JS is more lenient then many thing is good and that did help with extensibility,
You say that's not true but then you don't contradict what I'm saying. How were browsers able to do anything, while maintaining backwards compatibility, that needed standardizing later?
That's poor reasoning. The web now counts as strict but still extensible: you just have to clearly define how to handle unknown input. The web treats all unknowns as a div.
> you just have to clearly define how to handle unknown input.
That is being lenient. Allowing any unknown input is being lenient in what you accept. Not allowing unknown input at all is being strict.
This is already true for just UI vs. API. It’s incredible that we weren’t willing to put the effort into building good APIs, documentation, and code for our fellow programmers, but we are willing to do it for AI.
I think this can kinda be explained by the fact that agentic AI more or less has to be given documentation in order to be useful, whereas other humans working with you can just talk to you if they need something. There's a lack of incentive in the human direction (and in a business setting that means priority goes to other stuff, unfortunately).
In theory AI can talk to you too but with current interfaces that's quite painful (and LLMs are notoriously bad at admitting they need help).
> agentic AI more or less has to be given documentation in order to be useful, whereas other humans working with you can just talk to you if they need something. ... In theory AI can talk to you too but with current interfaces that's quite painful (and LLMs are notoriously bad at admitting they need help).
Another framing: documentation is talking to the AI, in a world where AI agents won't "admit they need help" but will read documentation. After all, they process documentation fundamentally the same way they process the user's request.
I also think it makes a difference that an AI agent can read the docs very quickly, and don't typically care about formatting and other presentation-level things that humans have to care about, whereas a human isn't going to read it all, and may read very little of it. I've been at places where we invested substantial time documenting things, only to have it be glanced at maybe a couple of times before becoming outdated.
The idea of writing docs for AI (but not humans) does feel a little reflexively gross, but as Spock would say, it does seem logical
The feedback loop from potential developer users of your API is excruciatingly slow and typically not a process that an API developer would want to engage in. Recruit a bunch of developers to read the docs and try it out? See how they used it after days/weeks? Ask them what they had trouble with? Organize a hackathon? Yuck. AI, on the other hand, gives you immediate feedback as to the usability of your “UAI”. It makes something, in under a minute, and you can see what mistakes it made. After you make improvements to the docs or API itself, you can effectively wipe its memory by cleaning out the context, and see if what you did helped. It’s the difference between debugging a punchcard based computing system and one that has a fully featured repl.
Yeah, this is so true. Well designed APIs are also already almost good enough for AI. There really was always a ton of value in good API design before LLMs. Yet a lot of people still said, for varying reasons, let's just ship slop and focus elsewhere.
We are only willing to have the Llm generate it for AI. Don’t worry people are writing and editing less.
And all those tenets of building good APIs, documentation, and code are opposite the incentive of building enshittified APIs, documentation, and code.
We recently released isagent.dev [1] exactly for this reason!
Internally at Stytch three sets of folks had been working on similar paths here, e.g. device auth for agents, serving a different documentation experience to agents vs human developers etc and we realized it all comes down to a brand new class of users on your properties: agents.
IsAgent was born because we wanted a quick and easy way to identify whether a user agent on your website was an agent (user permissioned agent, not a "bot" or crawler) or a human, and then give you a super clean <IsAgent /> and <IsHuman /> component to use.
Super early days on it, happy to hear others are thinking about the same problem/opportunity.
[1] GitHub here: http://github.com/stytchauth/is-agent
I feel OP is addressing the complementary, opposite use case in which behavior is to be unified across user agents.
> Mostly I’ve just been stressing to my team that we need to consider these interfaces for our application as equal surfaces for how our application will be used. As we build features, we need to make conscious decisions about whether and how those features are accessible and legible in all three of those interfaces.
I read this to mean that they're thinking about all three experiences (interfaces) on equal footing, not as a unified one experience for all three user agents.
if you want your app to be automated wouldn't you just publish your api and make that readily available? I understand the need for agentic UI navigation but obviously an api is still easier and less intensive right. The problem is that it isn't always available, and there ui agents can circumvent that. But you want to embrace the automation of your app so.. just work on your API? You can put an invisible node in your UI to tell agents to stop wasting compute and use the api.
This is true but also your API needs to actually implement all the use cases (often its only for a subset) and it needs to work well (often there are many nuances or inconsistencies). But I agree there are lots of overlap. No need to completwly reinvent the wheel here. Actually CQRS systems work incredibly well with LLMs already.
So, this gets to a fundamental or "death of the author" ie philosophical difference in how we define what an API is "for." Do I as its publisher have final say, to the extent of forbidding mechanically permissible uses? Or may I as the audience, whom the publisher exists to serve, exercise the machine to its not intentionally destructive limit, trusting its maker to prevent normal operation causing (even economic) harm?
The answer of course depends on the context and the circumstance, admitting no general answer for every case though the cognitively self-impoverishing will as ever seek to show otherwise. What is undeniable is that if you didn't specify your reservations API to reject impermissible or blackout dates, sooner or later whether via AI or otherwise you will certainly come to regret that. (Date pickers, after all, being famously among the least bug-prone of UI components...)
Insightful!!
All you need is AHP: https://ahp.nuts.services
Separating presentation layer from business logic has always been a best practice
https://mcpui.dev/ is worth checking out, really nice project; get the tools to bring dynamic ui to the agents.