I find automated tests are the only way to keep vibe coded projects on the rails, especially as you grow something beyond demo phase.

I find this impressive: in my experience, codex-rs loves to add tests even when not prompted. Of course, it’s a bit of a crap shoot as to whether the test tests useful behavior.

(My favorite so far: it created an empty file in /home/whatever and added a test to verify that some code it wrote would indeed fail when tested on this empty input and that it would fail with the correct error message. Never mind that this covered approximately none of the desired behavior and that the test would, of course, fail on any other system.

That would be really interesting. I doubt it's the case, actually probably the opposite? The harnesses seem very happy to write extensive test suits, without me having to ask much.

Tests is the new gold. You keep them to avoid a vibe coded fork.

And what would they test? This is a meaningless wrapper for Anthropic or OpenAI SDKs.

I use a custom framework for creating basic but useful tools that work with sensitive data. There are cases in my organization where I like the idea of people using Claude or similar to assist with a process, but Claude Desktop or Claude Code doesn't offer the safety or security we need (in part because the people using it are unconstrained, in part because the harnesses aren't perfect and the LLMs can make bad choices).

This provides a harness that's a state machine with very explicit directives, and it uses Deno as the runtime to constrain network, filesystem, environment, and other types of access at runtime as needed.

Kind of like using skills in Claude Code to teach it how to do something, but with extremely tight guard rails. Like, you can only write a specific file when in a specific state, otherwise that tool isn't even callable.

It requires understanding the problem that's being solved quite well. This often leads to realizing it can be automated without a harness. Finding cases where an LLM is genuinely crucial to enabling the automation is difficult.

A good example of one recently was getting a local LLM to define schemas for an internal tool based on existing research data. It looks at the data, figures out the semantics of the data, relationships, and how that maps to the target schema. This is impossible to automate without this semantic inference. It then uses duckdb to perform transformations from raw data to the appropriate schema, and finally, tests the schema in the validator with the data. It makes a very complex, often unappealing and confusing process very easy. Once it's done, the data is in better shape than we ever got it to by hand. This is partially because of a validator I created, but also because the LLM can identify patterns really well and retain a massive spec while it works.

You could do it with all kinds of existing harnesses but this one lets us comfortably define processes we trust and lets us operate on data our partners would never allow into the cloud or on OpenAI/Anthropic's servers in particular.

> I'm looking for examples that are real businesses, not toys.

These tools are used within a real business (specifically a coastal science NGO) and they aren't toys, so hopefully that's useful information. Based on my experience so far, and it could be my lack of imagination, I have no idea how you'd use these as the foundation for a business. I find more cases that can be automated without an LLM than I do with one, and they tend to be so niche and strange that no one else would ever need them and they can't be generalized.

We're building https://brooked.io/. In the same way that Cursor provides a lot of features on top of the base agents, we want to do the same for spreadsheets. There are many workflows that benefit from having an agent available - resolving cell values from a prompt, writing functions, sheet insights, alerting, debugging.

Yeah don’t understand. It’s a lot easier to just build your own.

It's the exact same problem solved by any library whatsoever

If you (or your agent) have to write less code, there's less room to write bugs. There will be less code to understand when it needs to be modified too.

I think the TypeScript ecosystem is more suitable for this.

I do not think Rust is a bad language. But the agent ecosystem changes very quickly, and in Rust, assembling and reshaping agent workflows is difficult.

Many people prefer Rust, and I understand why. It is a genuinely excellent language, and “Rust is a great language” is a strong message that attracts many developers. But as long as lifetimes exist, I think it will remain difficult.

The lifetime system assumes, in some sense, that humans can fully predict the lifecycle of values and resources. I am not sure that is truly possible in all domains. I am also not sure whether that model is linguistically suitable for the agent ecosystem.

In agent systems, requirements change constantly. Tools change, workflows change, providers change, schemas change, and failure policies change. In that kind of environment, I am not sure Rust is the right fit.

I like Rust a lot, and it is a language I genuinely want to learn. But I am not sure that applying Rust to everything is really the right answer.

I think Rust makes a lot of sense in relatively stable infrastructure ecosystems: operating systems, runtimes, sandboxes, and core low-level layers. But agent code usually requires high-level abstraction and rapid workflow composition. Doing that in Rust takes a tremendous amount of time.

Not ready for production yet but ive been working on https://wingman.actor for quite a while. Its a golang based portable agent runtime with minimal dependencies.

I am using TS sandboxed in deno for all our agent code generated from a UI builder (inspired by OpenAI's own agent builder, and spits out the same code output)

Uh... a flue is not a disease.

Mastra is a business, this seems to be a helpful lib

One big difference seems to be Mastra has tests and this project doesn't.

Why would I choose mastra over anthropic or cursor sdk ?

why not?

How would that actually work? Eventually the specification for the CPU has to be clear. For now, that means machine code.

Natural language is too vague, ambiguous, and inefficient.

That’s why we have _programming_ languages.

And once you specify everything you need, the “prompt” becomes a program.

Anything else is to lossy

Anders Hejlsberg, who designed C#, is now leading Typescript development. Why would I not join him at the frontiers of his creative and intellectual energies?

Go is a nice language, but it's not expressive the way typescript can be. I'm not convinced, either, that coroutines are all that snazzy an abstraction at the application level.

Kotlin and Scala too if you want the same type of strong type system as TypeScript

Not mentioning Java means we are already on the same wavelength.

I love C# too.

You're welcome to port anything over to those languages. LLMs can do it in a couple of days at most.

ADK comes in Go, Java, Python, and TS

Same framework, multiple languages, let people decide their preference while having consistency and interoperability

One is stuck in 80s and another doesn’t even have official open source debugger, are you serious?

What was offensive? Was it the use of Cloudflare? Or the way the import was written?