In my previous past, I talked about using LLMs to learn more about coding¹. I was curious about reversing my advice, so I decided to complete a project without writing a line of code. [Edit: see the repository if you're curious!]

I had Claude Sonnet 3.6 implement a browser extension that lets me add a row to my time-tracking spreadsheet. I set the following rules:

1. The process is asking Claude 3.6 to implement a file; I copy & paste the result into result; then I ask for edits.
2. I can test the code and give feedback, but no writing new lines.
3. I can add environment variables

My takeaways from the exercise:²

The Good:

• The project got done: As in, the project is in a workable state; it probably wouldn't have otherwise.
• The project got done quickly: ~700 LOC in 3 hours. Pretty much no amount of me being in the coding loop would have made that faster (at least for this size project).
• Making a "specification" was easy: I exchanged several messages describing what I wanted out of the extension, and then asked it to make a specification. This let me more quickly determine what exactly I wanted.³
• Fast iteration: Because I didn't need to worry about implementing the UI, I could iterate much faster on it.
• Clean(ish) code: The output is generally clean code, at least cleaner than I would have written in 3 hours.

The bad:

• Technical Debt: I have very little understanding of what's going on. If I needed to make a change myself (like change a category label), I would either have to learn the entire codebase from scratch, or ask an LLM. So I'm dependent which leads to ...
• Brittle changes: Because I'm dependent on the LLM, all I could do to change something was give the entire file (or most of the file) and hope the fix works. Occasionally, it took a couple of tries (or a new context) to get things working.
• Less fun: The things that were least automated about this process were filling out the OAuth form for google sheets. The fun feature scrunging was done for me, but I still had to figure out the bureaucracy.
• I learned a lot less: if I'd done this project myself. I would have learned a lot about UI, and the sheets API, and styling with CSS. Instead, the only thing I really learned was OAuth because that's what I did.

The interface

• Using chat: I was using the chat web app instead of something like Cursor or Aider! This was ehh. It didn't have great ergonomics.
• Managing contexts was crucial, both in getting Claude to pass along summaries to future instances, and in starting a fresh instance when getting stuck.
• Using onefilellm: this made it easy to communicate the entire codebase as an attached file.⁴
• Tree-based interface? This may not be the ideal coding interface, but I would love the ability to turn my LLM interactions into a tree. I often have two different questions for an LLM after its responses. And it would nice to be able to branch from one point (And eventually merge back together somehow). Almost like a chatbot git.

Big lessons:

• Human in the loop: At least at the moment, complete LLM dependence is not the most effective for me. It is faster at the beginning, but the technical debt builds up quickly. I don't necessarily need to understand every line perfectly, but I should understand the broad strokes.
• Actually using the LLM: On the other hand, it is freeing to give up some control. I often feel like I need to type every line but it is faster to pass off what I understand.⁵ So with something like the extension's manifest.json, I'm fine to have the LLM write the boilerplate.
• Volition and LLMs: At the Recurse Center: the directive of "coding at the edge of your ability" is balanced off with "building volitional muscles."⁶ This project did not have me coding at the edge of my abilities, but I still had to exercise a lot of volition! It was my idea to make the extension, and I specified how the UI was going to feel. Especially with "home-cooked software"⁷ knowing what you want rather than how to get it seems like it will become more and more important in the near term (6 months - 3 years). I don't know what comes after that, but for the moment, there's a lot of places to explore!⁸

Cheers!

Notes