So much of the discussion about AI is what it can do… but the more important question is what should we do with it. We have a lot of control as individuals and organizations over how we roll out AI in our workflow - and those choices really matter. And this isn’t about whether we automate whole jobs or industries - it’s about what individual tasks we think it is still important for humans to do.

It was a personal realization about how I have been using AI that got me thinking about this more. For many years, I’ve wanted to do more writing like this. But I usually lose momentum and motivation and end up stopping. I was hoping AI might help me out with that.

So I created a way for Claude to help me with my writing. I created a skill that would let me ramble for a few minutes and then structure those thoughts into a more clear written version. It seemed neat and fit nicely into my greater comfort in speaking versus writing. And at first, I was pretty good about editing and giving feedback on style and substance.

But then I started getting lazy. The technical term is “automation bias”. I started just accepting Claude’s first draft with minimal edits. Or maybe no edits. Or maybe without reading it that carefully. Or at all.

And I realized that in trying to make it easier to write, I had lost so much of the value of writing. The value of writing is that it helps me clarify my thoughts. The value is in the process as much as the output - and I had short-circuited the process.

Now I have a new mechanism for working with Claude on my writing. It has two parts. In the first, it will spar with me about my idea. It pushes me on weaknesses in my argument (for example, pointing out this earlier post as a counter to my initial argument for this one). When we come to agreement on a solid thesis and examples, it puts a few bullet points in a document. It will refuse to draft anything that looks like the prose - that work is left for me to do.

Then I write the draft in a basic text editor (I prefer Typora these days). Claude then has a second process for editing. It will review for missing points, poor grammar, structure that could be improved, typos. All of the feedback is submitted as comments in a Claude doc. I make the edits (except misspellings, I’ve given it permission to directly correct those). I decide how to improve awkward phrasing or combine two paragraphs that are repetitive.

I’ve enjoyed this switch back to drafting (though I admit it is slower and somewhat frustrating). Writing drafts myself has brought back the effort of thinking and writing - and the satisfaction you feel when you do it well. I also find that writing for myself improves the quality of my thinking.

I wanted to put a framework around what tasks we should and shouldn’t automate with AI and I came up with two categories - those where the output is worse when using AI and those where the human capabilities get worse when relying on AI.

The first category applies to both people and organizations - and will shrink as AI capabilities improve. The second is more difficult, because individuals have an incentive to hold on to these tasks while organizations have an incentive to automate them - and the tasks here are unaffected by changing AI capabilities.

You could think of the first category as things AI can’t do yet - or at least can’t do well. In the marketplace, lower quality products will lose. There is a tradeoff between the cost of production and the quality of output that might allow some firms to accept lower quality output at a lower cost. Other markets (or parts of markets) will not be willing or able to make that tradeoff. But if you’re thinking about what tasks you should or shouldn’t outsource to AI right now, the question of whether it improves or reduces the quality of the output is a good starting point.

The second category is what I saw in my writing. I might argue I saw both categories - but I viscerally felt my capability to write and the quality of my thinking declining because of the outsourcing. In an early study, similar skill losses have been seen with doctors performing colonoscopies.

And this sets up a challenging set of incentives. AI usage that results in an increase in quality of the output (at least in the short term) can increase productivity and profitability. But if it also negatively impacts the capabilities and effectiveness of your workforce over time - that’s a large price to pay. And unfortunately, our business climate often encourages short-term wins over longer-term decision-making.

Each of us needs to hold the line for ourselves on owning the tasks that help build our skills. This is exceedingly hard to do - as I found out myself by getting the balance wrong in my writing. Moreover, the best approaches will change over time as AI capabilities improve and we learn how to better take advantage of them. But I think individuals and organizations that are intentional and thoughtful about drawing both the can’t and shouldn’t lines will have a real competitive advantage.