Remember the last time somebody gave you feedback on your work, whether it be in writing or in person. I've always valued that more than the content. There's also the belief that the person giving the feedback has some sort of stake in me or the outcome. I often see them in the hallway. Many are friends and know my history. They care, or at least they should, if what they say helps me or sets me back. That investment has a real effect on the word they choose to share, the pauses, and every decision about what to say now vs. what to save for later.
AI can produce similar feedback. Maybe even better feedback, if better means more accurate or more comprehensive. But it can't feel the consequences of being wrong. AI isn't across the table from you the next day knowing it what it said might have been too harsh in that moment. It doesn't feel the satisfaction of seeing you try again and adjust based on it's advice. There's no real relationship at risk. AI's words might be right, but the absence of a person behind them changes what those words can do.
I recently watched the launch videos for Google's new model, Gemini 4 Argon, a few weeks ago. In one of them, a caterer hired for an event hands the job to an agent. The agent builds the custom presentation for the client and then communicate via email. As far as I could tell, the caterer never spoke to the person she was cooking for, at least from the marketing materials.
AI technology has become increasingly more impressive. I often deploy creative uses for it in my own work and classroom. But something about that specific demo bothered me, and I think it is this: the caterer had a reason to form that relationship, and the agent does not. The client was indeed placing an order and she was trusting someone to care about her event. That trust was directed at a person, and a person wasn't in the conversation.
When a student comes to my desk with a question, there's a reason. She is asking me, Timothy Cook, a trusted adult who knows her, what I think and how I can help. If a colleague stops me in the hallway and asks my opinion on something, he is not asking for a statistically likely opinion. He is asking for mine.
That different from a task thats more like checking a box such as filing a form, analyzing some data in a spreadsheet, or booking a hotel room. Nobody is asking for me in those situations, there just tasks. Hand them to a AI and nothing personable is lost.
The difference is whether there is expected human input the other person was counting on. A presentation for a client can be a box to check for a business but it can also be the moment a caterer learns that the bride's parent can't eat shellfish, that the family wants the dinner to feel like the one they had back in Korea, or that the person paying is nervous about the budget. All of that comes from conversation.
A message thread between a client, Ji-woo, and a caterer, Sara. The first messages are personal: her dad can’t eat shellfish, and she wants dinner to feel like the ones back in Korea. Then both sides hand off to AI agents. The avatars go empty, “Wedding dinner” becomes “Order #4417,” and the agents finish in three short messages.
When an agent handles that, the caterer loses the ability to read a request and tailor the experience to the people in the event, and that's what differentiates that business from all the others. That is a skill and it gets better with practice, or worse when it's neglected.
I believe we all know there's a difference in how it feels when something is made for you rather than made and offered for you to choose from. I ordered a pair of basketball shoes recently, directly from a website. I chose a style and a size from a menu of options, and that was exactly the experience I expected. Nobody pretended it was anything else.
Now imagine walking into a shoe store, or contacting a supplier about a bespoke design, because you want that other more personal experience. You want someone to understand what you need and make something to suit that request. And imagine that the replies you get for that bespoke service, the questions about your preferences, the reassurance that they understand, are coming from an AI agent.
A large part of every market transaction is trust between the buyer and the seller. We tend to talk about AI as if the focus should be on whether the output is good and how much added efficiency we can take from using it but the research suggests people care about something else too.
In a 2023 study in Scientific Reports, Jess Hohenstein and colleagues at Cornell had pairs of people discuss a policy issue over a chat platform with AI-generated smart replies. The AI replies actually made conversations more efficient and more positive. But when people suspected their partner was using AI responses, they rated that partner as less cooperative and felt less connected to them. And that happened regardless of whether the partner actually was.
So, human communication is not just the content but the perceived stake the other person has in you. This implies that suspicion that no person is there, whether true or not, destroys the value even when one is. The unfortunate outcome of this is that the societal cost will move beyond the people who use agents for human services, and spread. Once you know any reply might be generated, every reply becomes suspect, including the ones a human sat down and wrote for you. I can almost guarantee the caterer who still calls her clients herself will become way more valuable than the one who doesn't.
People are already viewed as commodities, not through some dramatic cultural shift, but through a slow change in what we assume is on the other side of an eventual human interaction. I fear the world where people become inputs to be handled and friendly request becomes a request to be cleared. We have already watched dating apps train people to treat each other as options to swipe through and take from. Agents that talk to people on our behalf could extend that habit into every relationship that has a transaction somewhere inside it, which is most of them.
I don’t think the answer is to keep AI out of work, and I’m not saying to keep AI out of the drafting process entirely. There is a difference between using AI in the process to help you communicate with another person and automating that communication entirely.
Transparency may be the key here. I often use AI to help with what I write, including this piece, but what you’re counting on when you read this is my argument, my beliefs, my examples and my judgment, from someone who will answer for everything written here. If I let an agent write this and send it under my name without saying so, I’d be the caterer who never met the client.
So let’s just get specific about which moments require a personal touch and which don’t. Amongst others, here are some thoughts:
Giving someone feedback: People want your feedback, not AI. If they wanted AI, they could ask themselves.
Responding to an email from someone who wrote to you, personally: That person wants a response from you, because you know them or you know something specifically that may be useful to them.
Writing a card: I know this is probably tough for many people, but writing a poorly written card that's personal is superior to a well written card that is empty.
Taking a meeting with a colleague/client: actually listen at the meeting and respond in real time. Let's not communicate with each other through AI.
Sure, an agent can write the birthday card. Will the person opening it want to read it?
These are just a few examples where I believe a person expects another person. This is significantly more important for children than anyone else because kids learn far more from watching how adults act than from being told what to do. They pick up much of their behavior through observation. If a child gets feedback from an AI instead of from her teacher, she does not only miss the feedback but learns that feedback wasn't worth an adult's effort. Children watching those adults may never learn that there was a skill there to develop. Sometimes, they are just asking for you.
I use Claude, Google AI Studio, and Kimi K3 in the scaffolding of this article from drafts, voice notes, speaking material and other published work, which I then rewrote and added to extensively. The arguments, the ideas and interpretations behind the piece, the examples and the reading of the source material all come from me. These are positions I can defend without AI in proximity. I take full responsibility for everything published here.
References
Bandura, A. (1977). Social learning theory. Prentice Hall.
Hohenstein, J., Kizilcec, R. F., DiFranzo, D., Aghajari, Z., Mieczkowski, H., Levy, K., Naaman, M., Hancock, J., & Jung, M. F. (2023). Artificial intelligence in communication impacts language and social relationships. Scientific Reports, 13, 5487. https://doi.org/10.1038/s41598-023-30938-9
