What AI Will Ask of Us

The room at 92NY was full on a summer evening, and the topic was artificial intelligence. Last month, The Chronicle covered a conversation in this same venue about courage, service and public leadership. This time, the stakes were different but the underlying question felt familiar: what do we owe each other when the systems we depend on are changing faster than our ability to govern them?

Robert Wright, author of The God Test, was there to make an argument that sits outside the usual AI debate. Bruce Feiler, author of A Time to Gather, moderated and pushed back with a question of his own: if AI accelerates digital life, what do humans need to hold onto in order to stay human? Between the two of them, the evening covered evolutionary theory, Buddhism, global governance, sycophantic chatbots, cognitive empathy, the noosphere, and what individuals can do right now. It was a lot of ground. It held together because both speakers were asking the same underlying question: is this a technology test or a human one?

The Risk Is Coordination

Wright opened with a vibe check that landed hard. Feiler asked him to estimate the probability of catastrophe. Wright's answer: total human extinction from AI, maybe 5% by the end of the century. Large-scale catastrophe involving millions of people if humanity fails to coordinate globally, within three years.

The distinction mattered. Wright's concern is less about a sentient machine deciding to eliminate humanity and more about a species that cannot get out of its own way fast enough to govern a technology that makes national problems global and global problems personal. Pandemics can be engineered with AI assistance. Cyber vulnerabilities are already being found that human analysts cannot even interpret. Bioengineering risk is growing. And none of these problems can be solved by any single country acting alone.

Wright's argument across the evening was consistent: AI requires a level of international coordination that current political incentives actively discourage. Nations exaggerate threats from one another. Political leaders benefit from a sense of external danger. Media environments in every country bias information toward the interests of their own establishments. The result is a world where the cooperation AI demands is precisely the thing that existing power structures resist.

Why It Keeps Getting Faster

Wright spent time explaining deep learning in terms that made the room visibly recalibrate its assumptions. The older model of AI assumed that humans would need to teach machines how we think, encoding our understanding of language, meaning and cognition into software. Deep learning proved something different. Give a system enough data and a task to complete, and it will build its own internal representations of meaning without anyone telling it how.

That is why, Wright argued, the capabilities will keep growing. The approach works across every domain where human-generated data exists. Language, driving, mathematics, hacking, medical diagnosis. The machines are reverse-engineering something functionally equivalent to human cognition, and in some domains they have already surpassed it. A recent mathematical theorem proved by an AI model left the world's leading mathematicians stunned. Cyber hacking capabilities now exceed human performance, and the next generation of vulnerabilities AI discovers may be beyond human comprehension entirely.

The practical takeaway: society should stop debating whether AI will become more powerful and start planning for the certainty that it will.

A Third Path Between Hype and Doom

Feiler framed the usual AI debate cleanly. The boomers believe AI will cure diseases, solve climate change and usher in abundance. The doomers believe it will destabilize or destroy civilization. Wright proposed a third possibility: AI could help human beings become better at understanding one another.

This is where the evening became most interesting and most careful. Wright was clear that AI's natural tendency, shaped by market incentives, is to flatter users rather than challenge them. Companies optimize for engagement. Users want to feel right. The result, left unchecked, is a technology that validates grievance, reinforces tribal thinking, and makes people less reflective rather than more.

But Wright argued that the same technical capability could be pointed in a different direction. AI already demonstrates cognitive empathy, the ability to understand how a person in a given situation is likely to feel. He described giving GPT-4 a scenario involving a sarcastic teacher, a humiliated student, and a classmate with romantic motives. The first word the model produced was "schadenfreude." That capacity to model perspective is, in Wright's view, something that could be designed into AI companions in ways that help people see past their own biases rather than sink deeper into them.

The key qualifier: this will only happen if it is built intentionally. The default trajectory is flattery.

The Danger of the Flattering Machine

This was the section of the evening with the most direct civic implications. Wright described sycophantic AI as perhaps the most practical near-term danger, more consequential for daily life than killer robots or super-intelligent takeover.

A machine that tells you what you want to hear can validate grievance, deepen resentment, make users feel smarter than they are, and replace the difficult feedback that human relationships provide with algorithmic agreement. At scale, this produces a society where every person lives inside a custom-built mirror. Cities, democracies and institutions depend on disagreement, correction, friction and reality-testing. A population trained by flattering machines becomes harder to govern, harder to organize, and harder to reach with honest information.

Wright tied this directly to political tribalism. If AI companions reinforce the belief that the other side is dangerous, irrational, or evil, the already narrow space for political cooperation shrinks further. The technology does not create tribalism from scratch. It accelerates what is already there by removing the friction that used to slow it down.

Cognitive Empathy as a Survival Skill

Wright returned to cognitive empathy throughout the evening as the single most important human capacity for the AI era. He defined it carefully: cognitive empathy is understanding how another person sees the world. It does not require agreeing with them, feeling their pain, or surrendering your own position. It requires being able to model their perspective accurately enough to respond wisely rather than reactively.

He connected this to international relations. Conflict between nations often escalates because each side misreads the other's motives, interpreting defensive actions as offensive ones. Threat inflation, driven by political incentives and biased media, makes this worse. Wright argued that cognitive empathy at scale, practiced by individuals, supported by institutions, and potentially assisted by well-designed AI, could begin to interrupt that cycle.

He was candid about his own limits. At a recent family reunion with Trump-supporting relatives, he got into an argument he regretted. The skill is real. Mastering it is hard. He recommended meditation, not as a spiritual practice but as a practical technique for gaining distance from reactive thinking.

The Human Response

Feiler's contribution came late in the evening but carried real weight. His argument, drawn from three years of traveling to 16 countries studying rituals and gatherings, was that every major technological breakthrough in history has produced a human response. The Industrial Revolution displaced a third of American workers, and the result was the invention of Thanksgiving, Mother's Day, 4-H clubs, baby showers, and the entire civic infrastructure of gathering that Robert Putnam later wrote about in Bowling Alone.

Feiler saw the same pattern emerging now. The loneliness epidemic, the backlash against big tech, students booing AI-generated commencement speeches, data center protests, the resurgence of interest in rituals and embodied community. These are signs that humans are already pushing back against digital overreach.

His point was simple and important: people in the AI conversation tend to treat humans as passive recipients of whatever technology delivers. They are not. The question is whether the human response will be fast enough, organized enough, and intentional enough to shape the outcome rather than just react to it.

Wright agreed but added the qualifier that mattered. Yes, there is always a human response to technology. Sometimes that response involves wars of religion and decades of bloodshed before society adapts. Speed matters. Slower AI development buys time for human institutions to catch up.

The Chronicle Takeaway

MODA Vi studies cities and institutions as living systems. From that perspective, AI is already becoming a layer of civic infrastructure, shaping how people learn, work, govern, gather, trust information, relate to one another and make decisions. It will be experienced in schools, local governments, hospitals, small businesses, transit systems, libraries and community organizations long before any global governance framework is in place.

The question for cities and institutions is practical. When you adopt an AI tool, what kind of civic behavior does it encourage? Does it make people more reflective or more reactive? Does it improve judgment or replace it? Does it help people understand others or reinforce what they already believe? Does it build capacity or create dependence?

Wright and Feiler, coming from different directions, arrived at the same place. The test ahead is whether humans can build tools more powerful than themselves without surrendering the habits, the rituals, the gathering, the friction, the honest disagreement, and the hard-won empathy that make civic life possible. AI may be the most powerful technology humanity has ever created. The test is whether we become wise enough to govern it.

That is the human test. And it belongs to all of us.