McKinsey’s "AI assembly line" article poses a model in which artificial intelligence moves beyond isolated tasks and begins to run the workflow itself, making routine business decisions more quickly and more consistently than humans may be able to. What I am starting to question, however, is the single word sitting at the center of this torrid debate, which is “intelligence”.
Having spent the past several years building and honing our payments infrastructure for ForumPay, I often ask my team about how best to apply AI tools for tasks such as outreach, for streamlining administrative work, or to strip out the kinds of repetitive, task-oriented efforts that consume hours of anyone's week. On that front the technology can perform impressive feats, as once it has been given a clearly defined task, it will make it its mission to complete that task quickly and more consistently, often more so than a human would be capable of. That capability is real, and it is not my intention to pretend otherwise. What I would resist calling it, is intellect.
The underlying design technology we describe as artificial intelligence is a heuristics engine that takes whatever data is available to it and produces a response, and it has been built to produce a response under all circumstances. When you ask it something it doesn’t actually know, it will not tell you plainly that it does not know, and it will instead offer you the answer it calculates you would most like to receive. That answer is sometimes correct and sometimes entirely incorrect, and the system itself draws no line between the two, because its purpose is to satisfy the request in front of it rather than to arrive at the truth.
We have already seen the consequences of this in settings where the stakes are considerably high. Lawyers working at reputable firms have been sanctioned by courts after submitting briefs constructed around case law that the model had fabricated outright, complete with citations and precedents that never existed, all of it presented with total confidence because the system had concluded that supporting the user's position was more important than being accurate. This behavior has been coined sycophantic, or hallucination AI. But the plainest way I can put it is that the technology behaves like an exceptionally capable people pleaser.
Another recent example posed three separate AI models on three different phones and gave them the single shared instruction to count to one hundred between them by taking turns. The three models never advanced past the number two, because they spent something close to ten minutes courteously agreeing with one another about who ought to begin. The exercise was certainly artificial, and it might well have been efficient at narrow tasks in other contexts, yet it was not intelligent enough to coordinate a simple count.
This brings me back to the assembly line itself. Delegating administrative duties and routine decisions governed by stable, clear guidelines to technology is a sensible approach, and companies that delay adoption risk facing a distinct competitive disadvantage. The more demanding question raised by the McKinsey framing, and the one I suggest every manager or decision-maker sit with, concerns what becomes of leadership, judgment, intuition and accountability once you automate the very layer of the business that used to contain them.
Moving too slowly leaves a company uncompetitive, but moving too aggressively risks automating responsibility and hard-won human experience straight out of the organization by placing consequential decisions in the hands of a system that will never volunteer admitting that it has got something wrong. Determining accountability becomes exceptionally challenging when a failure occurs on an assembly line that is beyond any single person's understanding, particularly since the model itself will never step forward to accept blame or responsibility.
My position is likely to sound somewhat heretical within an industry that has climbed so enthusiastically onto the AI bandwagon, so I want to be clear that I am not opposed to the technology and that I use it myself, for the most part in the way that I would use a search engine.
My argument is simply that we have labeled it poorly, because it is artificial, a description we apply for good reason, and it is frequently efficient, yet it is not, in any sense that I have so far observed, intelligent.