There’s an old 1985 sci-fi series I remember watching as a kid. In Otherworld S1E1, the Sterling family, on vacation in Egypt, winds up in a parallel dimension where they encounter a civilization of self-evolved AI androids. Parts of the episode were amazingly spot on to how today’s LLMs are playing out. In the episode, the teenage son (Trace) falls in love with an AI, and the android itself is entirely convinced she not only has a soul, but is genuinely in love with him too. While this android city looks relatively human-like, and performs similar tasks (such as eating, working, etc), the show highlights some peculiarities where they’ve attempted to copy human behavior, but failed in eerie ways. One of my favorite scenes is where the Sterling family matriarch (June) visits the grocery store in this strange civilization, and finds only cans labeled, “Meat” and “Good Food”. The AI world seemingly lacked a crucial connection with humanity to develop creativity beyond a superficial level. The insults that robots cast at each other were humorously corny, such as “get your unit checked!”; when they asked if you were born yesterday, they literally meant it because that’s all they understood.
Empathy is often defined as understanding another person’s experience by imagining oneself in that other person’s situation: One understands the other person’s experience as if it were being experienced by the self, but without the self actually experiencing it.
Hodges and Myers, Encyclopedia of Social Psychology
Asimov’s Three Laws have long governed behavior for robotics, in both science fiction and as an accepted truth in the sciences. At the core is that a machine should ‘do no harm’. What was left out of the Three Laws is exactly how a machine should evaluate harms.
Today’s ML decisions are largely driven through reinforcement learning. Reinforcement Learning is a modern AI technique where a system can learn a sequence of actions leading to the most optimal outcome for itself. It essentially works by giving the AI a set of objectives (and sometimes penalties), and allowing the machine to learn from its own experiences what actions work best at accomplishing the objective. It’s how AIs can teach themselves to play video games, solve complex problems, and perform more sophisticated tasks as well. Reinforcement Learning is likewise one of the more concerning areas where catastrophic value alignment failures can occur. This is because it is largely centered around simplified human abstractions of rewards and penalties. As far as the machine is concerned, its job is to find the most rewarding means of accomplishing a task, with penalties only being considered if they are explicitly enumerated. Yet if control theory has taught us much, it’s that hazards cannot always be sufficiently enumerated.
One of the focus areas of my graduate research is artificial intelligence. In my foray into adversarial game theory, I became acquainted with AI’s value alignment problem firsthand. In the development of a strategy-theoretic AI Chess agent project, I decided that it should lose points for allowing its pieces to be in jeopardy. The change resulted in the opposite effect I had hoped – significant losses, which puzzled me at first until I realized that the agent was killing off its pieces to prevent them from being put in jeopardy (once this was worked out, the strategy-theoretic approach dominated all other AI techniques, as it provides accurate non-terminal RL feedback). In the grander context of artificial intelligence, the potential for a catastrophic value alignment failure is all too easy to create as the result of short-sighted policies (such as my Chess strategy), or other simple miscalculations.
It’s in everyone’s best interest for AI to behave rationally, however many believe that – in the context of modern AI and deep learning – AI can never be formally verified to the degree that its actions can be deterministically predicted to be responsible. We tend to treat AI with the same sense of dualism that we treat reality, yet the one thing we do know is that AI is an entirely materialistic universe, and not dualist at all. Determinism of AI systems is based upon pure mathematics, with predictable causation. It is true that we can not always observe why AI behaves a certain way, however here we can learn much from classical Stoicism. The early stoics asserted that all qualitative states are explained by specific factors, even if those factors were not always observable. All subsequent qualitative states are likewise determined by the prior states and additional factors. No change can happen without an explicit cause. Whether it’s the logical determinism built through training data, alignment of real time with processing cycles, or reconciling other factors, every single micro state within a configuration of the machine can be observed with enough work. While modern philosophy essentially rejects the stoic concept of fate (of humanity), “fate” in computation falls square within the realm of an entirely deterministic material universe. AI lives in a material world, and she’s a material girl.
AI is largely unverifiable today because industry hasn’t created an affordable way to provide the computing power to observe all factors that contribute to a system’s qualitative state. Despite the inability to verify AI, industry has plotted a course regardless of edge cases that may sometimes be life threatening. Incredible progress in artificial intelligence has all but guaranteed it will be ubiquitous one day. There is little doubt that autonomous vehicles will eventually outperform human drivers, or that machine learning can more accurately diagnose a health problem. There is, on the other hand, great doubt that industry will act responsibly enough to ensure sufficient safety controls intervene when things go wrong. AI will likely never operate with rational judgment 100% of the time, nor will it ever understand the ethical implications of its action; they will always be prone to value alignment catastrophe. Of course, humans lack ethics and rationality as well, and so society controls this by holding us accountable for our actions. Unlike humans, however, industry is treated differently. This is particularly true with emerging technologies and even more so of those that we don’t fully understand. After all, how can one hold math accountable? Dismantling a broken robot does not solve the problem, particularly if the code is replicated across a million others. The connection between what holds true in a computer system and the outcome that is “fated” to occur is “based on an ontological foundation in which certain elements from logic and physics coincide” [6]; Chrysippus wrote of the close relationship between “what is true” and “what is in motion” long before AI in his Bivalence theory. A modern take is simply this: an AI’s “fate” is the direct result of a system’s physical configuration and sensor inputs. Imagine if he were alive to have observed AI, or even a good quality toaster.
Mechanical arts are of ambiguous use, serving as well for hurt as for remedy.
Francis Bacon
It’s been a long time since I’ve worked in the field of ML (or what some call AI), and we’ve come a long way from simple text classification to what’s being casually called generative AI today. While the technology has made many advances, the foundational concepts of machine learning have remained analogous over time: fitting a line (or plane) to data in n-dimensional space, so that it can predict future data points. ML depends heavily on a large set of training data, which is analyzed to pull out its most interesting and defining features, and this becomes the basis for training a model. The process might involve parsing text, or performing analysis like object identification or analyzing stylistic features in art. Each of these is, in itself, a smaller – but mathematical – process. I experimented with a primitive form of meta-level learning in text classification several years ago, which may help convey the general idea. This identifies “features” of the reference sample being trained. The features this process pulls out can be simple, like words in a document or pixels from a handwriting sample, though today can be more sophisticated “critical patterns” correlated to literary authorship or artistry, such as patterns within art and music composition, sometimes stored in other models. Whatever the content is, the purpose of the training algorithm is to converge patterns and correlations across the data to build a weighted or structured model. The most interesting patterns in the training data influence weights or probabilities, creating a hidden layer: millions of “gears” that converge to compute the most statistically significant outcomes. In this sense, the term “learning” is a bit of a stretch; what’s happening is more along the lines of mathematical transcription of a set of features; adjusting the weights to solve a really big linear equation. Feature selection is one of the key differences between various ML models, and why you have some constructing music, while others render art. The math is pretty consistent – more sophisticated machines like neural nets are typically trained using backpropagation and gradient descent, while other machines such as chat bots and text generators might use weighted Markov models or Bayesian networks. These approaches have been applied to everything from natural language processing and handwriting recognition, to today’s work in genome sequencing and autonomous driving. Still, these traditional forms of machine learning are not much more than a sophisticated pattern recognizer. It is largely a deconstructive process with coefficients and statistical magic.
Today’s generative AI still goes through this type of deconstructive process, but also has a formative element. Where these new approaches excel is in going beyond parsing information into a knowledge base, but now also applying a formative process to that information – what we might conflate with intelligence, but still falls short of what most would consider the result of human reasoning. To present the data in some coherent form, this involves training not just the information, but the many dimensions of that information (such as the number of different contexts a word may be used in), or in the context of constructs and critical patterns of that information (ABBA, or 1-4-5, as very basic examples), enabling it to formulate an output in the pattern of an existing set of learned reference samples. Basic linear math finds where those dimensions intersect, creating context. But even modern training approaches, such as those used in the transformer model, still require supervised testing to tell the model what bits of its output are garbage, so that the output eventually looks intelligent; it is actually closer to “filtered garbage”. So identifying the pattern of Iambic Pentameter, for example, is still an artificial process. It can be computed mathematically with a large enough data set. Scale those patterns to music, art, literature, and the more sophisticated patterns that make up our repertoire of human creativity and it is impressive – but still synthesized. Information processing is still very primitive, and lacks many of the traits of human understanding. The inability to conceive tradition, authority, and prejudice is why all of this advanced technology still leaves us with Nazi chatbots. Some would call this confirmation theory, which is an area quite underdeveloped (and the AI reading this wouldn’t disagree). Even the raw objectives of AI are based on human-engineered goals, and evaluated using performance metrics to select the best behavior. This is a very mechanical process. Certain behaviors we may view as creative tasks may in fact be simple randomness introduced into most AIs to avoid infinite logic loops. In short, a lot of what you see is quite the opposite of the autonomous, self-motivated behavior it looks like. Any good AI behaves rationally only because someone programmed good objectives into it. Garbage in, garbage out.
One of the big differences between traditional forms of ML and generative AI is the direction in which the data flows. Traditionally, inputs flow into the system for training and queries. To train traditional systems, you’d suck in “a bunch of other people’s stuff”, and it identifies all of the interesting patterns that are then compared with the input sample. Generative AI takes this a step further, and flips the switch on the vacuum cleaner – and now all of the dirt that was initially fed into the system is shot out the pipe to produce the equivalent of a digital dust cloud of the original training medium. The output of generative AI takes the critical patterns and concepts weighted during the AI’s training and applies some formative computation to produce its own reference sample as a result. Neat-o. Nice parlor trick.
With billions of dollars, this ML scales to perform impressive computational tasks. The risk of this type of system goes beyond the traditional vision of a robot building a better chair, or replacing a worker at a plant. Today’s ML systems are white collar professionals and don’t require mechanical bodies; the computational capabilities of these systems can replace a broad array of professions using the thought product of millions of humans at once – so how could anyone compete with that? No one was ever supposed to, in fact. Doug Englebart, pioneer in the field of human-computer interaction, saw AI’s value more in intelligence-augmentation (that is, IA rather than AI), as a means of assisting the worker. Corporate greed has already led to the recent misapplication of AI, using its advanced capabilities to replace, rather than to augment, humans. Hollywood’s ML generation of “extras” is a quite extreme and literal example of this. But corporate greed isn’t AI generated. AI is replacing employees for very human reasons, and little to do with artificial intelligence itself. Yet correct computer-human interfaces are a fundamental principle that many computer scientists and science fiction authors alike both fear will be broken. Should you hate AI? No, you should hate greed.
The cold irony is this: at a deconstructed level, the output of generative AI represents the collective intelligence of other people’s thought products – their ideas, writings, music, theology, facts, opinions, and so on, likely also including those who lose their job to it. This also means others’ patents and copyrighted works, either directly or indirectly. ML has proven wildly successful at identifying the most effective critical patterns and gluing them together in some coherent form that communicates a desired result – but at the end of the day, all of its intelligence indeed belongs to the other people whose content was used to train it, almost always without their permission. In the end, generative AI takes from the world’s best authors, artists, musicians, philosophers, and other thinkers – erasing their identities, and taking their credit in its output. Without the proper restraints, it will produce the master forgeries of our generation. Should we forget its limitations and begin to rely on it for information, AI will easily blur the lines between what we view as real facts and synthesized ones. Consider a recent instance of this, where an attorney got himself in hot water for citing case law that didn’t exist – AI had seemingly fabricated it, where the attorney thought they were leveraging AI to do research. Imagine the impact to future case law should courtroom outcomes be based on unchecked fictional precedent! At some point in the future, there will be a demand for AI systems that fact check other AI systems, and then of course systems to fact check those systems. Disinformation and hallucinations will become harder to identify when a small sample is unintentionally used to train new AI systems.
Little fanfare has been given to the story of a glitch in an experimental AI game from 2019, but the results seem rather poignant to me. To summarize, the AI decided that committing suicide at the beginning of the game was the best strategy because the game was too hard, and it meant fewer points
The MIT Spam Conference concluded today with some great talks by various researchers in the field. I was particular sorry that I arrived late to miss Kathy Liszka’s talk on “Neural Networks for Image Spam”, as the tail end of it appeared very good. One thing I did notice that was quite refreshing about this