26.0308 Lab notes (Gdrive) Git

Goal of this page is to explain what AI can not do. And to explain the hype and misconceptions.

TOC

• 1 AI limitations. This is the most important thing to understand about AI if you plan to use it for anything critical.
• 1b Acceptable AI applications (only where approximate outputs are acceptable).
• 2 AI hype. AI gurus are feeding you hype. Constantly. Applications of AI based on misconceptions are costly (and sometimes dangerous) to the end user (Elon Musk's much-hyped FSD is a prime example). AI is very useful, and very difficult to understand; such a situation can be very profitable to AI insiders.
• 3 Common misconceptions (errors) in AI videos.

• The LLM TF basically implements a complex universal function approximator (UFA) that can
  • Input text that does not exactly match any text it was trained on and
  • Spit out valid response text (that is only 1 of an infinite number of possible valid responses).
  • This process is based on massive statistical analysis computation. That is why AI requires nuclear power stations, while your brain requires 20W.
• The heart of an LLM is the transformer (TF). A TF is a UFA (universal function approximator) that produces outputs based on statistical analysis of inputs. While "approximators" have advantages, the drawbacks should be obvious.
• LLMs are programmed (not trained; they do not learn).
• LLM have no Intelligence

This diagram shows where the core "intelligence" is created and store in a TF.

AI can be used for such applications and perform so well not because AI has "intelligence" (it has none), but because AI implements UFAs (universal function approximators). APPROXIMATORS (AI algorithms can make a good output guess for an input combination it has never seen, based on a massive number of "training" input/ouput combinations that were used to program the AI model). AI can approximate any function, such as

• #1 INPUT = Sensor data / OUTPUT = Control signals for car (self-driving cars)
• #2 INPUT = Sensor data / OUTPUT = Control signals for powerful electric actuators (humanoid robots)
• #3 INPUT = Tokens (ASCII encodings of parts of words) / OUTPUT = Next token (LLMs)
• #4 INPUT = Sensor data and mission goals / OUTPUT = Strategic advice and weapons system commands (Operation Epic Fury)

Conclusions:

• Using an approximator for #1, #2, and #4 can be fatal.
• Mistakes can NOT be tolerated for #1 and #2.
• AI usage is only acceptable for #3 and #4.

AI gurus have spent a lifetime working on AI, and many stand to get rich from it. They answer to their stockholders. So they are motivated to say whatever it takes to the sell their product (makes sense). And their product is quite complex. But the basic concepts shown in this example (for GPT-3) are going to the same for even the latest LLM models. So it pays to understand the reality of AI.

AI hype started even before the semiconductor era. 70 years AI pioneers were "confident" (they knew better, and knew that their hype would fool the general public) that AI would surpass human intelligence in just a few years. 40 years ago my college classmates (engineering students) were making the same scam predictions. And so it is today. Below is my "spoiler alert" about the very significant (compared to real intelligence) limitations of the revolutionary engineering marvel that is man-made (artificial) "intelligence".

• The comparison of AI "neurons" to animal brain neurons is misleading (and intentionally so). I instead compare AI neurons to binary transistors (an analogy I am still working on).

Below: The UFA makes a bad approximation (marketing term:“hallucination”)

On 26.0303 after endless long chats with GPT we came to the conclusion that UFA and "intelligence" are not exactly the same. For details see docx #601_core_AI_demos_.docx.

A transformer (TF) neural network (NN) inside an LLM can accept previously unseen combinations of tokens and still produce a correct response (most of the time). This is made possible by TF UFAs (and a few other GPU-based components). However, this power comes from probability and approximation, not exact computation. That means it can sometimes fail, even when the input is valid. Deterministic functions do not fail in this way, but the must be programmed (not trained) and the range of valid inputs is much more restricted.

The 3D demos below

• On the left: Superimposes many individual 3D plots of inference output components. The resulting confusion needs no explanation.
• On the right: Plots training loss curves.
  • The z axis is loss squared.
  • I am not sure what the x,y axis are. Maybe x for one Wx, y for another Wx. Its meant to show that changes for one Wx causes changes of the loss curve for all Wx's (I've never seen a video that explains what the x,y axis are for).
  • The problem is such a complex surface would only be generated in a TF with a massive number of Wx's. Which would require a massive number of x,y axis.