The End (of an era)!

Editors Note: This is a modification of talk given by Carrot on the topic of AI and the future of education. The views of the author don’t necessarily represent the views of theDIHEDRAL.

Let’s start out with a couple of terms that will become increasingly familiar as more and more people come to think and talk about Artificial Intelligence.

Narrow AI = Ability to accomplish a narrow set of goals, e.g., play chess or Go.

Artificial General Intelligence (AGI) = A type of AI that matches or surpasses all human capabilities across a wide range of cognitive tasks.

Intel Co-Founder Gordon Moore is responsible for articulating what is known as Moore’s Law.  Essentially Moore’s Law which is more of a trend than a law states that the number of transistors that can be put onto a computer chip doubles every two years.  This trend has been remarkably accurate since it’s prediction back in 1965.  That doubling power has moved hand in hand with halving the cost, meaning we have been doubling the power of computing while halving the price for decades. These Decades of exponential growth in computing have led us to a place that most experts couldn’t have predicted and most lay-people can’t comprehend.

I want to take a brief look at a snippet of a fraction of the progress we’ve experienced over the last few years.

In 1997 Deep Blue defeated world chess champion Garry Kasparov.  Deep blue was a supercomputer filled with all the information about chess that its programmers could muster, it could do nothing else but play chess, but it could play chess better than anyone or anything else.

In 2011 a similar but much more advanced computer called Watson was trained on massive amounts of data including encyclopedias, dictionaries, and databases.  Due to Watson’s training, it was able to defeat Jeopardy champions Ken Jennings and Brad Rutter.

Soon, programmers turned their sites to a game called Go which is astronomically more complex that chess.

In 2012 Leading futurist Nick Bostrom ventured that Go would not be mastered until 2022.

In 2016 Alphabet subsidiary DeepMind used what is called “deep reinforcement learning” in a large neural net to train AlphaGo which went on to defeat Go world champion Ke Jie.  Starting with a huge number of recorded moves, AlphaGo went on to play itself over and over until it achieved the status of master.

A few months later, DeepMind released AlphaGo Zero.  Unlike AlphaGo, AlphaGo Zero was not given any human information about Go, except the rules of the game.  Within three days of playing against itself, it was able to defeat its human trained predecessor 100 games to 0. 

The reinforcement learning which allowed the program to become its own instructor achieved mastery in 21 days and was able to defeat sixty top professionals including world champion Ke Jie in three straight games.  In 40 days, it surpassed all versions of Go competitors (both human and digital) with no encoded human information or intervention.

In 2017 AlphaZero was released without any human knowledge or data, and within four hours was able to defeat all other chess players and machines, it also mastered Go and Shogi.  Thus, taking a step toward a more general intelligence.

In 2019 we get MuZero which was able to master any board game or deterministic video game without ever being given any rules.

From that point on, the idea is to take this learning capacity and apply it to situations beyond games, and into complex real-world scenarios.

Self-driving cars have driven over 44 million miles, and the thought that a person can drive safer or better than an AI would be like thinking a person could beat an AI at a game of chess.  We’ll return to this thought in a minute.

Conquering games and rule-based scenarios like driving is one thing, but without language comprehension taking the next step in AI integration would be challenging.

Transformers use tokens and sets of parameters to make predictions.  E.G.  One parameter – Rope = ? Two parameters Rope + Harness = ? Three Parameters – Rope + Harness + Climbing Shoes = ? 

With these three parameters we get a decent idea that we’re probably talking about rock climbing. 

The more parameters the higher likelihood of getting a prediction right.

Now, imagine the detail in our predictions if we had 1,000 parameters or 1,000,000 parameters.

Larger Language Models LLM’s do this with ever growing parameters leading to ever growing accuracy.

2019 GPT-2 (OpenAI) had 1.5 billion parameters.

2020 GPT-3 (OpenAI) had 175 billion parameters.

2020 Gopher (DeepMind) had 280 billion parameters.

2021 Switch (Google) had 1.6 trillion parameters.

2021 Multimodality allowed developers to move from text-based language to multiple forms of data in both input and output including audio, pictures, and coding.  Moving to a more generalized intelligence.

2022 Gato (DeepMind) 1.2 billion parameters. It can master games, chat via text and imaging, and control a robot.  Another major step toward generalization.

2022 GPT-3.5 (OpenAI Chat GPT) 175 billion parameters.  First time the general public was able to interact with an LLM.  100 million users within two months.

2023 GPT-4 (OpenAI Chat GPT) Parameters not publicly disclosed but estimated to be around 1 trillion.  It has the ability to pass academic exams including SAT, LSAT, AP exams, and the bar exam.  It can reason about hypothetical situations as well as understand relationships between objects and actions.  It can envision situations from different perspectives and keep track of objects spatially over time.

2023 – PaLM-E (Google) 562 billion parameters.  Combines the reasoning of an LLM in the embodiment of a robot, i.e., this system can take natural language instructions and carry them out in a physical environment.

All interesting things, and you can see the level of progression in the last 10 years essentially a 10-billion-fold increase since 2010.  Given Moore’s Law, think about this doubling effect every two years, moving forward, until FOOM!

FOOM = Sudden explosion of intelligence.

Once an AI has enough programming skills to give it even more programming skills and based on the speed at which computers can operate, humans will be cut out of the loop.  A loop of progressive improvement at unimaginable speeds…FOOM!

While politicians are deeply concerned about the advancement of deep fakes and teachers are equally concerned about plagiarism, what we are witnessing today is just the tip if the iceberg.

The speed at which change is taking place is unheralded, unimaginable, and non-linear. Change can be scary and exciting, but most importantly, change is unavoidable. So, don’t get too comfortable, because in this case, we are all in for the ride of our lives!

The future has always been uncertain, but for the first time in human history, we now have the ability to appreciate just how uncertain that future is!

1. Stats for this piece were taking from The Singularity Is Nearer: When We Merge with AI by Ray Kurzweil
2. Cover image generated by AI interpretation of this article.