In recent years, theartificial intelligence (AI) has gained enormous popularity and importance in various sectors, from technology to finance, marketing to healthcare. However, understanding the mechanisms underlying artificial intelligence can always be a truly arduous task.
We were really impressed by the project”Spreadsheets-are-all-you-need” that is to say “spreadsheets are all you need“. For what? But obviously for learn artificial intelligence and the basics of modern functioning generative models.
The name of the initiative appears as a clear reference to the historic Google document “Attention is all you need” of 2017 which explained the functioning of transformers, essential for the development of LLMs (Large Language Models), and the meaning of was clarified Attentionmechanism that allows you to assign a weight different to the various parts of the sequence received as input during the training phase so as to generate more accurate outputs.
How and why to learn artificial intelligence with a spreadsheet
Spreadsheets-are-all-you-need is a project that aims to make theAI learning accessible to all, using one approach low-code (therefore without resorting to programming code) using a “familiar” environment such as that of an Excel spreadsheet. The reference point is the precedent GPT-2 model of OpenAI which is implemented, for educational and informative purposes, in Excel using the set of standard spreadsheet functions.
Through the use of a spreadsheet, you can clearly visualize every step of the natural language processing process. This makes it easier to understand the complex concepts that underlie the functioning of the most advanced modern AI.
Currently, Spreadsheets-are-all-you-need features some video lessons covering topics such as GPT-2 architecture, coding Byte Pair and tokenization. Additional video lessons will then later be added to explore the further details of the AI models.
The GitHub repository contains the spreadsheet, in its most updated version, useful for to do experiments and follow the lessons published on YouTube.
The image is taken from the video “Demystifying how GPT works: From Architecture to…Excel!?!“.
How GPT generative models work
This first video offers, as mentioned previously, a truly innovative point of view to understand the GPT operation (Generative Pre-trained Transformer) through the use of an Excel spreadsheet. A clear demonstration of how the foundations of modern artificial intelligence can be understood through simple tools.
We start with the concept of tokenization, i.e. the process that divides the text into tokens (portions often made up of multiple words) using a predefined dictionary. This results in a distribution between individual words or fragments of them, depending on the complexity of the text.
L’embedding allows you to map each token into an array of numbers: the latter captures the meaning and position of the token in the context. GPT-2 uses a embedding Of 768 numbers to represent each token. The model therefore exploits the aforementioned attention mechanism to determine the token relevance in context, facilitating the understanding of the relationships between words and, in some way, acquiring information on semantics.
Lo schedule Multilayer perceptron results in a phase charged with further processing the information, refining the “meaning” of the tokens and improving the prediction of subsequent text based on the context provided by the tokens and attention.
Finally, the last step consists in generating the next tokenselected from the most probable, to complete or extend the text currently being produced.
Byte Pair Encoding for artificial intelligence: what it consists of
The second video published on YouTube offers a detailed explanation of the concept Byte Pair Encoding (BPE) and its role in natural language processing and artificial intelligence models such as GPT. Through the use of a spreadsheet, the author demonstrates how words and texts are transformed into numbers that AI models can understand and process.
The “understanding” of language by AI models begins with breaking down the text in “morphemes”. Do not morpheme it is the smallest unit of meaning within a word. In linguisticsa morpheme can be a root (the core meaning of a word), a prefix (added to the root of a word to change its meaning), a suffix (added to the end of a word to change its meaning), or an infix ( inserted inside the root of a word to change its meaning).
Using a spreadsheet as a practical example, it is possible to see how a language model can be implemented starting from a simple text input up to the prediction of a token.
We have already said previously that a crucial step consists in converting text to numbers. While tokenization breaks text down into smaller words or units, BPE allows you to efficiently handle unknown or rare words by reducing the size of the vocabulary needed. The BPE-based approach breaks down words into smaller units that can be recognized by the model, highlighting the advantages over tokenizationbased solely on whole words or characters.
A few things to know about implementing a generative model in Excel
As explained in the third in-depth video, the Excel processing capabilities however, they are limited when you have to interface with a generative model.
The implemented model is necessarily limited to the management of small workloads, with a context limit of only 10 token and words limited to 10 characters. Furthermore, you can only perform tasks inferenceexcluding the possibility of training the model.
However, despite moving within these important limitations, the proposed exercise is confirmed as extremely valuable educational opportunityillustrating key AI concepts in an accessible and easy-to-understand approach.
Spreadsheets-are-all-you-need represents a unique opportunity for anyone who wants to approach the world of AI. Using a common tool like Excel, the project demonstrates that AI isn’t just for humans programming experts e data scientistbut it can be understood and used by anyone with a good understanding of spreadsheets.
Opening image credit: iStock.com – Digital43