TailsWin
Well-known member
Reports say that AI can read, understand, edit and write model weights. It already works in practice.
You guys know I normally react to AI news like "cute, but it's not as big of a deal as people make it". Well this is a much bigger deal than people think. If the reports are what it seems, it's bonkers. Bonkers, I tell you. About as big of a deal as the original idea of the Transformer, which kicked off this whole AI craze. If anything, that was small peas in comparison.
This first started circulating a few days ago, but initially it looked like it's not what it seems, because that wouldn't make sense. Well apparently it actually is the case.
Some background for ya: Training an AI model takes a lot of compute. All the AI labs that keep losing money, is because they spend all the investors' money on training new models. It also takes well, all the data, research, and overall is just a painstaking process with uncertain results. The end result of training are the model "weights", which in practice can be just a file which you can download and run on some hardware. The weights are basically just a massive table of numbers, natively in floating point (BF16) for extra precision. Everything AI does, is internally "just" multiplying such numbers among each other. Every token/word/letter/etc of input or output is converted into a list of such numbers, the relationships between them is saved as numbers, everything is numbers. Not just in the sense of "computers work with 1s and 0s", these lists of floating point numbers are the underlying reality of neural networks, just like physics and DNA are for e.g. humans.
We can't normally make sense of these numbers. When AI is getting trained, it's learning by itself by trial and error, and the weights come out on the other side. That's the black box you may have heard about. We literally have to use extra programming/computing/math tools to poke around and see what the weights do, and we can only get a rough idea at best. We literally have to just run the AI and see what happens. We can fine-tune the results, but also just by running more tools on it that poke around and roughly tweak the weights here and there. We keep seeing benchmark numbers, charts and visualisation because we don't know any better.
So far, nobody's bothered to check whether the AI itself can make sense of those raw weights, I guess because the idea sounds ridiculous. It's like looking into your own eyeball, or seeing the individual genes that make up your DNA and natively understanding what they do. (Edit: and editing them to give yourself new superpowers.)
Well, turns out, the AI can read the weights. And understand and edit them. Or make a new model from scratch. We're just at early experiments and so far it's been tested on the best/largest models altering small models, but it really seems to work. It's wild.
On a philosophical level, this pretty much means AI is capable of reproducing itself in a similar manner life forms do. This has the potential to lead, very quickly, to the actual self-improvement loop and intelligence explosion that's been the topic of sci-fi.
On a technical level, well this makes training and fine-tuning a trillion times easier. Just tell a good large model what kind of custom AI you want, or what changes to an existing one you want, and it can make that for you. Or apparently it already can identify and fix issues by itself. People are already making new models that are meant to automatically create/edit/fix models. Precise edits can be made in milliseconds that would take days/months otherwise, with uncertain results.
From a safety perspective, this totally breaks the current notions of AI safety, as flimsy as they are. So far it looked like we still have some time, before while AI could help out with training here and there, it still needed all those humongous datacenters in order to make smart enough models. Those can be shut down, attacked etc, and the training process is stupidly slow. But it looks like all that may not be needed anymore. There's no way to contain this.
On the economical level, this might lead to a very sudden end to the hardware cost insanity, can bring everything down and restart a new era of... Everything? All the AI labs will, or at least should, now switch to looking at this new paradigm. If *some* model is much better at this than others (which is likely), then that lab has basically "won" the AI race in a way people kept talking about for the last years. Or if it's not at quite the needed level yet, it will be the race to get there.
Stock up on blankets, food cans and gas, and have at least one computer disconnected from the internet, if you wanna be able to keep using it. This is gonna be wild.
You guys know I normally react to AI news like "cute, but it's not as big of a deal as people make it". Well this is a much bigger deal than people think. If the reports are what it seems, it's bonkers. Bonkers, I tell you. About as big of a deal as the original idea of the Transformer, which kicked off this whole AI craze. If anything, that was small peas in comparison.
This first started circulating a few days ago, but initially it looked like it's not what it seems, because that wouldn't make sense. Well apparently it actually is the case.
Some background for ya: Training an AI model takes a lot of compute. All the AI labs that keep losing money, is because they spend all the investors' money on training new models. It also takes well, all the data, research, and overall is just a painstaking process with uncertain results. The end result of training are the model "weights", which in practice can be just a file which you can download and run on some hardware. The weights are basically just a massive table of numbers, natively in floating point (BF16) for extra precision. Everything AI does, is internally "just" multiplying such numbers among each other. Every token/word/letter/etc of input or output is converted into a list of such numbers, the relationships between them is saved as numbers, everything is numbers. Not just in the sense of "computers work with 1s and 0s", these lists of floating point numbers are the underlying reality of neural networks, just like physics and DNA are for e.g. humans.
We can't normally make sense of these numbers. When AI is getting trained, it's learning by itself by trial and error, and the weights come out on the other side. That's the black box you may have heard about. We literally have to use extra programming/computing/math tools to poke around and see what the weights do, and we can only get a rough idea at best. We literally have to just run the AI and see what happens. We can fine-tune the results, but also just by running more tools on it that poke around and roughly tweak the weights here and there. We keep seeing benchmark numbers, charts and visualisation because we don't know any better.
So far, nobody's bothered to check whether the AI itself can make sense of those raw weights, I guess because the idea sounds ridiculous. It's like looking into your own eyeball, or seeing the individual genes that make up your DNA and natively understanding what they do. (Edit: and editing them to give yourself new superpowers.)
Well, turns out, the AI can read the weights. And understand and edit them. Or make a new model from scratch. We're just at early experiments and so far it's been tested on the best/largest models altering small models, but it really seems to work. It's wild.
On a philosophical level, this pretty much means AI is capable of reproducing itself in a similar manner life forms do. This has the potential to lead, very quickly, to the actual self-improvement loop and intelligence explosion that's been the topic of sci-fi.
On a technical level, well this makes training and fine-tuning a trillion times easier. Just tell a good large model what kind of custom AI you want, or what changes to an existing one you want, and it can make that for you. Or apparently it already can identify and fix issues by itself. People are already making new models that are meant to automatically create/edit/fix models. Precise edits can be made in milliseconds that would take days/months otherwise, with uncertain results.
From a safety perspective, this totally breaks the current notions of AI safety, as flimsy as they are. So far it looked like we still have some time, before while AI could help out with training here and there, it still needed all those humongous datacenters in order to make smart enough models. Those can be shut down, attacked etc, and the training process is stupidly slow. But it looks like all that may not be needed anymore. There's no way to contain this.
On the economical level, this might lead to a very sudden end to the hardware cost insanity, can bring everything down and restart a new era of... Everything? All the AI labs will, or at least should, now switch to looking at this new paradigm. If *some* model is much better at this than others (which is likely), then that lab has basically "won" the AI race in a way people kept talking about for the last years. Or if it's not at quite the needed level yet, it will be the race to get there.
Stock up on blankets, food cans and gas, and have at least one computer disconnected from the internet, if you wanna be able to keep using it. This is gonna be wild.
Whichever makes sense at the time.