This tech is not less than a year old. The “tech” being used is literally decades old, the specific implementations marketed as LLMs are 3 years old.
People hyping the technology are looking at the dollar signs that come when you convince a bunch of C-levels that you can solve the unsolvable problem, any day now. LLMs are not, and will never be, AGI.
Yeah, I have friend who was a stat major, he talks about how transformers are new and have novel ideas and implementations, but much of the work was held back by limited compute power, much of the math was worked out decades ago. Before AI or ML it was once called Statistical Learning, there were 2 or so other names as well which were use to rebrand the discipline (I believe for funding, don’t take my word for it).
It’s refreshing to see others talk about its history beyond the last few years. Sometimes I feel like history started yesterday.
Yeah, when I studied computer science 10 years ago most of the theory implemented in LLMs was already widely known, and the academic literature goes back to at least the early 90’s. Specific techniques may improve the performance of the algorithms, but they won’t fundamentally change their nature.
Obviously most people have none of this context, so they kind of fall for the narrative pushed by the media and the tech companies. They pretend this is totally different than anything seen before and they deliberately give a wink and a nudge toward sci-fi, blurring the lines between what they created and fictional AGIs. Of course they have only the most superficially similarity.
the first implementations go back to the 60s - the neural net approach was abandoned in the 80s because building a large network was impractical and it was unclear how to train anything beyond a simple perceptron. there hadn’t been much progress in decades. that changed in the early oughts, especially when combined with statistical methods. this bore fruit in the teens and gave rise to recent LLMs.
Oh, I didn’t scroll down far enough to see that someone else had pointed out how ridiculous it is to say “this technology” is less than a year old. Well, I think I’ll leave my other comment, but yours is better! It’s kind of shocking to me that so few people seem to know anything about the history of machine learning. I guess it gets in the way of the marketing speak to point out how dead easy the mathematics are and that people have been studying this shit for decades.
“AI” pisses me off so much. I tend to go off on people, even people in real life, when they act as though “AI” as it currently exists is anything more than a (pretty neat, granted) glorified equation solver.
“AI” pisses me off so much. I tend to go off on people, even people in real life, when they act as though “AI” as it currently exists is anything more than a (pretty neat, granted) glorified equation solver.
Me too. The LLM hype riders want a real artificial waifu to actually love them so very badly that they’re pleased to describe living beings in as crude and coarse reductionist language as possible so their treat printers feel closer to real to them. The pathology is fucking glaringly obvious most of the time, especially when the “meat” talk gets rolled around or that hologram waifu from Blade Runner is literally brought up as an example of why we’re all ignorant barbarians because fiction is real amirite?
Well, I think I’ll leave my other comment, but yours is better! It’s kind of shocking to me that so few people seem to know anything about the history of machine learning.
I could be wrong but could it not also be defined as glorified “brute force”? I assume the machine learning part is how to brute force better, but it seems like it’s the processing power to try and jam every conceivable puzzle piece into a empty slot until it’s acceptable? I mean I’m sure the engineering and tech behind it is fascinating and cool but at a basic level it’s as stupid as fuck, am I off base here?
no, it’s not brute forcing anything. they use a simplified model of the brain where neurons are reduced to an activation profile and synapses are reduced to weights. neural nets differ in how the neurons are wired to each other with synapses - the simplest models from the 60s only used connections in one direction, with layers of neurons in simple rows that connected solely to the next row. recent models are much more complex in the wiring. outputs are gathered at the end and the difference between the expected result and the output actually produced is used to update the weights. this gets complex when there isn’t an expected/correct result, so I’m simplifying.
the large amount of training data is used to avoid overtraining the model, where you get back exactly what you expect on the training set, but absolute garbage for everything else. LLMs don’t search the input data for a result - they can’t, they’re too small to encode the training data in that way. there’s genuinely some novel processing happening. it’s just not intelligence in any sense of the term. the people saying it is misunderstand the purpose and meaning of the Turing test.
It’s pretty crazy to me how 10 years ago when I was playing around with NLPs and was training some small neural nets nobody I was talking to knew anything about this stuff and few were actually interested. But now you see and hear about it everywhere, even on TV lol. It reminds me of how a lot of people today seem to think that NVidia invented ray tracing.
This tech is not less than a year old. The “tech” being used is literally decades old, the specific implementations marketed as LLMs are 3 years old.
People hyping the technology are looking at the dollar signs that come when you convince a bunch of C-levels that you can solve the unsolvable problem, any day now. LLMs are not, and will never be, AGI.
Yeah, I have friend who was a stat major, he talks about how transformers are new and have novel ideas and implementations, but much of the work was held back by limited compute power, much of the math was worked out decades ago. Before AI or ML it was once called Statistical Learning, there were 2 or so other names as well which were use to rebrand the discipline (I believe for funding, don’t take my word for it).
It’s refreshing to see others talk about its history beyond the last few years. Sometimes I feel like history started yesterday.
Yeah, when I studied computer science 10 years ago most of the theory implemented in LLMs was already widely known, and the academic literature goes back to at least the early 90’s. Specific techniques may improve the performance of the algorithms, but they won’t fundamentally change their nature.
Obviously most people have none of this context, so they kind of fall for the narrative pushed by the media and the tech companies. They pretend this is totally different than anything seen before and they deliberately give a wink and a nudge toward sci-fi, blurring the lines between what they created and fictional AGIs. Of course they have only the most superficially similarity.
the first implementations go back to the 60s - the neural net approach was abandoned in the 80s because building a large network was impractical and it was unclear how to train anything beyond a simple perceptron. there hadn’t been much progress in decades. that changed in the early oughts, especially when combined with statistical methods. this bore fruit in the teens and gave rise to recent LLMs.
Oh, I didn’t scroll down far enough to see that someone else had pointed out how ridiculous it is to say “this technology” is less than a year old. Well, I think I’ll leave my other comment, but yours is better! It’s kind of shocking to me that so few people seem to know anything about the history of machine learning. I guess it gets in the way of the marketing speak to point out how dead easy the mathematics are and that people have been studying this shit for decades.
“AI” pisses me off so much. I tend to go off on people, even people in real life, when they act as though “AI” as it currently exists is anything more than a (pretty neat, granted) glorified equation solver.
Me too. The LLM hype riders want a real artificial waifu to actually love them so very badly that they’re pleased to describe living beings in as crude and coarse reductionist language as possible so their treat printers feel closer to real to them. The pathology is fucking glaringly obvious most of the time, especially when the “meat” talk gets rolled around or that hologram waifu from Blade Runner is literally brought up as an example of why we’re all ignorant barbarians because fiction is real amirite?
“AI winter? What’s that?”
I could be wrong but could it not also be defined as glorified “brute force”? I assume the machine learning part is how to brute force better, but it seems like it’s the processing power to try and jam every conceivable puzzle piece into a empty slot until it’s acceptable? I mean I’m sure the engineering and tech behind it is fascinating and cool but at a basic level it’s as stupid as fuck, am I off base here?
no, it’s not brute forcing anything. they use a simplified model of the brain where neurons are reduced to an activation profile and synapses are reduced to weights. neural nets differ in how the neurons are wired to each other with synapses - the simplest models from the 60s only used connections in one direction, with layers of neurons in simple rows that connected solely to the next row. recent models are much more complex in the wiring. outputs are gathered at the end and the difference between the expected result and the output actually produced is used to update the weights. this gets complex when there isn’t an expected/correct result, so I’m simplifying.
the large amount of training data is used to avoid overtraining the model, where you get back exactly what you expect on the training set, but absolute garbage for everything else. LLMs don’t search the input data for a result - they can’t, they’re too small to encode the training data in that way. there’s genuinely some novel processing happening. it’s just not intelligence in any sense of the term. the people saying it is misunderstand the purpose and meaning of the Turing test.
It’s pretty crazy to me how 10 years ago when I was playing around with NLPs and was training some small neural nets nobody I was talking to knew anything about this stuff and few were actually interested. But now you see and hear about it everywhere, even on TV lol. It reminds me of how a lot of people today seem to think that NVidia invented ray tracing.
it’s honestly been infuriating, lol. I hate that it got commoditized and mystified like this.