In a letter to the CEOs of OpenAI, Anthropic, and Meta, Sanders writes that almost every day brings a new story about how these companies are losing...
Those reports deserve independent scrutiny, particularly because some originate with interested companies.
This is where it all gets a little tinfoil hatty for me. The timeline is that huggingface detected a hack, shut down the hack, and reported it to the FBI. Days later, OpenAI discovered that their AI was the source of the attack. Are you implying that Huggingface and OpenAI were working together, and were willing to make a false FBI report for a PR stunt? That seems like a pretty extraordinary claim.
What remains inadequately established is the much broader claim that all frontier-model development constitutes one unified, winner-take-all arms race.
When it comes go AGI, the distinction between skeptics in the field and believers (because I don’t know a word that is an antonym of skeptic) in the field isn’t that this is race, it’s how long the race is going to be. Believers claim that AGI will be created within a decade, skeptics put it within 25 years. (These numbers are for example only; I haven’t kept up on the specific claims).
People that reject the possibility of AGI at all are, from my experience, using a definition of AGI that includes sentience or consciousness, but the accurate definition of AGI doesn’t mention these concepts at all.
My disbelief is that we’ll ever graduate from “potential emerging” to “actual” arms race.
Based on what?
I would support targeted agreements covering things such as autonomous nuclear-launch decisions, fully autonomous lethal targeting, attacks on civilian infrastructure and particular forms of autonomous cyber operation.
These things do not need a treaty. I would wholeheartedly support any legislation to this effect. I recall that one company, Anthropic I think, lost government contracts because it categorically refused to allow its AI to be used for public surveillance or weapon systems.
Which means the government wonted to do those things. Scary thought.
A treaty pausing “AI development” generally would be much harder to define and verify.
No more than if would be to police a country’s efforts to develop nuclear weapon technology.
An arms race creates an argument for maintaining capability. It does not prove that maximum aggregate speed is safer than targeted development, testing and regulation.
At the risk of being snarky, do you not understand what a race is?
I think I’ve assumed some shared knowledge that we may not have.
I focused on the current capability of AI in regard to hacking to demonstrate that it could be used offensively, but the danger that warrants a global pause in development isn’t strictly the ability to hack, though it is entwined with that concern.
The danger (many consider an existential danger) is creating an LLM that is better at developing LLMs than even the best humans. Once that happens, it becomes a positive feedback loop. If we allow it to help make a better version of itself (and I think we both agree that the incentives, both financial and strategic, would push them to do so) then we lose almost all control over AI.
Note that being better at improving LLMs is not AGI. That is to say that we can reach this point without arriving at anyone’s definition of AGI-- the artificial intelligence does not need to be general for this to happen.
Another assumption I made is that we both understood that no one on this planet really understands why LLMs work as they do. Chatting using natural language and receiving a coherent natural language answer is an emergent property of LLMs that we do not fully understand.
The “arms race” is based on the pretty solid assumption that AGI is possible using current transformer technology and that the first group to get to a place where their LLM is better than any human at improving LLMs will be the first to AGI. A side effect is that the skills that would make an LLM better at improving LLMs, also makes them better at finding and using exploits in software.
I’m not much concerned on whether people might consider the ability to improve itself as an “arm” and therefore comfortable calling this an arms race. The threat is arguably more dangerous to humanity than nuclear bombs could ever be.
That’s why we need a global treaty to halt AI development.
Are you familiar with AI 2027? As far as I know none of the people who worked on that are employed by AI companies. (Though I think some might have been in the past.)
There’s a really excellent third-party YouTube Video that I recommend, that walks through the narratives in the paper.
Those reports deserve independent scrutiny, particularly because some originate with interested companies.
This is where it all gets a little tinfoil hatty for me. The timeline is that huggingface detected a hack, shut down the hack, and reported it to the FBI. Days later, OpenAI discovered that their AI was the source of the attack.
I don’t know how to explain this without risk that I might be getting so pedantic that it comes off as patronizing. I’m not trying to, anyway.
HuggingFace detected an attack, took corrective action, and reported it. Days later, OpenAI released their statement. That’s typical behavior regardless of whether I’m right or not.
My point is that CEOs are financially incentivized. Look at what drives investment into AI companies — it’s a lot of hype. Hence the concern that we’re in an “AI bubble.” The “bubble” would be speculative investments based on hype.
My statements come from marrying the ideas that (1) CEOs are financially incentivized and (2) the value of their companies are current being driven by hype rather than demonstrated value. Therefore, what naturally leads is that you have CEOs with a financial incentive to maintain hype around the topic.
Are you implying that Huggingface and OpenAI were working together, and were willing to make a false FBI report for a PR stunt? That seems like a pretty extraordinary claim.
No, not at all. I’m implying (as I thought my prior questions made clear) that this could have been allowed to happen merely to demonstrate that it can happen. That might, for example, include having a “sandbox” (that isn’t properly air gapped) while doing “testing” (on penetration capabilities).
My argument could be inferred to suggest that OpenAI wouldn’t have cared if it were HuggingFace, Google, or some other actor.
This, to me, comes across much to the same tune as Mythos was being given by Anthropic. They touted its ability to find exploits in software, though it didn’t quite live up to scrutiny. The tout had an impact on stock valuation, though. Same with having Mythos pulled from foreign use for “national security” reasons, then only dropping Fable as the consumer facing version.
You can argue that these actions make sense when there’s an actual concern. Sure. My argument is that Anthropic, OpenAI, and others benefit from the concern, financially speaking. They want more concern. They want people to believe the shit they spew, because that maintains the value of their companies. They know that, which is why they frame these events as they do.
This isn’t a tinfoil hat conclusion. It’s a belief in uncoordinated, unethical behavior due to financial incentives. Furthermore, I’m pissed off that it caused so much more harm within communities than would be necessary. This is not an extraordinary claim. As I began in my earlier comment, it’s closer to “cultural stupidity.”
—-
I will have to give time to the rest of what you said. You make remarks insinuating that I dont know certain things, but honestly I in many cases would like to argue the point. It’s not that I “don’t know,” it’s that I believe differently than you do.
I’ll need to figure out how to respond without turning this into a Thomas Hobbes vs. John Bramhall situation.
This is where it all gets a little tinfoil hatty for me. The timeline is that huggingface detected a hack, shut down the hack, and reported it to the FBI. Days later, OpenAI discovered that their AI was the source of the attack. Are you implying that Huggingface and OpenAI were working together, and were willing to make a false FBI report for a PR stunt? That seems like a pretty extraordinary claim.
When it comes go AGI, the distinction between skeptics in the field and believers (because I don’t know a word that is an antonym of skeptic) in the field isn’t that this is race, it’s how long the race is going to be. Believers claim that AGI will be created within a decade, skeptics put it within 25 years. (These numbers are for example only; I haven’t kept up on the specific claims).
People that reject the possibility of AGI at all are, from my experience, using a definition of AGI that includes sentience or consciousness, but the accurate definition of AGI doesn’t mention these concepts at all.
Based on what?
These things do not need a treaty. I would wholeheartedly support any legislation to this effect. I recall that one company, Anthropic I think, lost government contracts because it categorically refused to allow its AI to be used for public surveillance or weapon systems.
Which means the government wonted to do those things. Scary thought.
No more than if would be to police a country’s efforts to develop nuclear weapon technology.
At the risk of being snarky, do you not understand what a race is?
I think I’ve assumed some shared knowledge that we may not have.
I focused on the current capability of AI in regard to hacking to demonstrate that it could be used offensively, but the danger that warrants a global pause in development isn’t strictly the ability to hack, though it is entwined with that concern.
The danger (many consider an existential danger) is creating an LLM that is better at developing LLMs than even the best humans. Once that happens, it becomes a positive feedback loop. If we allow it to help make a better version of itself (and I think we both agree that the incentives, both financial and strategic, would push them to do so) then we lose almost all control over AI.
Note that being better at improving LLMs is not AGI. That is to say that we can reach this point without arriving at anyone’s definition of AGI-- the artificial intelligence does not need to be general for this to happen.
Another assumption I made is that we both understood that no one on this planet really understands why LLMs work as they do. Chatting using natural language and receiving a coherent natural language answer is an emergent property of LLMs that we do not fully understand.
The “arms race” is based on the pretty solid assumption that AGI is possible using current transformer technology and that the first group to get to a place where their LLM is better than any human at improving LLMs will be the first to AGI. A side effect is that the skills that would make an LLM better at improving LLMs, also makes them better at finding and using exploits in software.
I’m not much concerned on whether people might consider the ability to improve itself as an “arm” and therefore comfortable calling this an arms race. The threat is arguably more dangerous to humanity than nuclear bombs could ever be.
That’s why we need a global treaty to halt AI development.
Are you familiar with AI 2027? As far as I know none of the people who worked on that are employed by AI companies. (Though I think some might have been in the past.)
There’s a really excellent third-party YouTube Video that I recommend, that walks through the narratives in the paper.
I don’t know how to explain this without risk that I might be getting so pedantic that it comes off as patronizing. I’m not trying to, anyway.
HuggingFace detected an attack, took corrective action, and reported it. Days later, OpenAI released their statement. That’s typical behavior regardless of whether I’m right or not.
My point is that CEOs are financially incentivized. Look at what drives investment into AI companies — it’s a lot of hype. Hence the concern that we’re in an “AI bubble.” The “bubble” would be speculative investments based on hype.
My statements come from marrying the ideas that (1) CEOs are financially incentivized and (2) the value of their companies are current being driven by hype rather than demonstrated value. Therefore, what naturally leads is that you have CEOs with a financial incentive to maintain hype around the topic.
No, not at all. I’m implying (as I thought my prior questions made clear) that this could have been allowed to happen merely to demonstrate that it can happen. That might, for example, include having a “sandbox” (that isn’t properly air gapped) while doing “testing” (on penetration capabilities).
My argument could be inferred to suggest that OpenAI wouldn’t have cared if it were HuggingFace, Google, or some other actor.
This, to me, comes across much to the same tune as Mythos was being given by Anthropic. They touted its ability to find exploits in software, though it didn’t quite live up to scrutiny. The tout had an impact on stock valuation, though. Same with having Mythos pulled from foreign use for “national security” reasons, then only dropping Fable as the consumer facing version.
You can argue that these actions make sense when there’s an actual concern. Sure. My argument is that Anthropic, OpenAI, and others benefit from the concern, financially speaking. They want more concern. They want people to believe the shit they spew, because that maintains the value of their companies. They know that, which is why they frame these events as they do.
This isn’t a tinfoil hat conclusion. It’s a belief in uncoordinated, unethical behavior due to financial incentives. Furthermore, I’m pissed off that it caused so much more harm within communities than would be necessary. This is not an extraordinary claim. As I began in my earlier comment, it’s closer to “cultural stupidity.”
—-
I will have to give time to the rest of what you said. You make remarks insinuating that I dont know certain things, but honestly I in many cases would like to argue the point. It’s not that I “don’t know,” it’s that I believe differently than you do.
I’ll need to figure out how to respond without turning this into a Thomas Hobbes vs. John Bramhall situation.
I will wait for your future response. Branching conversation threads quickly get out of hand. I’ll include the above comment in any reply.