The GNU Compiler Collection will reject copyright-significant contributions derived from LLM output, with limited exceptions for trivial changes and test cases.
Yeah, LLM code is fundamentally incompatible with FOSS licensing because you fundamentally cannot vet the codebases that it was trained from. The fact that this is ignored in most other places speaks to how well the LLM propaganda and astroturfing has worked.
I don’t think it is just propaganda. Most people (especially non coders) do not understand licensee and copyright, or Open Source and GPL at all. Or don’t care anyway. Sure there is some propaganda, especially from big tech companies, who don’t want to talk about the issues because they (think) to benefit from the tech. I just think its worse than “just” propaganda, with the previous mentioned issues.
The main confusing (or confused?) actor in the FOSS space in relation to this probably really is the Linux kernel.
I’ve seen so many projects reference significant LLM patches as acceptable “because the kernel does it”. It continues to blow my mind that the kernel would accept such a risk.
I’ve had similar trouble understanding this aspect in the past, so I’ll share how it now makes sense to me, within the frameworks of existing IP law. IANAL.
Copyright law protects a specific, tangible work from duplication, meaning that if someone directly derives an unauthorized copy from an original work (eg scanning a book), then that’s copyright infringement. However, copyright law does not simply look at whether a copy is identical to the original. There must be an analysis of the provenance of the copy: did the copy come from the original?
Two people can separately stand at the Grand Canyon visitor center and use their own cameras to take identical film photos from the same vantage point, such that the developed film is identical in every way. And yet, neither has violated each other’s copyright, because neither photo came from the other. Person A can sell their copy to a magazine, while Person B distributes copies for free with their holiday cards. This is all A-OK.
The trouble is when somebody suspects infringement, because they see a copy that looks unusual similar to an original. If they decide to raise a copyright lawsuit, then the defense would like a way to affirmatively show that “no, our copy didn’t come from your original”. How do they do this?
In software, the approach would be to methods like the clean room approach. This is explicitly a method for legal reverse engineering, precisely to avoid copyright concerns. It works because while copyright protects a tangible work, it does not protect the ideas recorded by that work. So if somebody was given only the idea, and tasked with writing the software that re-implements that idea but without ever seeing the original, then the resulting work cannot possibly have been based on the original work. Rarher, it’s just a take on the same idea, which is allowed to be similar, in the same way as the photo of the Grand Canyon. There is no straight line that connects the supposed copy to the original work, apart from the idea itself, which copyright does not cover.
With LLMs, there is zero assurance that any clean room approach was followed. By training directly on original, copyrighted works, there is a cloud of uncertainty about whether the LLM is regurgitating parts of the original (which would be infringement) or if it actually reduced the user’s prompt into an idea, and then independently generated code which implemented that idea.
Copyright infringement poisons everything that is derived thereafter, so even the spectre of a doubt causes problems, because if GNU or whoever else builds atop questionable copyrighted software, that foundation collapse suddenly if someone brings a lawsuit to enforce their copyright. And those are pricey to defend, even if it’s without merit.
The clean room method, when followed faithfully, is a way to discourage potential litigants from even filing a lawsuit, because it’s clearly obvious that they would fail to prove a connection to the original work. Patents are a different story (where ideas are protected).
LLMs basically have the same problem as when an engineer quits and goes to work for a competitor. Will they have (accidentally or not) taken trade secrets with them and implemented those in the competitor’s product? A well-advised company would go through great lengths to make sure that engineer isn’t working directly on a competing product, to avoid a costly lawsuit. LLMs do no such mitigation.
I expect that corporations will not sue for copyright infringement because that would set precedent that would make themselves susceptible to being sued for the same infringement. Bit contemporary copyright law looks like a trash fire to me, so I am in no position to predict anything
Corporations even before AI have sued each other, precisely on the premise that “your code looks like ours. We think you stole it” and then have a petty fight in court because their lawyers couldn’t hash out a settlement.
Now with so-called AI washing, it’s only going to get worse, because LLM generated code doesn’t have to be 100% identical to be considered a derivative copy of the original. It all goes back to the central question: did this copy come from this original work?
The corporate IP lawyers will be making bank off of the ambiguity.
We humans are not Ai. Don’t mix that up. In example just because you can write a script to scrape data from all Github, does not mean you can use the scraped data as you like without giving credit and respecting the license. But that is what Ai programs do. LLMs are not human and have not the same human rights.
I think the fundamental difference is that a human trained on something isn’t likely to reproduce the thing they were trained on exactly.
I know that some research I read a few years ago showed that LLMs generally produced novel text (rather than simply returning verbatim training material), but that that wasn’t always true, especially as the training sample got smaller, or a single training text got larger (something like a 1000 words).
So sure, for something simple like creating super standard CRUD or basic API endpoints in common languages, the sample size is significant enough that what you get is likely to be homogenized from thousands of samples available, and not likely to be an exact copy of someone else’s code. But the risk is that as the problem being solved gets more niche, and the training sample size gets smaller, you become more likely to get exact copies of someone else’s code, or something close enough that it might be considered plagiarism.
There’s also still grey areas where some courts in some areas have stated that LLM generated content can’t be copyrighted, since a human didn’t create it, creating issues with protecting sources with copyright.
I have non FOSS projects that my company writes / sells (I’ve written these for over 20 years, but only owned them for 3, and opening the existing source is complicated as it’s tangled deeply with non OSS projects).
I don’t allow LLM generated code in our products either, since I have to assume that most of training was done on public repos, and there’s no way to know what licenses existed on that code (in addition to all of the other valid arguments to not use that shit).
Yeah, LLM code is fundamentally incompatible with FOSS licensing because you fundamentally cannot vet the codebases that it was trained from. The fact that this is ignored in most other places speaks to how well the LLM propaganda and astroturfing has worked.
I don’t think it is just propaganda. Most people (especially non coders) do not understand licensee and copyright, or Open Source and GPL at all. Or don’t care anyway. Sure there is some propaganda, especially from big tech companies, who don’t want to talk about the issues because they (think) to benefit from the tech. I just think its worse than “just” propaganda, with the previous mentioned issues.
The main confusing (or confused?) actor in the FOSS space in relation to this probably really is the Linux kernel.
I’ve seen so many projects reference significant LLM patches as acceptable “because the kernel does it”. It continues to blow my mind that the kernel would accept such a risk.
I am far and away not an AI enthusiast but I don’t understand specifically the copyright issue with respect to “trained on x data”
Because we humans who contribute to FOSS are also generally trained on proprietary data (gaining experience with non-foss systems at companies).
I get the “who owns it” argument since the person didn’t create it.
Fuck LLMs for drying up groundwater and stealing electricity from families though.
I’ve had similar trouble understanding this aspect in the past, so I’ll share how it now makes sense to me, within the frameworks of existing IP law. IANAL.
Copyright law protects a specific, tangible work from duplication, meaning that if someone directly derives an unauthorized copy from an original work (eg scanning a book), then that’s copyright infringement. However, copyright law does not simply look at whether a copy is identical to the original. There must be an analysis of the provenance of the copy: did the copy come from the original?
Two people can separately stand at the Grand Canyon visitor center and use their own cameras to take identical film photos from the same vantage point, such that the developed film is identical in every way. And yet, neither has violated each other’s copyright, because neither photo came from the other. Person A can sell their copy to a magazine, while Person B distributes copies for free with their holiday cards. This is all A-OK.
The trouble is when somebody suspects infringement, because they see a copy that looks unusual similar to an original. If they decide to raise a copyright lawsuit, then the defense would like a way to affirmatively show that “no, our copy didn’t come from your original”. How do they do this?
In software, the approach would be to methods like the clean room approach. This is explicitly a method for legal reverse engineering, precisely to avoid copyright concerns. It works because while copyright protects a tangible work, it does not protect the ideas recorded by that work. So if somebody was given only the idea, and tasked with writing the software that re-implements that idea but without ever seeing the original, then the resulting work cannot possibly have been based on the original work. Rarher, it’s just a take on the same idea, which is allowed to be similar, in the same way as the photo of the Grand Canyon. There is no straight line that connects the supposed copy to the original work, apart from the idea itself, which copyright does not cover.
With LLMs, there is zero assurance that any clean room approach was followed. By training directly on original, copyrighted works, there is a cloud of uncertainty about whether the LLM is regurgitating parts of the original (which would be infringement) or if it actually reduced the user’s prompt into an idea, and then independently generated code which implemented that idea.
Copyright infringement poisons everything that is derived thereafter, so even the spectre of a doubt causes problems, because if GNU or whoever else builds atop questionable copyrighted software, that foundation collapse suddenly if someone brings a lawsuit to enforce their copyright. And those are pricey to defend, even if it’s without merit.
The clean room method, when followed faithfully, is a way to discourage potential litigants from even filing a lawsuit, because it’s clearly obvious that they would fail to prove a connection to the original work. Patents are a different story (where ideas are protected).
LLMs basically have the same problem as when an engineer quits and goes to work for a competitor. Will they have (accidentally or not) taken trade secrets with them and implemented those in the competitor’s product? A well-advised company would go through great lengths to make sure that engineer isn’t working directly on a competing product, to avoid a costly lawsuit. LLMs do no such mitigation.
I expect that corporations will not sue for copyright infringement because that would set precedent that would make themselves susceptible to being sued for the same infringement. Bit contemporary copyright law looks like a trash fire to me, so I am in no position to predict anything
Corporations even before AI have sued each other, precisely on the premise that “your code looks like ours. We think you stole it” and then have a petty fight in court because their lawyers couldn’t hash out a settlement.
Now with so-called AI washing, it’s only going to get worse, because LLM generated code doesn’t have to be 100% identical to be considered a derivative copy of the original. It all goes back to the central question: did this copy come from this original work?
The corporate IP lawyers will be making bank off of the ambiguity.
We humans are not Ai. Don’t mix that up. In example just because you can write a script to scrape data from all Github, does not mean you can use the scraped data as you like without giving credit and respecting the license. But that is what Ai programs do. LLMs are not human and have not the same human rights.
I think the fundamental difference is that a human trained on something isn’t likely to reproduce the thing they were trained on exactly.
I know that some research I read a few years ago showed that LLMs generally produced novel text (rather than simply returning verbatim training material), but that that wasn’t always true, especially as the training sample got smaller, or a single training text got larger (something like a 1000 words).
So sure, for something simple like creating super standard CRUD or basic API endpoints in common languages, the sample size is significant enough that what you get is likely to be homogenized from thousands of samples available, and not likely to be an exact copy of someone else’s code. But the risk is that as the problem being solved gets more niche, and the training sample size gets smaller, you become more likely to get exact copies of someone else’s code, or something close enough that it might be considered plagiarism.
There’s also still grey areas where some courts in some areas have stated that LLM generated content can’t be copyrighted, since a human didn’t create it, creating issues with protecting sources with copyright.
IMO proprietary code has this same risk.
I have non FOSS projects that my company writes / sells (I’ve written these for over 20 years, but only owned them for 3, and opening the existing source is complicated as it’s tangled deeply with non OSS projects).
I don’t allow LLM generated code in our products either, since I have to assume that most of training was done on public repos, and there’s no way to know what licenses existed on that code (in addition to all of the other valid arguments to not use that shit).