I didn't realize that the llama license forbids you from using its outputs to train other models. That's essentially a dealbreaker, synthetic data is going to be the most important type of training data from here on out. Any model that prohibits use of synthetic data to train new models is crippled.
Also the main business model of Google (and of search engines in general) is to republish rearranged snippets of copyrighted content and even serve whole copies of the content (googleusercontent cache), without prior authorization of the copyright holders, and for-profit.
It’s completely illegal if you think about it.
So why LLMs who crawl the internet to present snippets and information should be treated differently from Google ? (who also reproduce verbatim the same content without paying any compensation to the copyright owners (all types: text, image, code)
Google would argue (and they won in federal court versus the Author's Guild using this argument) that displaying snippets of publicly-crawlable websites constitutes "fair use." Profitability weighs against fair use but it doesn't discount it outright.
They would also probably cite robots.txt as an easy and widely-accepted "opt-out" method.
Overall, I'm not sure any court would rule against Google's use of snippets for search. And since Google's been around for over 20 years and they haven't lost a lawsuit over it, I don't think it's accurate to say "it's completely illegal if you think about it."
US copyright law is one of those things that might seem simple, but really isn't. Hence many of the copyright lawsuits clogging our judicial system.
If I was a gambling person I would say that interpretation of fair use is going to fall in the next 20 years as there is just too much weight put on it currently, and AI is just going to make it untenable in its current form.
In addition, the fair use test contains a pillar about the use not affecting the market for the copyright holder's works[1] which I think in google's case (and probably in the current openAI case too) seems obviously not to have worked out (ie google's use has demonstrably negatively affected the market for the original copyrighted work in cases such as news for example).
> ie google's use has demonstrably negatively affected the market for the original copyrighted work in cases such as news for example
Most news sites wouldn't get any traffic without search engines and aggegrators. Which is why they are now whining about FB et al no longer sending them traffic.
And let's not forget that both traditional and online news is no stranger to republishing other people's content - one of the reasons fair use exists in the first place.
I have no love for big tech but let's not pretent that this is about anything other than news publishers wanting more gibs.
Well it's because judges are humans and humans are fallible. Humans also "like google" because it makes their life easier. It's hard to punish an entity you like.
The result of that is either that they wouldn't show snippets or that they would pass the cost on to you. And do you think they profit from showing the snippets of results that are not the result you want to click on?
Not wanting to defend the likes of Google, but search engines link the original source (in contrast to LLMs). Their basic idea is to direct people to your content. There are countries where content companies didn't like what Google does: Google took them out of the index -> suddenly they where ok with it again so that Google put them in again. (extremely simplified story)
> Their basic idea is to direct people to your content.
This is less and less true, as evidenced by the progression of 0-click searchs.
> There are countries where content companies didn't like what Google does: Google took them out of the index -> suddenly they where ok with it again so that Google put them in again.
I over-simplified. It's about Google News. The news paper companies managed to lobby for a law that requires search providers to pay money to the news papers they link to (or for the tiny excerpt they show in the search results). So Google said they will discontinue Google News in those countries. Suddenly the news papers gave Google a free license to link to them. (still simplified story)
Because search engines do not create mishmash of this data to parrot some stuff about it. Also they don’t strip the source, the license, and stop scraping my site when I tell them.
LLMs scrape my site and code, strip all identifying information and license, and provide/sell that to others for profit, without my consent.
There's a standard for excluding content from indexing via the Robots Exclusion Standard using robots.txt (sitewide) or the <noindex> HTML meta header. The robots.txt standard has existed for nearly 30 years, being first proposed in February 1994.[1]
Should a publisher wish to be excluded from Google's, or any other web index's search and presentation, that's easy enough to specify.
That's not how copyright law works at all. It doesn't say "well if you didn't want someone to copy this thing you should have stopped them from doing it". It lays out 4 factors for a court to consider about whether something is fair use and none of them are around how easy it was to rip the work off.[1]
In the LLM space it seems even more clear because many/most of the works in the various corpora used for this training have very clear copyright terms which prevent digital storage and reproduction without the publishers permission (just look at the reverse of the title page of any book for the copyright notice if you don't believe me).
Finally, for LLMs many/most of the works are in corpora[2] that people just download so they aren't looking at a robots.txt file put up by teh original site. If you look at The Pile paper[3] for example they explicitly say that much of the material is under copyright and that they are relying on fair use.
Most critically, courts have put strong emphasis on the notion of transformative use of copyrighted works, and web indexing is transformative in the sense that it does not create a competing work, but provides a means of discovering and assessing the relevance of the indexed work itself.
As to web indexing, that (and associated factors including thumbnails and caching) have been ruled by courts to be fair-use adaptations of works:
Displaying a cached website in search engine results is a fair use and not an infringement. A “cache” refers to the temporary storage of an archival copy—often a copy of an image of part or all of a website. With cached technology it is possible to search Web pages that the website owner has permanently removed from display. An attorney/author sued Google when the company’s cached search results provided end users with copies of copyrighted works. The court held that Google did not infringe. Important factors: Google was considered passive in the activity—users chose whether to view the cached link. In addition, Google had an implied license to cache Web pages since owners of websites have the ability to turn on or turn off the caching of their sites using tags and code. In this case, the attorney/author knew of this ability and failed to turn off caching, making his claim against Google appear to be manufactured. (Field v. Google Inc., 412 F.Supp.2d 1106 (D. Nev., 2006).)
Or, to use your phrase, by common law (precedential case law), that is precisely "how copyright law works". Note particularly that the courts leaned on publishers' capabilities to indicate whether or not caching was or was not permitted "using tags and code".
There's a larger issue which I'm not aware of being explicitly raised in case law, which concerns how the World Wide Web is indexed as contrasted to how a print library is indexed. In the case of a library, an independent third party (the library cataloguer) assigns metadata to a work (standardised title, author(s), translator(s), illustrator(s), publisher(s), etc., as well as subject headings and call numbers. Additional indexing is provided through citations indices (both forward and reverse --- works cited by, and citing, other works). These largely don't rely on the text of the indexed work itself, though of course the cataloguer presumably is reading at least portions of the work to classify it. Critically: the works themselves are physical artefacts of fixed form which are virtually always read directly rather than interpreted through some mechanism.[1]
As it's evolved over the past quarter century or so, Web search doesn't rely strongly on metadata (though some of this is taken into consideration), and most particularly publisher-provided keywords are almost wholly ignored, largely due to flagrant abuse of that feature by some publishers. Instead, a combined approach of full-text indexing (that is: capturing the full text of a work and identifying keywords and tuples (multi-word phrases) which can be matched against queries entered by persons searching for documents, and an assessment of the overall relevance of that work, usually at a site (or sub-site) level based on other indicia, most famously (though somewhat less relevantly today) "PageRank", Google's original site-ranking algorithm.
Further, the entire mechanism of the Web is of creating copies of works on request. When an HTTP request is sent, the server responds by copying the requested work to an output stream, which is then received (and duplicated, often multiple times) by the client system as an integral part of the utilisation of that content. US copyright law does not have a section specifically referring to computer-network transmission, but there are multiple limitations on exclusive rights to copy (by authors) above and beyond the 107 Fair Use exemptions in sections 108 through 122 of 17 U.S.C, including specifically ephemeral recordings (108) and the case of computer programmes (117).
Large language model training is a new area of use and law (legislative or common) is yet to be determined, but there's at the very least existing statutory language as well as precedent which suggest that at least some uses might well be found to be fair use. As I'm watching the situation evolve, I'm reminded strongly of several articles copyright scholar Pamela Samuelson wrote in the 1990s over adapting copyright to the Internet age, and questions of what its future place might be: specific governance over the literal copying of expressive works, or a general doctrine against misappropriation. As always, there's a sharp tension between authors' rights (and, let's be brutally honest: publishers' profits) and the underlying Constitutional justification of US copyright law: "To promote the Progress of Science and useful Arts".
(Discussion here strongly reliant on US law. There's general international agreement on copyright through the Berne Convention, though significant national differences exist.)
________________________________
Notes:
1. There is a spectrum of works, e.g., print books, phonographs, CDs and DVDs (the latter containing anti-circumvention mechanisms), etc., but in general there's minimal if any intermediate copying and duplication of works, and in many cases none at all.
I appreciate the detail in your reply. Do you think the recent Warhol "Orange Prince" case[1] gives an inkling into possible future court treatment of the question of "transformative" use for generative AI models? There Warhol's silk screen print of the original Prince photo was deemed not transformative enough as I understand it. One of things about the stochastic nature of generative AI is can be rather hard to notice when the model spits out something very close to the training material.
Google respects the "robot.txt" and asks you to use it to opt out of their crawling.
Parent's point is if your own scaping army respects the "scaping.txt" and goes down on Google as they don't opt-out in their scraping.txt, it probably wouldn't fly.
I don't understand. What does "Rules for thee but not for me" mean if "google is allowed to scrape" whatever people allows Google to scrape but "you’re not allowed to scrape google" because using the same rules google.com/robots.txt says
There's an imbalance because the robot.txt rule is something Google pushed forward (didn't invent it, but made it standard) and is opt-out. So yes, Google made up their rules and won't let other people to make up their own self-beneficial rules in a similar way.
> Google [...] won't let other people to make up their own self-beneficial rules in a similar way.
What "other people"?
If it's the "you" who is not allowed to scrape google in https://news.ycombinator.com/item?id=36817237 then you can make your own "google is not allowed to scrape my thing" rules if you think that's beneficial for you.
If it's somehow related to LLM providers or users I doubt that's what the original comment was referring to.
To be clear, I understand the original comment as
LLM companies say "I can use your content and you cannot not prevent me from doing so, but I won't allow you to use the output of the LLM" just like Google says "I can scrape your content and you cannot not prevent me from doing so, but I won't allow you to scrape the output of the search engine"
You should change "you cannot prevent me from doing so" into "you'll need to setup your ressources in the way that I defined if you don't want me to slurp them".
I see it as the equivalent of the spam mail that require the user to login to disable them.
The belief that makes them consistent is that the authors of a million Reddit posts have no way to assert their rights while the big company that trained a Redditor model does.
Yes, they have to pick one or the other. Until then I'm going to assume that the model licence doesn't apply since the first point would be invalid and the model could not be built in the first place.
Those are perfectly consistent, despite what ideologically-driven people may want to believe.
Copyright is literally the right to copy. Arbitrary Internet data that is not copied does not have any copyright implications.
The difference is that LLaMa imposes additional contractual obligations that, for ideological reasons (Freedom #0), open source software does not.
This issue reminds me of the FSF/AGPL situation. At some point you just have to accept that copyright law, in and of itself, is not sufficient to control what people do with your software. If you want to do that, you have to limit end-user freedom with an EULA.
If someone uses LLaMa output to train models, it is unlikely they will be sued for copyright infringement. It is far more likely they will be sued for breach of contract.
> Arbitrary Internet data that is not copied does not have any copyright implications.
Training a model on model output isn't copying.
There's no way to phrase this where training a model on copyrighted human-generated images/text isn't copying, but training a model on computer-generated images/text is copying.
> If you want to do that, you have to limit end-user freedom with an EULA.
If you want to limit end-user freedom with a EULA, you have to figure out how to get users to sign it. Copyright is one way to force them to do so, but doesn't really seem relevant to this situation if training a model on copyrighted material is fair use.
And again, if somebody generates a giant dataset with LLaMA, if you want to argue that pushing that into another LLM to train with is making a copy of that data, then there's no way to get around the implication there that training on a human-generated image is also making a copy of that image.
> There's no way to phrase this where training a model on copyrighted human-generated images/text isn't copying, but training a model on computer-generated images/text is copying.
Literally nobody is saying that.
> If you want to limit end-user freedom with a EULA, you have to figure out how to get users to sign it.
That is not true. ProCD v. Zeidenberg, 86 F.3d 1447 (7th Cir. 1996).
You and others seem to have an over-the-top hostile reaction to the idea that contract law can do things copyright law cannot do. But it is objective and unarguable fact.
Okay? Apologies for making that assumption. But if you're not saying that, then your position here is even less defensible. Arguing that model output isn't copyrightable but that it's still covered by EULA if anyone anywhere tries to use it is even more absurd than arguing that it's covered by copyright. The interpretation that this is covered by copyright is arguably the charitable interpretation of what you wrote.
> That is not true. ProCD v. Zeidenberg, 86 F.3d 1447 (7th Cir. 1996).
ProCD is about shrinkwrap licenses, the court determined that buying the software and installing it was the equivalent of agreeing to the license.
In no way does that imply that licenses are enforceable on people who never agreed to the licenses. The court expanded what counts as agreement, it does not mean you don't have to get people to agree to the EULA. I mean, take pedantic issue with the word "sign" if you want (sure, other types of agreement exist, you're correct), but the basic point is still true -- if you want to restrict people with a EULA, they need to actually agree to the EULA. All that ProCD did was establish that buying a product and opening the package and installing it constituted agreement.
And that becomes a problem because if you don't have IP law as a way to block access to your stuff, then you don't really have a way to force people to agree to the EULA. Someone using LLaMA output to train a model may have never been in a position to agree to that EULA, and Facebook doesn't have the legal ability to say "hey, nobody can use output without agreeing to this" because they don't have copyright over that output. Can they get people to sign a EULA before downloading the weights from them? Sure. Is that enough to restrict everyone else who didn't download those weights? No.
To go a step further, if you don't believe that weights themselves are copyrightable, then putting a EULA in front of them is even less effective because people can just download the weights from someone else other than Facebook.
You can host a project Gutenberg book and get people to sign a EULA before they download it from you, even though you don't own the copyright. And that EULA would be binding, yes. But you cannot host a project Gutenberg book, put a EULA in front of it, and then claim that people who don't download it from you and instead just grab it off of a mirror are still bound by that EULA.
Your ability to control access is what gives you the ability to force people to sign the EULA. And that's kind of dependent on IP law. If someone sticks the LLaMA 2.0 weights on a P2P site, and those weights aren't covered by copyright or other IP law, then no, under no interpretation of US law would downloading those weights from a 3rd-party source constitute an agreement with Facebook.
But even if you don't take that position, even if you assume that model weights are copyrightable, if I download a dataset generated by LLaMA, there is still no shrinkwrap license on that data.
To your original point:
> If someone uses LLaMa output to train models, it is unlikely they will be sued for copyright infringement. It is far more likely they will be sued for breach of contract.
It is incredibly unlikely that someone using a 3rd-party database of LLaMA output would be found to be in violation of contract law unless at the very least they had actually agreed to the contract by downloading LLaMA themselves. A restriction on the usage of LLaMA does not mean anything for someone who is using LLaMA output but has not taken any action that would imply agreement to that EULA.
> You and others seem to have an over-the-top hostile reaction to the idea that contract law can do things copyright law cannot do. But it is objective and unarguable fact.
No, what we have a hostile reaction to is the objectively false idea that a EULA covers unrelated 3rd parties. That's not a thing, it's never been a thing.
I don't know what to say if you disagree with that other than that I'm putting a EULA in front of all of Shakespeare's works that says you now have to pay me $20 before you use them no matter where you get them from, and apparently that's a thing you believe I can do?
My "position" is the law, whether you like it or not.
Clickwrap agreements are enforceable, and legally enforceable agreements can place more restrictions on the use of a piece of software than copyright law alone can.
As a result, software that, for ideological reasons, does not restrict use will always have fewer protections than software with more restrictive terms.
Your off-topic rant about Shakespeare is irrelevant.
> My "position" is the law, whether you like it or not.
> Clickwrap agreements are enforceable, and legally enforceable agreements can place more restrictions on the use of a piece of software than copyright law alone can.
To take a page from your earlier comment, literally no one here is denying the existence of clickwrap agreements. Clickwrap agreements are completely irrelevant to the current conversation.
> Your off-topic rant about Shakespeare is irrelevant.
You can not enforce a EULA on someone interacting with a piece of work you do not own IP rights to if they did not agree to that EULA in some way.
I'm sorry, but agreement is part of contract law.
If you think you can force a EULA on a piece of content you don't own that will bind people who got the content from a 3rd-party and who never agreed to your EULA under any legal definition of agreement, then by all means, slap a EULA on Shakespeare. It makes just as much sense as what you're suggesting.
>> If you want to limit end-user freedom with a EULA, you have to figure out how to get users to sign it.
> literally no one here is denying the existence of clickwrap agreements.
You denied the enforceability of clickwrap agreements. You were wrong.
LLaMA uses a clickwrap agreement. "By clicking 'I Accept' below or by using or distributing any portion or element of the
Llama Materials, you agree to be bound by this Agreement."
That agreement covers its output: "You will not use the Llama Materials or any output or results of the Llama Materials to improve any other large language model (excluding Llama 2 or derivative works thereof)."
Your hypotheticals about third parties are off-topic and have zero bearing on this conversation.
The topic under discussion is whether it is logically "inconsistent" for Meta to claim its output is protected while other content is not. Those two positions are perfectly consistent in light of the fact that LLaMA output is protected by the terms of a clickwrap agreement.
Facebook absolutely factually does not have a clickwrap agreement over 3rd-party content generated with LLaMA; restrictions of users do not magically mean that output has its own universally enforceable EULA applied to everyone else. There is no interpretation of US contract law that says that 3rd-party data generated with LLaMA would be subject to LLaMA's license. There is no clickwrap agreement over LLaMA's output, and no legal precedent that argues that any restriction of LLaMA's usage would apply to 3rd-parties accessing that output. The output is not protected in the way you claim, and I fully stand by the fact that a clickwrap agreement over downloading LLaMA from Facebook would not be enforceable over people who did not download LLaMA and are merely using 3rd-party LLaMA output.
It's all but certainly copied, and not just in the "held in memory" sense but actually stored along with the rest of the training collection. What may not happen is distribution. There's a difference in scale/nature of copyright violation between the two but both could well be construed that way.
Additionally, I think there's a reasonable argument that use as training data is a novel one that should be treated differently under the law. And if there's not:
> If you want to do that, you have to limit end-user freedom with an EULA.
What will eventually happen -- at least without some kind of worldwide convention -- is that someone who can successfully dodge licensing obligations will be able to take and redistribute weight-data and/or clean-room code.
At least, if we're adopting a "because we can" approach to everything related.
But you can publish the output, right? And then a “third party” could train a different model on just that published material without copying it or ever agreeing to a EULA.
If you believe that courts will find your shell game convincing, you are free to try it and incur the legal risk. I recommend you consult with an attorney before doing so.
One of the common elements of training sets for these models (including LLama) is the Books3 dataset, which is a huge number of pirated books from torrents. That's exactly what you described.
Regardless, the lack of a license cannot give you more permission than a restrictive license. You're arguing that if take a book out of a bookstore without paying (or signing a contract), then I have more rights than if I sign a contract and then leave with the book.
I don't see how this would be enforceable in law without killing almost every AI company on the market today.
The whole legal premise of these models is that training on copyrighted material is fair use. If it's not, then... I mean is Facebook trying to claim that including copyrighted material in a dataset isn't fair use regardless of the author's wishes? Because I have bad news for LLaMA then.
"You need permission to train on this" is an interesting legal stance for any AI company to take.
From my non-legal-professional POV I can see an angle which may work:
Firstly, llama is not just the weights, but also the code alongside it. The weights may or may not be copyrightable, but the code is (and possibly also the network structure itself? that would be important if true but I don't know if it would qualify).
Secondly, you can write what you want in a copyright license: you could write that the license becomes null and void if the licensee eats too much blue cheese if you want.
Following from that, if you were to train on the outputs of the AI, you may not be guilty of copyright infringement in terms of doing the training (both because AI output is not copyrightable in the first place, something which seems pretty set in precedent already, and possibly also because even if it was, it gets established that it is fair use like any other data), but if it means your license to the original code is revoked then you will at the very least need to find another implementation that can use the weights, or (if the weights can be copyrighted, which I would argue is probably not the case, if you follow the argument that the training is fair use, especially if the reasoning is that the weights are simply a collection of facts about the training data, but it's very plausible that courts will rule differently here).
This could wind up with some strange situations where someone generating output with the intent of using it for training could be prosecuted (or at least forced to cease and desist) but anyone actually using that output for training would be in the clear.
I agree it is extremely "have your cake and eat it" on the part of the AI companies: They wish to both bypass copyright and also benefit from the restrictions of it (or, in the case of OpenAI, build a moat by lobbying for restrictions on the creation and use of the models themselves, by playing to fears of AI danger).
> This could wind up with some strange situations where someone generating output with the intent of using it for training could be prosecuted (or at least forced to cease and desist) but anyone actually using that output for training would be in the clear.
I'll add to this that it's not just output; say that someone is using another service built on top of LLaMA. Facebook itself launched LLaMA 2.0 with a public-facing playground that doesn't require any license agreement or login to use.
You can go right now and use their public-facing portal and generate as much training data as you can before they IP-block you, and... as far as I can tell you haven't done anything in that scenario that I can see that would bind you to this license agreement.
So I still feel like I'll be surprised if any AI company that's serious about wanting bootstrapping itself off of LLaMA is going to be too concerned about this license (whether that's a good idea to do just because the training data itself might be garbage is another conversation). It just seems so easy to get around any restrictions.
The code is largely irrelevant - it's all simple enough that it can be easily replaced, and most current users of LLaMA only use the weights in practice.
NN design is more interesting, but I don't think we're at the point yet where they are sufficiently complex to be copyrightable in general. Patentable, maybe.
> Following from that, if you were to train on the outputs of the AI, you may not be guilty of copyright infringement in terms of doing the training (both because AI output is not copyrightable in the first place, something which seems pretty set in precedent already, and possibly also because even if it was, it gets established that it is fair use like any other data), but if it means your license to the original code is revoked
Majority of the time, the code and weights are under independent license terms-while in theory the code license could say it is revoked or revocable if you violate the terms of the weights licenses, I think such a license term is rare in practice.
It is quite common even when the weights are under a restricted license for the code to be released under a standard open source license, and no open source license contains such a license term (and it would probably make the license non-open source were it included)
> The whole legal premise of these models is that training on copyrighted material is fair use.
Not to diminish the conversation here, but not even a Supreme Court Justice knows what the legality is. You’d have to be a whole 9 person Supreme Court to make an accurate statement here. I don’t think anyone really knows how Congress meant today’s laws to work in this scenario.
> I don’t think anyone really knows how Congress meant today’s laws to work in this scenario.
Congress, or more accurate, the drafters of the Constitution, intended that Congress would work to keep the Constitution updated to match the needs of modern times. Instead, Congress ossified to the point it's unable to pass basic laws because a bunch of far right morons hold the House GQP hostage and an absurd amount of leverage was passed to the executive and the Supreme Court as a result - with the active aid of both parties by the way, who didn't even think of passing actual laws to codify something as important as equitable access to elections, fair elections, or the right to have an abortion or to smoke weed when they held majorities. And on top of that your Supreme Court and many Federal court picks were hand-selected from a society that prefers a literal viewpoint of the constitution.
But fear not, y'all are not alone in this kind of idiocy, just look at us Germans and how we're still running on fax machines.
I'd say it's enforceable in the sense that if you agree to the license then violating those terms would be breach of contract regardless of whether use of the LLaMA v2 output is protected by copyright or not. But there's nothing stopping someone else who didn't agree to the license from using output you generate with LLaMA v2 to train their model.
I don't want to dip too much into the conversation of whether weights themselves are copyrightable, but note that it's very easy in the case of LLaMA 1.0 to get the weights and play with them without ever signing a contract.
If they turn out to be not copyrightable, then... all this would mean is downloading LLaMA 2.0 weights from a mirror instead of from Facebook.
"Yes, we train our models on a good chunk of the internet without asking permission, but don't you dare train on our models' output without our permission!"
In fact they can't (both Facebook and OpenAI) train their models without asking permission. Just wait for someone to start raising this concern. The EU is working on regulating these kind of aspects, for example this is not compliant at all with the GDPR (unless you train only on data that doesn't contain personal data, that is more rare than you would think).
Disgruntled current or former employee turning in their employer for the reward? That’s how Microsoft and the BSA used to bust people before the days of always online software.
Level1Techs "link show" (because we can't call it news anymore) kind of touched this topic.
I would like to read what you guys make of this:
> Supreme Court rejects Genius lawsuit claiming Google stole song lyrics
SCOTUS won't overturn ruling that US copyright law preempts Genius' claim.
> The song lyrics website Genius' allegations that Google "stole" its work in violation of a contract will not be heard by the US Supreme Court. The top US court denied Genius' petition for certiorari in an order list issued today, leaving in place lower-court rulings that went in Google's favor.
> Genius previously lost rulings in US District Court for the Eastern District of New York and the US Court of Appeals for the 2nd Circuit. In August 2020, US District Judge Margo Brodie ruled that Genius' claim is preempted by the US Copyright Act. The appeals court upheld the ruling in March 2022.
> "Plaintiff's argument is, in essence, that it has created a derivative work of the original lyrics in applying its own labor and resources to transcribe the lyrics, and thus, retains some ownership over and has rights in the transcriptions distinct from the exclusive rights of the copyright owners... Plaintiff likely makes this argument without explicitly referring to the lyrics transcriptions as derivative works because the case law is clear that only the original copyright owner has exclusive rights to authorize derivative works," Brodie wrote in the August 2020 ruling.
> Google search results routinely display song lyrics via the service LyricFind. Genius alleged that LyricFind copied Genius transcriptions and licensed them to Google.
> Brodie found that Genius' claim must fail even if one accepts the argument that it "added a separate and distinct value to the lyrics by transcribing them such that the lyrics are essentially derivative works." Since Genius "does not allege that it received an assignment of the copyright owners' rights in the lyrics displayed on its website, Plaintiff's claim is preempted by the Copyright Act because, at its core, it is a claim that Defendants created an unauthorized reproduction of Plaintiff's derivative work, which is itself conduct that violates an exclusive right of the copyright owner under federal copyright law," Brodie wrote.
The basic idea is whether an unauthorised derivative work is itself entitled to copyright protection: could the creator of the derivative work prevent copying by the original creator (or anyone else) of the work on which it is based, even though they themselves have no permission to distribute it? (if the work is authorised, this is generally considered to be the case). It looks like from this the conclusion is 'no', at the very least in this case. I'm not sure this matches most people's moral intuitions: every now and again a big company includes some fan art in their own official release without permission (usually not as a result of a general policy, but because of someone getting lazy and the rest of the system failing to catch it), and generally speaking the reaction is negative.
> whether an unauthorised derivative work is itself entitled to copyright protection
That is not what this court case was about. Genius had already settled the case of unauthorised transcriptions and had bought licences for its lyrics after a lawsuit 2014, so its own work was no longer unauthorised. In the case cited above, Genius was trying to enforce its claims against Google via contract law rather than copyright law. The court ruled that the alleged violations were covered by copyright law, so they could only pursued via copyright law, and that only the copyright holder (or assignee) of the lyrics that were copied could sue Google under it.
as a layman, i imagine for someone at the scale required it may not be worth the risk or the added effort vs paying or using a different model but it'd be funny if we see companies creating a subsidiary that just acts as a web-passthrough to "legalize" llama2 output as training data
Not that it's okay for this to be in the license, but I'm curious: what is the use case for synthetic data? Most of the discussion I've seen has been about how to avoid accidentally using LLM-generated data.
I'm not sure why anyone would even do that in the first place, LLama doesn't generate synthetic data that would be even remotely good enough. Even GPT 3.5 and 4 are already very borderline for it, with lots of wrong and censored answers. And at best you make a model that's as good as LLama is, i.e. not very.
Instruction-tuning is the obvious use case. That much has nothing to do with subjectivity, alignment or censorship, it's will-you-actually-show-this-as-JSON-if-asked.
That's tuning llama which is allowed from what I understand.
Otherwise why release it at all, it's not very functional in its initial state anyway. What that applies to is using llama outputs to train a completely new base model which makes no practical sense.
As for generating jsons, that's more of a inference runtime thing, since you need to pick the top tokens that result a valid json instead of just hoping it returns something that can be parsed. On top of extensive tuning of course.
It's exactly the opposite. We have better ways to combine the knowledge of several models together than sampling them. (i.e. mixture of experts, model merges, etc) Relying on synthetic data from one LLM to train another LLM is in general a terrible idea and will lead to a race to the bottom.
> forbids you from using its outputs to train other models.
I don't know how one can even forbid this. As a human, I'm a walking neural net, and I train myself on everything that I see, without a choice. The only difference is I'm a carbon-based neural net.
I would just do it anyway. In fact, I can release a suitably laundered version and you'd never know. If I release a few million, each with slight variation, there's no way provenance can be established. And then we're home-free.
A contract ordinarily has to have consideration. Since LLaMa weights are not copyrightable by Meta and are freely available, what exactly is the consideration? The bandwidth they provide?
