EleutherAI, an AI research organization, has released what it claims is one of the largest collections of licensed and open-domain text for training AI models.
The dataset, called The Common Pile v0.1, took around two years to complete in collaboration with AI startups Poolside, Hugging Face, and others, along with several academic institutions. Weighing in at 8 terabytes in size, The Common Pile v0.1 was used to train two new AI models from EleutherAI, Comma v0.1-1T and Comma v0.1-2T, that EleutherAI claims perform on par with models developed using unlicensed, copyrighted data.
AI companies, including OpenAI, are embroiled in lawsuits over their AI training practices, which rely on scraping the web — including copyrighted material like books and research journals — to build model training datasets. While some AI companies have licensing arrangements in place with certain content providers, most maintain that the U.S. legal doctrine of fair use shields them from liability in cases where they trained on copyrighted work without permission.
EleutherAI argues that these lawsuits have “drastically decreased” transparency from AI companies, which the organization says has harmed the broader AI research field by making it more difficult to understand how models work and what their flaws might be.
“[Copyright] lawsuits have not meaningfully changed data sourcing practices in [model] training, but they have drastically decreased the transparency companies engage in,” Stella Biderman, EleutherAI’s executive director, wrote in a blog post on Hugging Face early Friday. “Researchers at some companies we have spoken to have also specifically cited lawsuits as the reason why they’ve been unable to release the research they’re doing in highly data-centric areas.”
The Common Pile v0.1, which can be downloaded from Hugging Face’s AI dev platform and GitHub, was created in consultation with legal experts, and it draws on sources including 300,000 public domain books digitized by the Library of Congress and the Internet Archive. EleutherAI also used Whisper, OpenAI’s open-source speech-to-text model, to transcribe audio content.
EleutherAI claims Comma v0.1-1T and Comma v0.1-2T are evidence that the Common Pile v0.1 was curated carefully enough to enable developers to build models competitive with proprietary alternatives. According to EleutherAI, the models, both of which are 7 billion parameters in size and were trained on only a fraction of the Common Pile v0.1, rival models like Meta’s first Llama AI model on benchmarks for coding, image understanding, and math.
Parameters, sometimes referred to as weights, are the internal components of an AI model that guide its behavior and answers.
“In general, we think that the common idea that unlicensed text drives performance is unjustified,” Biderman wrote in her post. “As the amount of accessible openly licensed and public domain data grows, we can expect the quality of models trained on openly licensed content to improve.”
The Common Pile v0.1 appears to be in part an effort to right EleutherAI’s historical wrongs. Years ago, the company released The Pile, an open collection of training text that includes copyrighted material. AI companies have come under fire — and legal pressure — for using The Pile to train models.
EleutherAI is committing to releasing open datasets more frequently going forward in collaboration with its research and infrastructure partners.
Source link
#EleutherAI #releases #massive #training #dataset #licensed #open #domain #text
.PNG)
.PNG?ssl=1 "The Tovala Oven and Meal Kit Is Like a Robot Chef of Future PastA garlic-herb salmon with risotto was probably the best among the family meals I tried. The chopped asparagus was less than visually appealing when drizzled in garlic butter, but still tasty and a bit crisp. The salmon was tender and flaky. And the sweet pea risotto had no choice but to be delicious. There was so much cheese, butter, and lemon it was pretty much a concert of fats and acid.That chicken parm was likewise a mountain of cheese and salt. It reminded me, pleasantly, of countless family meals I had as a child in the 1980s: cheese-topped chicken, garlic bread, shells stuffed with ricotta and topped with even more cheese. The big difference is that there is simply no way my mother would have cooked this meal without a vegetable.Toval app via Matthew KorfhageAnd nutrition is where Toval runs aground a little. The nutritional notes on that chicken parm meal betray 2,300 milligrams of sodium per serving, pretty much the entire daily allowance for an adult human. This is also on par with comparable servings of Stouffer’s meat lasagna. The Tovala meal also carried about 10 times the cholesterol as Stouffer’s.Many other meals followed a similar pattern, loading up on fats and salt in order to make meals tasty. The net effect is that it’s a lot more like rich restaurant food than what most people prepare at home. Whether this is a good or a bad quality is up to you.Only one meal of the seven I tried failed utterly: I flagged a teriyaki chicken dinner to my editor as a possible cultural crime against Japan. The meal was sweet soy drenching pale and steaming chicken, with an implausible side of thick egg rolls and some loose, unseasoned broccoli. It felt like the “Japanese” food you’d get at a mall food court in the ’90s. But again, this was a rare major misstep.A more pernicious issue, in meals designed for the whole family, is the near-universal high-fat, cholesterol, and sodium content. Many with the income and inclination to eat hearty, low-effort meals like the ones from Tovala are either parents with children, or people in the retirement bracket. Each has their own reason to desire a little more nutrition, and less fat and salt.By the end of a couple of weeks of testing recipes, I’ll admit I felt a little relieved. I was grateful to feel my arteries slowly reopen. Tovala’s culinary model makes a lot of sense to me, as a smart way of splitting the difference between prepared meals and fresh food. And the company has proven it can cook well. It might be nice if they’d also cook a diet that felt more sustainable.Power up with unlimited access to WIRED. Get best-in-class reporting that’s too important to ignore. Includes unlimited digital access and exclusive subscriber-only content. Subscribe Today.#Tovala #Oven #Meal #Kit #Robot #Chef #Futurekitchen,food and drink,cooking,review,meal kits,shopping")
Post Comment