LICENSE
Hi @Datdanboi25
I like your new license for the Open SLM Leaderboard but i don't think you should include this "Copy metrics for competitor leaderboards or similar products"
Its just benchmark scores and they normally get copied everywhere, its just a score i don't think copying a score should not be allowed because its just a number.
Its just my opinion though.
Hi Banaxi,
This restriction is primarily in place to stop people from completely copying, pasting and rehosting all of our benchmark results that we have spent many hours gathering through our internal harness (which as we've seen can happen). It is not intended to stop people referencing, quoting, discussing or using these metrics in non-competitor means (such as a model readme, a hf post, etc.) and only restricts the bulk reproduction or reuse of our collected results or ranking metrics for competing leaderboards or substantially similar products.
Hope that clears up the intent behind the wording.
Any other questions or anything I can do, just let me know!
@Datdanboi25 Saying you added this clause because of the "many hours gathering results through our internal harness" is an open legal confession that your license relies on the debunked "sweat of the brow" doctrine.
As an Australian lab, you should know that Australian copyright law completely rejects this:
High Court Precedent (IceTV v Nine Network [2009] HCA 14): The High Court of Australia unanimously abolished the "sweat of the brow" doctrine. Expending labor, time, or compute to collect factual data does not grant you copyright over those facts.
Federal Court Precedent (Telstra v Phone Directories [2010] FCAFC 149): The Full Federal Court confirmed that automated, computer-generated data compilations (such as running test scripts through an evaluation harness) lack human authorial originality and attract zero copyright protection.
OSI violation. Banning the reproduction of data specifically for "competitor leaderboards" directly violates Criterion 5 (No Discrimination Against Groups) and Criterion 6 (No Discrimination Against Fields of Endeavor) of the Open Source Definition.
You cannot use a proprietary license to claim copyright ownership over objective mathematical numbers generated by an automated harness. When even your own partners are publicly in your comments telling you that benchmark scores are just public numbers, it is time to drop the legal fiction.
Closing
We're not claiming copyright ownership over individual numerical facts. The restriction expresses the permitted use of Axiomic Labs' leaderboard materials and is intended to prevent wholesale replication of the service/data compilation, not ordinary citation of individual results.
Ignoring a large chunk of my point doesn't make the remaining argument more applicable. I explicitly distinguished between referencing individual benchmark scores and reproducing the leaderboard's compiled dataset for a competing service. The former is permitted; the latter is what the restriction is aimed at.
@Datdanboi25 Pivoting to "compilation protection" still doesn't work under Australian law.
Australia doesn't have an EU-style database right. Under Australian High Court precedent (IceTV and Telstra), organizing factual benchmark numbers into standard functional columns (model, params, ARC, PIQA) is purely utilitarian. Automated test runs lack human creative authorship, so a standard table of evaluation scores has zero copyright protection in Australia.
Banning "competing services" completely breaks the Open Source Definition. OSD Criterion 5 (No Discrimination Against Groups) and Criterion 6 (Fields of Endeavor) explicitly forbid licenses from restricting competitors or specific use cases. That clause alone makes this a closed, proprietary license by definition.
Trying to dress up "sweat of the brow" as a protected data compilation doesn't change the law. Factual benchmark numbers in a standard table cannot be monopolized, and barring competitors means the project simply is not open source.
Im confused as to why you keep quoting OSD when the leaderboard is not under a strictly open-source license? Also holy chat gpt response
Im confused as to why you keep quoting OSD when the leaderboard is not under a strictly open-source license? Also holy chat gpt response
LIke its bad if GPTZero detects it but
EVEN MY OWN 0.8B Model KNows its AI
@Datdanboi25 We quote the OSD because your Space is literally titled the "OPEN" SLM Leaderboard. Thank you for finally admitting on the record that your leaderboard is not open-source. You just conceded the entire reason this discussion exists.
@Banaxi-Tech Pasting Australian High Court citations into GPTZero and an 800M classifier because neither of you can dispute the actual law is the ultimate white flag. AI detectors flag structured legal and academic writing by default.
Deflecting to an AI detector when you run out of legal arguments doesn't change the outcome: your own maintainer just admitted the leaderboard is closed-source, proving the "Open" title is pure open-washing.
Open SLM leaderboard means A Leaderboard for Open Small Language Models.
Not a Open Leaderboard for Small Language Models.
@Datdanboi25 We quote the OSD because your Space is literally titled the "OPEN" SLM Leaderboard. Thank you for finally admitting on the record that your leaderboard is not open-source. You just conceded the entire reason this discussion exists.
@Banaxi-Tech Pasting Australian High Court citations into GPTZero and an 800M classifier because neither of you can dispute the actual law is the ultimate white flag. AI detectors flag structured legal and academic writing by default.
Deflecting to an AI detector when you run out of legal arguments doesn't change the outcome: your own maintainer just admitted the leaderboard is closed-source, proving the "Open" title is pure open-washing.
Holy semantics lol, as ive said before and ill say again, since you clearly dont remember "The Open tag is in reference to the models, not the license". Gotta increase your chat gpt repetition penalty haha
Doing grammatical gymnastics over adjective placement just to defend open washing a proprietary leaderboard is the ultimate cope.
Importantly I'm not a lawyer (and clearly neither are you) and I have little interest in debating chat gpt. Your arguments are based on assumptions (Australian Copyright being the only applying ruleset being one).
Also "gramatical gymnastics" is a funny term given it was your further incorrect assumption, wait till you hear about openrouter haha
@Datdanboi25 The pivot from making aggressive copyright infringement threats to "Importantly I'm not a lawyer and I don't want to debate" is the fastest retreat on hf.
Let's address your claims directly:
Axiomic Labs is an Australian entity, and Hugging Face is governed by US law. Under Australian High Court precedent (IceTV, Telstra), US Supreme Court precedent (Feist v. Rural), and international treaties (Berne / TRIPS), mathematical facts and automated test outputs are uncopyrightable everywhere on Earth. Which fictional jurisdiction did you think granted you copyright over benchmark scores?
OpenRouter is a commercial, for-profit API proxy that sells token routing to models. Bringing up a paid commercial proxy service to justify open-washing an evaluation leaderboard on Hugging Face is pure desperation.
In less than 48 hours, you went from threatening copyright takedowns, to panic relicensing your repository, to demanding model deletions, to having your own collaborators publicly break away from you, only to end with "I'm not a lawyer and wait till you hear about OpenRouter."
The debate is completely over. Enjoy managing your closed leaderboard while the community builds genuine open-source tooling.
HOLY CHATGPTTTT EXPLAINING WHAT OPENROUTER IS XDDDDDDD
HOLY CHATGPTTTT EXPLAINING WHAT OPENROUTER IS XDDDDDDD
Spamming "XD" and crying ChatGPT every time you get hit with undeniable facts is the exact reason nobody in this community takes your group seriously.
HOLY CHATGPTTTT EXPLAINING WHAT OPENROUTER IS XDDDDDDD
Spamming "XD" and crying ChatGPT every time you get hit with undeniable facts is the exact reason nobody in this community takes your group seriously.
Nobody takes your group seriously.
Again, ignoring half my point, but I guess I'll debunk you one more time.
Axiomic Labs is not an Australian incorporated entity. We aren't a registered company; we're a collection of contributors. Absent an assignment, employment relationship or joint ownership arrangement, the relevant IP rights generally belong to the individual creator, in this case me for the leaderboard. Writing “Australian based” in a README does not somehow make every applicable IP question exclusively subject to Australian law. (Incorrect assumption #1.)
HF having US-governed ToS also does not mean US copyright law magically governs every underlying IP right worldwide. The governing law of a platform contract and the territorial law governing an IP right are separate questions. (Incorrect assumption #2.)
And ordinary copyright is not the only relevant protection. I also qualify for sui generis database protection, which can protect against substantial or systematic extraction of database contents. (Incorrect assumption #3.)
At this point, you're repeatedly arguing against claims I haven't made and building the rest of your argument on assumptions about Axiomic Labs that aren't true. I don't think there's much value in continuing this discussion further. So peace!
PS I’d also appreciate it if you stopped using our leaderboard discussions to continue this argument or advertise your own leaderboard. You’ve made your position clear, and so have I. I think it’s best we leave things here and avoid further interaction going forward.
Dan
@Datdanboi25 @Banaxi-Tech To close this thread for good, here is our final response addressing your last set of claims:
Claiming you qualify for "sui generis database protection" is legally embarrassing. Under Article 11 of EU Directive 96/9/EC, sui generis database rights are strictly restricted to EU/EEA nationals or companies with their registered office inside the European Union. There is zero reciprocity for Australian, US, or non-EU creators. Furthermore, European Court of Justice precedent (British Horseracing Board v William Hill) established that generating data through tests and automated computational runs does not qualify for database protection even inside Europe.
Shifting from claiming Axiomic is an "Australian lab" to claiming you are an un-incorporated individual creator does not change anything. Under Australian law (IceTV, Telstra), US law (Feist v. Rural), and international copyright treaties (Berne / TRIPS), mathematical facts and automated test outputs cannot be copyrighted anywhere on Earth. There is no jurisdiction where an individual owns copyright over benchmark scores.
Banaxi posting a screenshot of 124 followers after getting mathematically, architecturally, and legally disproven across every single topic is the ultimate sign of defeat. Follower counts do not rewrite contract law, open-source definitions, or High Court precedents.
In less than 48 hours, you went from threatening copyright infringement, to panic-relicensing your space, to demanding model takedowns, to having your own collaborators publicly break away from you, to admitting your board is not open-source, and finally to citing European database directives that you legally cannot use while flexing 100 followers.
The debate is officially settled. This is our final response on this repository. Enjoy managing your closed space while the community builds genuine open-source research on the Neo leaderboard.
Did you just assume I don't qualify for EU protections? That seems like quite the thing to claim given you dont actually know anything about me lol.
But since this is my repo Ill send one final message to just debunk a few things for any one else reading this.
Once again, you're mischaracterising how the database was created. It was not simply an automated dump of benchmark outputs; a substantial portion was manually compiled, verified, normalised and entered. That distinction is directly relevant to whether database protection can apply.
So your argument relies on two assumptions you haven't established: that I am ineligible for the relevant protections, and that the database is merely automated test output. Neither is true nor something you could actually know.
At this point you're repeatedly declaring legal conclusions based on facts you've invented about me and the project, then declaring the debate “settled” on top of those assumptions. If thats the level of certainty you're working with, I think anyone reading this thread can judge for themselves how seriously to take the rest of it.
Anyway, enjoy ArithMark 3, my ranking index, and everything else you've found useful from my work. I just hope one day you decide to try creating some stuff of your own, it's a lot more enjoyable and fulfilling for the soul. Peace!