Activity Feed

AI & ML interests

None defined yet.

Recent Activity

AtAndDevย 
posted an update about 1 month ago
view post
Post
5,314
@Banaxi-Tech stop hiding my comments. AND STOP STEALING PAPERS AND SPREADING MISINFORMATION.
your BGA blog is a copy of NSA (deepseek, 2025) branded under your name. literally the same top16 selected blocks, 512 local window, router over block summaries, all you did was change block size from 64 to 128.
you didnt cite NSA once but you put a โ€œplease cite BGAโ€ bibtex at the bottom.
i commented under your post and said that there is no way that you can support claims like: โ€œThe Accuracy Should BE WAy better than DSA but untested yet.โ€ you didnt run a single experiment. and the 256x isnt from BGA, its just n/2k with k=2048 so the exact same k DSA uses. if opus wrote this for you, at least read it before posting.
i commented again after you hid my comment despite it having constructive and correct feedback and you hid that too. and again.
you can hide the truth and just try to get hf post likes..... but is it really the thing that needs to be done? do you really want to take papers and make them yours while barely even changing the params?

admitting your mistakes and doing something about them needs humbleness, intelligence, humanness.
i encourage you to admit your mistakes and try to do better next time (at least read what blog your ai wrote or do proper experiments to back your stuff up).
  • 49 replies
ยท
GGUFGuyย 
published a Space about 2 months ago
AtAndDevย 
posted an update 2 months ago
view post
Post
2,965
SPECK 2 IS ALREADY OUT: specklabs/Speck2-140M

Pretrained on 4x more tokens than the previous releases (20b vs 5b).
Instruct tuned versions are coming soon.
Very interesting models are coming soon too (hint: super long context).

Thanks for everyone supporting!
  • 3 replies
ยท
AtAndDevย 
posted an update 2 months ago
view post
Post
221
SPECK1.5 IS COMING SOON!
Same 5B token budget but much better corpus quality.

Also getting a ton of downloads, thanks for everyone downloading and liking <3

specklabs
AtAndDevย 
posted an update 3 months ago
view post
Post
167
NEW SPECK UPDATES:

Just hit #14 and #15 with out FIRST models on Open SLM Leaderboard. The models were trained on 5B tokens, while competing with similarly sized models trained on more than 6-20x the data.

A new base model Speck1.5-140M being trained right now on a higher quality corpus and will be released soon.
SpeckChat3 is coming very soon with 1 million samples, specifically designed to post train small base models.

Also, just to clarify stuff, we will NOT release anything that is NOT MIT licensed EVER. Openness is needed in small language research.

Thanks to everyone supporting the project, and stay tuned for new releases!
AtAndDevย 
posted an update 3 months ago
view post
Post
1,900
SPECK UPDATES:
1 New instruct model tuned on top of Speck1-140M: specklabs/Speck1-140M-Instruct
2 Instruction tuning datasets
2 GGUFs

Much more coming soon:
Speck1.1-140M-Instruct that is post trained on SpeckChat2 will be coming very soon
New base model Speck1.5-140M is coming with a much higher quality corpus

Thanks to everyone who is already supporting the project, and stay tuned for new releases!
  • 3 replies
ยท
AtAndDevย 
posted an update 3 months ago
view post
Post
2,136
FIRST SPECK MODEL RELEASED:
specklabs/Speck1-140M

new models coming very soon (both instruct and much better models), with much much higher training scale as i am getting marenostrum5 access soon!
we will be looking at 100b-2t token budgets :)
  • 4 replies
ยท
s3nhย 
posted an update 4 months ago
view post
Post
575
Uncensoring Mistral,

give it a try
s3nh/Ministral-3-14B-Instruct-2512-BF16-abliterated
abidlabsย 
posted an update 5 months ago
s3nhย 
posted an update 5 months ago
view post
Post
613
Existing methods โ€” GPTQ, AWQ, llama.cpp's k-quants โ€” minimize empirical loss heuristically. None of them prove they are optimal in any information-theoretic sense. ICRB-Q builds a quantization scheme that is provably optimal via the Cramรฉr-Rao lower bound (CRB): no unbiased estimator of a weight can have lower variance than [F(ฮธ)]โปยน, where F is the Fisher information matrix.
  • 1 reply
ยท
hystsย 
in Gradio-Blocks/ViTPose 12 months ago

runtime error

2
#10 opened 12 months ago by
liangnanying
abidlabsย 
posted an update about 1 year ago
view post
Post
11,631
Why I think local, open-source models will eventually win.

The most useful AI applications are moving toward multi-turn agentic behavior: systems that take hundreds or even thousands of iterative steps to complete a task, e.g. Claude Code, computer-control agents that click, type, and test repeatedly.

In these cases, the power of the model is not how smart it is per token, but in how quickly it can interact with its environment and tools across many steps. In that regime, model quality becomes secondary to latency.

An open-source model that can call tools quickly, check that the right thing was clicked, or verify that a code change actually passes tests can easily outperform a slightly โ€œsmarterโ€ closed model that has to make remote API calls for every move.

Eventually, the balance tips: it becomes impractical for an agent to rely on remote inference for every micro-action. Just as no one would tolerate a keyboard that required a network request per keystroke, users wonโ€™t accept agent workflows bottlenecked by latency. All devices will ship with local, open-source models that are โ€œgood enoughโ€ and the expectation will shift toward everything running locally. Itโ€™ll happen sooner than most people think.
  • 8 replies
ยท
Norod78ย 
posted an update about 1 year ago
view post
Post
2,225
Multilingual Tokenization Showdown
Analyzing 12 LLM Tokenizers Across 204 Languages.

First, I've created a dataset with Wikipedia's "Cat" article text in 272 languages:
Norod78/WikiCat-Multilingual

For each language entry with at least 100 words, I tokenized the text using 12 tokenizers and calculated the "Characters per token" ratio and "Word per token" ratio. The higher this ratio is, the more information each token represents on average for that language (and perhaps allowing the llm to potentially learn more per-parameter if trained on a dataset of that language).

You can see a slideshow summary of the results here:
https://norod.github.io/wikicat-tokenizer-eval/tokenizer-slideshow.html

I hope I interpreted the results correctly, I've made the code available on GitHub so you can re-create the raw results jsonl with this repo:
https://github.com/Norod/wikicat-tokenizer-eval

Post on X:
https://x.com/Norod78/status/1984366900550266999

nouamanetaziย 
posted an update about 1 year ago
view post
Post
5,446
After training ๐’๐ฆ๐จ๐ฅ๐‹๐Œ๐Ÿ‘ on ๐Ÿ‘๐Ÿ–๐Ÿ’ ๐‡๐Ÿ๐ŸŽ๐ŸŽ๐ฌ for nearly a month, I've come to realize something most people overlook: ๐ข๐ง๐Ÿ๐ซ๐š๐ฌ๐ญ๐ซ๐ฎ๐œ๐ญ๐ฎ๐ซ๐ž ๐ข๐ฌ ๐ญ๐ก๐ž ๐ฆ๐š๐ค๐ž-๐จ๐ซ-๐›๐ซ๐ž๐š๐ค ๐Ÿ๐š๐œ๐ญ๐จ๐ซ ๐ข๐ง ๐‹๐‹๐Œ ๐ญ๐ซ๐š๐ข๐ง๐ข๐ง๐ . ๐Ÿ”ฅ

Everyone talks about model architecture and data quality. And yes, those matter immensely. But here's what nobody tells you: when your training run fails at 2 AM because of mysterious ๐๐‚๐‚๐‹ ๐ž๐ซ๐ซ๐จ๐ซ๐ฌ, or when your expensive GPU cluster is running at ๐Ÿ”๐ŸŽ% ๐ž๐Ÿ๐Ÿ๐ข๐œ๐ข๐ž๐ง๐œ๐ฒ, the problem isn't your model. It's most probably a ๐ฆ๐ข๐ฌ๐ฎ๐ฌ๐ž ๐จ๐Ÿ ๐ญ๐ก๐ž ๐ก๐š๐ซ๐๐ฐ๐š๐ซ๐ž. ๐Ÿ› ๏ธ

Questions that seemed simple but had no clear answers: Why is ๐Œ๐จ๐„ ๐ญ๐ซ๐š๐ข๐ง๐ข๐ง๐  ๐ฌ๐ฅ๐จ๐ฐ๐ž๐ซ ๐ญ๐ก๐š๐ง ๐๐ž๐ง๐ฌ๐ž ๐ฆ๐จ๐๐ž๐ฅ๐ฌ? Which ๐๐‚๐‚๐‹ ๐Ÿ๐ฅ๐š๐ ๐ฌ should we actually set? How often should we checkpoint without killing throughput?

That's why we built ๐“๐ก๐ž ๐’๐ฆ๐จ๐ฅ ๐“๐ซ๐š๐ข๐ง๐ข๐ง๐  ๐๐ฅ๐š๐ฒ๐›๐จ๐จ๐ค ๐Ÿ“–: a complete guide covering everything from model architecture and data curation to the SmolLM3 training marathon, post-training techniques, and crucially, the ๐ข๐ง๐Ÿ๐ซ๐š๐ฌ๐ญ๐ซ๐ฎ๐œ๐ญ๐ฎ๐ซ๐ž ๐ฅ๐š๐ฒ๐ž๐ซ that most teams get wrong.

We validated real vs theoretical bandwidth across the entire stack: ๐‡๐๐Œ๐Ÿ‘ ๐ก๐ข๐ญ๐ญ๐ข๐ง๐  ๐Ÿ‘ ๐“๐/๐ฌ, ๐๐•๐‹๐ข๐ง๐ค ๐Ÿ’.๐ŸŽ ๐ซ๐ž๐š๐œ๐ก๐ข๐ง๐  ๐Ÿ•๐Ÿ–๐Ÿ” ๐†๐/๐ฌ, ๐๐‚๐ˆ๐ž ๐†๐ž๐ง๐Ÿ’ ๐š๐ญ ๐Ÿ๐Ÿ’.๐Ÿ ๐†๐/๐ฌ. Then we ran collective operations across ๐Ÿ๐Ÿ๐Ÿ– ๐†๐๐”๐ฌ (16 nodes, 8xH100s each) and measured how performance degrades at scale: all-reduce drops from ๐Ÿ’๐Ÿ–๐ŸŽ ๐†๐/๐ฌ on a single node to ๐Ÿ‘๐Ÿ๐ŸŽ-๐Ÿ‘๐Ÿ“๐ŸŽ ๐†๐/๐ฌ across 16 nodes.

If you've ever wondered why your training runs are slower than they should be, or you're planning to scale up and want to avoid expensive mistakes, this guide might save you weeks of debugging.

๐“๐ก๐ž ๐’๐ฆ๐จ๐ฅ ๐“๐ซ๐š๐ข๐ง๐ข๐ง๐  ๐๐ฅ๐š๐ฒ๐›๐จ๐จ๐ค: https://lnkd.in/e5MKXUHS

Shared with โค๏ธ by the HuggingFace team
s3nhย 
posted an update about 1 year ago
view post
Post
952
Eduhelp with more empathy, based on model finetuned on
psychotheraputic preferences just landed on


Beck-8B as a base model, 13000 steps on educational dataset.
Time to go further and build more ๐Ÿฅฐ
s3nh/EduHelp_Beck_8B
Thanks to @basilic_ai for computations <3