Hugging Face's logo
Joseph Jones's picture
🔄 In a Training Loop

Joseph Jones PRO

KlondikeDev
18 1 14
wwewtech's profile picture John6666's profile picture jaithree's profile picture
·
https://oc.kunix.org/

AI & ML interests

Restoring the Digital Millennium. https://kunix.org/digital-millennium.html

Recent Activity

repliedto CodeSoft's post 1 day ago
Over the past week or so, I've been working on some models, those released being https://huggingface.co/CodeSoft/sorbet-25m and https://huggingface.co/CodeSoft/sorbet-v2-25m. In general, I'm a little confused because no matter what hyperparameters I change or datasets I add/remove, the benchmarks never move up. In a recent project, where I attached a TN-gram block to Sorbet-v2-25M, it still stayed the same on benchmarks despite the TN-gram clearly learning (due to the perplexity being lower with the TN-gram attached). When I changed the corpus to favor higher density text (the first paragraphs of Wikipedia articles and synthetic math), the benchmarks either stayed flat or went down. Does anyone have ideas on what I can do to improve my models? I'd really appreciate any feedback!
repliedto CodeSoft's post 1 day ago
Over the past week or so, I've been working on some models, those released being https://huggingface.co/CodeSoft/sorbet-25m and https://huggingface.co/CodeSoft/sorbet-v2-25m. In general, I'm a little confused because no matter what hyperparameters I change or datasets I add/remove, the benchmarks never move up. In a recent project, where I attached a TN-gram block to Sorbet-v2-25M, it still stayed the same on benchmarks despite the TN-gram clearly learning (due to the perplexity being lower with the TN-gram attached). When I changed the corpus to favor higher density text (the first paragraphs of Wikipedia articles and synthetic math), the benchmarks either stayed flat or went down. Does anyone have ideas on what I can do to improve my models? I'd really appreciate any feedback!
repliedto CodeSoft's post 1 day ago
Over the past week or so, I've been working on some models, those released being https://huggingface.co/CodeSoft/sorbet-25m and https://huggingface.co/CodeSoft/sorbet-v2-25m. In general, I'm a little confused because no matter what hyperparameters I change or datasets I add/remove, the benchmarks never move up. In a recent project, where I attached a TN-gram block to Sorbet-v2-25M, it still stayed the same on benchmarks despite the TN-gram clearly learning (due to the perplexity being lower with the TN-gram attached). When I changed the corpus to favor higher density text (the first paragraphs of Wikipedia articles and synthetic math), the benchmarks either stayed flat or went down. Does anyone have ideas on what I can do to improve my models? I'd really appreciate any feedback!
View all activity

Organizations

New Millennium Artificial Intelligence's profile picture OpenCerebral's profile picture Small Language Model Consortium's profile picture