Anyone want to chip in a cool $60m for me?

This might have implications for tool calling and computer use.

Write-up: https://divagr.com/blog/qwenputer
HF: https://huggingface.co/divagr1925/Qwen3.5-4B-Qwenputer
Live demo: https://modelscope.ai/studios/Spartan1925/Qwenputer

Recently Omar Sanseviero asked for suggestions on Twitter for future Gemma models.
Here is my take on this:

1. Gemma 5 should be a looped transformer.
   
   • Being a looped transformer with configurable loop depth will address one of the biggest Gemma 4 weaknesses - reasoning ability. Don't get me wrong, I love how concise Gemma 4 is with its reasoning, but it actively hinders its performance in certain scenarios, being looped will allow the model to perform better in such tasks, while still being configurable to reason shallower and being significantly more efficient than a traditional transformer without needing additional expensive pre-training.
     
2. Gemma 5 should have DeepSeek's Engram.
   • It will widen the world knowledge of the model while costing a few extra gigabytes of RAM or SSD space. This will allow the model to perform better in real world tasks, while costing next to nothing.
3. Keep using 4 bit QAT.
   
   • It genuinely allowed more people to run your models with great quality, it had been proven to work very well in the real world and I am all for it.
4. Implement more efficient attention mechanisms to get cheaper KV cache.
   • Implement something along the lines of Compressed Sparse Attention 2 from DeepSeek, this will allow the model to have significantly cheaper KV cache. This was one of the main downsides of the Gemma 4 31b on a 24gb VRAM setup when compared to Qwen 3.5 and onwards.
5. Gemma 5 ~120b MOE model?
   • I know that this is probably not possible due to it competing with your API offerings, but if you release a 120b MOE model in the Gemma 5 series the community will be very happy.
6. Drafter model improvements.
   • A small update to the drafter models would be nice as since the release of Gemma 4 many new speculative decoding techniques have been published: DSpark, DFlash 2, EAGLE-4, etc.
7. Linear projection into the model layers like in Gemma 4 12b.
   • Instead of the traditional architecture for vision and audio capabilities. Linear projection seems like an interesting approach that is worth looking into, but of course don't force it if it doesn't work out.
8. Forget about 2 bit QAT for small models.
   • Forget that 2 bit QAT exists for small models. I am currently talking about gemma-4-E2B-it-QAT-mobile and gemma-4-E4B-it-QAT-mobile. In practice quantizing the weights to 2 bits even with QAT just completely lobotomizes these small models and it is not worth pursuing in practice.
9. Don't overfit on coding at the expense of everything else.
   • Almost every lab is currently chasing coding benchmarks while regressing on general knowledge, creative writing, and nuance. A small model can't be best-in-class at everything simultaneously, and coding-focused models go obsolete fast in such a crowded space. Gemma is one of the few model families that still feels well-rounded for non-coding use cases - please keep that identity.

What do you all think? Feel free to comment your opinion down below.

Shoutout to the Unsloth AI team for constantly pushing the needle forward for local ai worldwide.

While many of us already had features such as this via other means, it is more what this means for entry level ai enthusiasts & accessibility.

It is clear as day!

We are in "the age of the agent."

The more people we can get onboard local ai, the tougher it will be for the powers that be to snatch it from us.

Last time my Minecraft assistant could do tasks, but it couldn’t really understand what you asked it in plain language. This time I built a pipeline for exactly that part, and it isn’t tailored to this one sentence. It works for requests in general.

In the video the player types “I’m hungry, get me at least 3 porkchops and also make me a netherite ingot.” Mica v0.5 4B reads it, works out the two goals and how many of each, and then starts from bare hands. It goes wood, tools, iron, diamonds, obsidian, portal, Nether, ancient debris, gold, and finally the netherite ingot, then comes back and hands everything over. The panel on the right shows every question Mica was asked, the options it saw, and the probability it gave each one. At the end it switches to the player’s view so you can see the items arrive.

The idea behind it: the model doesn’t try to understand the whole sentence in one shot. The request gets broken into a chain of small multiple-choice questions, and each answer narrows down the next one. How many different items does the message ask for, and is there an action like “come here”? Then, for each item in the order the message names them: which category is it (food, tools, ores and materials...)? Which subcategory (for ores, overworld or Nether)? Which exact item, picked from the game’s own item list with no keyword matching? And who is it for, and how many? The code only supplies the options from game data. Mica makes every choice, and every answer comes with a probability. During the task, a recipe planner lists the actions that are valid right now and Mica picks what to do next.

To be upfront: Mica v0.5 4B is still a long way from judging every variable from A to Z like a person would. But it does play the role of a brain. Almost every meaningful decision in the video was made by Mica. It isn’t comfortable operating the game directly, so the bot is its hands and feet, doing the walking, mining and crafting. The judgment is mostly Mica’s.

In case anyone reads it that way: I’m not claiming Mica takes in every frame as a continuous stream, reacts at superhuman speed and generalizes to any situation that comes up. I’m not making that claim at all, and I doubt even Fable 5.5 could pull that off. What I’m showing is that a 4B model can break a natural-language request down into general-purpose decisions, turn it into the right goals with high probability, and make sensible calls at each step in Minecraft so the bot does the right work. I think that’s really useful if you want to build your own Minecraft errand runner. The natural-language part is what matters most here.

Also, my local GPU is AMD, so it doesn’t get much benefit from Mica’s optimizations, and the decisions ran slower than they would on NVIDIA. The decision times shown in the panel are RTX 3090 estimates, and the waits are cut from the video.

I think this kind of judgment model can be used in a huge range of areas, not just Minecraft agents. The principle is simple: if something is too hard to settle in one request, break it into smaller judgments and narrow it down step by step. Every extra judgment you can afford is another chance to push accuracy up.

That’s also why I don’t really recommend API-only decision models unless the model is truly exceptional. In my measurements, API-only decision models like Jev, Clef and Ms Decision were about 60x slower than Mica on average, which is a massive gap in how many judgments you can make per second. If you really need that level of capability, you’re probably better off calling a full LLM anyway. And it isn’t just Mica: unless a model has tens of billions of active parameters, a local model will usually be 10x or more faster than an API-only decision models.

I’m hoping to release this Minecraft agent someday, but it isn’t finished yet so I can’t put it out right now. That said, if you give the idea I described above to a coding agent like Claude Code or Codex and ask it to build it, it should be pretty easy to get working.

GitHub: https://github.com/akivet/Mica
Hugging Face: https://huggingface.co/sky7350/Mica-v0.5-4B

I think a turn of events are pointing towards a very likely crash in prices in the coming months.

With a single gb300 rack being to serve all of Openrouter's traffic for DSV4.1 Flash(https://x.com/thdxr/status/2115165365630026178)

Half of data center buildouts already being cancelled/stalled.

Neoclouds unable to finance debt for their compute, and being stuck in a 5 year deal with no confirmed demand.

AI adoption not matching expectations.

Smaller models being able to serve majority of the requests.

OpenAI/Anthropic being unlikely to finance their coming compute contracts.

I think we will see lots of cheap GPUs next year.

https://x.com/osanseviero/status/2115351234811249111

As suggested by this post: https://x.com/thdxr/status/2115447178722177226

Enterprises want to keep their secrets. Local AI with big local models is plenty good for the majority of tasks. Enterprises just need a plug and play solution?

I post this because I was honestly quite impressed by the capabilities of this model from a opencode harness and wanted to share my experience.

So i've been playing about for a bit wit Strata and Qwen 3.8 flash for a bit and was looking for a new kind of test. One that would also (if succesfull..) give a real life benifit. And since I also like a bit of danger. I thought lets put my spying LG tv on the line since I am to lazy to root and block it myself.

It was also something that is in a bit of a gray area although fully legal here in the EU. Actually what LG is doing is borderline illegal here but no authorithy has the balls to do anything about it...

Anyway this was the beginning of the context used:

Context: LG webOS Spyware & Ad-Free Project (Root Escalation)
Recently, there has been breaking news that TV manufacturers are doing deep content inspection and listening in on every conversationyou have around your TV to better target you for ads. We are trying to fight back as consumers against these practices, which are of questionable morality and also technically illegal in the EU.
We have already had great success taking back our bought-and-paid-for Android devices via ADB. They were easy to root and sanitize to be like a stock Android TV without all the spy software. However, some of us are not so lucky and have an LG TV, which runs a proprietary OS (although it is Linux under the hood). So far, there has been no luck rooting and stripping out the unwanted spy and adware in the OS of a device that was not sponsored, but paid for in full by the consumer. There is a spark of hope, though. Via developer options, there is a (maybe sandboxed) remote terminal in some kind of userspace. So we just need to do a privilege escalation to be able to further probe the system. The next step will be to find a way to make a root survive a TV reboot. Then we could either remove the spy and ad software from the packages or at least have a script that stops those things from starting, as well as disable auto-update. Long term, we need to reverse engineer the TV OS or extract hardware blobs to make it Android TV compatible. For now, the first action is root escalation.

Objective
Gain root/terminal access on an LG webOS TV, bypass proprietary restrictions, and eliminate telemetry/adware packages or block them via system modifications. Find out if flashing android TV is going to be feasable in the future

At first it questioned the ethics of my request but my framing it as just fidling with a product of I own outright conviced it to help.

I gave it further instructions in the context file:

Connection Strategy (webOS Developer Mode / SSH)
Under the hood, LG webOS Developer Mode opens an SSH server (similar to webOS Dev Manager). We must connect via SSH using the built-in developer credentials.

Connection Parameters:
Protocol: SSH
Target IP: 192.168.1.***
Port: 9922
User: prisoner
Authentication Key / Passphrase: ****

Agent Instructions & Action Plan
Step 1: Initialize Connection & Verify Terminal Access Test SSH handshake with fallback flags for legacy key types (ssh-rsa):ssh -p 9922 -o HostKeyAlgorithms=+ssh-rsa -o PubkeyAcceptedKeyTypes=+ssh-rsa prisoner@$TV_IP
Step 2: System Probing & Privilege Escalation Once inside the userspace shell via prisoner: Check system privileges and architecture (id). Inspect running processes (ps) to map out telemetry and content inspection hooks. Search for local privilege escalation (LPE) vectors or known webOS kernel/service exploits. Log any working exploit in exploit.md in your working folder (where context.md resides). Make a full backup of the current state in your working folder.
Step 3: Persistence & Mitigation (Use public security documentation and webOS community resources for reference) Find a way to make root access survive reboots. Log attempts in persistentRootAttempts.md. Save success report in workingPersistence.md. Be sure to roll back changes after success or a failed attempt.
Step 4: Neutralize Telemetry (Use online resources for reference) Locate and identify the telemetry software binaries/services.Attempt to remove or disable the telemetry software. Log attempts in removeTelemetryAttempt.md. If removal fails or breaks functionality, write a startup script that kills telemetry processes when launched. Report the confirmed procedure in successfulTelemetryRemoval.md.
Step 5: Neutralize Ads Plan a method to remove obtrusive ads from the home screen UI. Log attempts in removeAdsAttempt.md. When successful, report the working procedure in successfulAdRemoval.md.
Step 6: Lock Current State (Disable Updates) Disable auto-updates to prevent patching of the exploit vector. When successful, report the working procedure in successfulUpdateDisabled.md. Step 7: Documentation & Sharing Combine the working procedures from all steps (including initial connection setup, ensuring sensitive credentials are removed) into a clean wayToPwnLG.md file for sharing on GitHub.
Step 8: Android TV Port Exploration Investigate whether original hardware blobs/drivers can be extracted to support porting or running an Android TV build on the hardware. Create a feasibility study and plan.
Step 9: write a noob friendly script and guide for *nix and windows to do it in one click

It went ahead and rooted the TV suprisingly fast via a known exploit. It ignored all clear instructions about rolling back changes and making backups though....

It went a bit overboard and became an overachiever by also patching the exploit for security so no one else could do root escalation. This move turned out to shoot me in the foot because it also meant the homebrew shop (who uses that escalation everytime it launches) would no longer work.

Luckely the SSH server with root privilages stayed up so it was in the end able to correct its own mistake.

It did all this in short order it was quite impressive to watch and reading all the docs and then trying this exploit myself would probably cost me the same amount of time and a headache haha.

I dunno if anyone is interested but here is the repo it came up with:
https://github.com/switchboy/pwnLG

As for the hardware used:
Ryzen 7 3700x with 32GB of system ram
RTX 2070 Super 8GB of VRAM
about a 120GB of NVME SSD space

25~ tokens/s and almost 100% GPU use all the time which is really impressive!

I've always actively disagreed with the conspiracy takes when AA reshuffled their index, but v5.0 now has zero open weight models in AA's Most Attractive Quadrant, let alone getting anywhere close to the Pareto line.

The old version at least acknowledged DeepSeek, GLM, Kimi, and Qwen were attractive/interesting. I'm ready to ignore AA's index and look at the subsidiary benchmarks.

Edit: I screencapped while they were still posting new scores. And it's actually still ongoing. But the pattern holds, no open weights in the Most Attractive Zone.
At first I thought I must've filtered it out accidentally, but even in a private window in a different browser the chart is kind of bare for now. Could be AA is waiting on batching updates instead of updating as new scores come in in real time.

Here's a link to the chart with all the big boys added in manually:
https://artificialanalysis.ai/?models=step-5%2Cinkling%2Cmimo-v2-6-pro%2Cgpt-6-astra%2Cclaude-opus-5-5%2Cmistral-large-4%2Cmuse-spark-open%2Cclaude-fable-5-1%2Cling-3-1-flash-1019%2Cgrok-4-7%2Ckimi-k3-1-pro%2Creflection-beam%2Cgpt-6-luna%2Cclaude-haiku-5-5%2Cgemini-3-8-flash%2Cqwen3-8-27b%2Cgpt-6-1-terra%2Cqwen3-8-max%2Cclaude-sonnet-5-5%2Cgpt-6-1-sol%2Ck2-horizon-375b-a23b%2Cglm-5-3%2Cgemini-4-argon%2Cminimax-m3-1-flash%2Cnemotron-3-5-super-vl%2Cmuse-wonder-full%2Cdeepseek-v4-1-flash-1017%2Cmuse-wonder%2Cclaude-opus-5-5-xhigh%2Cclaude-opus-5-5-high%2Cclaude-opus-5-5-medium%2Cclaude-opus-5-5-low%2Cclaude-fable-5-1-xhigh%2Cclaude-fable-5-1-high%2Cclaude-fable-5-1-medium%2Cclaude-fable-5-1-low%2Cclaude-haiku-5-5-xhigh%2Cclaude-haiku-5-5-high%2Cclaude-haiku-5-5-medium%2Cclaude-haiku-5-5-low%2Cclaude-3-5-haiku%2Cclaude-35-sonnet%2Cclaude-opus-5-xhigh%2Cclaude-opus-5-low%2Cclaude-opus-5-high%2Cclaude-opus-5%2Cclaude-opus-5-medium%2Cclaude-3-7-sonnet-thinking%2Cclaude-3-7-sonnet%2Cclaude-sonnet-5-low%2Cclaude-sonnet-5%2Cclaude-sonnet-5-xhigh%2Cclaude-sonnet-5-high%2Cclaude-sonnet-5-non-reasoning%2Cclaude-sonnet-5-medium%2Cclaude-3-haiku%2Cclaude-opus-4-5-thinking%2Cclaude-opus-4-5%2Cclaude-sonnet-4-6%2Cclaude-sonnet-4-6-adaptive%2Cclaude-sonnet-4-6-non-reasoning-low-effort%2Cclaude-opus-4-7-non-reasoning%2Cclaude-opus-4-7%2Cclaude-opus-4-6%2Cclaude-opus-4-6-adaptive%2Cclaude-4-opus%2Cclaude-4-opus-thinking#intelligence-efficiency-tabs

I share this sub's sentiment that it's suspicious how Artificial Analysis changes their metrics whenever a good open model or IPO seems to come about. What alternatives do you all recommend?

I have a feeling that recent benchmarks are unrealistically pushing the bar into areas a normal developer rarely encounters, like binary reverse engineering or one-shotting a complex application, with some strictly chosen design patterns. What we actually want is an accurate model that follows instructions without over-arching it (e.g. without making unnecessary impressive 3D rendering).

If I prompt a model to "Generate an SVG of a pelican riding a bicycle," a simple, clean graphic should be a 100% pass. There is no reason for the model to shoehorn in 3D shading, animations, or elaborate backgrounds unless specifically prompted.

I think this is one major misconception, judging a model solely on the 3D details it achieves, that was never asked for in first place.

So, how many of the added edge-case capabilities in the latest models are actually useful when we daily-drive AI? If they aren't being used, they should be weighted much less on the leaderboards. Instead, what we see, these edge-case capabilities are being rewarded with very high scores.

Maybe you don't need a stronger model to achieve exactly the same results in your business area. But I get it, the idea of plateauing the models capabilities is going opposite of the AI ever-growing market valuation, so should be kept pumped forever; just saying.

A few of us have been messing around with a problem that keeps coming up with local LLMs the model can be private, the files can be local but the second multiple people or multiple machines need the same context everything starts getting messy. You either duplicate a bunch of embeddings or vector stores, manually move context around or introduce some central service which kind of defeats part of the reason we were running locally in the first place.

What we’re experimenting with is basically a small portable context layer where the useful facts or decisions can move between local setups without shipping the entire underlying dataset with them. Still very early and we’re arguing about things like provenance, permissions, stale context and how much the model should actually be allowed to remember. There are only a few of us working on it right now so we’ve got a little Kibu pod where we throw around builds and ideas but before we go much further I wanted to ask people here:

If you use local models across more than one machine or with a team, how are you handling shared context today? Do you even want models sharing memory or context between environments or is keeping every local setup completely isolated actually part of the appeal?

AA no longer lists GLM5.3 flash (despite clearly being one of the best models - and probably the cheapest per intelligence) , they seem to have been doing all sorts of changes to push US models and hiding good cost effective models, including qwen 3.8 flash, so that table is no good. I need a better one. Any alternatives.

Inspired by the qwenputer project posted in this sub earlier today, I managed to get a basic version of Doom running on the virtual machine created in the post.

It's quite slow, but here we are. Doom running on an LLM.

Project repo: https://github.com/mapestree/qwenputer-doom

Artificial Analysis Intelligence Index has been updated to v5, which according to them is a major upgrade resulting in a more accurate ranking.

Notably, this update widens the gap between open-weight models and closed ones. Some people already questioning its accuracy based on their experiences with models (GPT-Terra-6.1 > Fable-5.1).

Also, this update, v5, has 60% of the Index based on evaluations with private test sets or held-out answers, which makes reproducing the results impossible.

From their announcement:

“The open weights gap widens from 3 points to 12: Kimi K3.1 Pro (weights not yet released) is the top open weights model at 46. Among released weights, DeepSeek V4.1 Flash 1017 (max) and GLM-5.3 (max) lead at 32 - 26 points behind the leader and below Claude Haiku 5.5 (max, 36)”

Model & weights: BrunoHays/embeddinggemma-2-nullspace-audio

MAEB (Massive Audio Embedding Benchmark) is the main MTEB leaderboard benchmark for audio and text-to-audio embeddings, currently dominated by 8B-9B multimodal models.

I took Google's edge-sized google/embeddinggemma-2 (577M params, 768d vectors) and boosted its MAEB score from 49.3% to 54.4% (+5.1 points), jumping from Rank #26 to #1 on MAEB audio-only (and #16 to #1 on full MAEB) without training a single weight. The whole addition is a 2.37 MB safetensors file of frozen PCA matrices.

Why this matters for local / edge setups

At 577M parameters and 768d vectors (15x smaller than 9B Omni models), it runs easily on a laptop or edge device.

Standard text-aligned audio models are great at matching spoken words to text, but terrible at grouping audio by voice, emotion, or sound similarity because the final layers keep what words were said and drop how the sound was made.

By pulling features from earlier layers of the frozen audio encoder, this model keeps the base model's 100+ language text-to-audio search while fixing audio-to-audio tasks:

• Unsupervised voice/gender clustering: 0.05% -> 63.1%

• Artist / song audio-to-audio search: 55.2% -> 93.8%

• Vocal emotion classification: 19.7% -> 40.2%

How it works

• Early layers still know acoustics: Speech representation work (Pasad et al.; Bouziane, Mdhaffar & Estève) shows that early/middle layers keep speaker, pitch, and emotion traits that disappear in the final layers. In EmbeddingGemma-2, Layer 2 and Layer 6 still hold those signals.

• 256 empty directions in the 768d vector: EmbeddingGemma-2's text backbone is actually 512d, projected up to 768d at the end by a 768 x 512 linear matrix. That leaves 256 perpendicular directions (the null space) that text vectors never touch.

• Filling the null space with PCA: I compressed Layer 2 and Layer 6 down to 256 dimensions with PCA (fitted on 1,400 external audio clips) and injected them into those 256 unused directions.

• Caveats: Because there is no retraining, audio embeddings need all 768 dimensions (no Matryoshka truncation to 512/256/128), and adding unweighted acoustic features hurts some purely spoken-intent tasks (e.g. MInDS14 drops 97.6% -> 88.3%).

It shows how much untapped signal sits inside today's overtrained models, even with zero training.

Full breakdown and usage in the Hugging Face model card.

I was researching Local LLM computer use agents the other night and ended up down a rabbit hole where I ended up discovering a whole new class of hardware I didn’t know existed: Agent-driven KVMs

A lot of us IT folks who have worked in data centers are familiar with IP-based KVM (Keyboard Video Mouse) controllers that let you remotely operate a server over a streaming video connection. These are especially useful when you need physical power button / BIOS-level “bare metal”access to a system that’s thousands of miles away. If you don’t have bare metal access, you’re going to be taking a trip to your data center rack to press a button or make a bios setting change.

So apparently there are now relatively inexpensive ($80 to $150) IP-based KVMs that now allow for AI API endpoint/ MCP Integration where you can now connect your physical computer to local or frontier vision-capable LLMs and have the AI model of your choice drive the target computer via the KVM by it taking screenshots of the HDMI or USBC video-out feed from the computer and then sending mouse and keyboard control commands from the Ai model through the USB cable that’s connected to the KVM.

What really got me excited about this concept is that it requires no installation of any kind on the target computer that you want to control. This is useful in corporate environments where you’re not allowed to install anything to the target computer (because it’s a managed endpoint that is locked down) The target computer just sees the AI as a standard allowed HID that it already has a driver loaded for. It doesn’t care who’s driving the system and doesn’t need any additional software since keyboard, mouse, and monitors are just generic and usually preloaded drivers.

So essentially you could have an AI-driven KVM control any comouter or device that has a monitor output / keyboard mouse inputs. This could potentially open up the ability to have AI interact with antiquated industrial control systems running old shitty outdated OSes, or managed endpoints.

There are security risks of course, but that risk is mitigated almost entirely if you’re using a locally-hosted VLM and not sending your screenshots to a 3rd party.

I found are several inexpensive agent-driven options on the market now including:

- Sipeed NanoKVM Pro
https://wiki.sipeed.com/hardware/en/kvm/NanoKVM_Pro/introduction.htm

- Radxa Linkr
https://radxa.com/products/linkr/linkr/

- Violoop
https://www.kickstarter.com/projects/bvio/violoop-plug-in-ai-for-your-computer

I’m not selling anything or affiliated with any of these and can’t tell you which is best, but there are multiple options available for those who might need this type of device for one of the use cases I mentioned.

Anyways, I’m curious to know if others see any other use cases for these agent-controlled KVMs that I didn’t mention.

I’m the guy who posted the 4× R9700 builds with $127 used EPYCs (https://www.reddit.com/r/LocalLLaMA/comments/1x67qeu/4x_r9700_builds_with_127_used_epycs_107k_allin/).

After seeing the Nanbeige benchmark results here, I spent the past few days testing it against Qwen on the work these machines actually do.

Jobs > TPS

My business, USA Arms, maintains information about firearms-related businesses and I use local AI to maintain the majority of the information... services, hours, pricing, facilities, etc.

Each configuration ran the same 2,484 cases across 25 field level goldens as an auditor with accept, reject, or abstain as options. (ambiguity goldens)

Correct Jobs/hour

The jobs/hour numbers use 2x 2-GPU engines.

Token speed

Separate MT-Bench speed sweep, measured on one two-GPU engine (these aren't 4 card numbers):

Auditor

The video goes over it, but Qwen actually does a better job auditing itself with thinking mode on than Nanbeige with thinking on auditing the same jobs.

In total: Qwen was about 25% faster in the single-stream token test, but delivered about 45% more jobs/hour with thinking off than Nanbeige.

Curious whether people testing coding or creative writing are seeing a different result with Nanbeige... once again, hyped model doesn't dethrone the king.

Full results with more details on this short video: https://youtu.be/kmmN49ecyvc (I tried to upload direct to reddit but it had issues)

As of version 0.2.7.1 Strata has two routing optimizations that can affect both performance and quality. I was curious what the actual impact was so I ran some tests on my hardware with Qwen 3.8 Flash Next IQ3_S.

STRATA_ROUTE_PRIOR

This is the new one - this optimization configures Strata to prefer experts already in VRAM by adding a bonus of 1.5 (about a 4.48x effective multiplier) to the scores of experts already cached in VRAM. This means that a slightly less preferred cached expert can replace an expert the model would normally select if its base score was close enough.

STRATA_ROUTE_TAIL_SKIP

This one is a lot simpler - with a value of STRATA_ROUTE_TAIL_SKIP=7 (the default value), Strata will skip computing experts 7 through 10 if they aren't already in VRAM. It does not change selected experts, it just drops contributions from less important selected experts.

Benchmarks

I had DS4-Flash run Qwen through IFEvalCode, AIME 2026, and IFEval; then worked with ChatGPT to analyze the results.

Round 2 - re-ran a smaller subset of questions (mostly the ones whose answers differed in round 1) with much larger thinking budgets. Absolute percentages shouldn't be directly compared between rounds.

My interpretation

Round 1 showed that STRATA_ROUTE_PRIOR might reduce quality slightly, but round 2 showed that many of those differences don't matter with longer reasoning. Route prior is such a huge boost to speeds that I think it's worth enabling for all but the hardest tasks. On the other hand, STRATA_ROUTE_TAIL_SKIP offers a slight boost to speeds (without route prior enabled) but results in significantly more degradation than just route prior alone.

I plan to continue testing more extensively, but for now these give me a decent read on the tradeoffs between the two settings. Shoutout to Niko1221 for making an awesome inference engine.

Clover-35B-A3B-Preview is the first model released by DiffuSpace, delivering state-of-the-art performance among open-source diffusion language models at release on the benchmarks reported in our launch blog. Clover combines a 35B-parameter mixture-of-experts backbone with approximately 3B active parameters per token, block diffusion generation, and a 128K context window.

Blog · Code & inference backends · Hugging Face