Tencent Shrinks Hunyuan Hy4 Preview From 1.5TB to 214GB: What On-Premise AI Now Costs Brands in China

On 1 September 2026, Tencent's Hunyuan team published a light build of Hy4 preview that compresses the model weights from roughly 1.5TB to about 214GB — an 86% cut — while keeping long-document understanding and multi-turn long-context retrieval within one or two points of the original BF16 model. The quantised GGUF weights, the base model, and the AngelSlim code repository are all public.

This is a hardware story, not a capabilities story. And for brands running marketing or content operations inside China, the hardware story is the one that has been blocking everything else.

1. What Changed in Four Days: From 1.5TB to 214GB

On 28 August 2026, Tencent released and open-sourced Hy4 preview: a 770-billion-parameter mixture-of-experts model that activates 49B parameters per token, handles a context window exceeding one million tokens, and ships under Apache 2.0 with no monthly-active-user threshold. Hosted API pricing through Tencent Cloud TokenHub and OpenRouter landed at ¥6 per million input tokens and ¥18 per million output tokens, with cached input at ¥0.3.

Four days later, the light build arrived. Community feedback after the original release had been consistent: the BF16 checkpoint was simply too large for most teams to load. Tencent's answer was a quantised build that fits in roughly one-seventh of the original footprint.

The compression ratio matters because of what it removes from the decision: the question is no longer "can we afford a GPU cluster," but "can we repurpose the hardware already in the office."

2. How the Compression Works: Sherry and MIX-STQ1_0

Two techniques are stacked, and they solve different halves of the problem.

Sherry sparse ternary quantisation. Sherry is a ternary scheme applied to the routed expert layers. Every group of four weights is quantised together into {-1, 0, +1}, with exactly one of the four forced to zero. That constraint yields 32 possible combinations per group, which can be encoded with a 5-bit index — so the arithmetic average is 1.25 bits per weight. Once block-shared scaling factors and other overhead are counted, the real on-disk cost lands at about 1.31 bpw. Tencent implemented an STQ1_0 inference kernel inside llama.cpp to make the format actually runnable; by their measurements it is comparable in speed to IQ1_M and faster than IQ1_S.

MIX-STQ1_0 mixed precision. Uniform bit width wastes precision on layers that do not need it and starves layers that do. MIX-STQ1_0 uses calibration data to judge each layer's sensitivity, then assigns bit width per layer: sensitive layers keep IQ2_XXS at 2.06 bits, insensitive layers take the more aggressive STQ1_0 at 1.31 bits. The result is lower quantisation error at the same average bit budget, and more than 5GB of additional storage saved versus a uniform UD-IQ1_M approach.

The combination is what makes the number interesting. Sherry does the heavy compression; MIX-STQ1_0 decides where the damage is allowed to land.

3. What You Give Up: The Benchmark Cost of 86% Compression

Tencent has been specific about the degradation, which is more than most quantisation releases offer.

Read the shape of that table rather than the individual numbers, all of which are vendor-reported. Retrieval-heavy and document-heavy work barely moves. Tool calling and code lose one to two points. Mathematics takes the clearest hit, which is the expected pattern for aggressive low-bit quantisation — reasoning chains are where accumulated rounding error shows up first.

Tencent states the light build covers everyday coding assistance, tool calling, long-document processing, and general question answering. Note what is not in that list: anything where a wrong intermediate step compounds silently.

The honest framing: this is a model for throughput, not for precision. If your use case is "read 400 pages of product documentation and produce a first-draft localisation," the trade is excellent. If it is "compute the correct media budget allocation across twelve markets," keep the BF16 or FP8 build as the reference and treat the light build as a screening pass.

4. Running It on Hardware You Already Own: The Heterogeneous Test

The more consequential announcement is the deployment experiment, run with the distributed cluster inference framework prima.cpp.

The setup: one laptop with a single RTX 4090, and one server with four A4000 cards. Combined, 80GB of VRAM and 64GB of system memory — 144GB total. The two machines were on separate local networks. That is 70GB short of the 214GB model.

It ran. prima.cpp's heterogeneous scheduler split the MoE layers across both machines and overlapped computation with weight loading, and the 214GB light build reached 1.02 tokens per second — about six times faster than offloading on the laptop alone.

What 1.02 token/s means in practice. For interactive chat, it is unacceptable. For batch work, it is entirely workable: roughly 3,700 tokens an hour, or about 88,000 tokens if you leave it running overnight. A 12,000-token localised landing page draft is three hours; a batch of thirty product descriptions across a weekend is trivial. The relevant question is not "is it fast" but "is my workload latency-sensitive."

Why the heterogeneous part matters more than the number. Most enterprise AI procurement conversations stall on a single line item: buy a matched cluster, or do nothing. This experiment demonstrates a third path — pool the mismatched hardware you already own, accept the throughput penalty, and defer the capital expense. For a regional marketing team in Shanghai that has a couple of workstations and a spare server, that is the difference between running a pilot this quarter and running it never.

5. Why On-Premise Matters More in China Than Anywhere Else

Here is where the technical release turns into a commercial argument, and it is the reason this belongs on a marketing agenda rather than an engineering one.

China's data regime makes cross-border transfer the exception rather than the default. Under the Personal Information Protection Law (PIPL), Articles 38 to 40 require a security assessment, certification, or filed standard contractual clauses before personal information leaves the mainland. The Data Security Law adds a data classification regime and a domestic-storage principle for "important data." The March 2024 Provisions on Promoting and Regulating Cross-Border Data Flows set the volume thresholds:

For a foreign brand operating in China, this collides with how global marketing technology is normally architected. The CRM export goes to a global instance. The campaign brief goes to a hosted LLM with endpoints in the US or Europe. The customer list goes into a lookalike audience tool. Each of those is a transfer, and each one needs a mechanism.

On-premise inference changes the equation structurally rather than contractually. If the model weights, the vector store, and the inference process all sit on infrastructure inside China, there is no cross-border transfer to assess. This is the same logic behind Tencent's own advice to foreign advertisers on landing pages — if your global site cannot carry China analytics and conversion code, move the landing page to a China-hosted platform instead of fighting the integration. Both are the same move: bring the processing to where the data is, rather than moving the data to the processing.

Two obligations survive localisation, and teams regularly miss both. The Interim Measures for Generative AI Services (in force since 15 August 2023) still apply if the output reaches users. And the Measures for Labelling AI-Generated Synthetic Content (in force since 1 September 2025) still require explicit and implicit labelling of AI-generated material — running the model locally does not exempt a brand from labelling the content it publishes.

6. What This Means for Content Production and Marketing Teams

Three concrete shifts.

The whole corpus fits in one pass. A one-million-token context window is roughly 750,000 words. That is the entire product manual, plus the last four quarters of campaign performance reports, plus the exported CRM notes, plus the competitor creative library — in a single prompt. Previously, brand-localisation work meant chunking documents, losing cross-reference context, and reconciling inconsistent terminology between chunks. With the full corpus in context, terminology stays consistent across a hundred-page deliverable.

Unreleased material becomes safe to process. Campaign creative that has not launched, pricing sheets, distributor terms, pre-launch product specifications — these are exactly the inputs that global AI policies prohibit sending to third-party hosted endpoints. A model running on a machine in your own office has no such restriction. For brand teams in China, this is often the difference between using AI for drafting and not using it at all.

Cost structure inverts at volume. Hosted API pricing at ¥6 and ¥18 per million input and output tokens is competitive and requires zero capital. But the cost scales linearly with usage forever. A self-hosted build converts that into a fixed asset plus electricity. The crossover point depends entirely on utilisation, and 1.02 token/s on repurposed hardware is not a high ceiling — but for a team running nightly batch jobs against a stable corpus, the marginal cost after month one is close to nothing.

Where this does not apply. Real-time customer service, ad-copy generation at campaign scale under time pressure, anything where a wrong number reaches a customer. Use the hosted API for those, and keep the local build for the batch work behind them.

7. A Practical Adoption Checklist

The Bottom Line

Tencent published a flagship-class model in August and removed the hardware excuse in September. Apache 2.0 licensing removes the legal friction. A 214GB footprint plus heterogeneous scheduling removes the capital friction. What remains is the only friction that was ever really there: deciding which of your workloads actually belongs on local infrastructure.

For most brands in China, the answer is narrower than the hype suggests and wider than zero. Batch content production, localisation against a fixed brand corpus, and processing of material that cannot lawfully cross a border — those are local-workload problems, and as of 1 September they have a workable answer.

Sources: Tencent Hunyuan Hy4 preview light build announcement, 1 September 2026 (IT之家 / 智东西 / 凤凰网科技 / 快科技); Tencent Hy4 preview release, 28 August 2026; Hugging Face tencent/Hy4-preview and AngelSlim/Hy4-preview-GGUF; GitHub Tencent/AngelSlim; prima.cpp heterogeneous inference framework. Regulatory references: PIPL Articles 38–40, Data Security Law, Provisions on Promoting and Regulating Cross-Border Data Flows (March 2024), Interim Measures for Generative AI Services (2023), Measures for Labelling AI-Generated Synthetic Content (2025).