Tiiny AI Pocket: 120B LLMs in 300g, and What It Means in Sri Lanka

The Tiiny AI Pocket is being pitched as the smallest edge AI device for local LLMs, and the headline claim is that it runs a 120B-parameter model from a box that weighs 300g and draws 30W. The Hacker News thread mostly argued about whether the website was a product or a slide deck. I think the better question is what the spec sheet says about where local AI is going, and whether any of it matters to a builder in Sri Lanka on a student budget.

I'm working from the company's own site, tiiny.ai, which is the source for every number below. I have not touched the device.

---

🔍 What the spec sheet actually says

The homepage is almost entirely images, which several commenters found odd. The product page underneath does list real specifications, and the memory figure is the whole story.

Spec	Tiiny AI Pocket (as listed by the vendor)
Memory	80GB LPDDR5X @ 6400 MT/s
Storage	1TB PCIe 4.0 NVMe SSD
Compute	Armv9.2 CPU + NPU (30 INT8 TOPS) plus a separate "dNPU" (160 INT8 TOPS)
Claimed output speed	18–40 tokens/s average
Power	30W TDP, 65W adapter required
Size and weight	142 × 80 × 22 mm, 300g
Ports	3 × USB-C, Wi-Fi 6, Bluetooth 5.3
Host OS	macOS and Windows only
Security	Hardware AES-256 full-disk encryption, keys in a secure enclave
Software	PowerInfer inference engine, OpenAI-compatible local API, developer SDK

The listed model lineup includes GPT-OSS-120B, GPT-OSS-20B, Llama 3.1 8B Instruct, Gemma 3 4B, Ministral 3 8B, Qwen3-30B-A3B, a Qwen3 reranker and the Z-Image-Turbo image model. The vendor's performance figures were measured with 512-token inputs generating up to 2048 output tokens.

Key takeaway: the compute numbers are ordinary for an ARM SoC. The unusual part is pairing that with 80GB of memory. A 120B model at roughly 4-bit precision needs on the order of 60GB just for weights, so memory capacity, not TOPS, is what decides whether the model loads at all.

---

⚡ Why 30W matters more than the TOPS figure

Commenters comparing this to a MacBook Pro are right that a well-specced laptop already runs large local models. What a laptop cannot do is run them at 30W as an appliance you leave on a shelf. That figure is the one Sri Lankan readers should care about, because electricity here is priced in blocks and a GPU workstation sits in the expensive ones.

Some back-of-envelope arithmetic, using nothing but the vendor's TDP:

• 30W for 8 hours a day is 0.24 kWh per day, or about 7.2 kWh a month.
• A desktop with a discrete GPU at 300W under the same duty cycle is roughly 72 kWh a month.
• A cloud API costs nothing in electricity but bills you per token, forever.

Put your own tariff into the Sri Lanka appliance electricity cost calculator to see what those two numbers cost on a CEB or LECO bill. For an always-on agent, the running cost of a low-power box is close to noise.

The catch is the 18–40 tokens/s claim, which one commenter called too slow for the money. For chat it is fine. For overnight batch jobs it is slow, but at this power draw "slow and unattended" is a legitimate strategy.

---

💰 The price problem, and what the site does and does not say

The site's FAQ says a deposit locks in a $1,299 price for the "Tiiny AI Pocket Lab", described as the lowest price it will ever offer. A Hacker News commenter cited $1,999. I could not find that figure on the vendor's site, so treat it as unconfirmed, but it suggests the deposit price is an early tier, not the shelf price.

Other facts from the FAQ that matter if you are outside the launch markets:

Question	Vendor's answer
Shipping priority	United States, Germany, UK, France, Italy, Spain, Netherlands, Singapore
Other regions	"May be supported with additional shipping fees"
Estimated delivery	August 2026
Subscription	None. One-time purchase, no token fees
Deposit refunds	On request by email, within 7 working days, transaction fees borne by you

Sri Lanka is not on the list. Singapore is the nearest, and anything shipped here meets customs on arrival. Convert the price with the LKR exchange rate tool, then add duty and freight on top.

Warning: the FAQ's estimated delivery date is August 2026, which has already passed as I write this in September 2026. The site does not say whether units have shipped, and a commenter pointed at the Kickstarter comments page as not looking encouraging. Do not pre-pay for hardware from a campaign you have not verified is shipping.

---

🛠️ What PowerInfer tells you about the design

The software side is the part I find credible. The site links to PowerInfer, an open-source inference engine from SJTU-IPADS at Shanghai Jiao Tong University. The vendor says version one ran 175B-parameter models on a single RTX 4090 at 90% of A100 speed, and that the device uses a newer generation of the same idea.

The core trick, in plain terms:

1. Most of a large model's neurons are "cold" for any given input and rarely fire.
2. PowerInfer keeps the frequently used "hot" neurons on the fast accelerator and streams the rest from slower memory on demand.
3. That lets a device with modest compute but lots of memory serve a model that would otherwise need a datacentre card.

That is why the spec sheet is memory-heavy and compute-light. It also explains why the listed models skew toward mixture-of-experts designs like GPT-OSS-120B and Qwen3-30B-A3B, where only a fraction of parameters are active per token.

Key takeaway: you do not need this box to benefit from the idea. PowerInfer is on GitHub. If you have a laptop with a lot of RAM and a small GPU, the same sparsity approach is what makes larger models usable on hardware you already own.

---

🌐 The cheaper path for a Sri Lankan builder

Nobody I know here will spend over a thousand dollars plus duty on a first-generation crowdfunded device. The useful lesson is about the shape of the workload, and you can act on it for free.

Your situation	What I would actually do
Student with an 8–16GB laptop	Run 4B–8B models locally. Gemma 3 4B and Llama 3.1 8B are on the vendor's own list and run on ordinary laptops.
Small team needing privacy	A used desktop with as much RAM as you can afford, PowerInfer or llama.cpp, and a local OpenAI-compatible endpoint.
Need a 120B-class model occasionally	Rent it by the hour or use a hosted API. Compare the per-token costs in the AI model comparison tool.
Building an always-on agent	This is the one case where a 30W appliance genuinely pays off. Watch this category, but buy the second generation.

The thing Tiiny gets right is the OpenAI-compatible local API. If your code talks to a standard chat-completions endpoint, you can develop against a free hosted tier today and point the same code at a local box later. The hardware question then becomes a deployment detail, not a rewrite.

---

💡 What this means for you

• Memory is the spec to watch. RAM capacity decides which models load. Compute decides how fast they run, and "slow but loads" beats "fast but won't fit".
• Low power is a real Sri Lankan advantage. With block tariffs, a 30W device running all day costs a fraction of a GPU rig.
• Do not pre-order across borders on a first-gen campaign. No Sri Lanka shipping priority, a lapsed delivery estimate and a thin website are three separate reasons to wait.
• Copy the architecture, not the purchase. Sparse inference, mixture-of-experts models and a local OpenAI-compatible endpoint are all open source and cost nothing to try this weekend.

I will be more interested when someone independent posts measured tokens-per-second on a real unit. Until then, the Tiiny AI Pocket is a useful signal about where local LLM hardware is heading, and a poor use of a Sri Lankan engineer's savings.