✨ Discover this awesome post from Hacker News 📖
📂 **Category**:
📌 **What You’ll Learn**:
Install the Python package. The inference engine is fetched once from Hugging Face and cached; there is nothing else to build.
Needle reads your tool descriptions to decide what to call and how to fill arguments, so describing them well is the whole game.
Simple: decorate a function. The signature gives the argument types, the docstring is the tool description, and run() completes the loop: the model picks the call, Needle executes your function, feeds the result back, and returns the final response with the executed tool results attached as results.
import needle @needle.tool def get_weather(city: str): "Get the current weather for a city." return ⚡ agent = needle.Needle(tools=[get_weather]) print(agent.run("what's it like in Lagos right now?")["results"]) # [💬]
Route by pattern: when a description cannot enumerate every phrasing, give a tool triggers, regular expressions matched against each request. A match restricts the decode to the matched tools and requires a call, so the request reaches the tool you named instead of being refused or misrouted, and the call ships even below the confidence floor. A match restricts the whole turn, so a catch-all should exclude the nouns other tools own, e.g. ^(?![\s\S]*\b(lights?|doors?)\b)[\s\S]*\b(turn|switch)\b[\s\S]*\b(on|off)\b; then “switch the fan on and dim the kitchen lights” still reaches both tools.
from typing import Literal @needle.tool(triggers=[r"\b(turn|switch|power|flip)\b.*\b(on|off)\b", r"\btoggle\b"]) def control_device(device: str, action: Literal["on", "off", "toggle"]): "Switch or toggle any named smart-home device." return 🔥 agent = needle.Needle(tools=[control_device, get_weather]) agent.complete("toggle the garage door") # function_calls [{"name": "control_device", "arguments": 🔥}]
Extraction: to pull structured data out of text, declare the shape and call extract(). Pass a Pydantic model and you get a typed object back.
from pydantic import BaseModel class Invoice(BaseModel): vendor: str total: float due_date: str invoice = needle.extract("Invoice from Acme Corp, $1,200.00, due 2026-09-01", Invoice) print(invoice.vendor, invoice.total) # -> Acme Corp 1200.0
Every turn returns one JSON object:
{
"type": "call",
"success": true,
"error": null,
"error_code": null,
"function_calls": [ { "name": "set_lights", "arguments": { "room": "living room", "on": true, "brightness": 30 } } ],
"reasoning": "'living room' -> room; 'dim' -> on true, brightness 30",
"confidence": 0.94,
"prefill_tps": 4300.0,
"decode_tps": 850.0,
"peak_ram_mb": 28.5
}
Confidence gating and routing: every response carries a confidence score from a calibrated head, and the engine already applies a floor of 0.1. Below it, the call is withheld into suppressed_calls and function_calls is empty. Above it, the score is yours to route on: act at once when it is high, show the call and ask when it is middling, and treat an empty result as a refusal. A tool with triggers always produces a call for a matching request, so the score is what tells you whether to run it or confirm it.
r = agent.complete(user_text) calls = r["function_calls"] held = r["suppressed_calls"] if calls and r["confidence"] >= 0.7: execute(calls) # sure: act elif calls or held: confirm(calls or held, r["reasoning"]) # unsure: show the call, ask else: say("I can't do that here") # nothing to do: refuse
Writing tools: the model reads a schema literally, so a narrow tool with a plain description beats a broad one. One tool per action, described by the actions it covers (“Turn a room’s lights on or off”) rather than a category. Name enum options after what a user says (action: ["increase", "decrease"]) and keep synonyms in the description. Give a required argument a default when a request may leave it out; a required argument with no default and no evidence in the request is withheld rather than guessed. Put value formats in descriptions ("City, ST", "e.g. T-1042"). Add triggers to intents that must always reach a tool, and keep the toolset per turn small, since every extra tool is a chance to misroute.
Fine-tune: the Python package is the quick path. LoRA on the frozen base at the full 20 layers, then a 4-bit .cact of any subnetwork that runs on the same engine.
needle finetune data.jsonl --epochs 10 --out adapter.safetensors needle build --lora adapter.safetensors --out tuned.cact needle build --lora adapter.safetensors --platform linux-arm64 --layers 2 --out ./device
{💬|⚡|🔥} **What’s your take?**
Share your thoughts in the comments below!
#️⃣ **#Needle #foundation #model #tiny #devices**
🕒 **Posted on**: 1789825639
🌟 **Want more?** Click here for more info! 🌟
