I had a tool description earlier but 'sh' as tool name seems to be sufficient, the agent behaviour was the same.
There might be performance gains if a description is added though, or worth trying different ways of telling the agent about what is available in the environment.
That said, the newer models are fairly good at driving a harness to explore the environment.
another reason why it’s great to have open weights: inference providers and your own hardware can ensure that you get the tokens you need even if a lab itself hits roadblocks (and there have been many in quality and availability)
So I've weighed it with my cheap kitchen scale, that says it weighs 1222g now. Sadly I did not weigh it before. I did look online and saw 1240g for the same model/specs.
Take of that what you will, but I can't imagine that I filed off that much. I'm assuming my kitchen scale is just not well calibrated
Most agent harnesses that have not been designed from scratch for current models are over-engineered.
Chances are whatever was needed to make earlier models perform well now either is no longer helping much or actively hurts performance (worse results, slower, uses more tokens …).
import json,sys,uuid;from subprocess import getoutput as sh;from urllib.request import Request as R,urlopen
b={"model":"gpt-5.6","prompt_cache_key":uuid.uuid4().hex,"input":[],"tools":[{"type":"custom","name":"shell"}]}
while prompt:=input("> "):
b["input"]+=[{"role":"user","content":prompt}]
while True:
o=(r:=json.load(urlopen(R(sys.argv[1],json.dumps(b).encode(),{"Content-Type":"application/json"}))))["output"]
b["input"]+=o;calls=[i for i in o if i["type"]=="custom_tool_call"];used=r["usage"]["total_tokens"]/10500
if not calls: print(o[-1]["content"][0]["text"],f'\n[{used:06.3f}%]'); break
b["input"]+=[{"type":"custom_tool_call_output","call_id":i["call_id"],"output":sh(i["input"])} for i in calls]
https://gist.github.com/tosh/61aca9ffa9ea115fa4df332407d7a9a...
reply