<?xml version="1.0" encoding="UTF-8"?><rss version="2.0" xmlns:content="http://purl.org/rss/1.0/modules/content/"><channel><title>dashgin — field reports</title><description>Writing on self-hosting, AI agents, cost, and shipping full-stack products.</description><link>https://www.dashgin.com</link><item><title>MCP Tool Bloat: Why 546 Tools Cost Me $1,557 (and the ~700-Token Fix)</title><link>https://www.dashgin.com/blog/mcp-tool-bloat-deferred-tools-caching</link><guid isPermaLink="true">https://www.dashgin.com/blog/mcp-tool-bloat-deferred-tools-caching</guid><description>The technical deep-dive behind the $1,557 agent bill — token forensics, why deferred tools + prompt caching change everything, and how I replaced a 546-tool MCP server with a curl-based skill that&apos;s ~700× smaller.</description><pubDate>Sat, 15 Aug 2026 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;This is the technical companion to &lt;em&gt;&lt;a href=&quot;https://www.dashgin.com/blog/self-hosted-ai-agent-burned-1557&quot;&gt;How My Self-Hosted AI Agent Burned $1,557 in 2 Days&lt;/a&gt;&lt;/em&gt;. If you want the token math, the root cause, and the actual fix, this is it. No infrastructure specifics — just the mechanics that apply to any agent.&lt;/p&gt;
&lt;h2&gt;The forensics&lt;/h2&gt;
&lt;p&gt;Two days of Hermes Agent on Claude Opus 5, billed through AWS Bedrock:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;3,187 model calls&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;350 million input tokens&lt;/strong&gt;, 1.86 million output tokens&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;~110,000 input tokens per call on average&lt;/strong&gt; — some at the 500k context ceiling&lt;/li&gt;
&lt;li&gt;Cost: &lt;strong&gt;~$1,557&lt;/strong&gt;, of which &lt;strong&gt;97% was input&lt;/strong&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Read that last line again. Almost none of the cost was the model &lt;em&gt;thinking&lt;/em&gt; (output). It was the same giant context being &lt;strong&gt;re-sent and re-billed on every single call&lt;/strong&gt;.&lt;/p&gt;
&lt;h2&gt;Where 110k tokens per call comes from&lt;/h2&gt;
&lt;p&gt;The Dokploy MCP server exposed &lt;strong&gt;546 tools&lt;/strong&gt;. In a naive agent setup, every one of those 546 tool definitions — name, description, and full JSON parameter schema — gets injected into the model&apos;s context on &lt;strong&gt;every&lt;/strong&gt; request.&lt;/p&gt;
&lt;p&gt;That&apos;s ~500k tokens of tool schemas riding along whether the user said &quot;deploy the staging service&quot; or just &quot;hi&quot;.&lt;/p&gt;
&lt;p&gt;At Opus 5 pricing — $5 per million input tokens — 100k tokens of tools per call = &lt;strong&gt;~$0.50 per call in tool overhead alone&lt;/strong&gt;, before the model reads a single word of the actual conversation. Multiply by thousands of agentic tool-loop iterations, add 3× retries on every error, and you get $1,557 in 48 hours.&lt;/p&gt;
&lt;h2&gt;Why the same tools + model are nearly free elsewhere&lt;/h2&gt;
&lt;p&gt;I use the &lt;em&gt;same&lt;/em&gt; 546-tool Dokploy MCP server with the &lt;em&gt;same&lt;/em&gt; Claude Opus 5 in &lt;strong&gt;Claude Code&lt;/strong&gt;, for a tiny fraction of the cost. Two mechanisms explain the entire gap:&lt;/p&gt;
&lt;h3&gt;1. Deferred tools (the #1 difference)&lt;/h3&gt;
&lt;p&gt;Claude Code doesn&apos;t dump all tool schemas into context. It keeps only the tool &lt;strong&gt;names&lt;/strong&gt; and a one-line description, then fetches a tool&apos;s full parameter schema &lt;strong&gt;on demand&lt;/strong&gt; — the first time the model actually wants to call it.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Naive agent: 546 full schemas in context, every call → ~500k tokens&lt;/li&gt;
&lt;li&gt;Deferred: ~546 names in context, schemas fetched as needed → a few k tokens&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Same capability. ~100× less context. This alone would have turned my $1,557 into ~$15.&lt;/p&gt;
&lt;h3&gt;2. Prompt caching&lt;/h3&gt;
&lt;p&gt;The static prefix of a request — system prompt + tool definitions — is identical across calls in a session. With prompt caching enabled, that block is written to cache once and &lt;strong&gt;read at ~10% of the input price&lt;/strong&gt; on subsequent calls (or skipped entirely on exact hits).&lt;/p&gt;
&lt;p&gt;My self-hosted agent had caching &lt;strong&gt;off&lt;/strong&gt; by default, so it paid full input price for the entire bloated prefix on every one of 3,187 calls.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Same model. Same tools. The cost difference was 100% context management.&lt;/strong&gt;&lt;/p&gt;
&lt;h2&gt;The real fix: a skill instead of an MCP server&lt;/h2&gt;
&lt;p&gt;Here&apos;s the insight that generalizes. &lt;strong&gt;You almost never need 546 tools. You need about 10.&lt;/strong&gt; Deploy, redeploy, restart, stop, logs, create, add-a-domain — that&apos;s the daily reality.&lt;/p&gt;
&lt;p&gt;So instead of an MCP server that injects 546 typed tools, I wrote a &lt;strong&gt;skill&lt;/strong&gt;: a ~700-token Markdown doc that teaches the agent to hit Dokploy&apos;s HTTP API with &lt;code&gt;curl&lt;/code&gt;, using the &lt;em&gt;one&lt;/em&gt; generic terminal tool it already has.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;&lt;/th&gt;
&lt;th&gt;MCP server&lt;/th&gt;
&lt;th&gt;Skill (curl + docs)&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Tools in context&lt;/td&gt;
&lt;td&gt;546 schemas (~500k tokens)&lt;/td&gt;
&lt;td&gt;1 generic tool + ~700-token doc&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;When loaded&lt;/td&gt;
&lt;td&gt;every call&lt;/td&gt;
&lt;td&gt;only when relevant (progressive disclosure)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Reliability&lt;/td&gt;
&lt;td&gt;high (typed params)&lt;/td&gt;
&lt;td&gt;slightly lower (agent builds the call)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Per-call cost&lt;/td&gt;
&lt;td&gt;~$2.50&lt;/td&gt;
&lt;td&gt;~$0.003&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;The skill documents the ~10 common operations as copy-paste &lt;code&gt;curl&lt;/code&gt; examples, and points to the full API spec &lt;strong&gt;for the rare cases&lt;/strong&gt; — so the 500+ other operations are fetched on demand and never sit in context. Roughly &lt;strong&gt;700× smaller&lt;/strong&gt; per call.&lt;/p&gt;
&lt;p&gt;This is the pattern for any fat MCP on a self-hosted agent: &lt;strong&gt;if a server exposes dozens of tools you mostly don&apos;t use, a skill that documents the API and uses the terminal is dramatically cheaper.&lt;/strong&gt; MCP&apos;s typed-tool reliability is worth it for a handful of tools, or in a client that defers schemas — not for 546 loaded on every call.&lt;/p&gt;
&lt;h2&gt;The rebuild checklist&lt;/h2&gt;
&lt;p&gt;If I were standing this agent back up, in order of impact:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;strong&gt;Don&apos;t default to the flagship model.&lt;/strong&gt; Use a cheap-but-capable model (Kimi K2.5, GLM-4.7-flash, Qwen3-Next) as the daily driver; reserve Opus for genuinely hard tasks. (5–70× cheaper.)&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Don&apos;t load a 546-tool MCP into a client that can&apos;t defer.&lt;/strong&gt; Replace it with a skill, or trim the toolset. (~100× less context.)&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Enable prompt caching.&lt;/strong&gt; Stop re-billing the static prefix. (~90% off repeat calls.)&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Cap the budget low.&lt;/strong&gt; Budget alarms lag ~a day; a $50 cap catches a runaway before a $200 one would.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Bound the loops.&lt;/strong&gt; Lower max-turns so an agentic loop can&apos;t spiral, and don&apos;t retry 3× into a wall.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;With all five, the same usage that cost $1,557 would run &lt;strong&gt;under $30&lt;/strong&gt;.&lt;/p&gt;
&lt;h2&gt;The one-line takeaway&lt;/h2&gt;
&lt;p&gt;&lt;strong&gt;Tokens are not just what the model writes — they&apos;re everything you make it read, every time.&lt;/strong&gt; A 546-tool MCP is 546 tools you pay for on every call. Manage the context, and a self-hosted agent is cheap. Ignore it, and it&apos;s a four-figure surprise.&lt;/p&gt;
</content:encoded><category>mcp</category><category>ai-agents</category><category>llm</category><category>context-engineering</category><category>cost</category></item><item><title>S3 Said 27 GB. The Bill Said 73.</title><link>https://www.dashgin.com/blog/s3-said-27gb-the-bill-said-73</link><guid isPermaLink="true">https://www.dashgin.com/blog/s3-said-27gb-the-bill-said-73</guid><description>46 GB of my bucket was invisible to every tool I had. Here&apos;s how S3 object metadata let me reconstruct a silent failure from two months earlier — and the design bug it exposed.</description><pubDate>Sat, 15 Aug 2026 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;I was checking storage costs on a digital asset manager I&apos;m building. &lt;code&gt;aws s3 ls&lt;/code&gt; said the bucket held 856 objects totalling &lt;strong&gt;26.7 GB&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;Cost Explorer said I was being billed for &lt;strong&gt;73.2 GB&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;Neither number was wrong. The 46 GB gap had been sitting there for two months, invisible to every tool I normally use, and the story of how it got there turned out to be more interesting than the storage cost — which, for the record, was $1.65 a month. This is not a post about saving money.&lt;/p&gt;
&lt;h2&gt;The part of S3 that doesn&apos;t show up in listings&lt;/h2&gt;
&lt;p&gt;When you upload a large file to S3, the client doesn&apos;t send it in one request. It calls &lt;code&gt;CreateMultipartUpload&lt;/code&gt;, gets an upload ID, pushes the file up in chunks, then calls &lt;code&gt;CompleteMultipartUpload&lt;/code&gt; to stitch them together into an object.&lt;/p&gt;
&lt;p&gt;Until that final call lands, the uploaded chunks are real, stored, and &lt;strong&gt;billed&lt;/strong&gt; — but they are not an object. &lt;code&gt;ListObjects&lt;/code&gt; doesn&apos;t return them. &lt;code&gt;aws s3 ls&lt;/code&gt; doesn&apos;t see them. The S3 console doesn&apos;t show them on the objects tab. They exist in a separate namespace you have to ask for explicitly:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;aws s3api list-multipart-uploads --bucket my-bucket
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;I ran it expecting nothing:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;incomplete multipart uploads: 47
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Forty-seven abandoned uploads. All the same 1.06 GB file. All initiated within a &lt;strong&gt;six-second window&lt;/strong&gt; on June 6th, at 21:17:59 UTC — 1:18 AM my time. Summing their parts came to &lt;strong&gt;48.3 GB&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;That was the missing storage.&lt;/p&gt;
&lt;h2&gt;Now the interesting question&lt;/h2&gt;
&lt;p&gt;I had the &lt;em&gt;what&lt;/em&gt;. I had no idea about the &lt;em&gt;why&lt;/em&gt;. There were no logs — this was a browser upload that died two months ago. No error report, no Sentry event, no server-side trace of the failure. Whatever happened, happened in a tab that no longer existed.&lt;/p&gt;
&lt;p&gt;But S3 keeps more metadata than people realise. It turned out to be enough.&lt;/p&gt;
&lt;h3&gt;Clue 1: the ETag says which code wrote the object&lt;/h3&gt;
&lt;p&gt;Alongside the 47 dead uploads, 19 &lt;em&gt;completed&lt;/em&gt; copies of the same video were sitting in the bucket. All 19 shared an ETag:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;&quot;b48ab5828a8e25a018b2095f098532d4-212&quot;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;For a multipart object, S3&apos;s ETag isn&apos;t the file&apos;s MD5. It&apos;s a hash of the concatenated part hashes, with &lt;strong&gt;the part count appended after a dash&lt;/strong&gt;. That &lt;code&gt;-212&lt;/code&gt; is a fingerprint of &lt;em&gt;how the file was uploaded&lt;/em&gt;, not just what it contains.&lt;/p&gt;
&lt;p&gt;1.06 GB in 212 parts means 5 MB chunks. My backend hardcodes exactly that:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;part_size = 5 * 1024 * 1024  # 5MB minimum for S3
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The AWS CLI defaults to 8 MB. rclone defaults differently again. That single suffix ruled out &quot;someone ran a CLI command&quot; and pinned every copy to my own application&apos;s upload endpoint. I hadn&apos;t fat-fingered a script — my app did this.&lt;/p&gt;
&lt;h3&gt;Clue 2: the gaps aren&apos;t random&lt;/h3&gt;
&lt;p&gt;&lt;code&gt;list-parts&lt;/code&gt; tells you which chunks made it. I expected uploads that died at scattered points — that&apos;s what a flaky connection looks like.&lt;/p&gt;
&lt;p&gt;Instead:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;21 of 47&lt;/strong&gt; had all &lt;strong&gt;212 of 212&lt;/strong&gt; parts uploaded.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;26 of 47&lt;/strong&gt; had &lt;strong&gt;210 of 212&lt;/strong&gt; — missing &lt;em&gt;exactly&lt;/em&gt; parts 211 and 212.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Nothing else. No upload stopped at part 47, or 130, or 8.&lt;/p&gt;
&lt;p&gt;Twenty-one uploads had every single byte in S3 and were one API call from being finished. The other twenty-six were two chunks short. A network problem doesn&apos;t distribute itself like that.&lt;/p&gt;
&lt;h3&gt;Clue 3: the 33-second window&lt;/h3&gt;
&lt;p&gt;Each part carries a &lt;code&gt;LastModified&lt;/code&gt;. I pulled the newest one from each of the 47 uploads and sorted them:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;earliest last-write:  2026-06-06T22:05:30Z
latest last-write:    2026-06-06T22:06:03Z
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Every one of the 47 uploads stopped writing inside the same &lt;strong&gt;33-second window&lt;/strong&gt;, after running for about 47 minutes.&lt;/p&gt;
&lt;p&gt;That settles it. Forty-seven independent failures don&apos;t synchronise to half a minute. This wasn&apos;t 47 uploads failing — it was &lt;strong&gt;one thing failing, once&lt;/strong&gt;, that all 47 were riding on. The tab was closed, or the machine went to sleep. At 2 AM, I&apos;d bet on sleep.&lt;/p&gt;
&lt;p&gt;And the parts still being there two months later proves the rest: &lt;strong&gt;my cleanup code never ran.&lt;/strong&gt;&lt;/p&gt;
&lt;h2&gt;The code&lt;/h2&gt;
&lt;p&gt;Here&apos;s what the browser was doing. Small files were carefully batched:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;// Small files: batched to avoid oversized requests
for (let i = 0; i &amp;lt; smallFiles.length; i += UPLOAD_URL_BATCH_SIZE) {
  const chunk = smallFiles.slice(i, i + UPLOAD_URL_BATCH_SIZE);
  // ...upload this chunk...
}
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;And immediately below it, large files were not:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;// Large files: multipart upload flow
const largePromises = largeFiles.map(async (file) =&amp;gt; {
  const init = await initMultipart(file);      // opens an S3 multipart upload
  const parts = await uploadMultipart(file);   // pushes ~212 chunks
  await completeMultipart(init, parts);        // finalises
});

await Promise.all(largePromises);               // ← every file, all at once
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Ten lines apart in the same function. One path bounded, the other unbounded.&lt;/p&gt;
&lt;p&gt;Select 47 large files and you open 47 concurrent S3 multipart uploads and try to push 48.5 GB through a single browser tab. It ran for 47 minutes and got &lt;em&gt;almost&lt;/em&gt; all the way — which is its own kind of unlucky.&lt;/p&gt;
&lt;p&gt;There &lt;em&gt;was&lt;/em&gt; per-file cleanup, and it was correct:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;} catch {
  if (fileKey &amp;amp;&amp;amp; uploadId) {
    abortMultipart({ fileKey, uploadId }).catch(() =&amp;gt; {});
  }
  return null;
}
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;It never executed. Not because it was buggy — because the failure mode was &lt;em&gt;the thing that runs the catch block ceasing to exist&lt;/em&gt;. There is no exception to catch when the JavaScript context is gone.&lt;/p&gt;
&lt;p&gt;One hypothesis I had to kill: presigned URLs expire, and mine were set to 3600 seconds. Tempting. But the uploads died at 47 minutes, comfortably inside the window. Expiry wasn&apos;t it.&lt;/p&gt;
&lt;h2&gt;The fix, in three layers&lt;/h2&gt;
&lt;p&gt;&lt;strong&gt;1. A lifecycle rule.&lt;/strong&gt; This is the one that actually matters:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;{
  &quot;Rules&quot;: [{
    &quot;ID&quot;: &quot;abort-incomplete-multipart-uploads&quot;,
    &quot;Filter&quot;: {},
    &quot;Status&quot;: &quot;Enabled&quot;,
    &quot;AbortIncompleteMultipartUpload&quot;: { &quot;DaysAfterInitiation&quot;: 7 }
  }]
}
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Seven lines of JSON. S3 now reaps abandoned uploads on its own, regardless of what any client does or fails to do. If I&apos;d had this from day one, the 46 GB would have cleaned itself up in a week and I&apos;d never have written this post.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;2. Bound the concurrency.&lt;/strong&gt; A small worker-pool helper with &lt;code&gt;Promise.allSettled&lt;/code&gt; semantics:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;export async function mapWithConcurrency&amp;lt;T, R&amp;gt;(
  items: readonly T[],
  limit: number,
  mapper: (item: T, index: number) =&amp;gt; Promise&amp;lt;R&amp;gt;,
): Promise&amp;lt;PromiseSettledResult&amp;lt;R&amp;gt;[]&amp;gt; {
  const results = new Array&amp;lt;PromiseSettledResult&amp;lt;R&amp;gt;&amp;gt;(items.length);
  let cursor = 0;

  const workers = Array.from({ length: Math.max(1, Math.min(limit, items.length)) }, async () =&amp;gt; {
    for (let index = cursor++; index &amp;lt; items.length; index = cursor++) {
      try {
        results[index] = { status: &apos;fulfilled&apos;, value: await mapper(items[index], index) };
      } catch (reason) {
        results[index] = { status: &apos;rejected&apos;, reason };
      }
    }
  });

  await Promise.all(workers);
  return results;
}
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Three files at a time instead of all of them. In-flight bytes drop from 48.5 GB to about 45 MB. A dead tab now strands three uploads, not forty-seven.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;3. Abort the existing 47.&lt;/strong&gt; Safe, because they could never be completed anyway — finalising a multipart upload requires the part ETags, and those only ever lived in a browser tab that died in June.&lt;/p&gt;
&lt;h2&gt;What I&apos;d actually take from this&lt;/h2&gt;
&lt;p&gt;&lt;strong&gt;Cleanup that lives only in the client&apos;s &lt;code&gt;catch&lt;/code&gt; block is not cleanup.&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;My abort code was well-written and correct and completely useless, because it assumed the client would survive long enough to run it. That assumption is invisible when you read the code — it looks like proper error handling. It only shows up when the client is the thing that dies.&lt;/p&gt;
&lt;p&gt;This generalises past S3. Anything you acquire remotely and release client-side has the same shape: open database transactions, distributed locks, Stripe payment intents, temp files on a server, reserved inventory. If the only thing that frees the resource is code running on the machine that just crashed, then you don&apos;t have cleanup — you have cleanup &lt;em&gt;most of the time&lt;/em&gt;, and the leaked cases accumulate silently precisely because nothing is around to report them.&lt;/p&gt;
&lt;p&gt;The fix is never a better &lt;code&gt;catch&lt;/code&gt;. It&apos;s a reaper on the other side that doesn&apos;t care whether your client is alive.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;And the smaller lesson:&lt;/strong&gt; the bounded loop and the unbounded one sat ten lines apart in the same file. Both looked fine on their own. I wrote both. I reviewed both. What made it visible in the end wasn&apos;t reading the code — it was a number in a billing console that didn&apos;t match a number in a file listing.&lt;/p&gt;
&lt;p&gt;Reconcile your numbers occasionally. The gap is where the interesting bugs live.&lt;/p&gt;
&lt;hr /&gt;
&lt;p&gt;&lt;em&gt;Postscript, for honesty&apos;s sake: this was me stress-testing my own upload path with a 10-hour 4K video at 1 AM, not a customer incident. Nobody&apos;s data was affected and the total cost of the mistake was about eighteen dollars a year. I&apos;m writing it up because the forensics were fun and the design bug is one I expect to meet again.&lt;/em&gt;&lt;/p&gt;
</content:encoded><category>aws</category><category>s3</category><category>debugging</category><category>forensics</category><category>post-mortem</category></item><item><title>How My Self-Hosted AI Agent Burned $1,557 in 2 Days</title><link>https://www.dashgin.com/blog/self-hosted-ai-agent-burned-1557</link><guid isPermaLink="true">https://www.dashgin.com/blog/self-hosted-ai-agent-burned-1557</guid><description>I put an always-on AI agent on my own server. Two days later AWS billed me $1,557. Here&apos;s exactly what happened, why, and how the same setup costs pennies when done right.</description><pubDate>Sat, 15 Aug 2026 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;I set out to self-host an AI agent — a little always-on assistant on my own server, wired into Telegram, web search, and my infrastructure. Two days later AWS had billed me &lt;strong&gt;$1,557&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;Here&apos;s exactly what happened, why, and how the &lt;em&gt;same setup&lt;/em&gt; costs pennies when done right.&lt;/p&gt;
&lt;h2&gt;The setup&lt;/h2&gt;
&lt;p&gt;I put &lt;strong&gt;Hermes Agent&lt;/strong&gt; on a small EC2 box and connected it to &lt;strong&gt;AWS Bedrock&lt;/strong&gt; for the model. Over an afternoon it grew into a proper little stack:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Claude Opus 5&lt;/strong&gt; as the brain&lt;/li&gt;
&lt;li&gt;A Telegram bot + self-hosted web search&lt;/li&gt;
&lt;li&gt;Vision, and a web dashboard behind a domain&lt;/li&gt;
&lt;li&gt;A handful of &lt;strong&gt;MCP servers&lt;/strong&gt; for tools — including the &lt;strong&gt;Dokploy MCP server&lt;/strong&gt;, which exposed &lt;strong&gt;546 tools&lt;/strong&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;It worked. It also, quietly, started printing money — for the cloud provider.&lt;/p&gt;
&lt;h2&gt;The bill&lt;/h2&gt;
&lt;p&gt;I checked the cost dashboard expecting maybe $20. Instead:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;Day 1:  $640
Day 2:  $916   ← and still climbing
─────────────
       $1,557  in 48 hours
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;A budget kill-switch I&apos;d set up &lt;em&gt;did&lt;/em&gt; fire and cut it off — but budget data lags almost a day, so it slammed the brakes only &lt;em&gt;after&lt;/em&gt; $1,557 had already gone through.&lt;/p&gt;
&lt;h2&gt;Why it happened (three mistakes, stacked)&lt;/h2&gt;
&lt;p&gt;The forensics were brutal and clear: &lt;strong&gt;3,187 model calls, 350 million input tokens&lt;/strong&gt; in two days. The average call sent &lt;strong&gt;110,000 tokens&lt;/strong&gt; — some hit &lt;strong&gt;500,000&lt;/strong&gt; (the model&apos;s ceiling). For simple &quot;hi&quot; messages. Three things multiplied together:&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;1. I used the most expensive model as the default.&lt;/strong&gt;
Claude Opus 5 is $5 per million input tokens, $25 per million output. Great model. Terrible default for a chatty, always-on agent.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;2. I loaded 546 tools into every single call.&lt;/strong&gt;
This is the big one. The Dokploy MCP server dumped all 546 of its tool definitions — roughly 500k tokens of JSON schemas — into the context of &lt;strong&gt;every request&lt;/strong&gt;. At Opus prices that&apos;s &lt;strong&gt;~$2.50 per call, just to say hello.&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;3. No caching, and retries on every error.&lt;/strong&gt;
The model re-read that giant context from scratch on every call (no prompt caching on Bedrock by default). And my error-heavy setup meant every failure retried 3× — three real, expensive calls each.&lt;/p&gt;
&lt;p&gt;Cheap model? No. Small context? No. Caching? No. It was the perfect storm.&lt;/p&gt;
&lt;h2&gt;The plot twist: I use the exact same thing every day, and it&apos;s basically free&lt;/h2&gt;
&lt;p&gt;Here&apos;s what stung. I use that &lt;em&gt;same&lt;/em&gt; Dokploy MCP server with the &lt;em&gt;same&lt;/em&gt; Opus 5 inside &lt;strong&gt;Claude Code&lt;/strong&gt; — constantly — and it costs me nothing close to this.&lt;/p&gt;
&lt;p&gt;Why? Two things Claude Code does that my self-hosted agent didn&apos;t:&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Deferred tools.&lt;/strong&gt; Claude Code keeps only the tool &lt;em&gt;names&lt;/em&gt; in context and fetches a tool&apos;s full schema &lt;strong&gt;only when it actually needs it&lt;/strong&gt;. It carries ~1% of those 546 definitions per call. My agent stuffed 100% of them into every request. Same MCP server — 100× the context.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Prompt caching.&lt;/strong&gt; The big static block of tools and system prompt gets cached, so repeat calls pay ~90% less for it. On Bedrock, with caching off, I paid full price every time.&lt;/p&gt;
&lt;p&gt;Same model. Same tools. The difference was entirely in &lt;em&gt;how the context was managed.&lt;/em&gt;&lt;/p&gt;
&lt;h2&gt;What the same usage would have cost&lt;/h2&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Model&lt;/th&gt;
&lt;th&gt;Same ~350M tokens&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Claude Opus 5 (what I ran)&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;~$1,800&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Kimi K2.5&lt;/td&gt;
&lt;td&gt;~$215&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Qwen3-Next&lt;/td&gt;
&lt;td&gt;~$50&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;GLM-4.7-flash&lt;/td&gt;
&lt;td&gt;~$25&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;And that&apos;s &lt;em&gt;before&lt;/em&gt; fixing the context bloat. A cheaper model &lt;strong&gt;+&lt;/strong&gt; deferred/trimmed tools &lt;strong&gt;+&lt;/strong&gt; caching would have turned the whole two-day binge into &lt;strong&gt;under $30&lt;/strong&gt;.&lt;/p&gt;
&lt;h2&gt;The lessons&lt;/h2&gt;
&lt;p&gt;If you self-host an AI agent with tools, tattoo these somewhere:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;strong&gt;Don&apos;t default to the flagship model.&lt;/strong&gt; A cheap-but-capable model (Kimi, GLM, Qwen) as the daily driver, Opus only for hard tasks.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Tools are not free — they ride in your context.&lt;/strong&gt; 546 tools in every call is 546 tools you pay for in every call. Trim them, or use a client that defers tool schemas.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Turn on prompt caching.&lt;/strong&gt; Static context (tools, system prompt) should be cached, not re-billed every call.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Your budget alarm lags ~a day.&lt;/strong&gt; Set the cap &lt;em&gt;low&lt;/em&gt;, because you&apos;ll blow past it before it notices.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Retries multiply mistakes.&lt;/strong&gt; An error-heavy config with 3× retries is 3× the burn.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;I killed the VM, kept encrypted backups, and rotated every key. The $1,557 was a one-time tuition payment. The agent itself was a good idea — I just handed it the priciest model, buried it in tools, and turned off every cost-saving feature at once.&lt;/p&gt;
&lt;p&gt;Do the opposite, and a self-hosted agent is genuinely cheap. Ask me how I know.&lt;/p&gt;
&lt;p&gt;&lt;em&gt;A deeper technical breakdown — the token math, the fix, and how I turned a 546-tool integration into a ~700-token skill — is in the &lt;a href=&quot;https://www.dashgin.com/blog/mcp-tool-bloat-deferred-tools-caching&quot;&gt;companion post&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;
</content:encoded><category>ai-agents</category><category>aws</category><category>bedrock</category><category>cost</category><category>post-mortem</category></item></channel></rss>