Ameba raises $7.1M Seed to bolster global supply chains with its AI platform
Ameba
Book a demo
All posts
AI Agents

The way we build cloud-based AI agents is broken

Adam Jarvis21 August 20268 min read

Photo by Kevin Ache on Unsplash

Cloud-based AI agents are easy to change locally but surprisingly tricky to test locally. You can edit the code on your laptop, but you cannot fully exercise the runtime that will host it until you build an image, deploy it, and wait for the response to come back.

At Ameba, we wanted to know what would happen if one specific agent's compute disappeared. Could its session resume while the wider multi-agent workflow continued? The code change took minutes. Testing it meant pushing an image to ECR, updating the AgentCore runtime, and waiting for the deployment to return. The next question sent us through the same loop again.

Flint closes that gap. It runs AgentCore Runtime images locally on your laptop using Docker, so we can stop a container mid-session, start a replacement, and see whether the session continues. All without needing to run a deployment to AWS.

The part that lives beyond your process

Amazon Bedrock AgentCore Runtime is the managed boundary around an agent. You package the code that runs the agent, expose the runtime protocol, and invoke it through AgentCore. Each request belongs to a runtimeSessionId. On first use, AgentCore provisions isolated compute (a Firecracker VM) for that session; later requests use the same logical session. Idle timeout, maximum lifetime, health, can cause the AWS control-plane to Reap the compute, as well as an explicit stop from you. When session storage is configured, a new compute environment can resume with the same files mounted back in. The AgentCore Runtime documentation describes the session as a dedicated microVM, separate from the application code running inside it.

Running an agent process on your laptop gives you a different boundary entirely. A local dev server exercises your agents request handler, but it doesn't reproduce a managed service deciding when session compute is idle, terminating it, starting a replacement, and restoring its workspace. AWS's own local development server covers that first kind of testing, but it still requires a cloud deployment to use the control-plane. Flint targets the second kind of testing. It covers the runtime API and the lifecycle around a packaged container image, both of which matter in a multi-agent application.

Make the lifecycle observable

The example runtime that ships with Flint does almost nothing. It returns a message, increments a counter, and stores that counter in /workspace. That is enough to tell the difference between a new process and a resumed session:

  1. Start a session. The first invocation omits the session header. Flint discovers the flint-runtime-example image, starts its container, creates a logical session, and returns a session ID. The response has "count": 1.
  2. Reuse the session. Send the next request with that session ID. Flint routes it to the existing container, which reads the same workspace and returns "count": 2.
  3. Remove the compute. Kill the container that Flint created for that session. This simulates the compute disappearing while the session still exists.
  4. Invoke again. Send another request with the original session ID. Flint starts a replacement container, mounts the session volume, and returns "count": 3.

The counter is deliberately boring. If Flint had created a new session in step four, the value would have gone back to 1. Continuing at 3 shows that the container was replaced without losing the session's workspace.

The lifecycle looks like this. Flint performs the same control-plane work locally that AgentCore performs in AWS, while the compute is replaceable:

Why choose a managed agent runtime?

A generic sandbox could have run our code just fine. The real question was which abstraction we wanted to own.

A generic sandbox gives us a machine or process boundary. We would still need to build the agent endpoint, session control, runtime lifecycle, and the integration around them. Other platforms provide more agent-native environments, coding-agent sessions, or infrastructure control. Those are useful products, but they solve a different problem.

For Ameba, the useful abstraction is a managed agent endpoint. AgentCore owns the runtime identity, invocation surface, sessions, and compute lifecycle. Our agent owns the work it performs inside that boundary. We do not want to build an agent control plane on top of a generic sandbox, or operate a small cloud platform just to keep agent sessions alive.

We also run multiple full agents in concert, not just one agent as a parent process with a few lightweight sub-agents inside it. Each agent often uses sub-agents for context managemtn, but these 'full-fat' agents need a real runtime boundary rather than a function call inside a shared process, while the wider workflow coordinates what they do. AgentCore gives us that service-level boundary.

Flint's job follows from that choice - it's a local implementation of the same AWS contract, so we can test our agent against session and container lifecycle before any of it reaches AWS. Keeping us moving just as fast as other more generic sandbox solutions out there.

Keep the runtime boundary intact

Flint does not ask the application to use a second local protocol. It discovers Docker images carrying a required runtime protocol label. Flint uses the final repository component of the image name, without its tag or digest, as the runtime ID:

LABEL ai.ameba.flint.runtime.protocol="HTTP"

A runtime can be this small

An example of an AgentCore runtime is just a Bun HTTP server. It keeps its counter in /workspace, answers the health check, and handles the invocation route that Flint forwards to:

const counterPath = "/workspace/invocation-count";
let counterQueue = Promise.resolve();

const incrementCount = () => {
  const operation = counterQueue.then(async () => {
    const text = await Bun.file(counterPath)
      .text()
      .catch(() => "0");
    const stored = Number.parseInt(text.trim(), 10);
    const count = Number.isSafeInteger(stored) && stored >= 0 ? stored + 1 : 1;
    await Bun.write(counterPath, `${count}\n`);
    return count;
  });

  counterQueue = operation.then(
    () => undefined,
    () => undefined,
  );
  return operation;
};

Bun.serve({
  hostname: "0.0.0.0",
  port: 8080,
  async fetch(request) {
    const path = new URL(request.url).pathname;

    if (request.method === "GET" && path === "/ping") {
      return Response.json({ status: "Healthy" });
    }

    if (request.method === "POST" && path === "/invocations") {
      await request.arrayBuffer();
      const count = await incrementCount();
      return Response.json({
        message: "hello from flint-runtime-example",
        count,
      });
    }

    return Response.json({ message: "not found" }, { status: 404 });
  },
});

In this example, the counter is only standing in for the agent 'session'. In a real runtime image, POST /invocations would be the entrypoint into an agent harness. Said harness might load session context and tools, call the model, and return the agent's response after a few runs. For a simpler runtime, it could call an inference API directly. Flint does not need to know which one. It starts the image, forwards the request, and preserves the session boundary around it.

The container image only needs to make that server reachable on port 8080 and tell Flint which protocol it provides:

FROM oven/bun:1-alpine

USER 0
RUN addgroup -S -g 10001 runtime \
  && adduser -S -D -H -u 10001 -G runtime runtime \
  && mkdir -p /workspace \
  && chown 10001:10001 /workspace

COPY --chown=10001:10001 server.ts /opt/runtime/server.ts
USER 10001:10001
WORKDIR /workspace
EXPOSE 8080
LABEL ai.ameba.flint.runtime.protocol="HTTP"
ENTRYPOINT ["bun", "/opt/runtime/server.ts"]

The checked-in example adds more error handling around file access and serialises counter updates. The important boundary is small. It is an HTTP server, a writable workspace, and a protocol label. Flint derives the runtime ID from the image name.

With the flint-runtime-example image tag, the runtime ID is flint-runtime-example. Flint handles the protocol-specific port and route, just as AgentCore does. For an HTTP runtime, Flint accepts POST /runtimes/flint-runtime-example/invocations and forwards the request body to the container's POST /invocations endpoint - matching the AgentCore Runtime service contract

A minimal local run looks like this:

docker build --tag flint-runtime-example examples/runtime
docker compose -f compose.example.yml up --build --wait

FLINT_ENDPOINT='http://127.0.0.1:35469'
FLINT_ACCOUNT_ID='000000000000'

curl --fail-with-body \
  --request POST \
  --header 'content-type: application/json' \
  --data '{"prompt":"hello"}' \
  "$FLINT_ENDPOINT/runtimes/flint-runtime-example/invocations?accountId=$FLINT_ACCOUNT_ID"

With the runtime image already on the local Docker host; Flint does not pull it from a registry. On the first request, Flint starts the runtime container and forwards the payload. You can configure any AgentCore SDK client to use the Flint's endpoint, and your applications still sees the same invocation contract as they would when running against your production cloud envrionment.

Test the awkward part first

A single successful invocation doesn't tell you much. What matters is whether you can test out your actual product locally, from a single Agent workflow to a multi-Agent orchestration - Flint can handle it all.

Flint is available on GitHub. The repository includes a small runnable example and Docker Compose setup, so you can put a local runtime image behind the same style of invocation endpoint, reuse a session, remove its container, and verify recovery before deploying to AWS.

AgentCore is still what runs in production. Flint just shortens the distance between changing agent code and finding out what happens when its compute disappears.

Flint is an independent project and is not affiliated with, sponsored by, or endorsed by AWS.

See it run on your own orders.

Pick a process and a few orders to run it on. In 30 minutes, we'll show it fully automated.

Book a demo