Becoming the assistant

When I decided to build Agent M, I thought that what I needed was something I could install in a repo that I was working with, and get a collection of skills and automation that would speed up working on a project. That’s exactly what I got, and it was great. As I grew more accustomed to the tools though, and as harnesses evolved, so did my needs.

So Agent M grew, first the number of skills, then I needed a place to group them, so I made a plugin system. Then I realized that the memory system was its own thing, so I split plugins (Crickets) and Agent M (memory), and it kept going. My feature list got longer and longer no matter how fast I built.

Then it hit me. Agent M doesn’t fit in a repo any more. I want to use it in every repo I work on. Not only that, I want to be able to keep plans and designs and work on them. I need roadmaps, and planning, and issue management and so many things. None of this really belongs in a repo. It should live alongside the harness itself. In my case, Claude Code or Antigravity. Those harnesses had developed to the point that they could easily support a setup like this.

By this point, Agent M and Crickets had grown to tens of thousands of lines of code. I had multiple plugins to manage all aspects of my development lifecycle, persistent memory with backed-up file storage, issue and project management, mirroring to Google Drive (again, for redundancy). It was becoming more useful in other things too. This blog, for example, was designed, planned and built in just three days with Agent M, which made it easy to take the lessons I’d learned from other projects, and through its evolving memory, avoid those mistakes when building something new.

So it came time to move Agent M to a new permanent home, the “user” level, or the host level. The best way for it to be useful was to split it in two. The harness code itself, the part that plugs into Claude and Antigravity, and an always-on companion (daemon) that can stay running in the background, taking care of the memory and file storage itself among other things. For a while, I thought I should support both a repo-installable version and a full install version, but as time went on, I realized that this would just provide a sub-par experience. So I’ve been pivoting since then to the full companion version, or said differently, I’ve been turning Agent M into the personal assistant I had imagined in the beginning.

Some call it an AgentOS, I call it a personal assistant, you might call it something else. Agent M is not done yet. My laptop isn’t always on, and I’d like for Agent M to always be available so it can perform tasks for me on a schedule, things like looking for the latest research on agents and harnesses and seeing what it can learn from it, or curating a list of articles from reputable websites for me to read in the morning as I get ready for work. My laptop can do these things, but it would be nicer if it didn’t have to. What I need belongs in a VM, a container, a VPS. Somewhere else.

So that’s where I’m going next. Agent M still needs a lot of work before we get there. Biggest is consistent reliable data gathering and learning from my usage. Turns out that’s a harder problem to solve than just asking the agent to summarize your day. But that’s a topic for another day. Time to get back to work.