My AI Assistant is Not Ready for Prime Time

I thought keeping track of a few calendar appointments and to-do items would be easy. I was wrong.

I feel like I've had better luck with AI than a lot of people. Generally, I find my interactions with ChatGPT, Claude, Gemini, etc. to be mostly useful and often productive and time-saving. But maybe that's because I've always kept my expectations in check and never looked to AI to do something like write a legal brief or design a bridge for me.

In fact, I recently used a couple of the "deep research" models to put together voluminous reports on potential vacation spots, down to specific hotels and resorts with a long list of prerequisites, and they all did an admirable job. Researching the top picks, I found these research reports to be mostly accurate and reliable. The one thing missing was an active agent mode that could click through and check specific availability dates for me.

So, I figured taking one of the more advanced consumer AI models and training it to be a sort of personal assistant would be easy. My idea was to drop in notes about upcoming appointments, assignments, due dates, random reminders, etc., just like I was jotting things down in my handy spiral-bound notebook. But in this case, I imagined, the notebook could talk back, remember what I wrote, put it in context, and help me remember and prioritize tasks.

For this project, I picked Gemini 2.5 Pro, and created what Google calls a "Gem," which is similar to a custom GPT or a Project in Claude. That means I could upload a certain amount of information that the bot would always remember. I trained it with a detailed description of what I wanted the bot to help me with, and a second document with background info about me and my work. Both of these were Google Docs, which I could easily link to Gemini. (You can also integrate your Google Drive with ChatGPT, Claude, and other AI tools.)

Unimaginatively, I named my AI helper AssistBot." Not my finest naming hour, but it's a placeholder.

So far, so good. AssistBot and I went over some initial ground rules and ideas for collaborating. Early on, AssistBot told me:

I understand the core mission: act as your proactive, efficient personal assistant, specifically attuned to the pace and demands of a newsroom environment. My focus is on managing your schedule, organizing tasks (including the editorial calendar and your provided to-do lists), and handling information effectively.

I was still stuck on how to get the most out of my robot assistant without manually typing in everything I needed to track. I really wanted a way to import calendar and email info, and ask it to remind me of appointments and to follow up on emails in a timely manner.

But, it was a hurdle to get that information from Outlook (I know, don't ask...) to AssistBot. The big problem with many AI tools is that they're not really integrated with the other tools we use every day, even if they're from the same company. Gemini can't do much to actual documents within the Google Docs/Sheets/etc. universe. Copilot can't go and fix most Windows problems for me, etc. Microsoft has made some strides in integrating Copilot with Office 365, but that requires a specific professional version of Office and a paid Microsoft 365 Copilot subscription.

So, I got the bright idea of grabbing screenshots of emails and calendar entries and giving those screenshots to AssistBot. That's generally a quick and easy way to grab some info...or so I thought.

Despite uploading clear screenshots, having a ton of successful experience using AI-based OCR, and carefully cropping my screenshots to exclude any extraneous or confusing information, AssistBot didn't actually read the information. Instead, it made up a fictional schedule for me, complete with imaginary meetings with imaginary employees. At a glance, everything looked just close enough to correct, and I had to give it a closer review to see that there was a problem. The training data I shared with AssistBot had apparently given it enough information about me to effectively fake what my calendar *might* look like. Here's an example of AssistBot’s output:

To be clear, none of these are real meetings, and none of the people cited are real people. But they do look like meetings I might have. The same thing happened when I asked it to absorb a rough editorial calendar via screenshot from a different tool. I got a lineup of invented articles by imaginary freelancers, but they all sounded like they *could* be real.

It was honestly so completely off-base, I felt embarrassed to ask AssistBot what went wrong. When I did, it apologized and attempted to re-read my screenshots, outputting an entirely different set of invented calendar invites.

I basically ended up with the Seinfeld episode where Kramer pretends to have an office job, eventually getting fired for knowing nothing about business and carrying around a briefcase full of crackers.

So AssistBot v1 was a dud, but I plan to keep experimenting. Maybe I'll to vibe code a new assistant app that interfaces with other tools. Maybe I'll try recreating AssistBot in another AI model. Though, at this point, that feels like I'd be falling into a sunk cost fallacy, when all I really want is a to-do list that talks back.

---

> Get my book here: The Tetris Effect
> Threads: threads.net/@danackerman
> Bsky: danackerman.bsky.social
> LinkedIn: linkedin.com/in/danackerman
> TikTok: tiktok.com/@danacknyc
> YouTube: youtube.com/danackerman
> IG: instagram.com/danack