AI is already smart, but it still doesn't know me well enough
I've been working on personal agents lately.
Before this, I worked on databases and LLMs, including model training and inference. Most of my attention went into making models smarter, improving performance, and lowering costs.
Now that I'm starting to work on agents, the problem feels a little different.
For example, I'd like AI to tell me each morning what meetings I have, which emails need replies, who I spoke with yesterday and what we discussed, and which promises I still haven't followed through on.
That doesn't sound particularly complicated.
Today's models generally have little trouble understanding an email or summarizing a meeting. Even a smaller, cheaper model can often do a good job.
The problem is that it has no idea what I did yesterday.
My email is in Gmail, my schedule in Google Calendar, and work discussions in Slack. Then there's WeChat, LINE, and all the files scattered across my computer.
However smart AI is, without that information it can only wait for me to tell it what to do.
Sometimes I feel that we've spent so many years making AI smarter, only to discover, when we actually try to use it, that what it lacks most may not be intelligence, but an understanding of us.
Does convenience mean handing over our information?
This brings me to an interesting contradiction.
Of course I want AI to do more for me.
Ideally, I wouldn't need to open my inbox and read every email one by one. It would know what matters and what doesn't. If someone asks about last week's discussion, it could find the record. It might even remind me of something I've forgotten, without being asked.
But to do those things, AI needs access to my emails, chat history, calendar, and perhaps even photos and private files.
When using ChatGPT before, I could choose what to tell it.
If I want a personal agent to work proactively, I can't keep providing every piece of information manually.
In other words, we want AI to know us as well as possible, yet we don't want others to know too much about us.
That's quite a contradiction.
We've made similar choices with internet services before. For convenience, we entrusted photos to cloud albums, emails to email providers, and plans to calendars.
But that information used to be scattered.
One service might know what I've photographed, another which meetings I've attended, and another what I've bought.
With personal agents, we actually want one AI to connect all of it.
Each piece of information, on its own, may seem unremarkable.
Put someone's emails, schedule, photos and conversations together, though, and it may become possible to infer things even they haven't noticed about themselves.
That does give me pause.
Does AI really need to know everything?
When building models, we always want more and better data.
But working on personal agents has made me wonder whether more is necessarily better.
To schedule tomorrow's meeting, for example, AI might only need to know which times in my calendar are already taken.
It doesn't necessarily need to know what every meeting is about.
Finding a photo from last year's holiday doesn't mean it needs to retain the contents of all my photos indefinitely.
Reminding me to reply to an email shouldn't mean it can send that reply without confirmation.
There may be an important distinction here:
Letting AI use my information doesn't mean I want to hand over control of it as well.
I didn't think much about this before. When researching models, I was mostly concerned with whether the data could be used and whether the model could learn from it.
Now that I'm trying to turn an agent into a product people can actually use, I have to think about these things.
Who can access this information? When? What remains after it's processed? If I switch to another AI, what happens to the old memories?
None of these questions sounds quite as exciting as model performance or benchmark rankings.
But increasingly, I think they may be what determines whether personal agents really become part of everyday life.
What if AI's memories didn't belong to AI?
There's another idea I've been thinking about lately.
Many AI products now emphasize memory: remembering a user's preferences, habits and past conversations.
That really is convenient.
But if those memories all live inside one AI service, what happens when I change services?
Suppose I've spent three years with an AI. It knows my work, my family, my habits, and the decisions I've made.
Three years later, a smarter, cheaper model comes along.
Of course I'd like to switch.
But the new AI knows nothing about me. We have to start again.
It's a bit like changing phones and discovering that your contacts can only stay on the old phone.
So I'm beginning to think that the core of a personal agent perhaps shouldn't be any particular model, but the information and memories a person has accumulated over time.
Models can change, agents can change, and even the applications we use can change.
But my memories should still be mine.
Seen this way, local storage, personal clouds, and data permissions we can manage ourselves may become important again.
Of course, keeping things locally doesn't guarantee security. Devices break, software can have vulnerabilities, and an agent may need to transmit some information when it calls an external model.
Many useful features also genuinely depend on cloud services.
So I don't think everything necessarily has to be locked inside our own devices.
What may really matter is whether we can understand where our data goes, decide what can be shared, and retain the ability to take it back later.
How much are we willing to hand over to AI?
Looking a little further ahead, the question seems to extend beyond personal information.
Many agents are moving beyond answering questions to carrying out tasks on our behalf.
Replying to emails, managing schedules, organizing files, even buying things.
Previously, AI gave us suggestions. We still decided what to do.
But what happens next?
If an agent handles dozens of little tasks for me every day, will I really check each one?
If it got the last hundred right, I might not bother looking at the hundred and first.
That trust builds gradually.
At some point, I may already have handed over a substantial share of decisions about my work and life without particularly noticing.
This makes me think that personal agents may ultimately have to address a question more complicated than model capability:
How much are we willing to hand over to AI?
I don't have a particularly clear answer yet.
As someone who researches models, I naturally want AI to become more capable. As someone trying to build products, I also want it to become a useful part of people's daily lives.
But as an ordinary user, I still hope that, even if AI eventually handles most things for me, I can always find out what it's doing and ask it to stop.
Perhaps a good personal agent won't be the AI that knows all my secrets and makes every decision for me.
Perhaps it will be one that understands me well enough while always leaving me in control.
As for me, I'll probably keep worrying about privacy while also finding it hard to resist handing more things over to AI.
There are, after all, just too many emails and messages to deal with every day.
Could technology let us benefit from AI while keeping control of our own data?
Illustrations generated with AI.