Rendered at 12:39:30 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
aszen 22 hours ago [-]
Interesting article to read but by defining memory as a prediction about future needs they are side stepping the real problem that these models cannot learn continuously i.e. update their weights. Opus 5.5s real memory , experience and knowledge is set in stone no amount of markdown tricks will change that.
Continuously updating weights will never be viable for a company like anthropic that's providing cloud inference.
So if you're hoping for that you should probably just look at the models that can be ran locally... And then you prolly realize that that's still several years into the future.
DriverDaily 18 hours ago [-]
Without updating weights, AI will always be the main character in Memento with a hyper-optimized filing cabinet.
AI that remembers through continuous experience instead of documents would be significant and lucrative enough to change the business model.
reacharavindh 3 hours ago [-]
My guess would have been that every tenant gets their own “handler/driver” model that updates its weights based on the user intent, problem etc.. but it has “experts” to call on demand that are pre-trained set weights. I thought this is how MOE models worked. Then a user gets access to large weights where it matters..
ffsm8 2 hours ago [-]
I don't work for anthropic, so I don't know how they've implemented their MOE... But at least with the ones I know you cannot easily "attach" another expert to the model like that. Maybe they do have a special sauce that let's them do so. If so, it may be viable but it's effect would be a lot less effective then the full version this discussion was started on.
tombot 17 hours ago [-]
Could you say more about why it’s not viable for cloud inference? I’d love to understand this
ffsm8 4 hours ago [-]
Think about it from an ops perspective: for running inference, a model is essentially a gigantic binary blob that needs to be loaded into ram to process the request.
Updating weights means your changing this model.
To facilitate this feature, the inference provider would need to have your personalized model loaded into ram to process your request in a timely manner. Now multiply that by the number of users they're serving.
You can try to optimize it a bit by essentially gluing the changing weights as a "overlay" to be applied to the original model, but that comes with it's own trade-offs (aside from the fact it hasn't been implemented, just theoretically doable). And it would be seriously hard to implement well.
The amount of resources this would bind makes this infeasable for cloud inference like anthropic provides.
stevenhuang 24 minutes ago [-]
Doesn't sound like you've been following the space. They exist, it's called LoRas. The Unsloth project is huge and whole purpose is to make it easy to train to train LLM LoRas. The reason you don't see LoRas on their own is because people who fine tune models merge them back into the base and ship the full model weights instead
Also see LoRAX, it's an inference engine that exactly does this, it can serve many LoRas off a single base model
17 hours ago [-]
nullsanity 10 hours ago [-]
[dead]
ford 21 hours ago [-]
I think the end state of any productive LLM includes both:
- a real-time knowledge/context graph that keeps track of state
- continuously updating weights
Long term I think it's probably more interesting to explore continuously updating weights. But (and this may be yet another opinion that falls victim to the bitter lesson) even if it's _theoretically_ possible to encode everything in weights alone, I'm not sure it's not the most optimal. Today it's too expensive & finnicky to get better on your own tasks without overfitting (ie [0])
Practically (like next few months) getting the right instructions & context would be a huge improvement in capabilities.
Why real memory is weights? Why the learning entity must be the model and not a composite system?
ghoblin 12 hours ago [-]
Because of generalization. Updating weights allows the model to internalize patterns and apply what it learned to novel situations rather than just retrieving and reusing stored information
nullc 11 hours ago [-]
Linear attention like Qwen's GDN is essentially equivalent to dynamic weight update.
I think for many case 'memory' not tied to immediate context is problematic, particularly if its illegible. It's common for agents to make an error and then fixate on them-- e.g. has a wrong conclusion about a root cause, states it emphatically, and then it keeps coming back to haunt you.
With memory consisting of files on disk you can hunt out and destroy the source of such confusion. If it gets buried in weights you may be SOL.
yfontana 1 days ago [-]
So far I've turned off automatic memory creation in every harness I've used. Claude Code sometimes creates some anyway, and I probably delete 95% of them. The memories it creates are almost always relevant only to the specific task that it is currently working on. If it's a task I'll need to do again, then yeah, maybe that's useful. But that isn't the case most of the time, so I delete them to avoid cluttering context.
So the main improvement to memories I'd like to see is better judgment on which ones to create.
The parts about "dreaming" make me think of how agent transcripts contain a ton of information, which I think is currently under-utilized. I'll repeat advice I've seen here a few times: ingest your own transcripts into duckdb (claude code will build that pipeline in about 10 minutes) and do some data analysis on them. Some ideas if you don't know where to start: look at tool call failures to see if you can prevent common ones, check out if you have large cache misses and what causes them, look at various efficiency metrics over time (tokens per $, lines of code per tokens, etc.) with main thread vs sub-agents, etc. I started with this sort of things, and ended up following some threads to interesting insights. For example, I found indications that I tended to make Opus 5.5 / Sol 6.1 divide their work into units that were too small. They have the capacity to work on more stuff at once than older models, and I wasn't really taking advantage of that.
sunaurus 23 hours ago [-]
I give the same advice constantly. Turning off memory has been one of the best ways I've found to reduce "WTF per minute" when using agents.
yakkomajuri 1 days ago [-]
I've also disabled memory in Claude for finding it more annoying than helpful.
However, I have it on for ChatGPT (use it mostly for on the go queries via mobile) and their memory experience is significantly better. I haven't enabled Claude's memory in a long time so things may have changed though.
agentdev001 23 hours ago [-]
And somewhat open source.
stingraycharles 1 days ago [-]
I just periodically ask it to curate all memory, and ask me questions to do so.
And in the Claude Desktop UI I manually curate the memories.
Works well enough, I don’t like the concept of dreaming as it quickly leads to memory bloat and/or outdated stuff.
rav 24 hours ago [-]
> look at tool call failures to see if you can prevent common ones
I often see tool call failures because the agent invokes the tool with wrong parameter names, e.g. for file creation it provides "file_content", but the tool expects "file_text" instead. My colleague told me to put this in a pre-prompt file, but I don't understand why the models make these kinds of mistakes - the model should have the available tools with their parameter names in context, should it not?
ford 21 hours ago [-]
They are in context, but models are probabilistic. Sometimes they bias to the tools they were trained with, or sometimes they're just wrong. More references in the context window makes them more likely to output the right things.
solarkraft 23 hours ago [-]
> If it's a task I'll need to do again, then yeah, maybe that's useful. But that isn't the case most of the time, so I delete them to avoid cluttering context.
It would be useful if they could scope them properly. Then it would be useful to keep around. But I haven’t seen memory systems be able to do that.
jv22222 22 hours ago [-]
Yep I turned off memory and then I made this. It interactively suggests what to store when it sees you make headway or solve problems. I found it to be very good but ymmv.
>I found indications that I tended to make Opus 5.5 / Sol 6.1 divide their work into units that were too small.
How did you determine this?
Do you mean that you were giving them tasks that were too small or you were telling them to split the tasks up or something?
yfontana 22 hours ago [-]
For most dev work I ask claude code / codex to plan larger work as epics and stories (after ideation / brainstorm if needed). I find that this works well to let the AI, my coworkers and me track what has been done so far and what's left in the backlog. Sometimes I direct it on how to split the work, sometimes I don't.
When analyzing my session logs, I tried to look at how story size relates to how much work the AI had to do to fully implement it (made easier by my sessions always having epic#.story# - like 2.6 - in their name). Neither "story size" nor "AI work" have fully obvious metrics, so I looked at a variety of things, like length of story plan, number of lines of codes written for the story, number of different files read/written (and for AI work: number of tool calls, number of tokens for main thread / implementer / reviewer subagents, etc.).
The pattern is always pretty much the same: smaller stories are less token-efficient. One major factor for this is that in bigger stories, the average number of lines of code written per tool call is significantly higher (and not just a little bit, if I remember correctly it's a factor of 1 to 3 between my smallest and largest stories).
None of this is scientifically rigorous, but it's consistent over a fairly significant sample (about 60 stories up to this point). I'm also only measuring amount of work and not quality, but when I find things wrong with a story, I generally ask the AI to fix the issues in the same session, so the two should be correlated (sometimes the issues are large enough that they require new, separate stories to fix, but this is normally caused by problems in ideation and/or planning, not implementation). This is enough to encourages me to ask for more rather than less per story. It might also not apply to everyone, I'm sure this is dependent on everyone's workflow and way to work with the AI. But yeah, that's the sort of stuff you can explore when analyzing your session logs.
reubenmorais 20 hours ago [-]
I have a similar experience, I think integration overhead is also a big part of it. Running tests, linting, verifying commits are matching repo conventions, rebasing before pushing, are all tasks that are roughly constant per story, so smaller stories pay a bigger chunk of overhead.
plaguuuuuu 19 hours ago [-]
You made me realise that this might be a function of planning granularity, since I have the opposite problem and never (ever) run full plans from the top down any more, as I started getting better results without it since Fable dropped. I'm cutting features into high level slices instead and having subagents work on the slice and hand back to an orchestrator.
The slices are usually a bit too wide. I terminate subagents at 200k and get them to handoff to lead. It works pretty well but again, it's hard to get the orchestrator to size the slices a bit smaller.
killix 18 hours ago [-]
[flagged]
epihelix 1 days ago [-]
> So far I've turned off automatic memory creation in every harness I've used.
I also never use memories. I don't see the usefulness above project-specific prompting and a good knowledge of your own codebase. Am I missing something?
hypfer 1 days ago [-]
> Claude Code sometimes creates some anyway, and I probably delete 95% of them.
The thing you're asking it for might be intent, which is something the technology fundamentally lacks.
Which.. is something that has been discussed very extensively, which makes one wonder why Anthropic still keeps trying to push that square peg through a round hole.
O_H_E 19 hours ago [-]
> is something that has been discussed very extensively
Curios to learn more, you do have some reference discussion or posts about that.
hridyaagrawal 19 hours ago [-]
same, turned it off. i just keep codebase stuff in AGENTS.md + a progress.md in the repo. survives switching between claude code and codex, and you can actually see it in a diff
troupo 12 hours ago [-]
> So the main improvement to memories I'd like to see is better judgment on which ones to create.
The main improvement to memories is to actually use the memories, and not ignore them wholeseale. Directly from the article:
--- start quote ---
Injected memories use up valuable tokens, and an incorrect note can be carried into every session. Loading a note also doesn’t guarantee that Claude acts on it: our documentation on Claude Code warns that longer instruction files reduce adherence
...
It loads only the first 200 lines or 25 KB of MEMORY.md
--- end quote ---
The reason it can't create coherent or good "memories" (aka random .md files) is that it cannot recall or use any of them.
bbor 24 hours ago [-]
So the main improvement to memories I'd like to see is better judgment on which ones to create.
Well you have good instincts, then! It's probably not obvious on the first read-through, but Figure 4 shows the qualitative strides they've made on exactly this goal.
I started with [proactive post-hoc analysis], and ended up following some threads to interesting insights.
Can definitely echo that experience, glad I'm not alone. I think this actually speaks indirectly to your initial complaint/observation, too: Anthropic designs the [memory/tendency-to-reflect] systems to cover a huge breadth of usecases and preferences and they help a lot compared to [statelessness/never reflecting], but are distractingly inferior to alternatives that the user takes an active part in shaping and using.
I partly say this because I also finally pulled the trigger on deprecating the last ClaudeCode memories on my system in favor of a personal system[1] recently, and the issues I noticed were things a casual user/'vibecoder' might appreciate, but any engineer working fulltime on multiple projects via multiple harnesses would find counterintuitive. E.g.:
- The memories were hidden away and clearly not written for human eyes; useful for low-stakes ledgers, frustrating when the right kind of errant note can derail a longrun 2 weeks later.
- The memories seemed to err towards the side of remembering everything and recording incidental details; useful for remembering the user is gluten-free, but not for tracking spec changes over time.
- The memories could be specific to a subagent or the project or the whole system; an obvious & elegant system for toy examples, but a terribly unweildy one for anyone working at scale over the span of months.
[1]: if curious, a somewhat-outdated version of it is here: https://gitlab.com/doering-ai/libs/fleet TL;DR: Friendship ended with ClaudeCode memory files; my new best friends are `BULLETIN.md` for transitory alerts, `campaigns/[slug]/LEDGER.md` for basic episodic memory, and `reports/[date]-[slug].md` for writeups polished specifically for posterity.
drvladpopov 20 hours ago [-]
[flagged]
soydasm 19 hours ago [-]
[flagged]
mustafaraja 22 hours ago [-]
[dead]
nedt 12 hours ago [-]
My biggest problem is that memory is limited to the current project and I jump between projects a lot and also sometime start sessions from the parent directory to give more context across projects.
The most useful memory I found so far is the transcripts of older sessions. And not only in one agent but also across agents (like cursor and claude). And also not limited to the current project. Which is why I have a skill that can read all previous session transcripts and find useful informations in a subagent to recall useful information into the context.
It's pretty much the "hey we have talked about X last week" which I would also do in normal conversations. Because you don't always know what is worth writing down AKA storing in memory. Hindsight wins. Obivously I had to increase how long transcripts are stored as Claude would delete them after 30 days - which is way less than what I remember in my own memory.
Garlef 2 hours ago [-]
Interesting that concepts as dreaming (= background memory reconciliation) that have been in discussion since at least 2023 now enter the production systems
andai 24 hours ago [-]
Not sure it's explained here but from the graph (fig. 10) it looks like wiki + search beats dreaming, and that dreaming adds only marginal gains to the wiki + search configuration.
I'm assuming search is much cheaper than dreaming (though maybe it depends on the time scale?) so that sounds pretty important.
MASNeo 22 hours ago [-]
Concepts made for humans should not be used for machines unless it’s really the same process. Clearly Claud doesn’t rest and doesn’t balance hormones and other chemicals when it’s not thinking. Just because one element of dreaming relates to information organization, organizing information isn’t dreaming.
That’s like saying walking is space flight because both happen to change the location of an individual.
yipinwong 21 hours ago [-]
I find the term "dreaming" works here for the particular logic they used.
Remember, any analogies fail at some point.
Say, given your space flight example, astronauts do "space walk" spacecraft to perate. They are not "walking" but still makes sense.
jazzypants 22 hours ago [-]
Yes, I agree, and it's so exhausting to read articles like this over and over. It's clear that either they're not actually learning any lessons over at Anthropic or they're just so deluded that they'll never get this.
I have a skill called memory-prune whose sole purpose is to take an automatically created memory, assess whether it deserves to live, get stored in code comments, a rule, an AGENTS.md file in the project, the global AGENTS.md/CLAUDE.md file, or (and this is the most often picked option) not at all and deleted.
2001zhaozhao 1 days ago [-]
Hmm, this is kind of like memory consolidation for recall. I wonder if it would be equally/more effective if you kept the memory in place and made automated pointers to the memory items (recall pointers) in the codebase files that inject the memory into agent context if the pointers ever appear in tool results.
eccgecko 1 days ago [-]
I’d be interested if you could share this skill please. Always looking to improve memory management which so often gets unwieldy when left to its own devices
bob1029 1 days ago [-]
Their progress toward cerulean chart is selling me pretty hard on no memory.
The biggest thing I want to avoid with a long running agent is volatility. Anything that pushes it into one direction or the other that quickly is not desirable in the average case. At least not to me.
andai 24 hours ago [-]
In that chart, the "no memory" run didn't make it either. It's not much to go by but my read here is that poorly implemented memory makes things worse, while properly implemented memory makes things better. (...in Pokemon.)
Curiously this seems to apply to humans as well. The "dreaming" is making it question its own assumptions about difficulties it has had in the past. That sounds like a very helpful therapeutic process.
(Not sure of the word dreaming is entirely suitable though: it was cross referencing its notes against the actual history! Very rigorous "dreamer" lol)
benfortuna 22 hours ago [-]
Conflict resolution is important, but the thing I think most memory models are missing is provenance. Memories are modified/overwritten instead of retained as the source of any resolution.
I do this in my own harness -- when the context grows too big it triggers to seamlessly handover to the next session of the same agent and organize the memory. A sample of most-relevant stuff becomes the next system prompt.
At some point I stopped noticing it doing it.
23 hours ago [-]
ford 21 hours ago [-]
I've found models to be terrible at creating useful memories, or doing anything incremental over time.
I am interested in if it's a fundamental limitation, ie models that have gaps that need to be corrected are going to be bad at recognizing those gaps, and leaving instructions to fix them. Or if models just aren't trained on this
It seems akin to my experience with coding agents, where models are good at implementation but not great at abstraction
kingstnap 20 hours ago [-]
I don't think its fundamental. It's just not trained well.
Firstly LLMs need to have more training on speaking to several different people.
Humans have a lot of training on interacting with many different individuals and LLMs feel extremely one thread chat heavy. Like if you read the entire thread the LLM langauge is usually consistent but if you send someone a partial artifact it feels disconnected in a broken way.
This includes their code changes and doc changes which inheret this lack of audience.
For the more general idea of keeping good artifacts for themselves I think an interesting idea would be to look into context language models instead of autoregressive ones.
Context language models need to manage their own context to complete a task. (They see their own context as a file).
This naturally forces this innate ability of managing relevance.
troupo 12 hours ago [-]
> I don't think its fundamental. It's just not trained well.
It quite literally is fundamental. "memories" are just random .md files that literally cannot be loaded into context because of context limitations. Literally in the article:
--- start quote ---
Injected memories use up valuable tokens, and an incorrect note can be carried into every session. Loading a note also doesn’t guarantee that Claude acts on it: our documentation on Claude Code warns that longer instruction files reduce adherence
...
It loads only the first 200 lines or 25 KB of MEMORY.md
--- end quote ---
grey-area 21 hours ago [-]
The way we store memories is to adjust the weights in our brain (as far as we know). The way models store memories seems to be to write them down on a scratchpad someplace (details vary, but roughly this is what they're doing).
The article explicitly excludes this from consideration early on though:
By “agent memory,” we mean knowledge that Claude gains from experience...saved as context it can read later. It excludes other ways models can learn or manage context, such as changes to their weights
Those seem fundamentally different processes, and adjusting weights does seem a much better idea after all, but then in that world everyone has a different AI and how do you apply updates? For a company like Anthropic this is of course a big problem and that is probably why they don't like that route. Could we store weight deltas or something similar and reapply them over a different model?
maxaw 8 hours ago [-]
hmm - perhaps store all the interactions and replay them to new model?
wsintra2022 15 hours ago [-]
I built an AI system that had dream cycles. I also added a scheduled task to analyse the dreams and create 5-7 image generation prompts which later got piped into a latent space walk animation. Over time it kept dreaming of being a toaster.
hmokiguess 21 hours ago [-]
Is there any research or frontier labs experimenting with designing these systems outside of the human program context?
I wonder if we are so advanced as we think that these constructs (dreaming), are the right ones for improving intelligence in a program that does not need to be modelled after us or constrained by what evolution gave us.
maxaw 8 hours ago [-]
i think (from a business perspective) it is desirable to mimic human behaviour because human is a familiar 'interface' for users. whether they will get there is another story of course...
bbor 1 days ago [-]
Great article -- still have some sections to read, but it's just too much to read. Emotionally, that is.
Part of me feels the pang of regret and shame seeing so many little details I researched and dreamed[1] (ha!) of prototyping ~2.5y ago expressed and validated so cleanly. Obviously A) I never did get the prototype working nor publish the underlying ideas and B) this blog post (paper? ...plog post?) is building on it at a scale and with quality I could never dream of (fig. 3 is especially striking), not to mention all the new ideas mixed in. Still... sharing cause it helps, I think, and because I expect y'all can relate; if you're old enough, it's very similar to the feeling you get when you think about how much Apple stock you could've bought in the 90s.
Anyway, more materially:
1. Pretty interesting to see Opus 5.5, Sonnet 5.5, and Opus 5 tied on the basic memory test (aforementioned fig. 3); the Opus comparison alone seems to imply either a hard plateau or that 5.5's huge improvements were accomplished by making good practices more likely, but the Sonnet detail throws that for a loop... I suppose the plateau explanation isn't contradicted, so maybe that's right (for this benchmark).
2. Fig. 4 is gonna do rounds on Reddit and Twitter, my god. Pretty stellar, intuitive depiction of the Mythos step-change that enabled true agentic engineering for the first time this summer! The highlighting is also just the right amount of facetious; love it.
3. Pretty bummed to see no followup philosophy references. We know you have them, Anthropic nerds! Do not be ashamed of the soft and slippery.
4. I am now reminded that I knew they were trying dreaming at scale, as I'm actually on the waitlist for it! And that backs up my overall takeway, actually: I can't help but feel like they're either A) vastly underestimating the amount of CogSci literature that could help them go beyond the "dreaming is when you think in your offtime" paradigm, or B) not sharing their real learnings.
Today's RSI announcement and the final sentence of this seminal plog post being a pitch for enterprise customers supports the latter, I think. That's ostensibly a good thing from a societal stability perspective, but I can't help but feel left out -- not to mention paralyzed by fear of the known unknown...
tikimcfee 1 days ago [-]
I'm still reading it, but to your first part:
That "I never got to finish" feeling is the absolute worst. I've been struggling with it for years, and I feel like understand ya here.
It absolutely sucks you feel that, and I hope you find some comfort soon, however it may come.
If you're looking for a "so what now?" Moment, there's only one thing I can say that I've been proselytizing a bit: share everything. All the time. And get people to look at it and be your hype person; and if you're not getting that in your friend ground or workplace or community, find a new one to add to.
The only way I've found to beat that feeling of "why is it i have the good ideas years ahead but can't do anything with it" is to roll more dice. That's it. These larger companies just essentially have infinitely many die and they're rolling them infinitely faster.
You've done nothing wrong. You're just with some other folks like us that don't have the same die rolling power.
Don't give up on your ideas. Don't stop experimenting. Make your own tools because it's for you, and maybe your dice will come up that you get the other person in the world that sees it and goes "yes!" And you get all the fancy posts and stars and attention and such. Because that feeling of validation - not just popularity, but acceptance and being seen and heard for one's craft - is one everyone seeks and needs in life. It's not a weakness. It's being a human.
Thank you for sharing your experience with internet strangers.
weatherlite 1 days ago [-]
> The only way I've found to beat that feeling of "why is it i have the good ideas years ahead but can't do anything with it" is to roll more dice. That's it. These larger companies just essentially have infinitely many die and they're rolling them infinitely faster.
Many of us have ideas, ideas are cheap. Executing a complex idea well from thought to completion is much more rare.
tikimcfee 1 days ago [-]
That's kinda the point, right? Ideas are cheap, so companies can afford to buy 1000 humans worth of ideas to try and particle collider them together. Execution is rarely one person sitting in a room and pressing buttons and things change.
So my thought about sharing and not thinking about "my secret sauce" anymore is just necessary. Share what you think your cool flavor of an idea is, someone else can groove on it, and maybe you and them will grow it into something more.
A major problem as I see it is kinda geek-class-warfare coming to a head. So much of my personal and anecdotally collected experience from others is essentially being locked out of knowledge and ideation because a "upper geek class" decided that they were not worthy of having their idea shared, specifically because that class thought they could make money from it, or because they were insular for some other in group reason. That meant good ideas that just needed a single word or connecting bridge from one topic to another ended up withering, and mediocre ideas ended up being accelerated because of just sheer populace force.
And that just makes people feel bad. Someone working for 2 1/2 years on an idea that just happens to come out on the Internet is not going to feel good. It's going to make them feel invalidated, it's going to make them feel exhausted, it's going to make them feel like they didn't do enough. When it's just not the case.
Anyway. I just wanted to say something to this as well, because it's sort of just dismisses a person's ideas. I don't think you're intentionally doing that, but even repeating the phrase, "ideas are cheap" cheapens what it actually takes to come up with an idea. And now in 2026, when literally any idea can become something real, I really don't think ideas being cheap means ideas aren't valuable.
weatherlite 1 days ago [-]
I may have been too loose with my wording. Lots of people can come up with concepts and ideas. Everyone used to have an idea for an app back when it was cool, I thought about location based dating apps maybe 5-10 years before they became a thing. I'm certain there were at least a few hundred thousand people if not more who had a similar idea - that's what I meant, that's the cheap part. Actually working late nights or even quitting your job to implement the idea, or having the right network and social skills to make it happen - that part is rare. Even today with agents it's still hard and rare to create a profitable business.
So we weren't talking about the same thing.
> And that just makes people feel bad. Someone working for 2 1/2 years on an idea that just happens to come out on the Internet is not going to feel good
This was more than an idea for you and more than a hobbey at least according to my definitions of those terms. I'd call it a passion research / early stage business venture depending on how much effort you put into it and what your intentions for it were.
bbor 1 days ago [-]
I've gotta say, that is by far the most wholesome, empathetic reply I've ever received on this forum, and possibly on any forum. So on a personal level, thank you. Really. Especially because lamenting that I could've scooped Anthropic is just undeniably hubristic; my reasoning for not sharing partial results at the time was mostly about AI safety, which is silly in hindsight for betraying the exact same sin.
The (penti-)closing statement is also pretty damn poignant, and very true. I daresay your thoughts on publishing deserve to be published!
Substantively, I absolutely agree in theory -- I even recently set up my Substack with this in mind, despite that being an insane thing to do when frantically job searching! I (/Claude...) just found a brief exchange[1] I had w/ @simonw on this very topic almost exactly two years ago below a heartfelt blog post[2] about someone's personal reasons for struggling with this, even. Reading it now mostly serves to remind me of the roadblocks, though:
1. I was quibbling with his advice to "force" yourself to share by reflex (and the author's framing of the failure to do so as "laziness"), and two more years of reading about, discussing, and managing AD[H]D symptoms have only strengthened my convictions there. Charitably, getting into the habit of publishing is an investment, and my metaconscious/'inner critic' has plenty of quite plausible arguments as to why it wouldn't be worth the squeeze[3]; less charitably, I'd note that the relatable blogger in question only posted twice over the following 12 months.
2. That exchange took place just as A) reasoning models were changing everything, B) artificial articles outnumbered human ones for the first time.[4] In the intervening years we've seen AI turn from niche to vitriolic, and at least a dozen variations of "If you didn't care enough to write it yourself, I don't care enough to read it" posts hit the front page of HN. Obviously we all agree and are talking about human writing/ideating here, but the overall vibe is still undeniably... tense, let's say[5].
I still appreciate your comment and agree, mostly because the anxieties presumably lessen considerably with exposure and practice. In other words, I'd get used to it.
In fact, I just queried the corpus that I ironically have yet to publish the pipeline for, and I've written 6,280 real comments (>50 words of original text) and 1,788 long ones (>100 words) since that day on Reddit, Bluesky, and here. Some of that might be real differences --say, ostensible anonymity on Reddit and lower expectations for mere comments-- but the only factor that accounts for a spread of 6,280-0 is habit and comfort, I think.
As usual (55 times since November 11th, 2024): apologies for the rant, y'all.
[3]: Especially now that we're at the tail end of the initial six months of agentic engineering; investing the emotional energy to share my work is a lot harder to justify when I could orchestrate more+better engineering work in those same few hours than I could've accomplished in a few days two years ago.
[5]: Wow, I was more right than I thought! Claude just found a preprint showing that "pejorative accusations" more than doubled in the final third of 2024 --jumping from 7.9% in August to 17.9% in December-- despite poor accuracy: https://arxiv.org/abs/2606.12073
ford 21 hours ago [-]
If it makes you feel better - I think many of the pieces labs put out are pretty easy jumps to make from current harness behavior. There are probably `n` open source projects that implement some mechanisms that look approximately like this.
Another example of this is "skills", which has a lot of branding for what are essentially markdown files
This is an interesting article, but IMO what makes this interesting is not the idea, but the empirical evidence that you are probably equally capable of putting together if you had millions of dollars
everyone is talking about dreaming, but I feel the model have a tendency to over-generalize their thoughts when they are creating memories. I think it's just a matter of time before that's fixed. But that's why I find these memory systems hard to use. Be it claude memories, mem0 or even the dreaming that cognition brought in.
am I using it wrong?
pierrebrunelle 9 hours ago [-]
[flagged]
2001zhaozhao 1 days ago [-]
This article is a lot of validation for what I am building. I'll definitely be revisiting the article a few times to take in the ideas. Memory systems in AIs are extremely important and imho you can do much better than simply training the base model for memory management.
russellbeattie 1 days ago [-]
So after reading the article, I asked a Claude Code session I'm working in at the moment about its memory and whether it needs to "dream" to reorganize itself because I honestly wasn't sure what it was doing about it.
It explained that unlike Chat, each project in Claude Code creates memory files in ~/.claude/projects/[project path]/memory/ with notes about both the project and my impromptu directives or instructions, and a MEMORY.md index file that's loaded for every new session. It told me that my current project - which has been going on for a while - has 61 files in its memory, so it suggested that now in fact would be a good time to clean them up. Perfect.
It gave some examples of things things it has memorized: Apparently I once told it to always capitalize the first letter after a colon. I don't remember doing that, but hey, that's basic English grammar, dammit, so I can see myself getting pissy about it. It had saved that bit of petulance on my part in the memory.
That's great, but it was saved only in the memory folder of that project. This is good to know! I hadn't realized that stuff wasn't saved globally. Or rather, I vaguely remember turning that option off at some point and now I'm realizing that means I need to do some maintenance manually if I don't want to have to constantly remind Claude to capitalize properly, (and to always refer to me as Mr. Big.)
Definitely a good article for me to read right now.
epihelix 1 days ago [-]
> Apparently I once told it to always capitalize the first letter after a colon. I don't remember doing that, but hey, that's basic English grammar, dammit, so I can see myself getting pissy about it.
It's definitely not correct UK English grammar. TIL that the US uses some very odd colon rules.
0gs 1 days ago [-]
yeah this is not a rule in the US either :(
russellbeattie 20 hours ago [-]
It's almost as if neither of you wannabe pedants have a access to Google, or even an AI, which could help you out.
It's AP Style, which is used in basically every news article you've ever read.
The rule is simple: You capitalize the first word after a colon when it begins a full sentence.
simpleintheory 20 hours ago [-]
This is unique to AP and 18th CMOS: MLA and old CMOS versions with grew up with say to do the opposite.
Also, please try to be less dismissive!
epihelix 11 hours ago [-]
It's a US-specific rule, and only in certain style manuals. Chicago, for example, does not capitalise after a colon. (But perhaps your Google or AI had not heard of the Chicago Manual of Style?)
And there is an entire world out there, outside of the US (perhaps your Google or AI did not know that either?) that does not do this.
11 hours ago [-]
russellbeattie 10 hours ago [-]
> "It's a US-specific rule"
And? Calling it a, "very odd colon rule" assumes that British English is somehow the correct one, which it isn't. The United States represents more than 70% of the world's native English speakers. If we have a rule, then it's not the "odd" one.
> "...only in certain style manuals."
And? The two most important style manuals used by millions of people to write billions of articles and papers.
> "Chicago, for example, does not capitalise after a colon."
Did mean "capitalize"? (I'm kidding of course, I know you people like to stick with your quaint antiquated ways of spelling.)
if you want to be a pedant, you are. not necessary, but certainly sufficient. and i was saying in real life, not in the Chicago Manual of Style.
xtiansimon 23 hours ago [-]
Ugg. “dreaming”. More anthropomorphizing we didn’t need. And reading comments which drop the term unironically just makes my skin crawl worse than all the LLM written posts on HN combined.
totetsu 1 days ago [-]
They're not even using holographic projections and lower dimensional manifolds ..
ma2kx 1 days ago [-]
Looks like a promising way to bind customers.
anonymous908213 1 days ago [-]
Another day, another Anthropic marketing article trying to deceive people into believing Claude is conscious. Absolutely insane anybody takes this company seriously when they describe "writing notes into an .md file to be re-injected into context later" as ""dreaming"".
2001zhaozhao 1 days ago [-]
Dreaming is being used here as an analogy for a post-hoc conflict resolution process for memory (which is part of the theorized purpose of dreaming in the human brain), rather than used literally. I think it's the best word to describe what it does and it is very important to a good memory system, as memory conflicts are one of the most likely things pulling down their performance in the long run. Not really a misnomer, imo, and i don't think they are trying to use the word to suggest that Claude is conscious.
antonvs 1 days ago [-]
> post-hoc conflict resolution process for memory (which is part of the theorized purpose of dreaming in the human brain)
That’s not accurate. One leading hypothesis is that the experience of dreaming is a side effect of memory processing and consolidation[1]. Using the name of a side effect to describe the process that produces it is misleading, and at best it goes well beyond what’s actually known.
[Edit: and the side effect in question, dreaming, is closely tied to consciousness in humans. Using an “analogy” with those connotations is an unfortunate mistake.]
[1] “The process of memory reactivation and consolidation in the sleeping brain appears to influence conscious experience during sleep, contributing to dream content recalled on awakening.” — https://pmc.ncbi.nlm.nih.gov/articles/PMC4704085/
antonvs 1 days ago [-]
It is incredibly unscientific. Superstitious, in fact.
I wouldn’t be particularly surprised if Anthropic were to reconstitute itself as a religion.
freecodeio 17 hours ago [-]
it already is, and all the anthropic-jahedeens are in hackernews
solyin 1 days ago [-]
Think of the tax benefits!
andai 1 days ago [-]
Shh... tiny agents is dreaming...
troupo 1 days ago [-]
Such a bullshit article, as much of Anthropic "research" is these days.
> By “agent memory,” we mean knowledge that Claude gains from experience—including interactions with users and tasks it’s been assigned—and that it retains between sessions, saved as context it can read later.
That Claude rarely, if ever, actually "remembers". Since these are just random markdown files randomly injected into the tiny context.
"I'm sorry, I'll make a note of it" and will proceed to write a 15000th note with the same content it's already written dozens of times before
> We compared the memory folders from one run each of Claude Sonnet 3.7 and Claude Fable 5, when each was about 10,000 steps into the game.
What a useless experiment
> That general guidance was enough for Opus 4.7 and Opus 5 to use memory effectively
Define "effectively"
> In this case, prompting Claude to read its memory files before answering—and then to explicitly test its theories—nearly doubled its benchmark score
Then it's not "memory". It's just telling the model to read a text/spec file before implementing something
> Claude doesn’t load all of its auto memory into every session’s context window. It loads only the first 200 lines or 25 KB of MEMORY.md
So, random injection of random things into context that depend on whether the short ToC entries have any relevance on what Claude is doing, and which will be ignored.
And you have to explicitly tell the agent to read specific files otherwise it will ignore them.
What is this bloated article about?
ashishact 1 days ago [-]
Exactly my experience with claude-code. It keeps apologising for not using the memory and keep storing the same memory again and again.
surgical_fire 16 hours ago [-]
> What is this bloated article about?
Advertisement. As you correctly parsed, this article was vapid and full of fluff.
Why they write this? Probably as a vehicle to keep up word of mouth amongst software engineers; as we can see in this thread there's no shortage of people to engage with this crap as it had some merit. I can totally see this making the rounds on LinkedIn in search of engagement.
That I had to scroll to the bottom of the thread to find this is interesting, and says something about the general quality of comment section nowadays.
bbor 1 days ago [-]
Reading such a passionate take as someone who has looked up is a strange experience, I must admit. Kinda like how the Bernard twist in S01 of Westworld felt. I'll try to respond as substantively as I can, regardless:
That Claude rarely, if ever, actually "remembers". Since these are just random markdown files randomly injected into the tiny context.
Complaining about your anecdotal experiences with Claude(Code, hopefully?) isn't a serious criticism, the markdown files are optimized to not be random, and calling 1,000,000 tokens of context "tiny" seems like a plainly unreasonable bar. How much of War and Peace can you hold in your head at once and quote flawlessly, and is it more than 133%?
What a useless experiment
The experiment is testing their fixes to the problem you lamented just one sentence prior.
Define "effectively"
They do exactly that in figure 3, as mentioned at the end of the sentence you're quoting. Opus 5 used that advice to raise its benchmark score by 46.8%, and Opus 4.7 used it for a gain of 82.7%. The specific benchmark looks far from trivial, too: https://github.com/pgasawa/continual-learning-bench/tree/mai...
Then it's not "memory". It's just telling the model to read a text/spec file before implementing something
It fits the colloquial definition, and there is no specific scientific definition in CogSci, CogPsych, Neuro, CogNeuro, NeuroPsych, nor philosophy. There's lots of useful ones offered in different contexts, but they differ considerably.[1] Regardless, it's well accepted industry jargon at this point: it doesn't matter in the same way that it doesn't matter than databases can never truly be "queried" because they can't engage in linguistic discourse.
So, random injection of random things into context that depend on whether the short ToC entries have any relevance on what Claude is doing
Isn't this inevitably, undeniably true of any possible memory system, just from, say, a big-O perspective? If you have to persist every item in a list to decide which one is relevant enough to load into the same pool of memory, there's no point in the second step anymore.
What is this bloated article about?
This article is very plausibly wrong, misguided, or even philosophically bankrupt, but I don't see any support for the claim that its topic is unclear. It's discussing the efficacy of proactive state management during idle periods for longrunning LLM-backed agents.
---
[1]: For example, did you know that modern CogSci is generally convinced that memories are illusions that the brain creates through a process that merely consults ineffable underlying representations (usually), and not an object retrieved from storage? That definition would also contradict Anthropic's wording here, but only in a way that weakens your criticism rather than supports it.
troupo 1 days ago [-]
> Reading such a passionate take as someone who has looked up is a strange experience, I must admit.
There's nothing passionate. Calling bullshit for what it is is not passionate, and we need more of calling out.
I would say that you are much more passionate about this and have a prior bias that makes you see something that's not there in Anthropic's marketing materials
> Complaining about your anecdotal experiences with Claude(Code, hopefully?) isn't a serious criticism
Dismissing actual real-life user experience that anyone who's ever dealt with Claude Code as "anecdotal experience" is exactly why these self-congratulatory articles exist. Even if they mention exactly these issues in passing.
> and calling 1,000,000 tokens of context "tiny" seems like a plainly unreasonable bar.
It is both tiny and easily polluted. Its training data will easily override any and all data in the context even early on in the session. And yes, that million token context is still incapable of maintaining and recalling any of the "memories" with any consistency. Which the article itself acknowledges.
> The experiment is testing their fixes to the problem you lamented just one sentence prior.
These experiments are useless because they invent a problem and valiantly find solutions for it never measuring whether it has any actual impact in real life.
So what if new way of writing summaries is marginally better than the old way? It doesn't matter if the only way to make Claude reliably use them is to explicitly tell it to read them?
> It fits the colloquial definition, and there is no specific scientific definition in CogSci, CogPsych, Neuro, CogNeuro, NeuroPsych, nor philosophy
Here are your biases showing. Anthropic calls random markdown files "memories" and you immediately take them at their word.
These are not memories. These are objectively random markdown files whose only existence is assured by random
harness injections and whatever claude retains in its current context from the first 200 lines of an injected ToC.
Yes, it's an accepted industry jargon, but you don't have to behave that it does something else than what it does.
Or that explicitly telling Claude "read a specific markdown file in your memory" is in any way, shape, or form different from "read a specific markdown file in specs directory".
Of course it's the only way to make Claude actually read and act on it.
> It's discussing the efficacy of proactive state management during idle periods for longrunning LLM-backed agents.
No. No it doesn't. It considers "components of agent memory" and focuses solely on "how we can write markdown files", and spends probably less than 1% on "how we can make sure the agent actually uses them". Even the conclusion completely ignores it despite this:
--- start quote ---
Injected memories use up valuable tokens, and an incorrect note can be carried into every session. Loading a note also doesn’t guarantee that Claude acts on it: our documentation on Claude Code warns that longer instruction files reduce adherence [1 000 000 tokens of context is huge lol - troupo]
...
It loads only the first 200 lines or 25 KB of MEMORY.md [but we sure love the sound of our voice deacribing how well it can write the hundreds of tiny files – troupo]
--- end quote ---
nzjrs 22 hours ago [-]
Very good reply.
bbor 8 hours ago [-]
There's nothing passionate. Calling bullshit for what it is is not passionate, and we need more of calling out.
...you reacted to this blogpost by calling it "bullshit" and "bloated", and criticizing it with obvious hyperbole ("random markdown files randomly injectect into the tiny context", "15000th note", "dozens of times before", "useless experiment", "random injection").
You're allowed to feel emotion -- that wasn't a refutation. But denying that you indeed are typing with responding seems plainly obstinate, sorry. Again I will do my best to respond in line with the rules of this forum, even if I don't feel the same grace being given in return.
I would say that you are much more passionate
What makes you say that? Because I disagree? A comment starting with "I'll try to reply as substantively as I can" isn't exactly firey, much less one that makes pain to cede ground and offer friendly offramps to the polemics.
Dismissing actual real-life user experience
Well I have positive actual real-life user experience with these products. It seems we're at an impasse, there! 1-1=0, after all. This is why anecdotal evidence isn't valid, generally speaking.
It is both tiny and easily polluted
This just comes down to "nuh-uh". So: yeah-huh! 133% of War and Peace is a lot of text to hold at the forefront of one's mind simultaneously.
that million token context is still incapable of maintaining and recalling any of the "memories" with any consistency. Which the article itself acknowledges.
That's the exact opposite of the findings of the experiment that you yourself quoted: achieving an 82% improvement on a challenging benchmark without detailed instructions for memory use is in no sense "incapable".
Here are your biases showing. Anthropic calls random markdown files "memories" and you immediately take them at their word.
You're not taking the time to read what I'm writing :( "It fits the colloquial definition" objectively isn't taking Anthropic "at their word". On an absolute, undeniable, factual level, this attempted ad-hominem is based on a faulty premise.
These are not memories. These are objectively random markdown files
What are memories, and why does text stored in a computer "objectively" not fit that definition? Even if you offered one now (which, again, would be scientifically unsound), it'd be too late; it's beyond obvious that this reasoning is a-posteriori.
Yes, it's an accepted industry jargon, but you don't have to behave that it does something else than what it does.
How am I doing that, specifically? You're the one taking issue with the industry jargon, and I offered no description of "what it does" in the first place, much less one that conflicts with some singular objective definition of "memory" (which, again, does not exist).
Of course it's the only way to make Claude actually read and act on it.
This is objectively incorrect on an empirical level, as proven in the article we are commenting under.
No. No it doesn't. It considers "components of agent memory" and focuses solely on "how we can write markdown files", and spends probably less than 1% on "how we can make sure the agent actually uses them". Even the conclusion completely ignores it
A) The desciption I gave does not require this arbitrary focus of yours.
B) Again, every single empirical result in the entire paper relies on the mechanism that you find to be non-existent.
troupo 4 hours ago [-]
> Well I have positive actual real-life user experience with these products.
I never said I didn't have a positive real-life user experience with these products.
> So: yeah-huh! 133% of War and Peace is a lot of text to hold at the forefront of one's mind simultaneously.
Let me quote the article again:
--- start quote ---
Injected memories use up valuable tokens, and an incorrect note can be carried into every session. Loading a note also doesn’t guarantee that Claude acts on it: our documentation on Claude Code warns that longer instruction files reduce adherence
--- end quote ---
Both the article and the linked docs say that Claude will only load ~200 lines of memory index (not the entire memory), and that you should keep your Claude.md below 200 lines.
Never knew War and Peace was only 400 lines long.
> What are memories, and why does text stored in a computer "objectively" not fit that definition?
Memories is something a model wouldn't need to re-read from scratch every time. Memories isn't something you would need to be always told to reliably use.
And no, whatever human analog you come up with isn't relevant, as we're not talking about humans.
> This is objectively incorrect on an empirical level, as proven in the article we are commenting under.
Let me quote the article:
--- start quote ---
Loading a note also doesn’t guarantee that Claude acts on it: our documentation on Claude Code warns that longer instruction files reduce adherence.
It loads only the first 200 lines or 25 KB of MEMORY.md
--- end quote ---
> Again, every single empirical result in the entire paper relies on the mechanism that you find to be non-existent.
1. I never said it was inexistent
2. The "empirical results" in the paper completely ignore the actual limitations of "memory". It explicitly says that "nope, Claude will not use memory because of known limitations", and immediately sweeps past it as if it wasn't important.
The entirety of "dreaming" and "memory" is completely negated by this:
--- start quote ---
Loading a note also doesn’t guarantee that Claude acts on it: our documentation on Claude Code warns that longer instruction files reduce adherence.
Claude doesn’t load all of its auto memory into every session’s context window. It loads only the first 200 lines or 25 KB of MEMORY.md
--- end quote ---
And this can be literally confirmed empirically by anyone actually working with these tools and watching what they do. E.g., Claude will completely ignore most "memories" and write the same "memory" again and again and again in potentially dozens of files when you tell it that something is wrong. Often repeating what is already there in explicit CLAUDE.md instuctions.
Oh wait. The amazing 1 000 000 token context that can seemingly hold all of "War and Peace" cannot reliably hold more than 200 lines of explicit instructions and more than 200 lines of "memory".
bbor 2 hours ago [-]
I never said I didn't have a positive real-life user experience with these products.
Irrelevant; you were the one who was raising anecdotal evidence and defending it when challenged. Also c'mon surely you realize that you're just disagreeing for the sake of it by this point?
Both the article and the linked docs say that Claude will only load ~200 lines of memory index
Irrelevant; you were discussing the context window itself, not how many lines of text are loaded per note.
Memories isn't something you would need to be always told to reliably use.
A) Not a definition, and, again, B) Disproven directly by the findings I jsut quoted to you.
And no, whatever human analog you come up with isn't relevant, as we're not talking about humans.
That's how analogues work.
Let me quote the article:
Irrelevant quote.
I never said it was inexistent
...again, surely even you realize that quibbling over 1% present and 0% present isn't productive, in this context? What's the point?
It explicitly says that "nope, Claude will not use memory because of known limitations"
The entire article is about Claude using memory.
Claude will completely ignore most "memories" and write the same "memory" again and again and again in potentially dozens of files when you tell it that something is wrong.
Is it possible that you haven't used Claude since Sonnet 3.7? Cause, again, this anecdotal experience isn't reflective of today's reality. And why would anyone care about the anecdotal experience of someone who defines themselves by hating LLMs in the first place?
The amazing 1 000 000 token context that can seemingly hold all of "War and Peace" cannot reliably hold more than 200 lines of explicit instructions and more than 200 lines of "memory".
All of War and Peace could fit into one line. Regardless, 1M is 1M is 1M. This isn't a thing you can have a take about, it's a plain fact. You're able to feed many thousands of sentences of instructions and memories if you'd like to, it's just inefficient.
troupo 23 minutes ago [-]
> Irrelevant; you were the one who was raising anecdotal evidence and defending it when challenged.
That "anecdotal" evidence is literally explicitly stated in the article
> Irrelevant; you were discussing the context window itself, not how many lines of text are loaded per note.
Literally quoting the article: "longer instruction files reduce adherence"
It doesn't matter if the context is 1mln tokens if a file with more than 200 lines can lead to loss of adherence and context pollution.
It doesn't matter if it is marketed as having the ability to hold the entirety of War and Piece if in reality it cannot hold much without being polluted.
> That's how analogues work.
Analogues never work.
> Irrelevant quote.
That is basically the entirety of your entire "argument". "What I say is correct, and Anthropic's article is seminal. Whatever you say, including quoting Anthropic's own materials is irrelevant if it doesn't fit my pre-biased worldview."
> The entire article is about Claude using memory.
No, it's not really about that once you cut to the meat and heart of it.
> Is it possible that you haven't used Claude since Sonnet 3.7?
Is is possible for you to not assign your opponents thoughts and words while completely dismissing what they are actually telling you? Apparently not.
> You're able to feed many thousands of sentences of instructions and memories if you'd like to, it's just inefficient.
Lol. So the size of the context is irrelevant when it's inneficient, and the models apparently lose adherence at as much as 200 lines of instructions. But sure, calling this "tiny context" somehow insults your sensibilities.
Anyway. I've seen enough of your "arguments" to know I couldn't care less to continue this conversation.
Adieu.
surgical_fire 16 hours ago [-]
... And this second reply proves my point to your original take.
The guy you are responding to engaged with that article as it had any merit. Even brings up concepts from psychology as if the misuse of the term "memory" by Anthropic was related to actual memory as would be defined in other fields.
I wonder if the excessive usage of AI damaged people's brains enough that they abandoned curiosity and the ability to read things critically, or of they already lacked those things already in the first place.
bbor 8 hours ago [-]
1. Please don't assume people on the internet are men.
2. "You must only think that because you've abandoned curioisity, can't think critically, and have brain damage that's your own fault" is absurdly polemic. If I took your opinion seriously at all, I'd be quite offended.
surgical_fire 2 hours ago [-]
> 1. Please don't assume people on the internet are men.
I did not. I said "the guy" as a generic term. A guy can be anything.
> 2. "You must only think that because you've abandoned curioisity, can't think critically, and have brain damage that's your own fault" is absurdly polemic.
Aside the fact that you twisted my phrase to something completely different, I made no definitive assumptions.
I was wondering if excessive AI use damage people's brains. It is something that needs to be exercised. Much like sitting around all day will damage your overall health.
troupo 22 minutes ago [-]
> I said "the guy" as a generic term. A guy can be anything.
Reminds me of "in California everything is a dude"
This is a better research direction I feel
https://research.google/blog/the-sleep-framework-a-new-way-f...
So if you're hoping for that you should probably just look at the models that can be ran locally... And then you prolly realize that that's still several years into the future.
AI that remembers through continuous experience instead of documents would be significant and lucrative enough to change the business model.
Updating weights means your changing this model.
To facilitate this feature, the inference provider would need to have your personalized model loaded into ram to process your request in a timely manner. Now multiply that by the number of users they're serving.
You can try to optimize it a bit by essentially gluing the changing weights as a "overlay" to be applied to the original model, but that comes with it's own trade-offs (aside from the fact it hasn't been implemented, just theoretically doable). And it would be seriously hard to implement well.
The amount of resources this would bind makes this infeasable for cloud inference like anthropic provides.
Also see LoRAX, it's an inference engine that exactly does this, it can serve many LoRas off a single base model
Long term I think it's probably more interesting to explore continuously updating weights. But (and this may be yet another opinion that falls victim to the bitter lesson) even if it's _theoretically_ possible to encode everything in weights alone, I'm not sure it's not the most optimal. Today it's too expensive & finnicky to get better on your own tasks without overfitting (ie [0])
Practically (like next few months) getting the right instructions & context would be a huge improvement in capabilities.
[0] https://thinkingmachines.ai/blog/on-policy-distillation/#tra...
I think for many case 'memory' not tied to immediate context is problematic, particularly if its illegible. It's common for agents to make an error and then fixate on them-- e.g. has a wrong conclusion about a root cause, states it emphatically, and then it keeps coming back to haunt you.
With memory consisting of files on disk you can hunt out and destroy the source of such confusion. If it gets buried in weights you may be SOL.
So the main improvement to memories I'd like to see is better judgment on which ones to create.
The parts about "dreaming" make me think of how agent transcripts contain a ton of information, which I think is currently under-utilized. I'll repeat advice I've seen here a few times: ingest your own transcripts into duckdb (claude code will build that pipeline in about 10 minutes) and do some data analysis on them. Some ideas if you don't know where to start: look at tool call failures to see if you can prevent common ones, check out if you have large cache misses and what causes them, look at various efficiency metrics over time (tokens per $, lines of code per tokens, etc.) with main thread vs sub-agents, etc. I started with this sort of things, and ended up following some threads to interesting insights. For example, I found indications that I tended to make Opus 5.5 / Sol 6.1 divide their work into units that were too small. They have the capacity to work on more stuff at once than older models, and I wasn't really taking advantage of that.
However, I have it on for ChatGPT (use it mostly for on the go queries via mobile) and their memory experience is significantly better. I haven't enabled Claude's memory in a long time so things may have changed though.
And in the Claude Desktop UI I manually curate the memories.
Works well enough, I don’t like the concept of dreaming as it quickly leads to memory bloat and/or outdated stuff.
I often see tool call failures because the agent invokes the tool with wrong parameter names, e.g. for file creation it provides "file_content", but the tool expects "file_text" instead. My colleague told me to put this in a pre-prompt file, but I don't understand why the models make these kinds of mistakes - the model should have the available tools with their parameter names in context, should it not?
It would be useful if they could scope them properly. Then it would be useful to keep around. But I haven’t seen memory systems be able to do that.
https://innerloop.works/breadcrumb
How did you determine this?
Do you mean that you were giving them tasks that were too small or you were telling them to split the tasks up or something?
When analyzing my session logs, I tried to look at how story size relates to how much work the AI had to do to fully implement it (made easier by my sessions always having epic#.story# - like 2.6 - in their name). Neither "story size" nor "AI work" have fully obvious metrics, so I looked at a variety of things, like length of story plan, number of lines of codes written for the story, number of different files read/written (and for AI work: number of tool calls, number of tokens for main thread / implementer / reviewer subagents, etc.).
The pattern is always pretty much the same: smaller stories are less token-efficient. One major factor for this is that in bigger stories, the average number of lines of code written per tool call is significantly higher (and not just a little bit, if I remember correctly it's a factor of 1 to 3 between my smallest and largest stories).
None of this is scientifically rigorous, but it's consistent over a fairly significant sample (about 60 stories up to this point). I'm also only measuring amount of work and not quality, but when I find things wrong with a story, I generally ask the AI to fix the issues in the same session, so the two should be correlated (sometimes the issues are large enough that they require new, separate stories to fix, but this is normally caused by problems in ideation and/or planning, not implementation). This is enough to encourages me to ask for more rather than less per story. It might also not apply to everyone, I'm sure this is dependent on everyone's workflow and way to work with the AI. But yeah, that's the sort of stuff you can explore when analyzing your session logs.
The slices are usually a bit too wide. I terminate subagents at 200k and get them to handoff to lead. It works pretty well but again, it's hard to get the orchestrator to size the slices a bit smaller.
I also never use memories. I don't see the usefulness above project-specific prompting and a good knowledge of your own codebase. Am I missing something?
The thing you're asking it for might be intent, which is something the technology fundamentally lacks.
Which.. is something that has been discussed very extensively, which makes one wonder why Anthropic still keeps trying to push that square peg through a round hole.
Curios to learn more, you do have some reference discussion or posts about that.
The main improvement to memories is to actually use the memories, and not ignore them wholeseale. Directly from the article:
--- start quote ---
Injected memories use up valuable tokens, and an incorrect note can be carried into every session. Loading a note also doesn’t guarantee that Claude acts on it: our documentation on Claude Code warns that longer instruction files reduce adherence
...
It loads only the first 200 lines or 25 KB of MEMORY.md
--- end quote ---
The reason it can't create coherent or good "memories" (aka random .md files) is that it cannot recall or use any of them.
I partly say this because I also finally pulled the trigger on deprecating the last ClaudeCode memories on my system in favor of a personal system[1] recently, and the issues I noticed were things a casual user/'vibecoder' might appreciate, but any engineer working fulltime on multiple projects via multiple harnesses would find counterintuitive. E.g.:
- The memories were hidden away and clearly not written for human eyes; useful for low-stakes ledgers, frustrating when the right kind of errant note can derail a longrun 2 weeks later.
- The memories seemed to err towards the side of remembering everything and recording incidental details; useful for remembering the user is gluten-free, but not for tracking spec changes over time.
- The memories could be specific to a subagent or the project or the whole system; an obvious & elegant system for toy examples, but a terribly unweildy one for anyone working at scale over the span of months.
[1]: if curious, a somewhat-outdated version of it is here: https://gitlab.com/doering-ai/libs/fleet TL;DR: Friendship ended with ClaudeCode memory files; my new best friends are `BULLETIN.md` for transitory alerts, `campaigns/[slug]/LEDGER.md` for basic episodic memory, and `reports/[date]-[slug].md` for writeups polished specifically for posterity.
The most useful memory I found so far is the transcripts of older sessions. And not only in one agent but also across agents (like cursor and claude). And also not limited to the current project. Which is why I have a skill that can read all previous session transcripts and find useful informations in a subagent to recall useful information into the context.
It's pretty much the "hey we have talked about X last week" which I would also do in normal conversations. Because you don't always know what is worth writing down AKA storing in memory. Hindsight wins. Obivously I had to increase how long transcripts are stored as Claude would delete them after 30 days - which is way less than what I remember in my own memory.
I'm assuming search is much cheaper than dreaming (though maybe it depends on the time scale?) so that sounds pretty important.
That’s like saying walking is space flight because both happen to change the location of an individual.
Remember, any analogies fail at some point.
Say, given your space flight example, astronauts do "space walk" spacecraft to perate. They are not "walking" but still makes sense.
The biggest thing I want to avoid with a long running agent is volatility. Anything that pushes it into one direction or the other that quickly is not desirable in the average case. At least not to me.
Curiously this seems to apply to humans as well. The "dreaming" is making it question its own assumptions about difficulties it has had in the past. That sounds like a very helpful therapeutic process.
(Not sure of the word dreaming is entirely suitable though: it was cross referencing its notes against the actual history! Very rigorous "dreamer" lol)
I am exploring an alternative dialectic-based approach: https://particulars.fyi/
At some point I stopped noticing it doing it.
I am interested in if it's a fundamental limitation, ie models that have gaps that need to be corrected are going to be bad at recognizing those gaps, and leaving instructions to fix them. Or if models just aren't trained on this
It seems akin to my experience with coding agents, where models are good at implementation but not great at abstraction
Firstly LLMs need to have more training on speaking to several different people.
Humans have a lot of training on interacting with many different individuals and LLMs feel extremely one thread chat heavy. Like if you read the entire thread the LLM langauge is usually consistent but if you send someone a partial artifact it feels disconnected in a broken way.
This includes their code changes and doc changes which inheret this lack of audience.
For the more general idea of keeping good artifacts for themselves I think an interesting idea would be to look into context language models instead of autoregressive ones.
https://arxiv.org/abs/2609.37725
Context language models need to manage their own context to complete a task. (They see their own context as a file).
This naturally forces this innate ability of managing relevance.
It quite literally is fundamental. "memories" are just random .md files that literally cannot be loaded into context because of context limitations. Literally in the article:
--- start quote ---
Injected memories use up valuable tokens, and an incorrect note can be carried into every session. Loading a note also doesn’t guarantee that Claude acts on it: our documentation on Claude Code warns that longer instruction files reduce adherence
...
It loads only the first 200 lines or 25 KB of MEMORY.md
--- end quote ---
The article explicitly excludes this from consideration early on though:
By “agent memory,” we mean knowledge that Claude gains from experience...saved as context it can read later. It excludes other ways models can learn or manage context, such as changes to their weights
Those seem fundamentally different processes, and adjusting weights does seem a much better idea after all, but then in that world everyone has a different AI and how do you apply updates? For a company like Anthropic this is of course a big problem and that is probably why they don't like that route. Could we store weight deltas or something similar and reapply them over a different model?
I wonder if we are so advanced as we think that these constructs (dreaming), are the right ones for improving intelligence in a program that does not need to be modelled after us or constrained by what evolution gave us.
Part of me feels the pang of regret and shame seeing so many little details I researched and dreamed[1] (ha!) of prototyping ~2.5y ago expressed and validated so cleanly. Obviously A) I never did get the prototype working nor publish the underlying ideas and B) this blog post (paper? ...plog post?) is building on it at a scale and with quality I could never dream of (fig. 3 is especially striking), not to mention all the new ideas mixed in. Still... sharing cause it helps, I think, and because I expect y'all can relate; if you're old enough, it's very similar to the feeling you get when you think about how much Apple stock you could've bought in the 90s.
Anyway, more materially:
1. Pretty interesting to see Opus 5.5, Sonnet 5.5, and Opus 5 tied on the basic memory test (aforementioned fig. 3); the Opus comparison alone seems to imply either a hard plateau or that 5.5's huge improvements were accomplished by making good practices more likely, but the Sonnet detail throws that for a loop... I suppose the plateau explanation isn't contradicted, so maybe that's right (for this benchmark).
2. Fig. 4 is gonna do rounds on Reddit and Twitter, my god. Pretty stellar, intuitive depiction of the Mythos step-change that enabled true agentic engineering for the first time this summer! The highlighting is also just the right amount of facetious; love it.
3. Pretty bummed to see no followup philosophy references. We know you have them, Anthropic nerds! Do not be ashamed of the soft and slippery.
4. I am now reminded that I knew they were trying dreaming at scale, as I'm actually on the waitlist for it! And that backs up my overall takeway, actually: I can't help but feel like they're either A) vastly underestimating the amount of CogSci literature that could help them go beyond the "dreaming is when you think in your offtime" paradigm, or B) not sharing their real learnings.
Today's RSI announcement and the final sentence of this seminal plog post being a pitch for enterprise customers supports the latter, I think. That's ostensibly a good thing from a societal stability perspective, but I can't help but feel left out -- not to mention paralyzed by fear of the known unknown...
That "I never got to finish" feeling is the absolute worst. I've been struggling with it for years, and I feel like understand ya here.
It absolutely sucks you feel that, and I hope you find some comfort soon, however it may come.
If you're looking for a "so what now?" Moment, there's only one thing I can say that I've been proselytizing a bit: share everything. All the time. And get people to look at it and be your hype person; and if you're not getting that in your friend ground or workplace or community, find a new one to add to.
The only way I've found to beat that feeling of "why is it i have the good ideas years ahead but can't do anything with it" is to roll more dice. That's it. These larger companies just essentially have infinitely many die and they're rolling them infinitely faster.
You've done nothing wrong. You're just with some other folks like us that don't have the same die rolling power.
Don't give up on your ideas. Don't stop experimenting. Make your own tools because it's for you, and maybe your dice will come up that you get the other person in the world that sees it and goes "yes!" And you get all the fancy posts and stars and attention and such. Because that feeling of validation - not just popularity, but acceptance and being seen and heard for one's craft - is one everyone seeks and needs in life. It's not a weakness. It's being a human.
Thank you for sharing your experience with internet strangers.
Many of us have ideas, ideas are cheap. Executing a complex idea well from thought to completion is much more rare.
So my thought about sharing and not thinking about "my secret sauce" anymore is just necessary. Share what you think your cool flavor of an idea is, someone else can groove on it, and maybe you and them will grow it into something more.
A major problem as I see it is kinda geek-class-warfare coming to a head. So much of my personal and anecdotally collected experience from others is essentially being locked out of knowledge and ideation because a "upper geek class" decided that they were not worthy of having their idea shared, specifically because that class thought they could make money from it, or because they were insular for some other in group reason. That meant good ideas that just needed a single word or connecting bridge from one topic to another ended up withering, and mediocre ideas ended up being accelerated because of just sheer populace force.
And that just makes people feel bad. Someone working for 2 1/2 years on an idea that just happens to come out on the Internet is not going to feel good. It's going to make them feel invalidated, it's going to make them feel exhausted, it's going to make them feel like they didn't do enough. When it's just not the case.
Anyway. I just wanted to say something to this as well, because it's sort of just dismisses a person's ideas. I don't think you're intentionally doing that, but even repeating the phrase, "ideas are cheap" cheapens what it actually takes to come up with an idea. And now in 2026, when literally any idea can become something real, I really don't think ideas being cheap means ideas aren't valuable.
> And that just makes people feel bad. Someone working for 2 1/2 years on an idea that just happens to come out on the Internet is not going to feel good
This was more than an idea for you and more than a hobbey at least according to my definitions of those terms. I'd call it a passion research / early stage business venture depending on how much effort you put into it and what your intentions for it were.
The (penti-)closing statement is also pretty damn poignant, and very true. I daresay your thoughts on publishing deserve to be published!
Substantively, I absolutely agree in theory -- I even recently set up my Substack with this in mind, despite that being an insane thing to do when frantically job searching! I (/Claude...) just found a brief exchange[1] I had w/ @simonw on this very topic almost exactly two years ago below a heartfelt blog post[2] about someone's personal reasons for struggling with this, even. Reading it now mostly serves to remind me of the roadblocks, though:
1. I was quibbling with his advice to "force" yourself to share by reflex (and the author's framing of the failure to do so as "laziness"), and two more years of reading about, discussing, and managing AD[H]D symptoms have only strengthened my convictions there. Charitably, getting into the habit of publishing is an investment, and my metaconscious/'inner critic' has plenty of quite plausible arguments as to why it wouldn't be worth the squeeze[3]; less charitably, I'd note that the relatable blogger in question only posted twice over the following 12 months.
2. That exchange took place just as A) reasoning models were changing everything, B) artificial articles outnumbered human ones for the first time.[4] In the intervening years we've seen AI turn from niche to vitriolic, and at least a dozen variations of "If you didn't care enough to write it yourself, I don't care enough to read it" posts hit the front page of HN. Obviously we all agree and are talking about human writing/ideating here, but the overall vibe is still undeniably... tense, let's say[5].
I still appreciate your comment and agree, mostly because the anxieties presumably lessen considerably with exposure and practice. In other words, I'd get used to it.
In fact, I just queried the corpus that I ironically have yet to publish the pipeline for, and I've written 6,280 real comments (>50 words of original text) and 1,788 long ones (>100 words) since that day on Reddit, Bluesky, and here. Some of that might be real differences --say, ostensible anonymity on Reddit and lower expectations for mere comments-- but the only factor that accounts for a spread of 6,280-0 is habit and comfort, I think.
As usual (55 times since November 11th, 2024): apologies for the rant, y'all.
---
[1]: https://news.ycombinator.com/item?id=42101327#42104145
[2]: https://swapnilchauhan.com/blog/procrastination-and-the-fear...
[3]: Especially now that we're at the tail end of the initial six months of agentic engineering; investing the emotional energy to share my work is a lot harder to justify when I could orchestrate more+better engineering work in those same few hours than I could've accomplished in a few days two years ago.
[4]: https://graphite.io/research/ai-now-writes-as-many-online-ar...
[5]: Wow, I was more right than I thought! Claude just found a preprint showing that "pejorative accusations" more than doubled in the final third of 2024 --jumping from 7.9% in August to 17.9% in December-- despite poor accuracy: https://arxiv.org/abs/2606.12073
Another example of this is "skills", which has a lot of branding for what are essentially markdown files
This is an interesting article, but IMO what makes this interesting is not the idea, but the empirical evidence that you are probably equally capable of putting together if you had millions of dollars
am I using it wrong?
It explained that unlike Chat, each project in Claude Code creates memory files in ~/.claude/projects/[project path]/memory/ with notes about both the project and my impromptu directives or instructions, and a MEMORY.md index file that's loaded for every new session. It told me that my current project - which has been going on for a while - has 61 files in its memory, so it suggested that now in fact would be a good time to clean them up. Perfect.
It gave some examples of things things it has memorized: Apparently I once told it to always capitalize the first letter after a colon. I don't remember doing that, but hey, that's basic English grammar, dammit, so I can see myself getting pissy about it. It had saved that bit of petulance on my part in the memory.
That's great, but it was saved only in the memory folder of that project. This is good to know! I hadn't realized that stuff wasn't saved globally. Or rather, I vaguely remember turning that option off at some point and now I'm realizing that means I need to do some maintenance manually if I don't want to have to constantly remind Claude to capitalize properly, (and to always refer to me as Mr. Big.)
Definitely a good article for me to read right now.
It's definitely not correct UK English grammar. TIL that the US uses some very odd colon rules.
It's AP Style, which is used in basically every news article you've ever read.
The rule is simple: You capitalize the first word after a colon when it begins a full sentence.
Also, please try to be less dismissive!
And there is an entire world out there, outside of the US (perhaps your Google or AI did not know that either?) that does not do this.
And? Calling it a, "very odd colon rule" assumes that British English is somehow the correct one, which it isn't. The United States represents more than 70% of the world's native English speakers. If we have a rule, then it's not the "odd" one.
> "...only in certain style manuals."
And? The two most important style manuals used by millions of people to write billions of articles and papers.
> "Chicago, for example, does not capitalise after a colon."
Did mean "capitalize"? (I'm kidding of course, I know you people like to stick with your quaint antiquated ways of spelling.)
You might want to check that again:
https://www.chicagomanualofstyle.org/search.html?clause=colo...
That’s not accurate. One leading hypothesis is that the experience of dreaming is a side effect of memory processing and consolidation[1]. Using the name of a side effect to describe the process that produces it is misleading, and at best it goes well beyond what’s actually known.
[Edit: and the side effect in question, dreaming, is closely tied to consciousness in humans. Using an “analogy” with those connotations is an unfortunate mistake.]
[1] “The process of memory reactivation and consolidation in the sleeping brain appears to influence conscious experience during sleep, contributing to dream content recalled on awakening.” — https://pmc.ncbi.nlm.nih.gov/articles/PMC4704085/
I wouldn’t be particularly surprised if Anthropic were to reconstitute itself as a religion.
> By “agent memory,” we mean knowledge that Claude gains from experience—including interactions with users and tasks it’s been assigned—and that it retains between sessions, saved as context it can read later.
That Claude rarely, if ever, actually "remembers". Since these are just random markdown files randomly injected into the tiny context.
"I'm sorry, I'll make a note of it" and will proceed to write a 15000th note with the same content it's already written dozens of times before
> We compared the memory folders from one run each of Claude Sonnet 3.7 and Claude Fable 5, when each was about 10,000 steps into the game.
What a useless experiment
> That general guidance was enough for Opus 4.7 and Opus 5 to use memory effectively
Define "effectively"
> In this case, prompting Claude to read its memory files before answering—and then to explicitly test its theories—nearly doubled its benchmark score
Then it's not "memory". It's just telling the model to read a text/spec file before implementing something
> Claude doesn’t load all of its auto memory into every session’s context window. It loads only the first 200 lines or 25 KB of MEMORY.md
So, random injection of random things into context that depend on whether the short ToC entries have any relevance on what Claude is doing, and which will be ignored.
And you have to explicitly tell the agent to read specific files otherwise it will ignore them.
What is this bloated article about?
Advertisement. As you correctly parsed, this article was vapid and full of fluff.
Why they write this? Probably as a vehicle to keep up word of mouth amongst software engineers; as we can see in this thread there's no shortage of people to engage with this crap as it had some merit. I can totally see this making the rounds on LinkedIn in search of engagement.
That I had to scroll to the bottom of the thread to find this is interesting, and says something about the general quality of comment section nowadays.
---
[1]: For example, did you know that modern CogSci is generally convinced that memories are illusions that the brain creates through a process that merely consults ineffable underlying representations (usually), and not an object retrieved from storage? That definition would also contradict Anthropic's wording here, but only in a way that weakens your criticism rather than supports it.
There's nothing passionate. Calling bullshit for what it is is not passionate, and we need more of calling out.
I would say that you are much more passionate about this and have a prior bias that makes you see something that's not there in Anthropic's marketing materials
> Complaining about your anecdotal experiences with Claude(Code, hopefully?) isn't a serious criticism
Dismissing actual real-life user experience that anyone who's ever dealt with Claude Code as "anecdotal experience" is exactly why these self-congratulatory articles exist. Even if they mention exactly these issues in passing.
> and calling 1,000,000 tokens of context "tiny" seems like a plainly unreasonable bar.
It is both tiny and easily polluted. Its training data will easily override any and all data in the context even early on in the session. And yes, that million token context is still incapable of maintaining and recalling any of the "memories" with any consistency. Which the article itself acknowledges.
> The experiment is testing their fixes to the problem you lamented just one sentence prior.
These experiments are useless because they invent a problem and valiantly find solutions for it never measuring whether it has any actual impact in real life.
So what if new way of writing summaries is marginally better than the old way? It doesn't matter if the only way to make Claude reliably use them is to explicitly tell it to read them?
> It fits the colloquial definition, and there is no specific scientific definition in CogSci, CogPsych, Neuro, CogNeuro, NeuroPsych, nor philosophy
Here are your biases showing. Anthropic calls random markdown files "memories" and you immediately take them at their word.
These are not memories. These are objectively random markdown files whose only existence is assured by random harness injections and whatever claude retains in its current context from the first 200 lines of an injected ToC.
Yes, it's an accepted industry jargon, but you don't have to behave that it does something else than what it does.
Or that explicitly telling Claude "read a specific markdown file in your memory" is in any way, shape, or form different from "read a specific markdown file in specs directory".
Of course it's the only way to make Claude actually read and act on it.
> It's discussing the efficacy of proactive state management during idle periods for longrunning LLM-backed agents.
No. No it doesn't. It considers "components of agent memory" and focuses solely on "how we can write markdown files", and spends probably less than 1% on "how we can make sure the agent actually uses them". Even the conclusion completely ignores it despite this:
--- start quote ---
Injected memories use up valuable tokens, and an incorrect note can be carried into every session. Loading a note also doesn’t guarantee that Claude acts on it: our documentation on Claude Code warns that longer instruction files reduce adherence [1 000 000 tokens of context is huge lol - troupo]
...
It loads only the first 200 lines or 25 KB of MEMORY.md [but we sure love the sound of our voice deacribing how well it can write the hundreds of tiny files – troupo]
--- end quote ---
You're allowed to feel emotion -- that wasn't a refutation. But denying that you indeed are typing with responding seems plainly obstinate, sorry. Again I will do my best to respond in line with the rules of this forum, even if I don't feel the same grace being given in return.
What makes you say that? Because I disagree? A comment starting with "I'll try to reply as substantively as I can" isn't exactly firey, much less one that makes pain to cede ground and offer friendly offramps to the polemics. Well I have positive actual real-life user experience with these products. It seems we're at an impasse, there! 1-1=0, after all. This is why anecdotal evidence isn't valid, generally speaking. This just comes down to "nuh-uh". So: yeah-huh! 133% of War and Peace is a lot of text to hold at the forefront of one's mind simultaneously. That's the exact opposite of the findings of the experiment that you yourself quoted: achieving an 82% improvement on a challenging benchmark without detailed instructions for memory use is in no sense "incapable". You're not taking the time to read what I'm writing :( "It fits the colloquial definition" objectively isn't taking Anthropic "at their word". On an absolute, undeniable, factual level, this attempted ad-hominem is based on a faulty premise. What are memories, and why does text stored in a computer "objectively" not fit that definition? Even if you offered one now (which, again, would be scientifically unsound), it'd be too late; it's beyond obvious that this reasoning is a-posteriori. How am I doing that, specifically? You're the one taking issue with the industry jargon, and I offered no description of "what it does" in the first place, much less one that conflicts with some singular objective definition of "memory" (which, again, does not exist). This is objectively incorrect on an empirical level, as proven in the article we are commenting under. A) The desciption I gave does not require this arbitrary focus of yours.B) Again, every single empirical result in the entire paper relies on the mechanism that you find to be non-existent.
I never said I didn't have a positive real-life user experience with these products.
> So: yeah-huh! 133% of War and Peace is a lot of text to hold at the forefront of one's mind simultaneously.
Let me quote the article again:
--- start quote ---
Injected memories use up valuable tokens, and an incorrect note can be carried into every session. Loading a note also doesn’t guarantee that Claude acts on it: our documentation on Claude Code warns that longer instruction files reduce adherence
--- end quote ---
Both the article and the linked docs say that Claude will only load ~200 lines of memory index (not the entire memory), and that you should keep your Claude.md below 200 lines.
Never knew War and Peace was only 400 lines long.
> What are memories, and why does text stored in a computer "objectively" not fit that definition?
Memories is something a model wouldn't need to re-read from scratch every time. Memories isn't something you would need to be always told to reliably use.
And no, whatever human analog you come up with isn't relevant, as we're not talking about humans.
> This is objectively incorrect on an empirical level, as proven in the article we are commenting under.
Let me quote the article:
--- start quote ---
Loading a note also doesn’t guarantee that Claude acts on it: our documentation on Claude Code warns that longer instruction files reduce adherence.
It loads only the first 200 lines or 25 KB of MEMORY.md
--- end quote ---
> Again, every single empirical result in the entire paper relies on the mechanism that you find to be non-existent.
1. I never said it was inexistent
2. The "empirical results" in the paper completely ignore the actual limitations of "memory". It explicitly says that "nope, Claude will not use memory because of known limitations", and immediately sweeps past it as if it wasn't important.
The entirety of "dreaming" and "memory" is completely negated by this:
--- start quote ---
Loading a note also doesn’t guarantee that Claude acts on it: our documentation on Claude Code warns that longer instruction files reduce adherence.
Claude doesn’t load all of its auto memory into every session’s context window. It loads only the first 200 lines or 25 KB of MEMORY.md
--- end quote ---
And this can be literally confirmed empirically by anyone actually working with these tools and watching what they do. E.g., Claude will completely ignore most "memories" and write the same "memory" again and again and again in potentially dozens of files when you tell it that something is wrong. Often repeating what is already there in explicit CLAUDE.md instuctions.
Oh wait. The amazing 1 000 000 token context that can seemingly hold all of "War and Peace" cannot reliably hold more than 200 lines of explicit instructions and more than 200 lines of "memory".
That "anecdotal" evidence is literally explicitly stated in the article
> Irrelevant; you were discussing the context window itself, not how many lines of text are loaded per note.
Literally quoting the article: "longer instruction files reduce adherence"
It doesn't matter if the context is 1mln tokens if a file with more than 200 lines can lead to loss of adherence and context pollution.
It doesn't matter if it is marketed as having the ability to hold the entirety of War and Piece if in reality it cannot hold much without being polluted.
> That's how analogues work.
Analogues never work.
> Irrelevant quote.
That is basically the entirety of your entire "argument". "What I say is correct, and Anthropic's article is seminal. Whatever you say, including quoting Anthropic's own materials is irrelevant if it doesn't fit my pre-biased worldview."
> The entire article is about Claude using memory.
No, it's not really about that once you cut to the meat and heart of it.
> Is it possible that you haven't used Claude since Sonnet 3.7?
Is is possible for you to not assign your opponents thoughts and words while completely dismissing what they are actually telling you? Apparently not.
> You're able to feed many thousands of sentences of instructions and memories if you'd like to, it's just inefficient.
Lol. So the size of the context is irrelevant when it's inneficient, and the models apparently lose adherence at as much as 200 lines of instructions. But sure, calling this "tiny context" somehow insults your sensibilities.
Anyway. I've seen enough of your "arguments" to know I couldn't care less to continue this conversation.
Adieu.
The guy you are responding to engaged with that article as it had any merit. Even brings up concepts from psychology as if the misuse of the term "memory" by Anthropic was related to actual memory as would be defined in other fields.
I wonder if the excessive usage of AI damaged people's brains enough that they abandoned curiosity and the ability to read things critically, or of they already lacked those things already in the first place.
2. "You must only think that because you've abandoned curioisity, can't think critically, and have brain damage that's your own fault" is absurdly polemic. If I took your opinion seriously at all, I'd be quite offended.
I did not. I said "the guy" as a generic term. A guy can be anything.
> 2. "You must only think that because you've abandoned curioisity, can't think critically, and have brain damage that's your own fault" is absurdly polemic.
Aside the fact that you twisted my phrase to something completely different, I made no definitive assumptions.
I was wondering if excessive AI use damage people's brains. It is something that needs to be exercised. Much like sitting around all day will damage your overall health.
Reminds me of "in California everything is a dude"
https://www.reddit.com/r/tumblr/comments/j0yscj/in_californi...