I reckon we can get a lot closer than an LLM in time. For one thing, the mind has particular understanding of interim steps whereas, as I understand it, the LLM has no real concept of meaning between the inputs and the output. Some of this interim is, I think, an important part of how we assess truthfulness of generated ideas before we put them into words.
I think that’s still different from what I’m thinking of of interim steps, though.
…but as I think how to explain I realize I’m about to blather about things I don’t understand, or at least haven’t had time to think about! So I’d better leave it there!