Does this mean review/benchmarking sites need to reevaluate all chips after os updates or at least publish the versions used while testing?

pretty much yeah as the performance gains are significant enough to impact a user’s buying decision.

To not update previous benchmarks is to do buyers a disservice as they would no longer be able to make their best informed buying decisions, the only ones who would benefit from a situation like this would be the corporations for no reason