NEW “Harmonized” Chain of Thought (CoT) Complexity

How do you pick the representative question from each cluster?

An interesting observation of this method comes to mind: It’s not a single wholesale method; it can be used in combination with other techniques. As an example, you could imagine not prompting individual models, but in fact prompting an entire agentic system. This method is cumulative.

Could you do a hierarchical ECHO system? I could very well imagine using it at every stage of an agentic pipeline, or targeted at stages that it performs well at. Ie: Suppose you had an existing pipeline that consistently broke logical problems down into mathematical steps, you might use ECHO at the stage where the actual math is done, or at the stage where you’re breaking the problem down, etc.

~2% doesn’t sound great. Keep in mind, that’s against a supervised baseline (CoT), so it’s higher against a zero shot prompt. Plus, you might imagine a system that uses it when it’s favourable, and ignores it when it’s not (perhaps you could train an encoder to identify the gains and losses, similar to learned routers for routing to small or large LLMs). Does that then make it a 3% gain? A 5%? Keep in mind, too, that the gains were all listed in relatively high scoring situations. Generally it’s harder to go from 60% to 65% than 5% to 10%. If it was used in more difficult situations (ie: a situation where the model scored 15% to begin with) would the effect be more dramatic?

But then the real magic happens: What if you intelligently incorporate it into a system that already provides a 10% increase in logical capabilities? Getting an extra 2-5% sounds quite interesting, there, because that’s enough to take you from 70 to 85%, which often feels like a different category of model.

Done , implemented it!! thanks, life saver. Its so important.

Hey where can we read latest research papets from?