DeepMind's SELF-DISCOVER Enhances LLM Problem-Solving
DeepMind’s SELF-DISCOVER is currently the most advanced LLM prompting framework. Here’s how it makes LLMs more human-like at solving problems.
Key Takeaways
DeepMind's SELF-DISCOVER framework enables LLMs to self-compose unique reasoning structures for each task, endowing them with more human-like problem-solving abilities.
Traditional LLM prompting methods like Chain of Thought (CoT) often fall short in handling complex reasoning problems. SELF-DISCOVER addresses this limitation by allowing LLMs to tailor their reasoning to the specific task.
The framework is built on atomic reasoning modules, which are combined to form a coherent reasoning structure.
Some of the main challenges are the framework's reliance on a predefined set of reasoning modules and the computational power required for its operation.
The SELF-DISCOVER framework has significant implications for complex workflows that require human-like intelligence.
This post is sponsored by Multimodal, a Generative AI company that helps organizations automate complex, knowledge-based workflows using AI Agents. Each AI Agent is trained on clients' proprietary data and for their specific workflows, enabling easy adoption and maximum efficiency.
If you’ve used an AI-powered chatbot like Gemini or ChatGPT, you know they seem great at answering basic questions and holding up routine conversations. But after a while, it’s impossible to not feel like you’re talking to a robot.
Here’s the truth: large language models are great at basic reasoning and problem-solving. Fine-tuning can make them excellent at solving complex problems as well. But without advanced, professional fine-tuning and tweaking, LLMs are still too far from having human-level cognition.
In the past year, most developments concerning LLMs have made them more efficient, faster, and closer to human cognition. In the same string comes DeepMind’s new SELF-DISCOVER prompting framework.
What Is SELF-DISCOVER?
DeepMind’s SELF-DISCOVER framework stands for "Self-Discovering Reasoning Structures". As the name suggests, the framework enables LLMs to self-compose reasoning structures for each task they encounter.
This means that instead of following a fixed, pre-defined approach to problem-solving, LLMs can dynamically create a tailored reasoning process based on the specific requirements of the task at hand.
This not only improves the performance of LLMs on complex reasoning tasks but also makes their decision-making process more interpretable and transparent – crucial for building trust in AI systems.
How Does SELF-DISCOVER Work?
The SELF-DISCOVER framework is based on the concept of atomic reasoning modules – basic building blocks of reasoning that can be combined in various ways to form a coherent reasoning structure.
These modules include components like critical thinking, step-by-step analysis, and logical deduction.
By selecting and assembling these modules, LLMs can construct a customized reasoning path for each problem, much like how a human would approach a task by breaking it down into smaller, manageable steps and tackling them one by one.
SELF-DISCOVER vs. Traditional LLM Prompting Methods
Traditional prompting methods in large language models typically involve either direct answering, where the model generates an answer without explicit reasoning steps, or Chain of Thought (CoT), where the model is prompted to generate a reasoning process leading to the final answer.
While these methods have shown success in various tasks, they often fall short in handling complex reasoning problems that require a deeper understanding and more structured approach.
SELF-DISCOVER addresses this limitation by enabling LLMs to come up with reasoning structures tailored to the specific task at hand. The framework operates in two stages:
Discover Task-Intrinsic Reasoning Structures: In this stage, the model is guided to select relevant atomic reasoning modules from a predefined set, such as "critical thinking" and "breaking the problem into sub-problems." The model then adapts these modules to be more specific to the task and finally implements them into a structured, actionable plan.
Solve Problems Using Discovered Structure on Instance-Level: In the second stage, the model uses the discovered reasoning structure to solve individual instances of the task. The structure is presented in a key-value format, which the model fills in step-by-step during decoding to arrive at the final answer.
The key advantage of SELF-DISCOVER over traditional prompting methods is its ability to dynamically create a task-specific reasoning structure.
This leads to more effective and interpretable problem-solving, as the model is not constrained to a single a priori reasoning method like Chain of Thought (CoT). Instead, it can combine multiple reasoning approaches in a coherent and structured manner.
Here’s an analogy to understand the difference between traditional prompting methods and SELF-DISCOVER:
Traditional Prompting Methods (e.g., Chain of Thought)
Imagine a construction team is tasked with building a house. With typical prompting methods, the team follows a standard blueprint for every project, regardless of the unique requirements of each site or the preferences of the homeowner.
While this approach can work for straightforward projects, it might not be suitable for more complex or customized constructions, leading to suboptimal outcomes.
SELF-DISCOVER Framework
Now, consider the SELF-DISCOVER framework as a more advanced construction approach. Instead of following a fixed blueprint, the construction team has access to a library of modular building components (like the multiple atomic reasoning modules) and methods.
For each new project, the team selects and adapts these components and methods to create a customized blueprint tailored to the specific needs of the site and the homeowner's preferences. This approach allows for more flexibility, precision, and efficiency in the construction process, resulting in a better-finished product that meets the unique requirements of each project.
In this analogy, the construction team's ability to dynamically select and adapt modular building components to create a customized blueprint mirrors the way the SELF-DISCOVER framework enables LLMs to self-compose logical reasoning structures tailored to each task. This results in a more effective and interpretable problem-solving.
How Human Are LLMs Right Now?
Existing LLMs like the latest GPT, PaLM, and Gemini models are already pretty good at human-level cognition. A research group developed a novel psychometric assessment focusing on Emotion Understanding (EU), a core component of Emotional Intelligence (EI), suitable for both humans and LLMs.
The results showed that most mainstream LLMs achieved above-average EQ scores, with GPT-4 exceeding 89% of human participants with an EQ of 117.
However, the study also noted that some LLMs did not rely on a human-like mechanism to achieve human-level performance, as their representational patterns were qualitatively distinct from humans.
The SELF-DISCOVER framework significantly enhances the human-like reasoning capabilities of Large Language Models (LLMs). With this framework, LLMs can compose reasoning structures tailored to specific tasks. This approach allows LLMs to break down complex problems into smaller, manageable parts, apply logical reasoning, and integrate information from different sources to arrive at a coherent solution.
The Business Case For SELF-DISCOVER and Human-Like LLMs
The use-case for large language models in various industries mostly revolves around automating routine tasks that have traditionally required humans. The more complex a business workflow or task is, the more human-like the LLM needs to be to execute it with the same or better performance and efficiency as a human worker would.
The present solutions do not achieve optimum efficiency precisely because they’re not exactly like human employees. While 100% human-level intelligence for LLMs is far in the future, SELF-DISCOVER helps bring LLMs closer to being human-like than ever before, making them perform superiorly on challenging reasoning benchmarks.
Workflows that require human-level reasoning are those that involve complex decision-making, critical thinking, and the ability to process, plan, and solve nuanced information. Here are some industries where SELF-DISCOVER would help significantly.
Insurance
Claim Processing: SELF-DISCOVER can help LLMs better understand the nuances of insurance claims, leading to more accurate assessments and faster processing times.
Risk Assessment: The framework can enable LLMs to analyze complex risk factors more effectively, improving underwriting accuracy and efficiency.
Finance
Investment Analysis: By self-composing reasoning structures, LLMs can better analyze financial data and market trends, leading to more informed investment decisions.
Fraud Detection: SELF-DISCOVER can enhance fraud detection by enabling LLMs to reason more effectively about patterns of normal and abnormal behavior.
Healthcare
Diagnosis and Treatment Planning: The framework can improve medical data interpretation and develop more accurate diagnoses and treatment plans.
Clinical Decision Support: SELF-DISCOVER can enhance the support provided to clinicians by enabling LLMs to synthesize information from various sources more effectively.
DeepMind SELF-DISCOVER Framework: The Challenges
A major issue with SELF-DISCOVER is the reliance on a predefined set of reasoning modules. While this approach allows for flexibility and adaptability, it also constrains the model's reasoning capabilities to the scope of these modules. If the set is not comprehensive, the model may struggle to solve complex problems that fall outside its programmed parameters.
Additionally, the effectiveness of SELF-DISCOVER is heavily dependent on the quality of the atomic reasoning modules. If these modules are not well-designed, the overall framework may produce suboptimal results.
It’s worth noting that SELF-DISCOVER is computationally more efficient than typical reasoning techniques. According to the paper,
“SELF-DISCOVER achieves better performance while requiring 10-40x fewer inference computers compared to self-consistency or majority voting.”
Despite this, however, the computational power required for this framework is still pretty high and can be expensive for businesses to deploy.
I also host an AI podcast and content series called “Pioneers.” This series takes you on an enthralling journey into the minds of AI visionaries, founders, and CEOs who are at the forefront of innovation through AI in their organizations.
To learn more, please visit Pioneers on Beehiiv.
Wrapping Up
SELF-DISCOVER does bring LLMs closer to human cognition. With such prompting frameworks and an overall improvement in the architecture, training, and deployment of large language models, most industries and individuals are poised to find excellent utilization for AI. As more improvements are made in this space, we remain positive and hopeful that routine tasks will no longer consume human hours and businesses will become greatly efficient.