Favorite prompt series - Unlocking the Mysteries: Exploring the Chain of Why's

 

Introduction 

Have you ever wondered why things happen the way they do? The human mind has an inherent curiosity to understand the underlying reasons behind events and phenomena. In this blog post, we embark on looking at an example of a Prompt pattern, which can allow you to dig deeper into any phenomenon, and discover the root causes that underlie a particular observation.

“Image generated with the assistance of Bing Copilot AI”

Prompt

I am going to provide a simple prompt which allows you to describe a phenomena, and then have the language model ask Why recursively, and then peel out the layers that get to the root cause of that phenomena.

The prompt is as follows:

"

I will give you a statement - [STATEMENT]. For the statement, type the question why does it happen, then answer that Why yourself. Then review your own response, and then ask why for the response to yourself, and answer that question. Repeat this process [N] times. Ask me for the starting statement - [STATEMENT], as well as for the Number [N]. Note that the user will only enter the initial statement, and you have to answer the remaining Why's yourself without any inputs from the user.

"

In other words, the model will now ask us for two inputs:

[STATEMENT] = The observation or phenomena we want to get a deeper understanding of.

[N] = The number of times the model needs to ask why the phenomena happens.

Prompt Usage

Let's go to the ChatGPT interface ( free version is fine ) and type the above prompt. 

Once above is typed, the model responds as follows

We can then input for example the following text, where [STATEMENT] is represented by "North pole magnets repel each other", and 10 represents the number of introspection rounds for the LLM.

We will then obtain the following answer from ChatGPT.

"

Starting Statement: North pole magnets repel each other

1. Why does this happen? This happens because like magnetic poles repel each other due to the alignment of their magnetic fields.

2. Why do like magnetic poles repel each other? Like magnetic poles repel each other because of the fundamental property of magnetism known as the Lorentz force, which causes similar magnetic poles to repel each other.

3. Why does the Lorentz force cause similar magnetic poles to repel each other? The Lorentz force causes similar magnetic poles to repel each other because when two magnetic fields with the same polarity interact, the force between them is such that they push away from each other, aligning themselves to reduce the overall magnetic potential energy.

4. Why do magnetic fields align to reduce overall magnetic potential energy? Magnetic fields align to reduce overall magnetic potential energy because this configuration corresponds to a more stable state according to the laws of electromagnetism and thermodynamics.

5. Why does a more stable state correspond to aligned magnetic fields? A more stable state corresponds to aligned magnetic fields because this configuration minimizes the energy stored in the magnetic field, resulting in a lower overall energy state, which is favored according to the principles of physics.

6. Why is a lower overall energy state favored according to the principles of physics? A lower overall energy state is favored according to the principles of physics because systems naturally tend toward states of minimum energy to achieve greater stability and equilibrium, in accordance with the second law of thermodynamics.

7. Why do systems tend toward states of minimum energy? Systems tend toward states of minimum energy because this state represents a condition of greater stability and equilibrium, which is a fundamental principle governing the behavior of physical systems.

8. Why is stability and equilibrium desirable for physical systems? Stability and equilibrium are desirable for physical systems because they allow the system to maintain its structure and function over time, resisting external disturbances and ensuring its integrity and functionality.

9. Why do physical systems strive to maintain their integrity and functionality? Physical systems strive to maintain their integrity and functionality because it enables them to effectively interact with their environment, fulfill their purpose or function, and contribute to the overall order and organization of the universe.

10. Why is it important for physical systems to contribute to the overall order and organization of the universe? It is important for physical systems to contribute to the overall order and organization of the universe because the universe operates according to fundamental laws and principles that govern the behavior of all matter and energy within it, and the interactions of physical systems play a crucial role in maintaining this cosmic balance and harmony.

"

Note that the prompt is flexible in that you can easily control the phenomena you want to get to the bottom to, and how deep you want to dig, in order to get to where you want to get to.

Conclusion and Summary: In conclusion, the above prompt allows us to facilitate a deeper understanding for a specific topic of interest.

Kickstarting projects via GitHub CoPilot

 

Introduction

In my experience with GitHub CoPilot, it tends to produce very relevant suggestions when CoPilot knows what you are trying to accomplish. By providing a clear project structure and goals for each file, we can leverage Copilot’s context-aware capabilities to generate highly relevant and tailored code suggestions.

Example Prompt

You can write the above suggestions, and goal decomposition yourself, or alternatively, leverage LLMs to do that for you.

For example, suppose you wanted to implement the game of chess. Then you can prompt an LLM as:

“

I want to develop a Chess game in Python. Please recommend a modular way to implement it, using different Python implementation files. For each file, provide a list of points about the purpose and goals of the file.

”

Below is a sample response:

“

Creating a modular Chess game in Python is an excellent approach for better organization and maintainability. Here’s a recommended modular structure along with the purpose and goals of each file:

board.py

• Purpose: Define the chessboard and its properties.
• Goals:
• Define the Board class to represent the chessboard.
• Implement methods to initialize the board, display it, and update it after each move.
• Include functions to validate moves and check for check/checkmate.
• Handle board state manipulation, such as resetting the board or loading a specific position.

pieces.py

• Purpose: Define the different chess pieces and their behaviors.
• Goals:
• Create classes for each type of chess piece (Pawn, Rook, Knight, Bishop, Queen, King).
• Implement methods to calculate valid moves for each piece.
• Define behaviors for capturing opponent pieces and special moves like castling and pawn promotion.
• Include functions to check for piece presence and movement legality.

player.py

• Purpose: Manage player actions and interactions with the game.
• Goals:
• Define the Player class to represent each player.
• Implement methods to handle player moves, including input validation.
• Manage player turns and switching between players.
• Allow players to offer draws, resign, or request undo moves (if applicable).

game.py

• Purpose: Control the flow of the game and orchestrate interactions between components.
• Goals:
• Define the Game class to manage the overall game state.
• Initialize the board and players, and start the game loop.
• Handle player input and update the board accordingly.
• Implement game-specific logic such as detecting checkmate, stalemate, and draw conditions.
• Provide options for saving and loading game states.

main.py

• Purpose: Entry point for the application; start the game.
• Goals:
• Initialize the game instance from the game.py module.
• Start the game loop and handle any exceptions or errors gracefully.
• Allow players to start new games or exit the application.

”

Next, you can go ahead and create an outline of a project structure using the above layout, and include the recommended points as called out above. Also feel free to edit the project structure and / or the goals for each file.

With the above in-place you should be able to generate very targeted and relevant recommendations, when you engage GitHub CoPilot to execute on some of the functionality.

Conclusion

By providing a clear and modular project structure with well-defined goals for each file, you can leverage GitHub Copilot’s context-aware capabilities to generate highly relevant and targeted code suggestions. This approach will streamline the development process when you work collaboratively with CoPilot.

Happy Coding !

The Knowledge Worker’s Playbook: Navigating the Challenges and Opportunities of the LLM Revolution

 

Introduction

The rise of Large Language Models (LLMs) like ChatGPT, Bard, and others has sparked concerns about the future relevance of knowledge workers. These powerful AI systems have demonstrated remarkable capabilities in areas traditionally dominated by human expertise, such as writing, research, and problem-solving.

However, rather than viewing LLMs as a threat, knowledge workers can leverage these technologies to enhance their own skills and stay relevant in the evolving job market.

1. Embrace LLMs as Assistants, Not Replacements

LLMs should be viewed as powerful tools that can augment and enhance the work of knowledge workers, not replace them entirely. By learning to effectively prompt and collaborate with these models, knowledge workers can boost their productivity, generate higher-quality content, and focus on higher-level tasks that require human creativity and judgment.

2. Develop Prompting and Prompt Engineering Skills

Mastering the art of prompting is crucial for knowledge workers to get the most out of LLMs. This involves crafting precise, well-structured prompts that elicit the desired responses from the models. Prompt engineering skills can help knowledge workers generate more accurate, relevant, and coherent content.

3. Focus on Higher-Order Skills

As LLMs take over more routine knowledge work, knowledge workers should focus on developing skills that are less easily replicated by AI, such as critical thinking, problem-solving, emotional intelligence, and creativity. These higher-order skills will be increasingly valuable in the LLM era.

Some examples of higher-order skills, for example from a finance industry perspective, are as follows:

Critical Thinking:

• Analyzing complex financial models and challenging the underlying assumptions
• Evaluating the validity and reliability of financial data and research reports
• Drawing logical conclusions about market trends and making informed investment decisions

Problem-Solving:

• Identifying and addressing issues in financial processes or systems
• Devising creative solutions to complex financial problems, such as managing risk or optimizing portfolio performance
• Breaking down ambiguous financial challenges into manageable components

Creativity and Ideation:

• Generating innovative investment strategies or financial products to meet evolving market needs
• Developing novel approaches to financial analysis and decision-making
• Exploring unconventional ways to leverage financial technologies and data

Emotional Intelligence:

• Effectively communicating financial information to clients and stakeholders with empathy
• Managing emotions and maintaining composure during periods of market volatility
• Collaborating with cross-functional teams to align financial objectives with broader organizational goals

Adaptability and Lifelong Learning:

• Continuously updating knowledge and skills to stay current with changes in financial regulations, technologies, and industry best practices
• Embracing new analytical tools and techniques to enhance financial decision-making
• Demonstrating a willingness to learn from mistakes and adapt strategies accordingly

Strategic Thinking:

• Aligning financial strategies with the organization’s long-term vision and objectives
• Anticipating and preparing for potential market shifts or economic disruptions
• Considering the broader implications of financial decisions on the organization and its stakeholders

Data Analysis and Interpretation:

• Extracting meaningful insights from large, complex financial data sets
• Identifying patterns, trends, and anomalies that inform investment decisions or risk management strategies
• Effectively communicating data-driven insights to inform business strategy

4. Become Subject Matter Experts

Knowledge workers can differentiate themselves by becoming true subject matter experts in their fields. This involves continuously learning, staying up-to-date with the latest developments, and developing a deep understanding of the nuances and complexities within their domains.

5. Collaborate with LLMs Effectively

Knowledge workers should learn to effectively collaborate with LLMs, using them as tools to enhance their own work. This may involve using LLMs for tasks like research, ideation, and content generation, while maintaining a critical eye and ensuring the quality and accuracy of the output.

Beware the Pitfalls: Evaluating Large Language Models with Other LLMs

 

Why does it matter?

Evaluating the capabilities and limitations of LLMs is crucial as they become more widely adopted, especially in high-stakes domains like healthcare and law.

Evaluating LLMs with LLMs offers a convenient and scalable approach to evaluating LLMs.

However, using other LLMs to evaluate LLMs can lead to systematic biases and unreliable results. Understanding the potential pitfalls of this approach is important to ensure accurate and responsible assessment of these powerful AI systems.

The Pitfalls

The key pitfalls of using LLMs to evaluate other LLMs include:

1. Biases in LLM-generated test items: LLMs may generate test items that resemble their training data, leading to inflated performance scores that do not reflect true understanding.1 The systematic assessment of such biases remains an open challenge.
2. Interdependence of generation and evaluation: The ability of an LLM to generate the correct answer and the ability to determine if an answer is correct may not be independent, potentially leading to misleading evaluations.1
3. Lack of transparency and control with closed LLMs: It is difficult to verify the training data, fine-tuning, and model changes of closed-source LLMs, making it challenging to conduct rigorous evaluations.1
4. Inconsistency of LLM-based evaluations: LLM-based evaluation metrics like G-Eval can be inconsistent, undermining the reliability of the scores.2
5. Potential for amplifying existing biases: LLM-based evaluations may amplify biases present in the training data, leading to unfair and inaccurate assessments.4

In summary, while using LLMs for evaluation may seem convenient, it requires extreme caution and extensive validation to avoid systematic biases and unreliable results. Researchers should consider alternative evaluation approaches, such as using diverse human-curated test sets, to ensure the accurate and responsible assessment of LLM capabilities.

The Mitigations

To mitigate these issues, researchers should consider the following strategies:

1. Leveraging Diverse Human-Curated Test Sets:

• Use test sets curated by domain experts that cover a wide range of topics and difficulty levels, rather than relying solely on LLM-generated test items.*
• Employ techniques like adversarial testing to identify and address biases in the test sets.

2. Incorporating Transparency and Interpretability:

• Develop open-source LLMs with transparent training processes and model architectures to enable rigorous evaluation and validation.*
• Utilize interpretability techniques, such as feature importance analysis and saliency maps, to understand the inner workings of LLMs and identify potential sources of bias.*

3. Continuous Evaluation and Monitoring:

• Implement continuous evaluation frameworks that monitor LLM performance over time and across diverse datasets, detecting deviations and anomalies.*
• Leverage tools like FiddlerAI that provide comprehensive LLM evaluation capabilities, including bias detection and performance optimization*

4. Addressing Interdependence of Generation and Evaluation:

• Explore evaluation approaches that decouple the generation and evaluation components, such as using separate models or human evaluators for these tasks.*
• Develop evaluation metrics that account for the interdependence between generation and evaluation, ensuring more reliable and unbiased assessments.*

5. Incorporating Domain Knowledge and Human Oversight:

• Involve domain experts in the evaluation process to provide context-specific insights and ensure the relevance and accuracy of the assessments.*
• Implement human-in-the-loop evaluation frameworks that leverage the strengths of both LLMs and human experts.*

Sources and Citations

1 Hagendorff, T. (2023). Running cognitive evaluations on large language models: The do’s and don’ts. arXiv preprint arXiv:2312.01276.

2 Confident AI. (2024). LLM Evaluation Metrics: Everything You Need for LLM Evaluation. https://www.confident-ai.com/blog/llm-evaluation-metrics-everything-you-need-for-llm-evaluation

[3] Intro to AI Series: Evaluating LLMs and Potential Pitfalls — YouTube. (2024). https://www.youtube.com/watch?v=e0IqkNkq1qE

[4] Madireddy, S., Lusch, B., & Ngom, M. (2023). Large language models in medicine: the potentials and pitfalls. arXiv preprint arXiv:2309.00087.