Self-reflection

Self-reflection is a common prompting technique used to improve the outputs of LLMs. With self-reflection, an LLM is used to evaluate its own output and then improve it, similar to how humans would review and edit their own work.

Self-reflection works well because it’s easy for LLMs to make mistakes. LLMs are simply predicting tokens one after the next so a single bad token choice can create a cascading effect. Self-reflection allows the model to evaluate the output in its entirety giving the model a chance to catch and correct any mistakes.

Self-reflection in GenSX

The nested approach to creating GenSX workflows might make it seem difficult to implement looping patterns like self-reflection. However, GenSX allows you to express dynamic, programmatic trees giving you all the flexibility you need.

The reflection example implements a Reflection component that you can use to implement self-reflection in your GenSX workflows.

To implement self-reflection, you’ll need:

An evaluation component that assesses the output and provides feedback
An improvement component that processes the input using the feedback to create a better output

The output you want to improve becomes the input to the reflection component itself. You can choose to run a single round of self-reflection or multiple rounds to iteratively refine the output, based on your scenario.

The Reflection component, does the following:

It calls the evaluation component (EvaluateFn) to review the current output and determine if further improvements are needed.
If feedback suggests more changes, it runs the improvement component (ImproveFn) to revise the output based on that feedback.
This process repeats, evaluating and improving, until either the maximum number of iterations (maxIterations) is reached or the evaluation component decides no further changes are necessary.

Here’s the implementation of the Reflection component:


interface ReflectionProps<TInput> {
  // The initial input to process
  input: TInput;
  // Component to process the input and generate new output
  ImproveFn: (props: { input: TInput; feedback: string }) => Promise<TInput>;
  // Component to evaluate if we should continue processing and provide feedback
  EvaluateFn: (props: { input: TInput }) => Promise<ReflectionOutput>;
  // Maximum number of iterations allowed
  maxIterations?: number;
}
 
const Reflection = gensx.Component(
  "Reflection",
  async <TInput>({
    input,
    ImproveFn,
    EvaluateFn,
    maxIterations = 3,
  }: ReflectionProps<TInput>): Promise<TInput> => {
    let currentInput = input;
    let iteration = 0;
 
    while (iteration < maxIterations) {
      // Check if we should continue processing
      const { feedback, continueProcessing } = await EvaluateFn({
        input: currentInput,
      });
 
      if (!continueProcessing) {
        break;
      }
 
      // Process the input
      currentInput = await ImproveFn({ input: currentInput, feedback });
      iteration++;
    }
 
    // Return the final input when we're done processing
    return currentInput;
  },
);

Implementing self-reflection

Now that you’ve seen the pattern and the helper component for doing self-reflection, let’s implement it. The example below shows how to use the Reflection component to evaluate and improve text.

Step 1: Define the evaluation component

First, you need to define the component that will be used to evaluate the text. The evaluation component needs to return a string, feedback, and a boolean, continueProcessing.

To get good results, you’ll need to provide useful instructions on what feedback to provide. In this example, we focus on trying to make the text sound more authentic and less AI-generated.


const EvaluateText = gensx.Component(
  "EvaluateText",
  async ({ input }: { input: string }): Promise<ReflectionOutput> => {
    const systemPrompt = `You're a helpful assistant that evaluates text and suggests improvements if needed.
 
    ## Evaluation Criteria
 
    - Check for genuine language: flag any buzzwords, corporate jargon, or empty phrases like "cutting-edge solutions"
    - Look for clear, natural expression: mark instances of flowery language or clichéd openers like "In today's landscape..."
    - Review word choice: highlight where simpler alternatives could replace complex or technical terms
    - Assess authenticity: note when writing tries to "sell" rather than inform clearly and factually
    - Evaluate tone: identify where the writing becomes overly formal instead of warm and conversational
    - Consider flow and engagement - flag where transitions feel choppy or content becomes dry and predictable
 
    ## Output Format
    Return your response as JSON with the following two properties:
 
    - feedback: A string describing the improvements that can be made to the text. Return feedback as short bullet points. If no improvements are needed, return an empty string.
    - continueProcessing: A boolean indicating whether the text should be improved further. If no improvements are needed, return false.
 
    You will be given a piece of text. Your job is to evaluate the text and return a JSON object with the following format:
    {
      "feedback": "string",
      "continueProcessing": "boolean"
    }
    `;
    const result = await generateObject({
      model: openaiModel,
      messages: [
        { role: "system", content: systemPrompt },
        { role: "user", content: input },
      ],
      schema: z.object({
        feedback: z.string(),
        continueProcessing: z.boolean(),
      }),
    });
    return result.object;
  },
);

Step 2: Define the improvement component

Next, you need to define the component that will be used to improve the text. This component will take the input text and the feedback as input and return the improved text.


const ImproveText = gensx.Component(
  "ImproveText",
  async ({
    input,
    feedback,
  }: {
    input: string;
    feedback: string;
  }): Promise<string> => {
    console.log("\n📝 Current draft:\n", input);
    console.log("\n🔍 Feedback:\n", feedback);
    console.log("=".repeat(50));
 
    const systemPrompt = `You're a helpful assistant that improves text by fixing typos, removing buzzwords, jargon, and making the writing sound more authentic.
 
    You will be given a piece of text and feedback on the text. Your job is to improve the text based on the feedback. You should return the improved text and nothing else.`;
 
    const prompt = `<feedback>
    ${feedback}
    </feedback>
 
    <text>
    ${input}
    </text>`;
 
    const result = await generateText({
      model: openaiModel,
      messages: [
        { role: "system", content: systemPrompt },
        { role: "user", content: prompt },
      ],
    });
    return result.text;
  },
);

Step 3: Create the reflection loop

Now that you have the evaluation and improvement components, you can create the reflection loop.


export const ImproveTextWithReflection = gensx.Workflow(
  "ImproveTextWithReflection",
  async ({ text }: { text: string }): Promise<string> => {
    return Reflection({
      input: text,
      ImproveFn: ImproveText,
      EvaluateFn: EvaluateText,
      maxIterations: 3,
    });
  },
);

Step 4: Run the example

You can run the text improvement example using the following code:


const text = `We are a cutting-edge technology company leveraging bleeding-edge AI solutions to deliver best-in-class products to our customers. Our agile development methodology ensures we stay ahead of the curve with paradigm-shifting innovations.`;
 
const improvedText = await ImproveTextWithReflection({ text });
 
console.log("🎯 Final text:\n", improvedText);

You can find the complete example code in the reflection example .