Skip to Content
PatternsStructured outputs

Structured outputs

Workflows regularly require getting structured outputs (JSON) from LLMs. This guide shows how to use structured outputs with both @gensx/openai and @gensx/vercel-ai. You can also find similar examples in OpenAI examples  and Vercel AI examples  in the GitHub repo.

Structured outputs with the Vercel AI SDK

The @gensx/vercel-ai package provides two ways to get structured outputs from LLMs: generateObject and streamObject. Two key benefits of using the Vercel AI SDK  are that you can stream the output as it’s generated, and you can use it with models that don’t natively support structured outputs.

Using generateObject

Start by defining the Zod schema for the output format you want:

import { z } from "zod";
 
const ExtractEntitiesSchema = z.object({
  people: z.array(z.string()),
  places: z.array(z.string()),
  organizations: z.array(z.string()),
});

Then define the component and set the schema param on the generateObject function:

interface ExtractEntitiesInput {
  text: string;
}
 
const ExtractEntities = gensx.Component(
  "ExtractEntities",
  ({ text }: ExtractEntitiesInput) => {
    const prompt = `Please review the following text and extract all the people, places, and organizations mentioned.
 
  <text>
  ${text}
  </text>
 
  Please return JSON with the following format:
  {
    "people": ["person1", "person2", "person3"],
    "places": ["place1", "place2", "place3"],
    "organizations": ["org1", "org2", "org3"]
  }`;
    const result = generateObject({
      model: openai("gpt-4o-mini"),
      schema: ExtractEntitiesSchema,
      messages: [{ role: "user", content: prompt }],
    });
 
    return result.object;
  },
);

When you run this component, it will return the structured output directly matching the type of the ExtractEntitiesSchema with no extra parsing required.

Using streamObject

The streamObject function is similar to generateObject but it streams the output as it’s generated.

const ExtractEntitiesStreaming = gensx.Component(
  "ExtractEntitiesStreaming",
  ({ text }: ExtractEntitiesInput) => {
    const prompt = `Please review the following text and extract all the people, places, and organizations mentioned.
 
  <text>
  ${text}
  </text>
 
  Please return JSON with the following format:
  {
    "people": ["person1", "person2", "person3"],
    "places": ["place1", "place2", "place3"],
    "organizations": ["org1", "org2", "org3"]
  }`;
    const result = streamObject({
      model: openai("gpt-4o-mini"),
      schema: ExtractEntitiesSchema,
      messages: [{ role: "user", content: prompt }],
    });
 
    const generator = async function* () {
      for await (const chunk of result.partialObjectStream) {
        yield chunk;
      }
    };
 
    return generator();
  },
);

Then you can consume the output of the component like this:

const structuredStreamResult = await ExtractEntitiesStreaming({
  text: "John Doe is a software engineer at Google.",
});
console.log("Response:");
for await (const chunk of structuredStreamResult) {
  console.clear();
  console.log(chunk);
}

Structured outputs with the OpenAI SDK

You can also use the @gensx/openai package to get structured outputs from OpenAI models and compatible APIs.

The OpenAI SDK provides a parse method that you can use to automatically parse the response of the structured output.

Here’s an example of how to extract entities using the OpenAI SDK:

import { OpenAI } from "@gensx/openai";
import { zodResponseFormat } from "openai/helpers/zod.mjs";
import { z } from "zod";
 
const openai = new OpenAI();
 
const EntityExtractionSchema = z.object({
  people: z.array(z.string()).describe("List of people mentioned in the text"),
  places: z.array(z.string()).describe("List of places mentioned in the text"),
  organizations: z
    .array(z.string())
    .describe("List of organizations mentioned in the text"),
});
 
const ExtractEntities = gensx.Component(
  "ExtractEntities",
  async ({ text }: { text: string }) => {
    const result = await openai.beta.chat.completions.parse({
      model: "gpt-4",
      messages: [
        {
          role: "user",
          content: `Please extract all people, places, and organizations from this text:\n\n${text}`,
        },
      ],
      response_format: zodResponseFormat(
        EntityExtractionSchema,
        "entityExtraction",
      ),
    });
 
    return result.choices[0].message.parsed!;
  },
);

When you run this component, it will return a typed object matching the EntityExtractionSchema structure. The parse method ensures that the response is properly validated against your schema before returning it.

You can use it like this:

const result = await ExtractEntities({
  text: "John works at Google in New York City.",
});
 
console.log(result);
// Output:
// {
//   people: ["John"],
//   places: ["New York City"],
//   organizations: ["Google"]
// }

Alternatively, you can just call openai.chat.completions.create and then parse the response yourself with Zod:

const parsed = EntityExtractionSchema.parse(result.choices[0].message.content);
Last updated on
Self-reflectionTools