Structured outputs

Workflows regularly require getting structured outputs (JSON) from LLMs. This guide shows how to use structured outputs with both @gensx/openai and @gensx/vercel-ai. You can also find similar examples in OpenAI examples and Vercel AI examples in the GitHub repo.

Structured outputs with the Vercel AI SDK

The @gensx/vercel-ai package provides two ways to get structured outputs from LLMs: generateObject and streamObject. Two key benefits of using the Vercel AI SDK are that you can stream the output as it’s generated, and you can use it with models that don’t natively support structured outputs.

Using `generateObject`

Start by defining the Zod schema for the output format you want:


import { z } from "zod";
 
const ExtractEntitiesSchema = z.object({
  people: z.array(z.string()),
  places: z.array(z.string()),
  organizations: z.array(z.string()),
});

Then define the component and set the schema param on the generateObject function:


interface ExtractEntitiesInput {
  text: string;
}
 
const ExtractEntities = gensx.Component(
  "ExtractEntities",
  ({ text }: ExtractEntitiesInput) => {
    const prompt = `Please review the following text and extract all the people, places, and organizations mentioned.
 
  <text>
  ${text}
  </text>
 
  Please return JSON with the following format:
  {
    "people": ["person1", "person2", "person3"],
    "places": ["place1", "place2", "place3"],
    "organizations": ["org1", "org2", "org3"]
  }`;
    const result = generateObject({
      model: openai("gpt-4o-mini"),
      schema: ExtractEntitiesSchema,
      messages: [{ role: "user", content: prompt }],
    });
 
    return result.object;
  },
);

When you run this component, it will return the structured output directly matching the type of the ExtractEntitiesSchema with no extra parsing required.

Using `streamObject`

The streamObject function is similar to generateObject but it streams the output as it’s generated.


const ExtractEntitiesStreaming = gensx.Component(
  "ExtractEntitiesStreaming",
  ({ text }: ExtractEntitiesInput) => {
    const prompt = `Please review the following text and extract all the people, places, and organizations mentioned.
 
  <text>
  ${text}
  </text>
 
  Please return JSON with the following format:
  {
    "people": ["person1", "person2", "person3"],
    "places": ["place1", "place2", "place3"],
    "organizations": ["org1", "org2", "org3"]
  }`;
    const result = streamObject({
      model: openai("gpt-4o-mini"),
      schema: ExtractEntitiesSchema,
      messages: [{ role: "user", content: prompt }],
    });
 
    const generator = async function* () {
      for await (const chunk of result.partialObjectStream) {
        yield chunk;
      }
    };
 
    return generator();
  },
);

Then you can consume the output of the component like this:


const structuredStreamResult = await ExtractEntitiesStreaming({
  text: "John Doe is a software engineer at Google.",
});
console.log("Response:");
for await (const chunk of structuredStreamResult) {
  console.clear();
  console.log(chunk);
}