Hacker News analyzer

The Hacker News Analyzer  example uses GenSX to fetch, analyze, and extract trends from the top Hacker News posts. It shows how to combine data fetching, parallel analysis, and content generation in a single workflow to create two outputs: a detailed report and a tweet of the trends.

Workflow

The Hacker News Analyzer workflow is composed of the following steps:

1. Fetch the top 500 Hacker News posts and filters down to text posts (FetchHNPosts)
2. Process each post in parallel (AnalyzeHNPosts)
   • Summarize the content (SummarizePost)
   • Analyze the comments (AnalyzeComments)
3. Writes a detailed report identifying the key trends across all posts (GenerateReport)
4. Edits the report into the style of Paul Graham (EditReport)
5. Generates a tweet in the voice of Paul Graham (WriteTweet)

Running the example

# Navigate to the example directory
cd examples/hacker-news-analyzer
 
# Install dependencies
pnpm install
 
# Set your OpenAI API key
export OPENAI_API_KEY=<your_api_key>
 
# Run the example
pnpm run dev

The workflow will create two files:

• hn_analysis_report.md: A detailed analysis report
• hn_analysis_tweet.txt: A tweet-sized summary of the analysis

Key patterns

Parallel processing

The AnalyzeHNPosts component processes each post in parallel and does two types of analysis in parallel as well. This is achieved using Promise.all to concurrently process multiple posts, with each post’s analysis being handled by separate components.

const AnalyzeHNPosts = gensx.Component(
  "AnalyzeHNPosts",
  async ({ stories }: AnalyzeHNPostsProps) => {
    const analyses = await Promise.all(
      stories.map(async (story) => {
        const [summary, commentAnalysis] = await Promise.all([
          SummarizePost({ story }),
          AnalyzeComments({
            postId: story.id,
            comments: story.comments,
          }),
        ]);
        return { summary, commentAnalysis };
      }),
    );
    return { analyses };
  },
);

The component returns an array of analyses that looks like this:

{
  analyses: [
    { summary: "...", commentAnalysis: "..." },
    { summary: "...", commentAnalysis: "..." },
    // ...
  ];
}

Additional resources

Check out the other examples in the GenSX Github Repo .