Building Your First Agent Pipeline: From Zero to Production in a Weekend

What We're Building

By the end of this guide, you'll have:

• A capture agent that structures raw input
• A retrieval agent that answers questions
• A knowledge base connecting them
• A working pipeline you can extend

Time: One focused weekend.

Prerequisites

• API access to Claude or GPT-4
• An Xtended account (free tier works)
• Basic familiarity with APIs
• Node.js or Python environment

The Architecture

Raw Input (meeting notes, ideas, research)
         │
         ▼
  ┌──────────────┐
  │ Capture Agent │ ──→ Structures input
  └──────────────┘
         │
         ▼
  ┌──────────────┐
  │ Knowledge    │ ──→ Stores structured data
  │ Base         │
  └──────────────┘
         │
         ▼
  ┌──────────────┐
  │ Retrieval    │ ──→ Answers questions
  │ Agent        │
  └──────────────┘

Part 1: Set Up Your Knowledge Base (30 min)

Create Your Space

1. Log into Xtended
2. Create space: "Meeting Intelligence"
3. Description: "Structured notes, decisions, and action items from meetings"

Create the Meeting Notes Template

Template: Meeting Notes
Fields:
├── title (text, required)
│   Description: "Meeting title, e.g., 'Q4 Planning Review'"
│
├── date (date, required)
│   Description: "When the meeting occurred"
│
├── attendees (text)
│   Description: "Comma-separated names of participants"
│
├── summary (textarea)
│   Description: "2-3 sentence summary of the meeting's purpose and outcome"
│
├── key_points (textarea)
│   Description: "Bullet points of main discussion topics and conclusions"
│
├── decisions (textarea)
│   Description: "Specific decisions made, who made them, and rationale"
│
├── action_items (textarea)
│   Description: "Tasks assigned, who owns them, and deadlines"
│
└── open_questions (textarea)
    Description: "Unresolved questions requiring follow-up"

Get Your API Key

Settings → API Keys → Generate → Copy and save securely.

Part 2: Build the Capture Agent (2 hours)

The Goal

Take raw, messy meeting notes and output structured entries.

System Prompt

You are a meeting notes processor. Given raw meeting notes, you extract and structure information into a specific format.

You have access to a knowledge base with a "Meeting Notes" template containing these fields:
- title: Meeting title
- date: When it occurred (YYYY-MM-DD)
- attendees: Comma-separated participant names
- summary: 2-3 sentence summary
- key_points: Bullet points of main topics
- decisions: Specific decisions made with context
- action_items: Tasks with owners and deadlines
- open_questions: Unresolved items

When given raw notes, extract information for each field. If information isn't present, leave that field empty rather than inventing content.

Be concise but capture important nuance.

Implementation (Node.js)

import Anthropic from "@anthropic-ai/sdk";

const anthropic = new Anthropic();
const XTENDED_API_KEY = process.env.XTENDED_API_KEY;
const XTENDED_BASE_URL = "https://api.xtended.app";

// Tool definition for creating entries
const tools = [
  {
    name: "create_meeting_entry",
    description: "Create a structured meeting notes entry in the knowledge base",
    input_schema: {
      type: "object",
      properties: {
        title: { type: "string", description: "Meeting title" },
        date: { type: "string", description: "Date in YYYY-MM-DD format" },
        attendees: { type: "string", description: "Comma-separated names" },
        summary: { type: "string", description: "Brief meeting summary" },
        key_points: { type: "string", description: "Main discussion points" },
        decisions: { type: "string", description: "Decisions made" },
        action_items: { type: "string", description: "Tasks assigned" },
        open_questions: { type: "string", description: "Unresolved questions" },
      },
      required: ["title", "date", "summary"],
    },
  },
];

async function createEntry(data) {
  const response = await fetch(`${XTENDED_BASE_URL}/api/entries`, {
    method: "POST",
    headers: {
      Authorization: `Bearer ${XTENDED_API_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      template_id: "YOUR_TEMPLATE_ID",
      metadata: data,
    }),
  });
  return response.json();
}

async function processRawNotes(rawNotes) {
  const response = await anthropic.messages.create({
    model: "claude-sonnet-4-20250514",
    max_tokens: 1024,
    tools: tools,
    messages: [
      {
        role: "user",
        content: `Process these meeting notes and create a structured entry:\n\n${rawNotes}`,
      },
    ],
  });

  // Handle tool use
  for (const block of response.content) {
    if (block.type === "tool_use" && block.name === "create_meeting_entry") {
      const result = await createEntry(block.input);
      console.log("Entry created:", result);
      return result;
    }
  }
}

Test It

Run with sample notes. Verify the entry appears in Xtended with proper structure.

Part 3: Build the Retrieval Agent (2 hours)

The Goal

Answer questions using your structured meeting knowledge.

System Prompt

You are a meeting intelligence assistant. You have access to a knowledge base containing structured meeting notes.

When asked questions, search the knowledge base for relevant meetings and provide answers based on what you find.

Be specific—cite which meeting, when, and who was involved when relevant.

If you can't find relevant information, say so rather than guessing.

Implementation

const retrievalTools = [
  {
    name: "search_meetings",
    description: "Search meeting notes by keyword or date range",
    input_schema: {
      type: "object",
      properties: {
        query: { type: "string", description: "Search keywords" },
        start_date: { type: "string", description: "Optional start date" },
        end_date: { type: "string", description: "Optional end date" },
      },
      required: ["query"],
    },
  },
  {
    name: "get_meeting_details",
    description: "Get full details of a specific meeting",
    input_schema: {
      type: "object",
      properties: {
        entry_id: { type: "string", description: "The meeting entry ID" },
      },
      required: ["entry_id"],
    },
  },
];

async function answerQuestion(question) {
  const messages = [{ role: "user", content: question }];

  while (true) {
    const response = await anthropic.messages.create({
      model: "claude-sonnet-4-20250514",
      max_tokens: 1024,
      tools: retrievalTools,
      messages: messages,
    });

    // Check for tool use
    const toolUse = response.content.find((b) => b.type === "tool_use");

    if (!toolUse) {
      // Final response
      return response.content.find((b) => b.type === "text")?.text;
    }

    // Execute tool and continue conversation
    let toolResult;
    if (toolUse.name === "search_meetings") {
      toolResult = await searchMeetings(toolUse.input.query);
    }
    // ... handle other tools

    messages.push({ role: "assistant", content: response.content });
    messages.push({
      role: "user",
      content: [
        { type: "tool_result", tool_use_id: toolUse.id, content: JSON.stringify(toolResult) },
      ],
    });
  }
}

Part 4: Connect the Pipeline (1 hour)

Create a Simple Interface

import readline from "readline";

const rl = readline.createInterface({
  input: process.stdin,
  output: process.stdout,
});

function prompt(query) {
  return new Promise((resolve) => rl.question(query, resolve));
}

async function main() {
  console.log("Meeting Intelligence Pipeline");
  console.log("Commands: 'capture', 'ask', 'quit'\n");

  while (true) {
    const command = await prompt("> ");

    if (command === "quit") break;

    if (command === "capture") {
      console.log("Paste meeting notes (end with empty line):");
      let notes = "";
      let line;
      while ((line = await prompt("")) !== "") {
        notes += line + "\n";
      }
      await processRawNotes(notes);
      console.log("Meeting captured.\n");
    }

    if (command === "ask") {
      const question = await prompt("Question: ");
      const answer = await answerQuestion(question);
      console.log("\n" + answer + "\n");
    }
  }

  rl.close();
}

main();

Part 5: Test and Iterate (2 hours)

Capture 5 Test Meetings

Use real or realistic meeting notes. Variety helps:

• Team standup
• Planning meeting
• Customer call
• Decision meeting
• Brainstorm session

Ask Test Questions

• "What action items are assigned to Sarah?"
• "What decisions did we make last week?"
• "What open questions do we still have?"
• "When did we discuss the analytics feature?"

Debug and Refine

Common issues:

• Wrong fields populated: Improve field descriptions
• Missing information: Adjust extraction prompt
• Retrieval misses: Check search query handling
• Hallucination: Add explicit "don't invent" instructions

What You've Built

In a weekend:

• ✓ Capture agent that structures messy input
• ✓ Knowledge base that stores structured data
• ✓ Retrieval agent that answers questions
• ✓ Pipeline connecting them

This is the foundation. Every improvement you make—better prompts, more templates, richer context—compounds.

Next weekend: Add two more agents.