Bulk Operations for Agents: Moving from Single Calls to Batch Processing

The Single-Call Trap

Most agent tutorials show this pattern:

for (const item of items) {
  await agent.createEntry(item); // One at a time
}

Simple. Works. Until you have 500 items.

The math:

• 500 items × 200ms per call = 100 seconds
• 500 context reconstructions
• 500 API roundtrips
• 500 opportunities for rate limiting

With bulk:

• 1 call × 500ms = 0.5 seconds
• 1 context reconstruction
• 1 API roundtrip
• No rate limiting concerns

Bulk is 200x faster. At scale, this is the difference between usable and unusable.

The Latency Stack

Every single API call incurs:

┌─────────────────────────────────────┐
│ Network latency          ~50ms      │
├─────────────────────────────────────┤
│ TLS handshake            ~30ms      │
├─────────────────────────────────────┤
│ Server routing           ~10ms      │
├─────────────────────────────────────┤
│ Auth validation          ~20ms      │
├─────────────────────────────────────┤
│ Database connection      ~15ms      │
├─────────────────────────────────────┤
│ Query execution          ~25ms      │
├─────────────────────────────────────┤
│ Serialization            ~10ms      │
├─────────────────────────────────────┤
│ Network return           ~50ms      │
└─────────────────────────────────────┘
Total overhead: ~210ms per call

For bulk, most of this happens once:

Single overhead: ~210ms
+ N × ~25ms (query execution per item)

100 items single: 21,000ms
100 items bulk:     2,710ms (87% reduction)

Designing Bulk-Capable Tools

Tool: Single Create

{
  name: "create_entry",
  description: "Create a single entry",
  input_schema: {
    type: "object",
    properties: {
      template_id: { type: "string" },
      metadata: { type: "object" }
    }
  }
}

Tool: Bulk Create

{
  name: "bulk_create_entries",
  description: "Create multiple entries in a single operation. Use for batch imports or multi-item creation.",
  input_schema: {
    type: "object",
    properties: {
      template_id: { type: "string" },
      entries: {
        type: "array",
        items: { type: "object" },
        maxItems: 100,
        description: "Array of metadata objects to create"
      }
    }
  }
}

Tool: Bulk Update

{
  name: "bulk_update_entries",
  description: "Update multiple entries matching criteria",
  input_schema: {
    type: "object",
    properties: {
      filter: {
        type: "object",
        description: "Criteria to match entries"
      },
      changes: {
        type: "object",
        description: "Fields to update on all matching entries"
      }
    }
  }
}

When Agents Should Use Bulk

Train your agent to recognize bulk scenarios:

System prompt addition:

When you need to create, update, or delete multiple items:
- 1-3 items: Individual operations are fine
- 4-10 items: Prefer bulk if available
- 11+ items: Always use bulk operations

Signs you should use bulk:
- User says "all", "every", "batch", "import"
- List of items provided
- Pattern-based updates ("all inactive customers")
- Migration or cleanup tasks

Implementation Patterns

Chunked Bulk Operations

For very large batches, chunk into manageable sizes:

async function bulkCreateChunked(templateId, entries, chunkSize = 100) {
  const results = [];

  for (let i = 0; i < entries.length; i += chunkSize) {
    const chunk = entries.slice(i, i + chunkSize);
    const result = await api.bulkCreate(templateId, chunk);
    results.push(...result.created);

    // Optional: Progress tracking
    console.log(`Created ${Math.min(i + chunkSize, entries.length)}/${entries.length}`);
  }

  return results;
}

Streaming Bulk Responses

For large result sets:

async function* bulkQueryStreaming(filter) {
  let cursor = null;

  do {
    const response = await api.bulkQuery({
      filter,
      cursor,
      limit: 100,
    });

    for (const item of response.items) {
      yield item;
    }

    cursor = response.nextCursor;
  } while (cursor);
}

// Usage
for await (const entry of bulkQueryStreaming({ status: "active" })) {
  processEntry(entry);
}

Partial Success Handling

Bulk operations may partially succeed:

async function bulkCreateWithRecovery(templateId, entries) {
  const result = await api.bulkCreate(templateId, entries);

  if (result.errors.length > 0) {
    console.log(`${result.created.length} succeeded, ${result.errors.length} failed`);

    // Analyze failures
    const retryable = result.errors.filter((e) => e.retryable);
    const permanent = result.errors.filter((e) => !e.retryable);

    if (retryable.length > 0) {
      // Retry with backoff
      await sleep(1000);
      const retryResult = await bulkCreateWithRecovery(
        templateId,
        retryable.map((e) => e.entry)
      );
      result.created.push(...retryResult.created);
    }

    if (permanent.length > 0) {
      // Log for manual review
      console.error("Permanent failures:", permanent);
    }
  }

  return result;
}

Performance Comparison

Benchmark: Create 1000 entries

Benchmark: Update matching records

Agent Prompting for Bulk

Good: Explicit bulk instruction

User: "Import these 50 customer records"
Agent: I'll use bulk_create_entries to import all 50 records efficiently.
[Calls bulk_create_entries with array of 50 records]

Bad: Loop without bulk

User: "Import these 50 customer records"
Agent: I'll create each record one by one.
[Calls create_entry 50 times]

Teaching bulk recognition

In your system prompt:

For batch operations:
- "Import [list]" → Use bulk_create
- "Update all [criteria]" → Use bulk_update
- "Delete all [criteria]" → Use bulk_delete
- "Tag everything with [X]" → Use bulk_update

Never loop through single operations when bulk alternatives exist.

Error Handling in Bulk

All-or-nothing

// Transaction semantics
const result = await api.bulkCreate(entries, {
  atomic: true, // Rollback all if any fails
});

Best-effort

// Process all, report failures
const result = await api.bulkCreate(entries, {
  atomic: false, // Continue on failure
  stopOnError: false,
});
// result.created = successfully created
// result.errors = failed entries with reasons

The Efficiency Mindset

Every time your agent processes multiple items, ask:

1. Is there a bulk alternative? If yes, use it.
2. Can I batch the singles? Collect items, then bulk process.
3. Am I repeating context? Bulk eliminates redundant context setup.
4. What's my items-per-second? If low, investigate bulk options.

Single operations are for single items. Everything else is bulk.