Complete Cloudflare Workers AI Complete Guide | Running AI Models on Edge, Vectorize, D1

Key Takeaways

Complete guide to running AI models on Edge with Cloudflare Workers AI. Covers practical examples using Workers AI, Vectorize, D1, R2 and production deployment.

Real-World Experience: Sharing experience of switching global service AI inference infrastructure to Cloudflare Workers AI, reducing average worldwide response time from 300ms to 50ms and monthly server costs from $8000 to $200.

Introduction: “I Want to Run AI Fast Globally”

Real-World Problem Scenarios

Scenario 1: Global Latency
Running AI on US server causes 300ms delay for Korean users. Edge AI is 50ms. Scenario 2: Server Cost Explosion
GPU server costs $10,000/month. Workers AI is sufficient at $200/month. Scenario 3: Scaling Issues
Must manually scale servers during traffic spikes. Workers auto-scales. Here’s a detailed implementation using mermaid. Please review the code to understand the role of each part.

flowchart TB
    subgraph Traditional[Traditional Server AI]
        A1[User] --> A2[Nearest Server]
        A2 --> A3[US GPU Server]
        A3 --> A2 --> A1
        A4[Latency: 300ms]
        A5[Cost: $10k/month]
    end
    subgraph Edge[Cloudflare Workers AI]
        B1[User] --> B2[Nearest Edge]
        B2 --> B3[AI Execution]
        B3 --> B2 --> B1
        B4[Latency: 50ms]
        B5[Cost: $200/month]
    end

---

1. What is Cloudflare Workers AI?

Core Concepts

Cloudflare Workers AI is a service that can run AI models on Edge in 330+ cities worldwide. Key Features:

• Workers AI: 80+ models including LLM, image generation, speech recognition
• Vectorize: Vector database (RAG implementation)
• D1: SQLite-based Edge database
• R2: S3-compatible object storage
• KV: Key-Value store Pricing (As of 2026):
• Workers AI: $0.011 / 1000 neurons (very cheap)
• Vectorize: $0.04 / 1M dimensions per query
• D1: Free reads, $0.001 / 1000 writes

---

2. Getting Started

Installation

npm install -g wrangler
wrangler login

Create Project

npm create cloudflare@latest my-ai-app
cd my-ai-app

First AI Worker

Here’s an implementation example using TypeScript. Perform tasks efficiently with async processing. Please review the code to understand the role of each part.

// src/index.ts
export default {
  async fetch(request: Request, env: Env): Promise<Response> {
    const response = await env.AI.run('@cf/meta/llama-3-8b-instruct', {
      messages: [
        { role: 'user', content: 'Hello!' }
      ],
    });
    return Response.json(response);
  },
};

Here’s an implementation example using bash. Try running the code directly to see how it works.

# Run locally
wrangler dev
# Deploy
wrangler deploy

---

3. Real Example: Text Summarization API

Here’s a detailed implementation using TypeScript. Define classes to encapsulate data and functionality, perform tasks efficiently with async processing, ensure stability with error handling, perform branching with conditionals. Please review the code to understand the role of each part.

// src/index.ts
interface Env {
  AI: any;
}
export default {
  async fetch(request: Request, env: Env): Promise<Response> {
    // CORS
    if (request.method === 'OPTIONS') {
      return new Response(null, {
        headers: {
          'Access-Control-Allow-Origin': '*',
          'Access-Control-Allow-Methods': 'POST',
          'Access-Control-Allow-Headers': 'Content-Type',
        },
      });
    }
    if (request.method !== 'POST') {
      return new Response('Method Not Allowed', { status: 405 });
    }
    try {
      const { text } = await request.json();
      if (!text || text.length < 100) {
        return Response.json(
          { error: 'Text must be at least 100 characters' },
          { status: 400 }
        );
      }
      // Summarize with AI
      const response = await env.AI.run('@cf/facebook/bart-large-cnn', {
        input_text: text,
        max_length: 150,
      });
      return Response.json({
        summary: response.summary,
        original_length: text.length,
        summary_length: response.summary.length,
      });
    } catch (error) {
      return Response.json(
        { error: 'Internal Server Error' },
        { status: 500 }
      );
    }
  },
};

---

4. Implement RAG with Vectorize

Create Vectorize

# Create vector index
wrangler vectorize create my-vectors --dimensions=768 --metric=cosine

RAG Implementation

Here’s a detailed implementation using TypeScript. Define classes to encapsulate data and functionality, perform tasks efficiently with async processing, process data with loops. Please review the code to understand the role of each part.

// src/rag.ts
interface Env {
  AI: any;
  VECTORIZE: VectorizeIndex;
}
export default {
  async fetch(request: Request, env: Env): Promise<Response> {
    const { question } = await request.json();
    // 1. Convert question to vector
    const embedding = await env.AI.run('@cf/baai/bge-base-en-v1.5', {
      text: question,
    });
    // 2. Search similar documents
    const matches = await env.VECTORIZE.query(embedding.data[0], {
      topK: 3,
    });
    // 3. Use retrieved documents as context
    const context = matches.matches
      .map(m => m.metadata.text)
      .join('\n\n');
    // 4. Generate answer with LLM
    const response = await env.AI.run('@cf/meta/llama-3-8b-instruct', {
      messages: [
        {
          role: 'system',
          content: `Answer based on the following documents:\n\n${context}`
        },
        {
          role: 'user',
          content: question
        }
      ],
    });
    return Response.json({
      answer: response.response,
      sources: matches.matches.map(m => m.metadata),
    });
  },
};

Document Embedding and Storage

Here’s a detailed implementation using TypeScript. Perform tasks efficiently with async processing, process data with loops. Please review the code to understand the role of each part.

// scripts/embed-docs.ts
const documents = [
  { id: '1', text: 'Cloudflare Workers run on Edge.' },
  { id: '2', text: 'Workers AI provides 80+ models.' },
  { id: '3', text: 'Vectorize is a vector database.' },
];
for (const doc of documents) {
  // Generate embedding
  const embedding = await env.AI.run('@cf/baai/bge-base-en-v1.5', {
    text: doc.text,
  });
  // Store in Vectorize
  await env.VECTORIZE.upsert([
    {
      id: doc.id,
      values: embedding.data[0],
      metadata: { text: doc.text },
    },
  ]);
}

---

5. D1 Database Integration

Create D1

wrangler d1 create my-database

Here’s an implementation example using TOML. Try running the code directly to see how it works.

# wrangler.toml
[[d1_databases]]
binding = "DB"
database_name = "my-database"
database_id = "your-database-id"

Create Schema

Here’s a detailed implementation using SQL. Please review the code to understand the role of each part.

-- schema.sql
CREATE TABLE users (
  id INTEGER PRIMARY KEY AUTOINCREMENT,
  email TEXT UNIQUE NOT NULL,
  name TEXT NOT NULL,
  created_at DATETIME DEFAULT CURRENT_TIMESTAMP
);
CREATE TABLE conversations (
  id INTEGER PRIMARY KEY AUTOINCREMENT,
  user_id INTEGER NOT NULL,
  message TEXT NOT NULL,
  response TEXT NOT NULL,
  created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
  FOREIGN KEY (user_id) REFERENCES users(id)
);

# Apply schema
wrangler d1 execute my-database --file=schema.sql

Use in Worker

Here’s a detailed implementation using TypeScript. Define classes to encapsulate data and functionality, perform tasks efficiently with async processing. Please review the code to understand the role of each part.

// src/index.ts
interface Env {
  AI: any;
  DB: D1Database;
}
export default {
  async fetch(request: Request, env: Env): Promise<Response> {
    const { userId, message } = await request.json();
    // Generate AI response
    const aiResponse = await env.AI.run('@cf/meta/llama-3-8b-instruct', {
      messages: [{ role: 'user', content: message }],
    });
    // Save conversation
    await env.DB.prepare(
      'INSERT INTO conversations (user_id, message, response) VALUES (?, ?, ?)'
    )
      .bind(userId, message, aiResponse.response)
      .run();
    return Response.json({ response: aiResponse.response });
  },
};

---

6. Performance Optimization

Streaming Response

Here’s a detailed implementation using TypeScript. Perform tasks efficiently with async processing. Please review the code to understand the role of each part.

export default {
  async fetch(request: Request, env: Env): Promise<Response> {
    const { messages } = await request.json();
    const stream = await env.AI.run('@cf/meta/llama-3-8b-instruct', {
      messages,
      stream: true,
    });
    return new Response(stream, {
      headers: {
        'Content-Type': 'text/event-stream',
        'Cache-Control': 'no-cache',
      },
    });
  },
};

Caching

Here’s a detailed implementation using TypeScript. Define classes to encapsulate data and functionality, perform tasks efficiently with async processing, perform branching with conditionals. Please review the code to understand the role of each part.

// Cache responses with KV
interface Env {
  AI: any;
  CACHE: KVNamespace;
}
export default {
  async fetch(request: Request, env: Env): Promise<Response> {
    const { prompt } = await request.json();
    
    // Check cache
    const cached = await env.CACHE.get(prompt);
    if (cached) {
      return Response.json({ response: cached, cached: true });
    }
    // Run AI
    const response = await env.AI.run('@cf/meta/llama-3-8b-instruct', {
      messages: [{ role: 'user', content: prompt }],
    });
    // Store in cache (1 hour)
    await env.CACHE.put(prompt, response.response, {
      expirationTtl: 3600,
    });
    return Response.json({ response: response.response, cached: false });
  },
};

---

7. Cost Calculation

Workers AI Pricing

Here’s an implementation example using TypeScript. Please review the code to understand the role of each part.

// Example: Text generation
// LLaMA-3-8B: 8B parameters = 8 billion neurons
// Cost per request: 8B / 1000 * $0.011 = $0.088
// 10,000 requests/month
// Total cost: $0.088 * 10,000 = $880
// vs OpenAI GPT-4
// Average $0.03 per request (500 input tokens, 500 output tokens)
// 10,000 requests/month: $300
// Workers AI may be more expensive, but has Edge latency advantage

Cost Optimization

Here’s an implementation example using TypeScript. Perform tasks efficiently with async processing, process data with loops. Please review the code to understand the role of each part.

// 1. Use smaller model
const response = await env.AI.run('@cf/meta/llama-2-7b-chat-int8', {
  // Quantized model (cheaper)
});
// 2. Caching
await env.CACHE.put(key, value, { expirationTtl: 3600 });
// 3. Batch processing
const responses = await Promise.all(
  prompts.map(p => env.AI.run(model, { messages: [{ role: 'user', content: p }] }))
);

---

Summary and Checklist

Key Summary

• Cloudflare Workers AI: Run AI models on Edge
• 330+ cities worldwide: Average response within 50ms
• 80+ models: LLM, image generation, speech recognition, etc.
• Vectorize: Implement RAG with vector DB
• D1: Edge database
• Cost Efficient: 90% savings possible vs traditional servers

Production Checklist

• Create Cloudflare account
• Install and login wrangler
• Set up Workers AI binding
• Choose appropriate model
• Implement error handling
• Establish caching strategy
• Set up cost monitoring
• Production deployment

---

Related Articles

• Complete WebAssembly AI Guide | Running LLM in Browser
• Complete ChatGPT API Guide
• Complete Cloudflare Pages Guide

---

Keywords Covered

Cloudflare, Workers AI, Edge AI, Serverless, LLM, Vectorize, D1, Edge Computing

Frequently Asked Questions (FAQ)

Q. How much does Cloudflare Workers AI cost?

A. $0.011 / 1000 neurons. One LLaMA-3-8B model execution costs about $0.088. Free plan provides up to 10,000 neurons per day.

Q. What models can be used?

A. Provides 80+ models including LLaMA, Mistral, BERT, Stable Diffusion, Whisper. See Cloudflare documentation for full list.

Q. OpenAI API vs Workers AI, which is better?

A. OpenAI API is more powerful but expensive. Workers AI is cheap and fast but has limited model selection. Workers AI recommended for simple tasks.

Q. Is it fast in Korea?

A. Yes, Cloudflare has datacenter in Seoul, enabling responses within 50ms.