feat: stream gpt-image generation via SSE with keepalive
- /api/generate now responds with text/event-stream end-to-end - forwards upstream image_generation.* / image_edit.* partial+completed events - 20s keepalive comments survive Cloudflare's 120s proxy-read timeout - falls back to non-streaming when upstream rejects stream/partial_images - drops @ai-sdk/openai-compatible, @ai-sdk/react, ai (unused) - frontend consumes SSE via fetch+ReadableStream, shows progressive preview
This commit is contained in:
@@ -1,7 +1,8 @@
|
||||
# AGENTS.md
|
||||
|
||||
Bun + TypeScript single-file server that exposes an OpenAI-compatible image
|
||||
generation endpoint and serves a small vanilla HTML/JS playground.
|
||||
Bun + TypeScript single-file server that proxies an OpenAI-compatible image
|
||||
endpoint and serves a small vanilla HTML/JS playground. The whole pipeline is
|
||||
SSE end-to-end so it survives Cloudflare's 120s proxy-read timeout.
|
||||
|
||||
## Runtime
|
||||
|
||||
@@ -19,7 +20,7 @@ generation endpoint and serves a small vanilla HTML/JS playground.
|
||||
`noEmit: true`, so plain `bunx tsc` works too).
|
||||
- Tests / lint / formatter: none configured. If adding tests, use `bun test`.
|
||||
|
||||
The server binds `0.0.0.0` (see `index.ts:61`), so it is reachable from other
|
||||
The server binds `0.0.0.0` (see `index.ts:175`), so it is reachable from other
|
||||
hosts on the network when running locally — be mindful when entering API keys.
|
||||
|
||||
## Architecture
|
||||
@@ -27,25 +28,37 @@ hosts on the network when running locally — be mindful when entering API keys.
|
||||
- `index.ts` — the entire backend. One `Bun.serve` instance with:
|
||||
- `/` serves `index.html` via Bun's HTML import (`import index from "./index.html"`).
|
||||
- `POST /api/generate` accepts
|
||||
`{ baseURL, apiKey, model, prompt, size, referenceImages? }`. It returns
|
||||
`{ images: string[] }` where each entry is a `data:` URL (base64).
|
||||
- Two code paths inside the handler:
|
||||
1. No `referenceImages` → uses `@ai-sdk/openai-compatible` + `generateImage`
|
||||
from `ai`.
|
||||
2. `referenceImages` present → hand-rolled `multipart/form-data` POST to
|
||||
`${baseURL}/images/edits` (see `generateWithReference`). The AI SDK
|
||||
does not currently expose image edits for OpenAI-compatible providers,
|
||||
so this path bypasses it on purpose. The edits endpoint is gpt-image
|
||||
series only (see UI hint in `index.html`).
|
||||
`{ baseURL, apiKey, model, prompt, size, referenceImages? }` and **always
|
||||
responds with `text/event-stream`**. Emitted events:
|
||||
- `event: partial` — `{ image: dataUrl, index }` for each `partial_image`
|
||||
- `event: final` — `{ image: dataUrl }` for the completed image
|
||||
- `event: done` — empty payload, sent right before close
|
||||
- `event: error` — `{ message }` for any failure
|
||||
- SSE comments `: keepalive` every 20s while waiting for upstream, so
|
||||
Cloudflare's 120s proxy-read timeout never fires.
|
||||
- Upstream dispatch:
|
||||
- `referenceImages` present → `POST {baseURL}/images/edits` as
|
||||
`multipart/form-data` (image blobs decoded from data URLs).
|
||||
- Otherwise → `POST {baseURL}/images/generations` as JSON.
|
||||
- Both calls send `stream: true, partial_images: 2` first. If upstream
|
||||
returns a 400 mentioning `stream` or `partial_images`,
|
||||
`isStreamingUnsupportedError` triggers a single retry with
|
||||
`stream: false` and the response is replayed as one `final` event via
|
||||
`forwardUpstreamJSON`. Any other 4xx/5xx propagates as `error`.
|
||||
- Targets the **gpt-image series only** (gpt-image-2 is the default). Do
|
||||
not reintroduce DALL·E-only fields like `response_format` — gpt-image
|
||||
always returns `b64_json`.
|
||||
- `index.html` — self-contained UI: inline CSS, plain DOM JS, no build step.
|
||||
Text fields (`baseURL`, `apiKey`, `model`, `size`, `prompt`) persist in
|
||||
`localStorage` under the `aip:<field>` prefix. Reference images are kept
|
||||
in an in-memory `refImages` array as base64 data URLs and are **not**
|
||||
persisted — refreshing the page drops them. There is no React code despite
|
||||
Reads the SSE response via `fetch` + `ReadableStream` (not `EventSource`,
|
||||
because the API is `POST`). Partials overwrite a single `<img>` so the
|
||||
preview animates in place. Text fields (`baseURL`, `apiKey`, `model`,
|
||||
`size`, `prompt`) persist in `localStorage` under the `aip:<field>` prefix.
|
||||
Reference images are kept in an in-memory `refImages` array as base64 data
|
||||
URLs and are **not** persisted. There is no React code despite
|
||||
`react` / `react-dom` / `@types/react*` being in `package.json` — treat
|
||||
those deps as latent. Do not invent a React frontend unless asked.
|
||||
- No router, no DB, no auth. API key is supplied per-request by the browser
|
||||
and never stored server-side.
|
||||
- No router, no DB, no auth, no AI SDK. API key is supplied per-request by
|
||||
the browser and never stored server-side.
|
||||
|
||||
## TypeScript conventions
|
||||
|
||||
@@ -60,15 +73,17 @@ hosts on the network when running locally — be mindful when entering API keys.
|
||||
|
||||
## When extending the API
|
||||
|
||||
- Add new routes inside the `routes` object in `index.ts`; keep the
|
||||
- Add routes inside the `routes` object in `index.ts`; keep the
|
||||
`{ POST: async (req) => … }` shape used by `/api/generate`.
|
||||
- Return JSON with `Response.json(...)`. Validate the request body shape
|
||||
explicitly — the existing handler asserts required fields and returns 400
|
||||
before calling the model.
|
||||
- The AI SDK image type is loose; the current handler casts to
|
||||
`{ mediaType?: string; base64?: string }`. Mirror that pattern rather than
|
||||
trusting field presence.
|
||||
- For anything the AI SDK does not cover (e.g. image edits, masks, variations),
|
||||
follow `generateWithReference`: build `FormData` with `Blob`s decoded from
|
||||
the incoming data URLs and `fetch` the upstream endpoint directly with the
|
||||
caller's `Authorization: Bearer <apiKey>`.
|
||||
- For any long-running upstream call, mirror the SSE-with-keepalive pattern:
|
||||
build a `ReadableStream<Uint8Array>`, start a 20s `: keepalive` comment
|
||||
timer in `start()`, do work inside `try`, always `clearInterval` and
|
||||
`controller.close()` in `finally`. Helpers `sseEvent` / `sseComment`
|
||||
already exist.
|
||||
- Stay defensive about upstream capabilities: many OpenAI-compatible
|
||||
providers reject unknown params. Send the optimistic request first, then
|
||||
detect the specific 400 (see `isStreamingUnsupportedError`) and retry with
|
||||
a degraded body rather than feature-detecting up front.
|
||||
- Decode incoming data URLs with `decodeDataUrl` (returns `Buffer` + mime)
|
||||
and pass them as `Blob` parts to `FormData` — same pattern as the edits
|
||||
path.
|
||||
|
||||
@@ -5,9 +5,6 @@
|
||||
"": {
|
||||
"name": "ai-playground",
|
||||
"dependencies": {
|
||||
"@ai-sdk/openai-compatible": "^2.0.47",
|
||||
"@ai-sdk/react": "^3.0.186",
|
||||
"ai": "^6.0.184",
|
||||
"react": "^19.2.6",
|
||||
"react-dom": "^19.2.6",
|
||||
},
|
||||
@@ -20,20 +17,6 @@
|
||||
},
|
||||
},
|
||||
"packages": {
|
||||
"@ai-sdk/gateway": ["@ai-sdk/gateway@3.0.115", "", { "dependencies": { "@ai-sdk/provider": "3.0.10", "@ai-sdk/provider-utils": "4.0.27", "@vercel/oidc": "3.2.0" }, "peerDependencies": { "zod": "^3.25.76 || ^4.1.8" } }, "sha512-xonmGfN9pt54WdKqMzWe68BRYS3rsYvraBzioyA0gfNcecHs8Ir5qk/X8grJSyZ95hghjWiOphrK6bAc11E6SA=="],
|
||||
|
||||
"@ai-sdk/openai-compatible": ["@ai-sdk/openai-compatible@2.0.47", "", { "dependencies": { "@ai-sdk/provider": "3.0.10", "@ai-sdk/provider-utils": "4.0.27" }, "peerDependencies": { "zod": "^3.25.76 || ^4.1.8" } }, "sha512-Enm5UlL0zUCrW3792opk5h7hRWxZOZzDe6eQYVFqX9LUOGGCe1h8MZWAGim765nwzgnjlpeYOsuzZmLtRsTPlg=="],
|
||||
|
||||
"@ai-sdk/provider": ["@ai-sdk/provider@3.0.10", "", { "dependencies": { "json-schema": "^0.4.0" } }, "sha512-Q3BZ27qfpYqnCYGvE3vt+Qi6LGOF9R5Nmzn+9JoM1lCRsD9mYaIhfJLkSunN48nfGXJ6n+XNV0J/XVpqGQl7Dw=="],
|
||||
|
||||
"@ai-sdk/provider-utils": ["@ai-sdk/provider-utils@4.0.27", "", { "dependencies": { "@ai-sdk/provider": "3.0.10", "@standard-schema/spec": "^1.1.0", "eventsource-parser": "^3.0.8" }, "peerDependencies": { "zod": "^3.25.76 || ^4.1.8" } }, "sha512-ubkAJ+xODouwtmN1tYlvTPphH1hPOBfZaEQe8U7skGvFAnIRs9PPpsq57bC2+Ky/MB4yzhd6YOsxTAx9sGpazw=="],
|
||||
|
||||
"@ai-sdk/react": ["@ai-sdk/react@3.0.186", "", { "dependencies": { "@ai-sdk/provider-utils": "4.0.27", "ai": "6.0.184", "swr": "^2.2.5", "throttleit": "2.1.0" }, "peerDependencies": { "react": "^18 || ~19.0.1 || ~19.1.2 || ^19.2.1" } }, "sha512-fy8wuy8pBghYD1ECw/M5vAsGsZp2D3y/oSTp1iOlAnJqRXzvz4rWLBz1n+rjL+aHZNgJK3kR3NHlnifoKYERfA=="],
|
||||
|
||||
"@opentelemetry/api": ["@opentelemetry/api@1.9.1", "", {}, "sha512-gLyJlPHPZYdAk1JENA9LeHejZe1Ti77/pTeFm/nMXmQH/HFZlcS/O2XJB+L8fkbrNSqhdtlvjBVjxwUYanNH5Q=="],
|
||||
|
||||
"@standard-schema/spec": ["@standard-schema/spec@1.1.0", "", {}, "sha512-l2aFy5jALhniG5HgqrD6jXLi/rUWrKvqN/qJx6yoJsgKhblVd+iqqU4RCXavm/jPityDo5TCvKMnpjKnOriy0w=="],
|
||||
|
||||
"@types/bun": ["@types/bun@1.3.14", "", { "dependencies": { "bun-types": "1.3.14" } }, "sha512-h1hFqFVcvAvD9j9K7ZW7vd82aSA+rTdznZa+5bwvCwqSB1jmmfLcbIWhOLx1/+boy/xmjgCs/OMUL8hRJSmnPw=="],
|
||||
|
||||
"@types/node": ["@types/node@25.9.0", "", { "dependencies": { "undici-types": ">=7.24.0 <7.24.7" } }, "sha512-AOQwYUNolgy3VosiRqXrACUXTN8nJUtPl7FJXMqZVyxiiCLhQuG3jXKvCS1ALr+Y2OmZhzzLVlYPEqJaiqkaJQ=="],
|
||||
@@ -42,36 +25,18 @@
|
||||
|
||||
"@types/react-dom": ["@types/react-dom@19.2.3", "", { "peerDependencies": { "@types/react": "^19.2.0" } }, "sha512-jp2L/eY6fn+KgVVQAOqYItbF0VY/YApe5Mz2F0aykSO8gx31bYCZyvSeYxCHKvzHG5eZjc+zyaS5BrBWya2+kQ=="],
|
||||
|
||||
"@vercel/oidc": ["@vercel/oidc@3.2.0", "", {}, "sha512-UycprH3T6n3jH0k44NHMa7pnFHGu/N05MjojYr+Mc6I7obkoLIJujSWwin1pCvdy/eOxrI/l3uDLQsmcrOb4ug=="],
|
||||
|
||||
"ai": ["ai@6.0.184", "", { "dependencies": { "@ai-sdk/gateway": "3.0.115", "@ai-sdk/provider": "3.0.10", "@ai-sdk/provider-utils": "4.0.27", "@opentelemetry/api": "^1.9.0" }, "peerDependencies": { "zod": "^3.25.76 || ^4.1.8" } }, "sha512-j//zHkKvj5ra27l8izHco8cj1g1Pr7vx1ZK+hrzrkHvndgIRmdfZKOb6+RAPpvbk42qGIsuYvlYbGlVAu3erNQ=="],
|
||||
|
||||
"bun-types": ["bun-types@1.3.14", "", { "dependencies": { "@types/node": "*" } }, "sha512-4N0ig0fEomHt5R0KCFWjovxow98rIoRwKolrYdCcknNwMekCXRnWEUvgu5soYV8QXtVsrUD8B95MBOZGPvr6KQ=="],
|
||||
|
||||
"csstype": ["csstype@3.2.3", "", {}, "sha512-z1HGKcYy2xA8AGQfwrn0PAy+PB7X/GSj3UVJW9qKyn43xWa+gl5nXmU4qqLMRzWVLFC8KusUX8T/0kCiOYpAIQ=="],
|
||||
|
||||
"dequal": ["dequal@2.0.3", "", {}, "sha512-0je+qPKHEMohvfRTCEo3CrPG6cAzAYgmzKyxRiYSSDkS6eGJdyVJm7WaYA5ECaAD9wLB2T4EEeymA5aFVcYXCA=="],
|
||||
|
||||
"eventsource-parser": ["eventsource-parser@3.0.8", "", {}, "sha512-70QWGkr4snxr0OXLRWsFLeRBIRPuQOvt4s8QYjmUlmlkyTZkRqS7EDVRZtzU3TiyDbXSzaOeF0XUKy8PchzukQ=="],
|
||||
|
||||
"json-schema": ["json-schema@0.4.0", "", {}, "sha512-es94M3nTIfsEPisRafak+HDLfHXnKBhV3vU5eqPcS3flIWqcxJWgXHXiey3YrpaNsanY5ei1VoYEbOzijuq9BA=="],
|
||||
|
||||
"react": ["react@19.2.6", "", {}, "sha512-sfWGGfavi0xr8Pg0sVsyHMAOziVYKgPLNrS7ig+ivMNb3wbCBw3KxtflsGBAwD3gYQlE/AEZsTLgToRrSCjb0Q=="],
|
||||
|
||||
"react-dom": ["react-dom@19.2.6", "", { "dependencies": { "scheduler": "^0.27.0" }, "peerDependencies": { "react": "^19.2.6" } }, "sha512-0prMI+hvBbPjsWnxDLxlCGyM8PN6UuWjEUCYmZhO67xIV9Xasa/r/vDnq+Xyq4Lo27g8QSbO5YzARu0D1Sps3g=="],
|
||||
|
||||
"scheduler": ["scheduler@0.27.0", "", {}, "sha512-eNv+WrVbKu1f3vbYJT/xtiF5syA5HPIMtf9IgY/nKg0sWqzAUEvqY/xm7OcZc/qafLx/iO9FgOmeSAp4v5ti/Q=="],
|
||||
|
||||
"swr": ["swr@2.4.1", "", { "dependencies": { "dequal": "^2.0.3", "use-sync-external-store": "^1.6.0" }, "peerDependencies": { "react": "^16.11.0 || ^17.0.0 || ^18.0.0 || ^19.0.0" } }, "sha512-2CC6CiKQtEwaEeNiqWTAw9PGykW8SR5zZX8MZk6TeAvEAnVS7Visz8WzphqgtQ8v2xz/4Q5K+j+SeMaKXeeQIA=="],
|
||||
|
||||
"throttleit": ["throttleit@2.1.0", "", {}, "sha512-nt6AMGKW1p/70DF/hGBdJB57B8Tspmbp5gfJ8ilhLnt7kkr2ye7hzD6NVG8GGErk2HWF34igrL2CXmNIkzKqKw=="],
|
||||
|
||||
"typescript": ["typescript@6.0.3", "", { "bin": { "tsc": "bin/tsc", "tsserver": "bin/tsserver" } }, "sha512-y2TvuxSZPDyQakkFRPZHKFm+KKVqIisdg9/CZwm9ftvKXLP8NRWj38/ODjNbr43SsoXqNuAisEf1GdCxqWcdBw=="],
|
||||
|
||||
"undici-types": ["undici-types@7.24.6", "", {}, "sha512-WRNW+sJgj5OBN4/0JpHFqtqzhpbnV0GuB+OozA9gCL7a993SmU+1JBZCzLNxYsbMfIeDL+lTsphD5jN5N+n0zg=="],
|
||||
|
||||
"use-sync-external-store": ["use-sync-external-store@1.6.0", "", { "peerDependencies": { "react": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0" } }, "sha512-Pp6GSwGP/NrPIrxVFAIkOQeyw8lFenOHijQWkUTrDvrF4ALqylP2C/KCkeS9dpUM3KvYRQhna5vt7IL95+ZQ9w=="],
|
||||
|
||||
"zod": ["zod@4.4.3", "", {}, "sha512-ytENFjIJFl2UwYglde2jchW2Hwm4GJFLDiSXWdTrJQBIN9Fcyp7n4DhxJEiWNAJMV1/BqWfW/kkg71UDcHJyTQ=="],
|
||||
}
|
||||
}
|
||||
|
||||
+56
-12
@@ -335,31 +335,75 @@
|
||||
status.textContent = "Generating...";
|
||||
result.innerHTML = "";
|
||||
|
||||
let preview = null;
|
||||
const ensurePreview = () => {
|
||||
if (!preview) {
|
||||
preview = document.createElement("img");
|
||||
preview.alt = body.prompt;
|
||||
result.appendChild(preview);
|
||||
}
|
||||
return preview;
|
||||
};
|
||||
|
||||
try {
|
||||
const res = await fetch("/api/generate", {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify(body),
|
||||
});
|
||||
const data = await res.json();
|
||||
if (!res.ok) throw new Error(data.error || ("HTTP " + res.status));
|
||||
|
||||
if (!data.images?.length) {
|
||||
status.textContent = "No images returned.";
|
||||
status.className = "status error";
|
||||
const contentType = res.headers.get("content-type") || "";
|
||||
if (!contentType.includes("event-stream")) {
|
||||
const data = await res.json().catch(() => ({}));
|
||||
throw new Error(data.error || ("HTTP " + res.status));
|
||||
}
|
||||
if (!res.body) throw new Error("No response body");
|
||||
|
||||
const reader = res.body.getReader();
|
||||
const decoder = new TextDecoder();
|
||||
let buffer = "";
|
||||
|
||||
const handleBlock = (raw) => {
|
||||
let event = "message";
|
||||
const dataLines = [];
|
||||
for (const line of raw.split("\n")) {
|
||||
if (line.startsWith(":")) continue;
|
||||
if (line.startsWith("event:")) event = line.slice(6).trim();
|
||||
else if (line.startsWith("data:")) dataLines.push(line.slice(5).trim());
|
||||
}
|
||||
if (dataLines.length === 0) return;
|
||||
let payload;
|
||||
try {
|
||||
payload = JSON.parse(dataLines.join("\n"));
|
||||
} catch {
|
||||
return;
|
||||
}
|
||||
|
||||
for (const src of data.images) {
|
||||
const img = document.createElement("img");
|
||||
img.src = src;
|
||||
img.alt = body.prompt;
|
||||
result.appendChild(img);
|
||||
}
|
||||
if (event === "partial") {
|
||||
ensurePreview().src = payload.image;
|
||||
status.textContent = "Receiving preview " + ((payload.index ?? 0) + 1) + "...";
|
||||
} else if (event === "final") {
|
||||
ensurePreview().src = payload.image;
|
||||
status.textContent = "Done.";
|
||||
} else if (event === "error") {
|
||||
throw new Error(payload.message || "Unknown error");
|
||||
}
|
||||
};
|
||||
|
||||
while (true) {
|
||||
const { value, done } = await reader.read();
|
||||
if (done) break;
|
||||
buffer += decoder.decode(value, { stream: true });
|
||||
let idx;
|
||||
while ((idx = buffer.indexOf("\n\n")) !== -1) {
|
||||
handleBlock(buffer.slice(0, idx));
|
||||
buffer = buffer.slice(idx + 2);
|
||||
}
|
||||
}
|
||||
if (buffer.trim()) handleBlock(buffer);
|
||||
} catch (err) {
|
||||
status.textContent = "Error: " + (err.message || String(err));
|
||||
status.className = "status error";
|
||||
if (preview && !preview.src) preview.remove();
|
||||
} finally {
|
||||
btn.disabled = false;
|
||||
}
|
||||
|
||||
@@ -1,71 +1,8 @@
|
||||
import { createOpenAICompatible } from "@ai-sdk/openai-compatible";
|
||||
import { generateImage } from "ai";
|
||||
import index from "./index.html";
|
||||
|
||||
type Size = `${number}x${number}`;
|
||||
|
||||
async function generateWithReference({
|
||||
baseURL,
|
||||
apiKey,
|
||||
model,
|
||||
prompt,
|
||||
size,
|
||||
referenceImages,
|
||||
}: {
|
||||
baseURL: string;
|
||||
apiKey: string;
|
||||
model: string;
|
||||
prompt: string;
|
||||
size: Size;
|
||||
referenceImages: string[];
|
||||
}): Promise<string[]> {
|
||||
const form = new FormData();
|
||||
form.append("model", model);
|
||||
form.append("prompt", prompt);
|
||||
form.append("size", size);
|
||||
|
||||
for (let i = 0; i < referenceImages.length; i++) {
|
||||
const dataUrl = referenceImages[i];
|
||||
if (!dataUrl) continue;
|
||||
const match = dataUrl.match(/^data:([^;]+);base64,(.+)$/);
|
||||
if (!match) continue;
|
||||
const mime = match[1]!;
|
||||
const b64 = match[2]!;
|
||||
const bytes = Buffer.from(b64, "base64");
|
||||
const ext = mime.split("/")[1] ?? "png";
|
||||
form.append("image", new Blob([bytes], { type: mime }), `ref-${i}.${ext}`);
|
||||
}
|
||||
|
||||
const url = `${baseURL.replace(/\/+$/, "")}/images/edits`;
|
||||
const res = await fetch(url, {
|
||||
method: "POST",
|
||||
headers: { Authorization: `Bearer ${apiKey}` },
|
||||
body: form,
|
||||
});
|
||||
|
||||
if (!res.ok) {
|
||||
const text = await res.text().catch(() => "");
|
||||
throw new Error(`Upstream ${res.status}: ${text || res.statusText}`);
|
||||
}
|
||||
|
||||
const data = (await res.json()) as {
|
||||
data?: Array<{ b64_json?: string }>;
|
||||
};
|
||||
|
||||
return (data.data ?? [])
|
||||
.map((item) => (item.b64_json ? `data:image/png;base64,${item.b64_json}` : null))
|
||||
.filter((s): s is string => s !== null);
|
||||
}
|
||||
|
||||
const server = Bun.serve({
|
||||
hostname: "0.0.0.0",
|
||||
routes: {
|
||||
"/": index,
|
||||
"/api/generate": {
|
||||
POST: async (req) => {
|
||||
try {
|
||||
const { baseURL, apiKey, model, prompt, size, referenceImages } =
|
||||
(await req.json()) as {
|
||||
type GenerateRequest = {
|
||||
baseURL?: string;
|
||||
apiKey?: string;
|
||||
model?: string;
|
||||
@@ -74,50 +11,249 @@ const server = Bun.serve({
|
||||
referenceImages?: string[];
|
||||
};
|
||||
|
||||
type SSEController = ReadableStreamDefaultController<Uint8Array>;
|
||||
|
||||
const encoder = new TextEncoder();
|
||||
|
||||
function sseEvent(controller: SSEController, event: string, data: unknown): void {
|
||||
controller.enqueue(
|
||||
encoder.encode(`event: ${event}\ndata: ${JSON.stringify(data)}\n\n`),
|
||||
);
|
||||
}
|
||||
|
||||
function sseComment(controller: SSEController, text: string): void {
|
||||
controller.enqueue(encoder.encode(`: ${text}\n\n`));
|
||||
}
|
||||
|
||||
function decodeDataUrl(dataUrl: string): { bytes: Buffer; mime: string } | null {
|
||||
const match = dataUrl.match(/^data:([^;]+);base64,(.+)$/);
|
||||
if (!match) return null;
|
||||
const mime = match[1]!;
|
||||
const b64 = match[2]!;
|
||||
return { bytes: Buffer.from(b64, "base64"), mime };
|
||||
}
|
||||
|
||||
async function callUpstream(args: {
|
||||
baseURL: string;
|
||||
apiKey: string;
|
||||
model: string;
|
||||
prompt: string;
|
||||
size: Size;
|
||||
referenceImages: string[];
|
||||
stream: boolean;
|
||||
}): Promise<Response> {
|
||||
const { baseURL, apiKey, model, prompt, size, referenceImages, stream } = args;
|
||||
const isEdit = referenceImages.length > 0;
|
||||
const url = `${baseURL.replace(/\/+$/, "")}/images/${isEdit ? "edits" : "generations"}`;
|
||||
|
||||
if (isEdit) {
|
||||
const form = new FormData();
|
||||
form.append("model", model);
|
||||
form.append("prompt", prompt);
|
||||
form.append("size", size);
|
||||
if (stream) {
|
||||
form.append("stream", "true");
|
||||
form.append("partial_images", "2");
|
||||
}
|
||||
for (let i = 0; i < referenceImages.length; i++) {
|
||||
const dataUrl = referenceImages[i];
|
||||
if (!dataUrl) continue;
|
||||
const decoded = decodeDataUrl(dataUrl);
|
||||
if (!decoded) continue;
|
||||
const ext = decoded.mime.split("/")[1] ?? "png";
|
||||
form.append(
|
||||
"image",
|
||||
new Blob([decoded.bytes], { type: decoded.mime }),
|
||||
`ref-${i}.${ext}`,
|
||||
);
|
||||
}
|
||||
return fetch(url, {
|
||||
method: "POST",
|
||||
headers: { Authorization: `Bearer ${apiKey}` },
|
||||
body: form,
|
||||
});
|
||||
}
|
||||
|
||||
const body: Record<string, unknown> = { model, prompt, size };
|
||||
if (stream) {
|
||||
body.stream = true;
|
||||
body.partial_images = 2;
|
||||
}
|
||||
return fetch(url, {
|
||||
method: "POST",
|
||||
headers: {
|
||||
Authorization: `Bearer ${apiKey}`,
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
body: JSON.stringify(body),
|
||||
});
|
||||
}
|
||||
|
||||
function parseSSEBlock(raw: string): { event: string; data: string } | null {
|
||||
let eventName = "message";
|
||||
const dataLines: string[] = [];
|
||||
for (const line of raw.split("\n")) {
|
||||
if (line.startsWith(":")) continue;
|
||||
if (line.startsWith("event:")) eventName = line.slice(6).trim();
|
||||
else if (line.startsWith("data:")) dataLines.push(line.slice(5).trim());
|
||||
}
|
||||
if (dataLines.length === 0) return null;
|
||||
return { event: eventName, data: dataLines.join("\n") };
|
||||
}
|
||||
|
||||
async function forwardUpstreamSSE(
|
||||
upstream: Response,
|
||||
controller: SSEController,
|
||||
): Promise<void> {
|
||||
if (!upstream.body) throw new Error("Upstream returned no body");
|
||||
const reader = upstream.body.getReader();
|
||||
const decoder = new TextDecoder();
|
||||
let buffer = "";
|
||||
|
||||
const handle = (raw: string) => {
|
||||
const block = parseSSEBlock(raw);
|
||||
if (!block) return;
|
||||
if (block.data === "[DONE]") return;
|
||||
let parsed: {
|
||||
type?: string;
|
||||
b64_json?: string;
|
||||
partial_image_index?: number;
|
||||
};
|
||||
try {
|
||||
parsed = JSON.parse(block.data);
|
||||
} catch {
|
||||
return;
|
||||
}
|
||||
const type = parsed.type ?? block.event;
|
||||
const b64 = parsed.b64_json;
|
||||
if (!b64) return;
|
||||
if (type.endsWith(".partial_image")) {
|
||||
sseEvent(controller, "partial", {
|
||||
image: `data:image/png;base64,${b64}`,
|
||||
index: parsed.partial_image_index ?? 0,
|
||||
});
|
||||
} else if (type.endsWith(".completed")) {
|
||||
sseEvent(controller, "final", {
|
||||
image: `data:image/png;base64,${b64}`,
|
||||
});
|
||||
}
|
||||
};
|
||||
|
||||
while (true) {
|
||||
const { value, done } = await reader.read();
|
||||
if (done) break;
|
||||
buffer += decoder.decode(value, { stream: true });
|
||||
let idx: number;
|
||||
while ((idx = buffer.indexOf("\n\n")) !== -1) {
|
||||
handle(buffer.slice(0, idx));
|
||||
buffer = buffer.slice(idx + 2);
|
||||
}
|
||||
}
|
||||
if (buffer.trim().length > 0) handle(buffer);
|
||||
}
|
||||
|
||||
async function forwardUpstreamJSON(
|
||||
upstream: Response,
|
||||
controller: SSEController,
|
||||
): Promise<void> {
|
||||
const data = (await upstream.json()) as {
|
||||
data?: Array<{ b64_json?: string }>;
|
||||
};
|
||||
for (const item of data.data ?? []) {
|
||||
if (!item.b64_json) continue;
|
||||
sseEvent(controller, "final", {
|
||||
image: `data:image/png;base64,${item.b64_json}`,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
function isStreamingUnsupportedError(errText: string): boolean {
|
||||
return /\b(stream|partial_images)\b/i.test(errText);
|
||||
}
|
||||
|
||||
const server = Bun.serve({
|
||||
hostname: "0.0.0.0",
|
||||
routes: {
|
||||
"/": index,
|
||||
"/api/generate": {
|
||||
POST: async (req) => {
|
||||
const body = (await req.json()) as GenerateRequest;
|
||||
const { baseURL, apiKey, model, prompt, size, referenceImages } = body;
|
||||
if (!baseURL || !apiKey || !model || !prompt) {
|
||||
return Response.json(
|
||||
{ error: "baseURL, apiKey, model, prompt are required" },
|
||||
{ status: 400 },
|
||||
);
|
||||
}
|
||||
|
||||
if (Array.isArray(referenceImages) && referenceImages.length > 0) {
|
||||
const images = await generateWithReference({
|
||||
const refs = Array.isArray(referenceImages) ? referenceImages : [];
|
||||
const args = {
|
||||
baseURL,
|
||||
apiKey,
|
||||
model,
|
||||
prompt,
|
||||
size: size ?? "1024x1024",
|
||||
referenceImages,
|
||||
});
|
||||
return Response.json({ images });
|
||||
size: size ?? ("1024x1024" as Size),
|
||||
referenceImages: refs,
|
||||
};
|
||||
|
||||
const stream = new ReadableStream<Uint8Array>({
|
||||
async start(controller) {
|
||||
const keepalive = setInterval(() => {
|
||||
try {
|
||||
sseComment(controller, "keepalive");
|
||||
} catch {}
|
||||
}, 20_000);
|
||||
|
||||
try {
|
||||
let upstream = await callUpstream({ ...args, stream: true });
|
||||
|
||||
if (!upstream.ok && upstream.status === 400) {
|
||||
const errText = await upstream.text().catch(() => "");
|
||||
if (isStreamingUnsupportedError(errText)) {
|
||||
upstream = await callUpstream({ ...args, stream: false });
|
||||
} else {
|
||||
throw new Error(`Upstream 400: ${errText || upstream.statusText}`);
|
||||
}
|
||||
}
|
||||
|
||||
const provider = createOpenAICompatible({
|
||||
name: "custom",
|
||||
apiKey,
|
||||
baseURL,
|
||||
});
|
||||
if (!upstream.ok) {
|
||||
const errText = await upstream.text().catch(() => "");
|
||||
throw new Error(
|
||||
`Upstream ${upstream.status}: ${errText || upstream.statusText}`,
|
||||
);
|
||||
}
|
||||
|
||||
const { images } = await generateImage({
|
||||
model: provider.imageModel(model),
|
||||
prompt,
|
||||
size: size || "1024x1024",
|
||||
});
|
||||
const contentType = upstream.headers.get("content-type") ?? "";
|
||||
if (contentType.includes("event-stream")) {
|
||||
await forwardUpstreamSSE(upstream, controller);
|
||||
} else {
|
||||
await forwardUpstreamJSON(upstream, controller);
|
||||
}
|
||||
|
||||
const out = images.map((img) => {
|
||||
const mediaType = (img as { mediaType?: string }).mediaType ?? "image/png";
|
||||
const base64 = (img as { base64?: string }).base64;
|
||||
return base64 ? `data:${mediaType};base64,${base64}` : null;
|
||||
}).filter(Boolean);
|
||||
|
||||
return Response.json({ images: out });
|
||||
sseEvent(controller, "done", {});
|
||||
} catch (err) {
|
||||
const message = err instanceof Error ? err.message : String(err);
|
||||
console.error("[generate] error:", err);
|
||||
return Response.json({ error: message }, { status: 500 });
|
||||
try {
|
||||
sseEvent(controller, "error", { message });
|
||||
} catch {}
|
||||
} finally {
|
||||
clearInterval(keepalive);
|
||||
try {
|
||||
controller.close();
|
||||
} catch {}
|
||||
}
|
||||
},
|
||||
});
|
||||
|
||||
return new Response(stream, {
|
||||
headers: {
|
||||
"Content-Type": "text/event-stream",
|
||||
"Cache-Control": "no-cache, no-transform",
|
||||
"X-Accel-Buffering": "no",
|
||||
Connection: "keep-alive",
|
||||
},
|
||||
});
|
||||
},
|
||||
},
|
||||
},
|
||||
development: {
|
||||
|
||||
@@ -14,9 +14,6 @@
|
||||
"typescript": "^6.0.3"
|
||||
},
|
||||
"dependencies": {
|
||||
"@ai-sdk/openai-compatible": "^2.0.47",
|
||||
"@ai-sdk/react": "^3.0.186",
|
||||
"ai": "^6.0.184",
|
||||
"react": "^19.2.6",
|
||||
"react-dom": "^19.2.6"
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user