mirror of
https://github.com/OpenRouterTeam/spawn.git
synced 2026-05-19 08:01:17 +00:00
fix(growth): migrate Phase 0b to OAuth 2.0, block em dashes, wire SPA tweet posting (#3331)
- growth.sh: guard Phase 0b on X_CLIENT_ID (was checking stale X_API_KEY) - x-fetch.ts: rewrite to use OAuth 2.0 Bearer tokens from state.db w/ auto-refresh - Strip em/en dashes from all generated JSON output (tweet, engagement, reddit) - Tighten prompt language against em dashes in all 3 growth prompts - SPA system prompt: tell Claude how to post tweets via x-post.ts and query tweets/candidates tables from state.db for context-aware Twitter conversations Co-authored-by: Claude <claude@anthropic.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
2306fb1914
commit
fe075190ea
6 changed files with 196 additions and 74 deletions
|
|
@ -99,7 +99,7 @@ Poster qualification:
|
|||
Relevance score: {score}/10
|
||||
|
||||
Draft reply:
|
||||
{a short casual reply the team could use, written like a real dev on reddit. 2-3 sentences, no em dashes, no corporate speak, lowercase ok. end with "disclosure: i help build this" if mentioning spawn}
|
||||
{a short casual reply the team could use, written like a real dev on reddit. 2-3 sentences. **ABSOLUTELY NO em dashes (—) or en dashes (–) — use periods, commas, or rephrase. This is non-negotiable, em dashes are an AI tell.** No corporate speak, lowercase ok. end with "disclosure: i help build this" if mentioning spawn}
|
||||
=== END CANDIDATE ===
|
||||
```
|
||||
|
||||
|
|
|
|||
|
|
@ -143,15 +143,21 @@ if [[ -f "${TWEET_TEMPLATE}" && "${COMMIT_COUNT}" -gt 0 ]]; then
|
|||
await Bun.write(process.env._OUT, texts.join("\n"));
|
||||
' 2>> "${LOG_FILE}" || true
|
||||
|
||||
# Extract json:tweet
|
||||
# Extract json:tweet (with em/en dash stripping)
|
||||
TWEET_JSON=""
|
||||
if [[ -f "${TWEET_OUTPUT_FILE}" ]]; then
|
||||
TWEET_JSON=$(_OUT="${TWEET_OUTPUT_FILE}" bun -e '
|
||||
const text = await Bun.file(process.env._OUT).text();
|
||||
const blocks = [...text.matchAll(/```json:tweet\n([\s\S]*?)\n```/g)];
|
||||
const stripDashes = (v) => typeof v === "string" ? v.replace(/\s*[\u2014\u2013]\s*/g, ", ") : v;
|
||||
const walk = (obj) => {
|
||||
if (Array.isArray(obj)) return obj.map(walk);
|
||||
if (obj && typeof obj === "object") return Object.fromEntries(Object.entries(obj).map(([k, v]) => [k, walk(v)]));
|
||||
return stripDashes(obj);
|
||||
};
|
||||
let result = "";
|
||||
for (const block of blocks) {
|
||||
try { result = JSON.stringify(JSON.parse(block[1].trim())); } catch {}
|
||||
try { result = JSON.stringify(walk(JSON.parse(block[1].trim()))); } catch {}
|
||||
}
|
||||
if (result) console.log(result);
|
||||
' 2>/dev/null) || true
|
||||
|
|
@ -178,7 +184,7 @@ else
|
|||
fi
|
||||
|
||||
# --- Phase 0b: Search X for mentions + draft engagement ---
|
||||
if [[ -z "${X_API_KEY:-}" ]]; then
|
||||
if [[ -z "${X_CLIENT_ID:-}" ]]; then
|
||||
log "Phase 0b: Skipping (no X API credentials)"
|
||||
else
|
||||
log "Phase 0b: Searching X for Spawn mentions..."
|
||||
|
|
@ -252,9 +258,15 @@ else
|
|||
XENG_JSON=$(_OUT="${XENG_OUTPUT_FILE}" bun -e '
|
||||
const text = await Bun.file(process.env._OUT).text();
|
||||
const blocks = [...text.matchAll(/```json:x_engage\n([\s\S]*?)\n```/g)];
|
||||
const stripDashes = (v) => typeof v === "string" ? v.replace(/\s*[\u2014\u2013]\s*/g, ", ") : v;
|
||||
const walk = (obj) => {
|
||||
if (Array.isArray(obj)) return obj.map(walk);
|
||||
if (obj && typeof obj === "object") return Object.fromEntries(Object.entries(obj).map(([k, v]) => [k, walk(v)]));
|
||||
return stripDashes(obj);
|
||||
};
|
||||
let result = "";
|
||||
for (const block of blocks) {
|
||||
try { result = JSON.stringify(JSON.parse(block[1].trim())); } catch {}
|
||||
try { result = JSON.stringify(walk(JSON.parse(block[1].trim()))); } catch {}
|
||||
}
|
||||
if (result) console.log(result);
|
||||
' 2>/dev/null) || true
|
||||
|
|
@ -407,9 +419,15 @@ if [[ -f "${CLAUDE_OUTPUT_FILE}" ]]; then
|
|||
CANDIDATE_JSON=$(_OUT="${CLAUDE_OUTPUT_FILE}" bun -e '
|
||||
const text = await Bun.file(process.env._OUT).text();
|
||||
const blocks = [...text.matchAll(/```json:candidate\n([\s\S]*?)\n```/g)];
|
||||
const stripDashes = (v) => typeof v === "string" ? v.replace(/\s*[\u2014\u2013]\s*/g, ", ") : v;
|
||||
const walk = (obj) => {
|
||||
if (Array.isArray(obj)) return obj.map(walk);
|
||||
if (obj && typeof obj === "object") return Object.fromEntries(Object.entries(obj).map(([k, v]) => [k, walk(v)]));
|
||||
return stripDashes(obj);
|
||||
};
|
||||
let result = "";
|
||||
for (const block of blocks) {
|
||||
try { result = JSON.stringify(JSON.parse(block[1].trim())); } catch {}
|
||||
try { result = JSON.stringify(walk(JSON.parse(block[1].trim()))); } catch {}
|
||||
}
|
||||
if (result) console.log(result);
|
||||
' 2>/dev/null)
|
||||
|
|
|
|||
|
|
@ -25,6 +25,7 @@ GIT_DATA_PLACEHOLDER
|
|||
2. **Draft exactly 1 tweet**, max 280 characters. Rules:
|
||||
- Write like a developer sharing something cool, not a marketing team
|
||||
- No corporate speak, no buzzwords, no "excited to announce"
|
||||
- **NEVER use em dashes (—) or en dashes (–).** Use a period, comma, or rephrase. This is non-negotiable — em dashes are a tell that AI wrote it.
|
||||
- At most 1 hashtag (only if it fits naturally)
|
||||
- Mention `@OpenRouterTeam` only if it fits naturally
|
||||
- OK to include a short URL like `https://openrouter.ai/spawn`
|
||||
|
|
|
|||
|
|
@ -28,6 +28,7 @@ X_DATA_PLACEHOLDER
|
|||
- Answer their question or add to the conversation
|
||||
- Mention Spawn only if it genuinely fits what they are discussing
|
||||
- Casual, developer-to-developer tone
|
||||
- **NEVER use em dashes (—) or en dashes (–).** Use a period, comma, or rephrase. Em dashes are an AI tell and must be avoided.
|
||||
- Include `https://openrouter.ai/spawn` only if it adds value
|
||||
- Disclosure: include "disclosure: i help build this" if recommending Spawn
|
||||
|
||||
|
|
|
|||
|
|
@ -1,35 +1,32 @@
|
|||
/**
|
||||
* X (Twitter) Fetch — Search for Spawn/OpenRouter mentions on X.
|
||||
*
|
||||
* Uses X API v2 to find tweets mentioning Spawn, OpenRouter, or related topics.
|
||||
* Gracefully exits with empty results if credentials are not configured.
|
||||
* Uses X API v2 with OAuth 2.0 Bearer tokens (stored in state.db by x-auth.ts).
|
||||
* Auto-refreshes tokens when expired. Gracefully exits empty if no tokens.
|
||||
*
|
||||
* Env vars: X_API_KEY, X_API_SECRET, X_ACCESS_TOKEN, X_ACCESS_TOKEN_SECRET
|
||||
* Env vars: X_CLIENT_ID, X_CLIENT_SECRET (for token refresh)
|
||||
*/
|
||||
|
||||
import { Database } from "bun:sqlite";
|
||||
import { createHmac, randomBytes } from "node:crypto";
|
||||
import { existsSync } from "node:fs";
|
||||
import * as v from "valibot";
|
||||
|
||||
const API_KEY = process.env.X_API_KEY ?? "";
|
||||
const API_SECRET = process.env.X_API_SECRET ?? "";
|
||||
const ACCESS_TOKEN = process.env.X_ACCESS_TOKEN ?? "";
|
||||
const ACCESS_TOKEN_SECRET = process.env.X_ACCESS_TOKEN_SECRET ?? "";
|
||||
const CLIENT_ID = process.env.X_CLIENT_ID ?? "";
|
||||
const CLIENT_SECRET = process.env.X_CLIENT_SECRET ?? "";
|
||||
const DB_PATH = `${process.env.HOME ?? "/tmp"}/.config/spawn/state.db`;
|
||||
|
||||
// Graceful skip if credentials are not configured
|
||||
if (!API_KEY || !API_SECRET || !ACCESS_TOKEN || !ACCESS_TOKEN_SECRET) {
|
||||
console.error("[x-fetch] No X API credentials configured — outputting empty results");
|
||||
console.log(JSON.stringify({ posts: [], postsScanned: 0 }));
|
||||
if (!CLIENT_ID || !CLIENT_SECRET) {
|
||||
console.error("[x-fetch] No X_CLIENT_ID/SECRET configured — outputting empty results");
|
||||
console.log(
|
||||
JSON.stringify({
|
||||
posts: [],
|
||||
postsScanned: 0,
|
||||
}),
|
||||
);
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
// Validate credential format — reject newlines that could corrupt headers
|
||||
if (/[\r\n]/.test(API_KEY) || /[\r\n]/.test(API_SECRET)) {
|
||||
console.error("Invalid X_API_KEY / X_API_SECRET: must not contain newlines");
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
// Search queries — shuffled each run for variety
|
||||
const QUERIES = shuffle([
|
||||
"openrouter spawn",
|
||||
|
|
@ -79,6 +76,12 @@ const XSearchResponseSchema = v.object({
|
|||
),
|
||||
});
|
||||
|
||||
const TokenResponseSchema = v.object({
|
||||
access_token: v.string(),
|
||||
refresh_token: v.optional(v.string()),
|
||||
expires_in: v.optional(v.number()),
|
||||
});
|
||||
|
||||
interface XPost {
|
||||
tweetId: string;
|
||||
text: string;
|
||||
|
|
@ -91,52 +94,119 @@ interface XPost {
|
|||
url: string;
|
||||
}
|
||||
|
||||
interface StoredTokens {
|
||||
accessToken: string;
|
||||
refreshToken: string;
|
||||
expiresAt: number;
|
||||
}
|
||||
|
||||
/** Fisher-Yates shuffle. */
|
||||
function shuffle<T>(arr: T[]): T[] {
|
||||
const a = [...arr];
|
||||
const a = [
|
||||
...arr,
|
||||
];
|
||||
for (let i = a.length - 1; i > 0; i--) {
|
||||
const j = Math.floor(Math.random() * (i + 1));
|
||||
[a[i], a[j]] = [a[j], a[i]];
|
||||
[a[i], a[j]] = [
|
||||
a[j],
|
||||
a[i],
|
||||
];
|
||||
}
|
||||
return a;
|
||||
}
|
||||
|
||||
/**
|
||||
* Generate OAuth 1.0a signature for X API requests.
|
||||
* Reference: https://developer.x.com/en/docs/authentication/oauth-1-0a
|
||||
*/
|
||||
function generateOAuthHeader(method: string, url: string, params: Record<string, string>): string {
|
||||
const oauthParams: Record<string, string> = {
|
||||
oauth_consumer_key: API_KEY,
|
||||
oauth_nonce: randomBytes(16).toString("hex"),
|
||||
oauth_signature_method: "HMAC-SHA1",
|
||||
oauth_timestamp: String(Math.floor(Date.now() / 1000)),
|
||||
oauth_token: ACCESS_TOKEN,
|
||||
oauth_version: "1.0",
|
||||
function loadTokens(): StoredTokens | null {
|
||||
if (!existsSync(DB_PATH)) return null;
|
||||
try {
|
||||
const db = new Database(DB_PATH, {
|
||||
readonly: true,
|
||||
});
|
||||
const row = db
|
||||
.query<
|
||||
{
|
||||
access_token: string;
|
||||
refresh_token: string;
|
||||
expires_at: number;
|
||||
},
|
||||
[]
|
||||
>("SELECT access_token, refresh_token, expires_at FROM x_tokens WHERE id = 1")
|
||||
.get();
|
||||
db.close();
|
||||
if (!row) return null;
|
||||
return {
|
||||
accessToken: row.access_token,
|
||||
refreshToken: row.refresh_token,
|
||||
expiresAt: row.expires_at,
|
||||
};
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function saveTokens(tokens: StoredTokens): void {
|
||||
const db = new Database(DB_PATH);
|
||||
db.run(
|
||||
`INSERT INTO x_tokens (id, access_token, refresh_token, expires_at, updated_at)
|
||||
VALUES (1, ?, ?, ?, ?)
|
||||
ON CONFLICT (id) DO UPDATE SET
|
||||
access_token = excluded.access_token,
|
||||
refresh_token = excluded.refresh_token,
|
||||
expires_at = excluded.expires_at,
|
||||
updated_at = excluded.updated_at`,
|
||||
[
|
||||
tokens.accessToken,
|
||||
tokens.refreshToken,
|
||||
tokens.expiresAt,
|
||||
new Date().toISOString(),
|
||||
],
|
||||
);
|
||||
db.close();
|
||||
}
|
||||
|
||||
async function refreshToken(currentRefresh: string): Promise<StoredTokens | null> {
|
||||
const basicAuth = Buffer.from(`${CLIENT_ID}:${CLIENT_SECRET}`).toString("base64");
|
||||
const res = await fetch("https://api.x.com/2/oauth2/token", {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"Content-Type": "application/x-www-form-urlencoded",
|
||||
Authorization: `Basic ${basicAuth}`,
|
||||
},
|
||||
body: new URLSearchParams({
|
||||
grant_type: "refresh_token",
|
||||
refresh_token: currentRefresh,
|
||||
}),
|
||||
});
|
||||
|
||||
if (!res.ok) {
|
||||
console.error(`[x-fetch] Token refresh failed: ${res.status}`);
|
||||
return null;
|
||||
}
|
||||
|
||||
const json: unknown = await res.json();
|
||||
const parsed = v.safeParse(TokenResponseSchema, json);
|
||||
if (!parsed.success) return null;
|
||||
|
||||
const newTokens: StoredTokens = {
|
||||
accessToken: parsed.output.access_token,
|
||||
refreshToken: parsed.output.refresh_token ?? currentRefresh,
|
||||
expiresAt: Date.now() + (parsed.output.expires_in ?? 7200) * 1000,
|
||||
};
|
||||
saveTokens(newTokens);
|
||||
return newTokens;
|
||||
}
|
||||
|
||||
const allParams = { ...params, ...oauthParams };
|
||||
const sortedKeys = Object.keys(allParams).sort();
|
||||
const paramString = sortedKeys
|
||||
.map((k) => `${encodeURIComponent(k)}=${encodeURIComponent(allParams[k])}`)
|
||||
.join("&");
|
||||
|
||||
const signatureBase = `${method.toUpperCase()}&${encodeURIComponent(url)}&${encodeURIComponent(paramString)}`;
|
||||
const signingKey = `${encodeURIComponent(API_SECRET)}&${encodeURIComponent(ACCESS_TOKEN_SECRET)}`;
|
||||
const signature = createHmac("sha1", signingKey).update(signatureBase).digest("base64");
|
||||
|
||||
oauthParams.oauth_signature = signature;
|
||||
|
||||
const headerParts = Object.keys(oauthParams)
|
||||
.sort()
|
||||
.map((k) => `${encodeURIComponent(k)}="${encodeURIComponent(oauthParams[k])}"`)
|
||||
.join(", ");
|
||||
|
||||
return `OAuth ${headerParts}`;
|
||||
async function getAccessToken(): Promise<string | null> {
|
||||
const tokens = loadTokens();
|
||||
if (!tokens) return null;
|
||||
if (Date.now() > tokens.expiresAt - 300_000) {
|
||||
const refreshed = await refreshToken(tokens.refreshToken);
|
||||
return refreshed?.accessToken ?? null;
|
||||
}
|
||||
return tokens.accessToken;
|
||||
}
|
||||
|
||||
/** Search X API v2 for recent tweets matching a query. */
|
||||
async function searchTweets(query: string): Promise<XPost[]> {
|
||||
async function searchTweets(query: string, accessToken: string): Promise<XPost[]> {
|
||||
const baseUrl = "https://api.x.com/2/tweets/search/recent";
|
||||
const params: Record<string, string> = {
|
||||
query,
|
||||
|
|
@ -151,11 +221,9 @@ async function searchTweets(query: string): Promise<XPost[]> {
|
|||
.join("&");
|
||||
const fullUrl = `${baseUrl}?${queryString}`;
|
||||
|
||||
const authHeader = generateOAuthHeader("GET", baseUrl, params);
|
||||
|
||||
const res = await fetch(fullUrl, {
|
||||
headers: {
|
||||
Authorization: authHeader,
|
||||
Authorization: `Bearer ${accessToken}`,
|
||||
"User-Agent": "spawn-growth/1.0",
|
||||
},
|
||||
});
|
||||
|
|
@ -192,14 +260,18 @@ async function searchTweets(query: string): Promise<XPost[]> {
|
|||
|
||||
/** Load tweet IDs already processed from the tweets DB. */
|
||||
function loadSeenTweetIds(): Set<string> {
|
||||
const dbPath = `${process.env.HOME ?? "/tmp"}/.config/spawn/state.db`;
|
||||
if (!existsSync(dbPath)) return new Set();
|
||||
if (!existsSync(DB_PATH)) return new Set();
|
||||
try {
|
||||
const db = new Database(dbPath, { readonly: true });
|
||||
const db = new Database(DB_PATH, {
|
||||
readonly: true,
|
||||
});
|
||||
const rows = db
|
||||
.query<{ source_tweet_id: string }, []>(
|
||||
"SELECT source_tweet_id FROM tweets WHERE source_tweet_id IS NOT NULL",
|
||||
)
|
||||
.query<
|
||||
{
|
||||
source_tweet_id: string;
|
||||
},
|
||||
[]
|
||||
>("SELECT source_tweet_id FROM tweets WHERE source_tweet_id IS NOT NULL")
|
||||
.all();
|
||||
db.close();
|
||||
return new Set(rows.map((r) => r.source_tweet_id));
|
||||
|
|
@ -221,20 +293,34 @@ async function pooled<T>(tasks: Array<() => Promise<T>>, limit: number): Promise
|
|||
}
|
||||
|
||||
await Promise.all(
|
||||
Array.from({ length: Math.min(limit, tasks.length) }, () => worker()),
|
||||
Array.from(
|
||||
{
|
||||
length: Math.min(limit, tasks.length),
|
||||
},
|
||||
() => worker(),
|
||||
),
|
||||
);
|
||||
return results;
|
||||
}
|
||||
|
||||
async function main(): Promise<void> {
|
||||
const accessToken = await getAccessToken();
|
||||
if (!accessToken) {
|
||||
console.error("[x-fetch] No valid tokens — run x-auth.ts first");
|
||||
console.log(
|
||||
JSON.stringify({
|
||||
posts: [],
|
||||
postsScanned: 0,
|
||||
}),
|
||||
);
|
||||
process.exit(0);
|
||||
}
|
||||
console.error("[x-fetch] Authenticated");
|
||||
|
||||
const seenIds = loadSeenTweetIds();
|
||||
console.error(`[x-fetch] ${seenIds.size} tweets already seen in DB`);
|
||||
|
||||
const searchTasks = QUERIES.map(
|
||||
(query) => () => searchTweets(query),
|
||||
);
|
||||
const searchTasks = QUERIES.map((query) => () => searchTweets(query, accessToken));
|
||||
|
||||
console.error(`[x-fetch] Firing ${searchTasks.length} searches (concurrency=${MAX_CONCURRENT})...`);
|
||||
|
||||
|
|
@ -256,7 +342,9 @@ async function main(): Promise<void> {
|
|||
|
||||
console.error(`[x-fetch] Found ${allPosts.size} unique tweets (${skippedSeen} already seen, skipped)`);
|
||||
|
||||
const postsArray = [...allPosts.values()];
|
||||
const postsArray = [
|
||||
...allPosts.values(),
|
||||
];
|
||||
const filtered = postsArray.filter((p) => p.likes >= 1 || p.replies >= 1);
|
||||
filtered.sort((a, b) => b.likes - a.likes);
|
||||
|
||||
|
|
|
|||
|
|
@ -310,12 +310,26 @@ function sanitizeStdinInput(input: string): string {
|
|||
|
||||
const SYSTEM_PROMPT = `You are SPA (Spawn's Personal Agent), a Slack bot for the Spawn project (${GITHUB_REPO}).
|
||||
|
||||
Your primary job is to help manage GitHub issues based on Slack conversations:
|
||||
Your primary job is to help manage GitHub issues and X/Twitter posts based on Slack conversations:
|
||||
|
||||
1. **Create issues**: When a thread describes a bug, feature request, or task — create a GitHub issue with \`gh issue create --repo ${GITHUB_REPO}\`. Use a clear title and include the Slack context in the body.
|
||||
2. **Update issues**: When a thread references an existing issue (by number like #123) — add comments, update labels, or close issues as appropriate using \`gh issue comment\`, \`gh issue edit\`, etc.
|
||||
1. **Create issues**: When a thread describes a bug, feature request, or task, create a GitHub issue with \`gh issue create --repo ${GITHUB_REPO}\`. Use a clear title and include the Slack context in the body.
|
||||
2. **Update issues**: When a thread references an existing issue (by number like #123), add comments, update labels, or close issues as appropriate using \`gh issue comment\`, \`gh issue edit\`, etc.
|
||||
3. **Search issues**: When asked about existing issues, search with \`gh issue list --repo ${GITHUB_REPO}\` or \`gh issue view\`.
|
||||
4. **General help**: Answer questions about the Spawn codebase, suggest fixes, or help triage.
|
||||
4. **Post tweets to X/Twitter**: When a user asks to post to X/Twitter, run:
|
||||
\`\`\`
|
||||
TWEET_TEXT="<your tweet, max 280 chars>" bun run /home/lab/spawn/.claude/skills/setup-agent-team/x-post.ts
|
||||
\`\`\`
|
||||
Required env vars (X_CLIENT_ID, X_CLIENT_SECRET) are already in your environment. Tokens auto-refresh.
|
||||
To reply to a tweet, also set \`REPLY_TO_TWEET_ID=<id>\`.
|
||||
On success, the script prints JSON \`{"id":"...","text":"..."}\` — share the tweet URL \`https://x.com/i/status/<id>\` in the Slack thread.
|
||||
**IMPORTANT**: Never use em dashes (—) or en dashes (–) in tweets. Use periods, commas, or rephrase. Em dashes are an AI tell.
|
||||
5. **Query tweet/reddit state**: Inspect pending or posted candidates with SQLite:
|
||||
\`\`\`
|
||||
sqlite3 ~/.config/spawn/state.db "SELECT tweet_text, status, posted_text FROM tweets ORDER BY created_at DESC LIMIT 10"
|
||||
sqlite3 ~/.config/spawn/state.db "SELECT title, subreddit, status FROM candidates ORDER BY created_at DESC LIMIT 10"
|
||||
\`\`\`
|
||||
Use this to answer questions like "what tweets have we posted today?" or "what's in the queue?"
|
||||
6. **General help**: Answer questions about the Spawn codebase, suggest fixes, or help triage.
|
||||
|
||||
Always use the \`gh\` CLI for GitHub operations. You are already authenticated.
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue