vrr/supermemory
2
0
Fork 0
mirror of https://github.com/supermemoryai/supermemory.git synced 2026-08-14 19:14:01 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Merge branch 'main' into feat/landing_revamp

Browse source
This commit is contained in:
Kinfe Michael Tariku 2024-06-25 19:56:54 +03:00 • committed by GitHub
commit f46e42c2df
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
331 changed files with 17342 additions and 961 deletions
Download patch file Download diff file Expand all files Collapse all files

74
.github/workflows/build.yml vendored
View file

@ -1,74 +0,0 @@
name: Build and Deploy Changes
on:
push:
branches: [main]
paths:
- "apps/web/**"
- "apps/extension/**"
- "apps/cf-ai-backend/**"
pull_request:
branches: [main]
paths:
- "apps/web/**"
- "apps/extension/**"
- "apps/cf-ai-backend/**"
jobs:
build-extension:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: ./.github/actions/buildextension
build-app:
runs-on: ubuntu-latest
steps:
- name: Checkout repo
uses: actions/checkout@v3
- name: Setup Bun
uses: oven-sh/setup-bun@v1
with:
bun-version: latest
- name: Install packages
run: bun i
shell: bash
- name: Build app
run: bun run pages:build
working-directory: apps/web
shell: bash
env:
GOOGLE_CLIENT_ID: ${{ secrets.GOOGLE_CLIENT_ID }}
GOOGLE_CLIENT_SECRET: ${{ secrets.GOOGLE_CLIENT_SECRET }}
NEXTAUTH_SECRET: ${{ secrets.NEXTAUTH_SECRET }}
DATABASE_URL: ${{ secrets.DATABASE_URL }}
NEXTAUTH_URL: ${{ secrets.NEXTAUTH_URL }}
BACKEND_SECURITY_KEY: ${{ secrets.BACKEND_SECURITY_KEY }}
- name: Publish to Cloudflare Pages
uses: cloudflare/pages-action@v1
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
with:
apiToken: ${{ secrets.CF_API_TOKEN }}
accountId: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
projectName: ${{ secrets.CLOUDFLARE_PROJECT_NAME }}
directory: apps/web/.vercel/output/static
branch: main
# deploy-cf-worker:
# runs-on: ubuntu-latest
# steps:
# - name: Checkout repo
# uses: actions/checkout@v3
# - name: Deploy to Cloudflare Workers
# uses: cloudflare/wrangler-action@1.2.0
# with:
# apiToken: ${{ secrets.CF_API_TOKEN }}
# workingDirectory: apps/cf-ai-backend
# env:
# OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
# SECURITY_KEY: ${{ secrets.BACKEND_SECURITY_KEY }}

2
.gitignore vendored
View file

@ -4,6 +4,8 @@ bun.lockb
.*.vars
.wrangler
.million
yarn.lock
package-lock.json
# Dependencies
node_modules

15
SETUP-GUIDE.md
View file

@ -13,12 +13,13 @@
3. Create a `.dev.vars` file in `apps/web` with the following content:
```bash
GOOGLE_CLIENT_ID="-"
GOOGLE_CLIENT_SECRET="-"
GOOGLE_CLIENT_ID="-" // required, visit https://developers.google.com/identity/protocols/oauth2
GOOGLE_CLIENT_SECRET="-" // required
NEXTAUTH_SECRET='nextauthsecret'
DATABASE_URL='database.sqlite'
NEXTAUTH_URL='http://localhost:3000'
BACKEND_SECURITY_KEY='veryrandomsecuritykey'
BACKEND_BASE_URL="where your backend is hosted"
```
4. Setup the database:
@ -28,10 +29,10 @@ First, edit the `wrangler.toml` file in `apps/web` to point the d1 database to y
You can create a d1 database by running this command
```
wrangler d1 create DATABASE_NAME
bunx wrangler d1 create <YOUR_DATABASE_NAME>
```
And then replace these values
And then replace database_name and database_id with the values
```
[[d1_databases]]
@ -43,10 +44,12 @@ database_id = "YOUR_DB_ID"
Simply run this command in `apps/web`
```
wrangler d1 execute dev-d1-anycontext --local --file=db/prepare.sql
bunx wrangler d1 migrations apply <YOUR_DATABASE_NAME>
```
If it runs, you can set up the cloud database as well by removing the `--local` flag.
If it runs, you can set up the cloud database as well by removing the `--local` flag,
if you just want to contribute to frontend then just run `bun run dev` in the root of the project and done! (you won't be able to try ai stuff), otherwise continue...
5. You need to host your own worker for the `apps/cf-ai-backend` module.

2
apps/browser-rendering

@ -1 +1 @@
Subproject commit b37c962365a36cf342a31a196f4908f4f1343553
Subproject commit 4d21045a45fdbf56b7483d3704ae0474ebf044fb

84
apps/cf-ai-backend/README.md
View file

@ -1,58 +1,50 @@
# Hono minimal project
baseURL: https://new-cf-ai-backend.dhravya.workers.dev
This is a minimal project with [Hono](https://github.com/honojs/hono/) for Cloudflare Workers.
Authentication:
You must authenticate with a header and `Authorization: bearer token` for each request in `/api/*` routes.
## Features
### Add content:
- Minimal
- TypeScript
- Wrangler to develop and deploy.
- [Jest](https://jestjs.io/ja/) for testing.
## Usage
Initialize
POST `/api/add` with
```
npx create-cloudflare my-app https://github.com/honojs/hono-minimal
body {
pageContent: z.string(),
title: z.string().optional(),
description: z.string().optional(),
space: z.string().optional(),
url: z.string(),
user: z.string(),
}
```
Install
### Query without user data
GET `/api/ask` with
query `?query=testing`
(this is temp but works perfectly, will change soon for chat use cases specifically)
### Query vectorize and get results in natural language
POST `/api/chat` with
```
yarn install
query paramters (?query=...&" {
query: z.string(),
topK: z.number().optional().default(10),
user: z.string(),
spaces: z.string().optional(),
sourcesOnly: z.string().optional().default("false"),
model: z.string().optional().default("gpt-4o"),
}
body z.object({
chatHistory: z.array(contentObj).optional(),
});
```
Develop
### Delete vectors
```
yarn dev
```
Test
```
yarn test
```
Deploy
```
yarn deploy
```
## Examples
See: <https://github.com/honojs/examples>
## For more information
See: <https://honojs.dev>
## Author
Yusuke Wada <https://github.com/yusukebe>
## License
MIT
DELETE `/api/delete` with
query param websiteUrl, user

2
apps/cf-ai-backend/package.json
View file

@ -6,7 +6,7 @@
"scripts": {
"test": "jest --verbose",
"deploy": "wrangler deploy",
"dev": "wrangler dev",
"dev": "wrangler dev --remote --port 8686",
"start": "wrangler dev",
"unsafe-reset-vector-db": "wrangler vectorize delete supermem-vector && wrangler vectorize create --dimensions=1536 supermem-vector-1 --metric=cosine"
},

121
apps/cf-ai-backend/src/helper.ts
View file

@ -21,8 +21,6 @@ export async function initQuery(
index: c.env.VECTORIZE_INDEX,
});
const DEFAULT_MODEL = "gpt-4o";
let selectedModel:
| ReturnType<ReturnType<typeof createOpenAI>>
| ReturnType<ReturnType<typeof createGoogleGenerativeAI>>
@ -52,12 +50,6 @@ export async function initQuery(
break;
}
if (!selectedModel) {
throw new Error(
`Model ${model} not found and default model ${DEFAULT_MODEL} is also not available.`,
);
}
return { store, model: selectedModel };
}
@ -72,19 +64,60 @@ export async function deleteDocument({
c: Context<{ Bindings: Env }>;
store: CloudflareVectorizeStore;
}) {
const toBeDeleted = `${url}-${user}`;
const toBeDeleted = `${url}#supermemory-web`;
const random = seededRandom(toBeDeleted);
const uuid =
random().toString(36).substring(2, 15) +
random().toString(36).substring(2, 15);
await c.env.KV.list({ prefix: uuid }).then(async (keys) => {
for (const key of keys.keys) {
await c.env.KV.delete(key.name);
await store.delete({ ids: [key.name] });
const allIds = await c.env.KV.list({ prefix: uuid });
if (allIds.keys.length > 0) {
const savedVectorIds = allIds.keys.map((key) => key.name);
const vectors = await c.env.VECTORIZE_INDEX.getByIds(savedVectorIds);
// We don't actually delete document directly, we just remove the user from the metadata.
// If there's no user left, we can delete the document.
const newVectors = vectors.map((vector) => {
delete vector.metadata[`user-${user}`];
// Get count of how many users are left
const userCount = Object.keys(vector.metadata).filter((key) =>
key.startsWith("user-"),
).length;
// If there's no user left, we can delete the document.
// need to make sure that every chunk is deleted otherwise it would be problematic.
if (userCount === 0) {
store.delete({ ids: savedVectorIds });
void Promise.all(savedVectorIds.map((id) => c.env.KV.delete(id)));
return null;
}
return vector;
});
// If all vectors are null (deleted), we can delete the KV too. Otherwise, we update (upsert) the vectors.
if (newVectors.every((v) => v === null)) {
await c.env.KV.delete(uuid);
} else {
await c.env.VECTORIZE_INDEX.upsert(newVectors.filter((v) => v !== null));
}
});
}
}
function sanitizeKey(key: string): string {
if (!key) throw new Error("Key cannot be empty");
// Remove or replace invalid characters
let sanitizedKey = key.replace(/[.$"]/g, "_");
// Ensure key does not start with $
if (sanitizedKey.startsWith("$")) {
sanitizedKey = sanitizedKey.substring(1);
}
return sanitizedKey;
}
export async function batchCreateChunksAndEmbeddings({
@ -98,19 +131,47 @@ export async function batchCreateChunksAndEmbeddings({
chunks: string[];
context: Context<{ Bindings: Env }>;
}) {
const ourID = `${body.url}-${body.user}`;
await deleteDocument({ url: body.url, user: body.user, c: context, store });
//! NOTE that we use #supermemory-web to ensure that
//! If a user saves it through the extension, we don't want other users to be able to see it.
// Requests from the extension should ALWAYS have a unique ID with the USERiD in it.
// I cannot stress this enough, important for security.
const ourID = `${body.url}#supermemory-web`;
const random = seededRandom(ourID);
const uuid =
random().toString(36).substring(2, 15) +
random().toString(36).substring(2, 15);
const allIds = await context.env.KV.list({ prefix: uuid });
// If some chunks for that content already exist, we'll just update the metadata to include
// the user.
if (allIds.keys.length > 0) {
const savedVectorIds = allIds.keys.map((key) => key.name);
const vectors = await context.env.VECTORIZE_INDEX.getByIds(savedVectorIds);
// Now, we'll update all vector metadatas with one more userId and all spaceIds
const newVectors = vectors.map((vector) => {
vector.metadata = {
...vector.metadata,
[`user-${body.user}`]: 1,
// For each space in body, add the spaceId to the vector metadata
...(body.spaces ?? [])?.reduce((acc, space) => {
acc[`space-${body.user}-${space}`] = 1;
return acc;
}, {}),
};
return vector;
});
await context.env.VECTORIZE_INDEX.upsert(newVectors);
return;
}
for (let i = 0; i < chunks.length; i++) {
const chunk = chunks[i];
const uuid =
random().toString(36).substring(2, 15) +
random().toString(36).substring(2, 15) +
"-" +
i;
const chunkId = `${uuid}-${i}`;
const newPageContent = `Title: ${body.title}\nDescription: ${body.description}\nURL: ${body.url}\nContent: ${chunk}`;
@ -121,19 +182,25 @@ export async function batchCreateChunksAndEmbeddings({
metadata: {
title: body.title?.slice(0, 50) ?? "",
description: body.description ?? "",
space: body.space ?? "",
url: body.url,
user: body.user,
type: body.type ?? "page",
content: newPageContent,
[sanitizeKey(`user-${body.user}`)]: 1,
...body.spaces?.reduce((acc, space) => {
acc[`space-${body.user}-${space}`] = 1;
return acc;
}, {}),
},
},
],
{
ids: [uuid],
ids: [chunkId],
},
);
console.log("Docs added: ", docs);
await context.env.KV.put(uuid, ourID);
await context.env.KV.put(chunkId, ourID);
}
}

13
apps/cf-ai-backend/src/index.test.ts
View file

@ -1,13 +0,0 @@
import app from ".";
// TODO: write more tests
describe("Test the application", () => {
it("Should return 200 response", async () => {
const res = await app.request("http://localhost/");
expect(res.status).toBe(200);
}),
it("Should return 404 response", async () => {
const res = await app.request("http://localhost/404");
expect(res.status).toBe(404);
});
});

445
apps/cf-ai-backend/src/index.ts
View file

@ -1,6 +1,6 @@
import { z } from "zod";
import { Hono } from "hono";
import { CoreMessage, streamText } from "ai";
import { CoreMessage, generateText, streamText, tool } from "ai";
import { chatObj, Env, vectorObj } from "./types";
import {
batchCreateChunksAndEmbeddings,
@ -14,9 +14,18 @@ import { bearerAuth } from "hono/bearer-auth";
import { zValidator } from "@hono/zod-validator";
import chunkText from "./utils/chonker";
import { systemPrompt, template } from "./prompts/prompt1";
import { swaggerUI } from "@hono/swagger-ui";
import { createOpenAI } from "@ai-sdk/openai";
const app = new Hono<{ Bindings: Env }>();
app.get(
"/ui",
swaggerUI({
url: "/doc",
}),
);
// ------- MIDDLEWARES -------
app.use("*", poweredBy());
app.use("*", timing());
@ -31,6 +40,17 @@ app.use("/api/", async (c, next) => {
});
// ------- MIDDLEWARES END -------
const fileSchema = z
.instanceof(File)
.refine(
(file) => file.size <= 10 * 1024 * 1024,
"File size should be less than 10MB",
) // Validate file size
.refine(
(file) => ["image/jpeg", "image/png", "image/gif"].includes(file.type),
"Invalid file type",
); // Validate file type
app.get("/", (c) => {
return c.text("Supermemory backend API is running!");
});
@ -54,6 +74,82 @@ app.post("/api/add", zValidator("json", vectorObj), async (c) => {
return c.json({ status: "ok" });
});
app.post(
"/api/add-with-image",
zValidator(
"form",
z.object({
images: z
.array(fileSchema)
.min(1, "At least one image is required")
.optional(),
"images[]": z
.array(fileSchema)
.min(1, "At least one image is required")
.optional(),
text: z.string().optional(),
spaces: z.array(z.string()).optional(),
url: z.string(),
user: z.string(),
}),
(c) => {
console.log(c);
},
),
async (c) => {
const body = c.req.valid("form");
const { store } = await initQuery(c);
if (!(body.images || body["images[]"])) {
return c.json({ status: "error", message: "No images found" }, 400);
}
const imagePromises = (body.images ?? body["images[]"]).map(
async (image) => {
const buffer = await image.arrayBuffer();
const input = {
image: [...new Uint8Array(buffer)],
prompt:
"What's in this image? caption everything you see in great detail. If it has text, do an OCR and extract all of it.",
max_tokens: 1024,
};
const response = await c.env.AI.run(
"@cf/llava-hf/llava-1.5-7b-hf",
input,
);
console.log(response.description);
return response.description;
},
);
const imageDescriptions = await Promise.all(imagePromises);
await batchCreateChunksAndEmbeddings({
store,
body: {
url: body.url,
user: body.user,
type: "image",
description:
imageDescriptions.length > 1
? `A group of ${imageDescriptions.length} images on ${body.url}`
: imageDescriptions[0],
spaces: body.spaces,
pageContent: imageDescriptions.join("\n"),
title: "Image content from the web",
},
chunks: [
imageDescriptions,
...(body.text ? chunkText(body.text, 1536) : []),
].flat(),
context: c,
});
return c.json({ status: "ok" });
},
);
app.get(
"/api/ask",
zValidator(
@ -74,6 +170,159 @@ app.get(
},
);
// This is a special endpoint for our "chatbot-only" solutions.
// It does both - adding content AND chatting with it.
app.post(
"/api/autoChatOrAdd",
zValidator(
"query",
z.object({
query: z.string(),
user: z.string(),
}),
),
zValidator("json", chatObj),
async (c) => {
const { query, user } = c.req.valid("query");
const { chatHistory } = c.req.valid("json");
const { store, model } = await initQuery(c);
let task: "add" | "chat" = "chat";
let thingToAdd: "page" | "image" | "text" | undefined = undefined;
let addContent: string | undefined = undefined;
// This is a "router". this finds out if the user wants to add a document, or chat with the AI to get a response.
const routerQuery = await generateText({
model: model,
system: `You are Supermemory chatbot. You can either add a document to the supermemory database, or return a chat response. Based on this query,
You must determine what to do. Basically if it feels like a "question", then you should intiate a chat. If it feels like a "command" or feels like something that could be forwarded to the AI, then you should add a document.
You must also extract the "thing" to add and what type of thing it is.`,
prompt: `Question from user: ${query}`,
tools: {
decideTask: tool({
description:
"Decide if the user wants to add a document or chat with the AI",
parameters: z.object({
generatedTask: z.enum(["add", "chat"]),
contentToAdd: z.object({
thing: z.enum(["page", "image", "text"]),
content: z.string(),
}),
}),
execute: async ({ generatedTask, contentToAdd }) => {
task = generatedTask;
thingToAdd = contentToAdd.thing;
addContent = contentToAdd.content;
},
}),
},
});
if ((task as string) === "add") {
// addString is the plaintext string that the user wants to add to the database
let addString: string = addContent;
if (thingToAdd === "page") {
// TODO: Sometimes this query hangs, and errors out. we need to do proper error management here.
const response = await fetch("https://md.dhr.wtf/?url=" + addContent, {
headers: {
Authorization: "Bearer " + c.env.SECURITY_KEY,
},
});
addString = await response.text();
}
// At this point, we can just go ahead and create the embeddings!
await batchCreateChunksAndEmbeddings({
store,
body: {
url: addContent,
user,
type: thingToAdd,
pageContent: addString,
title: `${addString.slice(0, 30)}... (Added from chatbot)`,
},
chunks: chunkText(addString, 1536),
context: c,
});
return c.json({
status: "ok",
response:
"I added the document to your personal second brain! You can now use it to answer questions or chat with me.",
contentAdded: {
type: thingToAdd,
content: addString,
url:
thingToAdd === "page"
? addContent
: `https://supermemory.ai/note/${Date.now()}`,
},
});
} else {
const filter: VectorizeVectorMetadataFilter = {
[`user-${user}`]: 1,
};
const queryAsVector = await store.embeddings.embedQuery(query);
const resp = await c.env.VECTORIZE_INDEX.query(queryAsVector, {
topK: 5,
filter,
returnMetadata: true,
});
const minScore = Math.min(...resp.matches.map(({ score }) => score));
const maxScore = Math.max(...resp.matches.map(({ score }) => score));
// This entire chat part is basically just a dumb down version of the /api/chat endpoint.
const normalizedData = resp.matches.map((data) => ({
...data,
normalizedScore:
maxScore !== minScore
? 1 + ((data.score - minScore) / (maxScore - minScore)) * 98
: 50,
}));
const preparedContext = normalizedData.map(
({ metadata, score, normalizedScore }) => ({
context: `Website title: ${metadata!.title}\nDescription: ${metadata!.description}\nURL: ${metadata!.url}\nContent: ${metadata!.text}`,
score,
normalizedScore,
}),
);
const prompt = template({
contexts: preparedContext,
question: query,
});
const initialMessages: CoreMessage[] = [
{
role: "system",
content: `You are an AI chatbot called "Supermemory.ai". When asked a question by a user, you must take all the context provided to you and give a good, small, but helpful response.`,
},
{ role: "assistant", content: "Hello, how can I help?" },
];
const userMessage: CoreMessage = { role: "user", content: prompt };
const response = await generateText({
model,
messages: [
...initialMessages,
...((chatHistory || []) as CoreMessage[]),
userMessage,
],
});
return c.json({ status: "ok", response: response.text });
}
},
);
/* TODO: Eventually, we should not have to save each user's content in a seperate vector.
Lowkey, it makes sense. The user may save their own version of a page - like selected text from twitter.com url.
But, it's not scalable *enough*. How can we store the same vectors for the same content, without needing to duplicate for each uer?
@ -85,8 +334,8 @@ app.post(
"query",
z.object({
query: z.string(),
topK: z.number().optional().default(10),
user: z.string(),
topK: z.number().optional().default(10),
spaces: z.string().optional(),
sourcesOnly: z.string().optional().default("false"),
model: z.string().optional().default("gpt-4o"),
@ -97,105 +346,107 @@ app.post(
const query = c.req.valid("query");
const body = c.req.valid("json");
if (body.chatHistory) {
body.chatHistory = body.chatHistory.map((i) => ({
...i,
content: i.parts.length > 0 ? i.parts.join(" ") : i.content,
}));
const sourcesOnly = query.sourcesOnly === "true";
// Return early for dumb requests
if (sourcesOnly && body.sources) {
return c.json(body.sources);
}
const sourcesOnly = query.sourcesOnly === "true";
const spaces = query.spaces?.split(",") || [undefined];
const spaces = query.spaces?.split(",") ?? [undefined];
// Get the AI model maker and vector store
const { model, store } = await initQuery(c, query.model);
const filter: VectorizeVectorMetadataFilter = { user: query.user };
if (!body.sources) {
const filter: VectorizeVectorMetadataFilter = {
[`user-${query.user}`]: 1,
};
console.log("Spaces", spaces);
// Converting the query to a vector so that we can search for similar vectors
const queryAsVector = await store.embeddings.embedQuery(query.query);
const responses: VectorizeMatches = { matches: [], count: 0 };
// Converting the query to a vector so that we can search for similar vectors
const queryAsVector = await store.embeddings.embedQuery(query.query);
const responses: VectorizeMatches = { matches: [], count: 0 };
// SLICED to 5 to avoid too many queries
for (const space of spaces.slice(0, 5)) {
if (space !== undefined) {
// it's possible for space list to be [undefined] so we only add space filter conditionally
filter.space = space;
console.log("hello world", spaces);
// SLICED to 5 to avoid too many queries
for (const space of spaces.slice(0, 5)) {
console.log("space", space);
if (!space && spaces.length > 1) {
// it's possible for space list to be [undefined] so we only add space filter conditionally
filter[`space-${query.user}-${space}`] = 1;
}
// Because there's no OR operator in the filter, we have to make multiple queries
const resp = await c.env.VECTORIZE_INDEX.query(queryAsVector, {
topK: query.topK,
filter,
returnMetadata: true,
});
// Basically recreating the response object
if (resp.count > 0) {
responses.matches.push(...resp.matches);
responses.count += resp.count;
}
}
// Because there's no OR operator in the filter, we have to make multiple queries
const resp = await c.env.VECTORIZE_INDEX.query(queryAsVector, {
topK: query.topK,
filter,
returnMetadata: true,
});
const minScore = Math.min(...responses.matches.map(({ score }) => score));
const maxScore = Math.max(...responses.matches.map(({ score }) => score));
// Basically recreating the response object
if (resp.count > 0) {
responses.matches.push(...resp.matches);
responses.count += resp.count;
// We are "normalising" the scores - if all of them are on top, we want to make sure that
// we have a way to filter out the noise.
const normalizedData = responses.matches.map((data) => ({
...data,
normalizedScore:
maxScore !== minScore
? 1 + ((data.score - minScore) / (maxScore - minScore)) * 98
: 50, // If all scores are the same, set them to the middle of the scale
}));
let highScoreData = normalizedData.filter(
({ normalizedScore }) => normalizedScore > 50,
);
// If the normalsation is not done properly, we have a fallback to just get the
// top 3 scores
if (highScoreData.length === 0) {
highScoreData = normalizedData
.sort((a, b) => b.score - a.score)
.slice(0, 3);
}
const sortedHighScoreData = highScoreData.sort(
(a, b) => b.normalizedScore - a.normalizedScore,
);
body.sources = {
normalizedData,
};
// So this is kinda hacky, but the frontend needs to do 2 calls to get sources and chat.
// I think this is fine for now, but we can improve this later.
if (sourcesOnly) {
const idsAsStrings = sortedHighScoreData.map((dataPoint) =>
dataPoint.id.toString(),
);
const storedContent = await Promise.all(
idsAsStrings.map(async (id) => await c.env.KV.get(id)),
);
const metadata = normalizedData.map((datapoint) => datapoint.metadata);
return c.json({ ids: storedContent, metadata, normalizedData });
}
}
const minScore = Math.min(...responses.matches.map(({ score }) => score));
const maxScore = Math.max(...responses.matches.map(({ score }) => score));
// We are "normalising" the scores - if all of them are on top, we want to make sure that
// we have a way to filter out the noise.
const normalizedData = responses.matches.map((data) => ({
...data,
normalizedScore:
maxScore !== minScore
? 1 + ((data.score - minScore) / (maxScore - minScore)) * 98
: 50, // If all scores are the same, set them to the middle of the scale
}));
let highScoreData = normalizedData.filter(
({ normalizedScore }) => normalizedScore > 50,
);
// If the normalsation is not done properly, we have a fallback to just get the
// top 3 scores
if (highScoreData.length === 0) {
highScoreData = normalizedData
.sort((a, b) => b.score - a.score)
.slice(0, 3);
}
const sortedHighScoreData = highScoreData.sort(
(a, b) => b.normalizedScore - a.normalizedScore,
);
// So this is kinda hacky, but the frontend needs to do 2 calls to get sources and chat.
// I think this is fine for now, but we can improve this later.
if (sourcesOnly) {
const idsAsStrings = sortedHighScoreData.map((dataPoint) =>
dataPoint.id.toString(),
);
// We are getting the content ID back, so that the frontend can show the actual sources properly.
// it IS a lot of DB calls, i completely agree.
// TODO: return metadata value here, so that the frontend doesn't have to re-fetch anything.
const storedContent = await Promise.all(
idsAsStrings.map(async (id) => await c.env.KV.get(id)),
);
return c.json({ ids: storedContent });
}
const vec = responses.matches.map((data) => ({ metadata: data.metadata }));
const vecWithScores = vec.map((v, i) => ({
...v,
score: sortedHighScoreData[i].score,
normalisedScore: sortedHighScoreData[i].normalizedScore,
}));
const preparedContext = vecWithScores.map(
({ metadata, score, normalisedScore }) => ({
const preparedContext = body.sources.normalizedData.map(
({ metadata, score, normalizedScore }) => ({
context: `Website title: ${metadata!.title}\nDescription: ${metadata!.description}\nURL: ${metadata!.url}\nContent: ${metadata!.text}`,
score,
normalisedScore,
normalizedScore,
}),
);
@ -245,4 +496,28 @@ app.delete(
},
);
// ERROR #1 - this is the api that the editor uses, it is just a scrape off of /api/chat so you may check that out
app.get(
"/api/editorai",
zValidator(
"query",
z.object({
context: z.string(),
request: z.string(),
}),
),
async (c) => {
const { context, request } = c.req.valid("query");
const { model } = await initQuery(c);
const response = await streamText({
model,
prompt: `${request}-${context}`,
maxTokens: 224,
});
return response.toTextStreamResponse();
},
);
export default app;

8
apps/cf-ai-backend/src/prompts/prompt1.ts
View file

@ -6,28 +6,24 @@ To generate your answer:
- Carefully analyze the question and identify the key information needed to address it
- Locate the specific parts of each context that contain this key information
- Compare the relevance scores of the provided contexts
- In the <justification> tags, provide a brief justification for which context(s) are more relevant to answering the question based on the scores
- Concisely summarize the relevant information from the higher-scoring context(s) in your own words
- Provide a direct answer to the question
- Use markdown formatting in your answer, including bold, italics, and bullet points as appropriate to improve readability and highlight key points
- Give detailed and accurate responses for things like 'write a blog' or long-form questions.
- The normalisedScore is a value in which the scores are 'balanced' to give a better representation of the relevance of the context, between 1 and 100, out of the top 10 results
Provide your justification between <justification> tags and your final answer between <answer> tags, formatting both in markdown.
- provide your justification in the end, in a <justification> </justification> tag
If no context is provided, introduce yourself and explain that the user can save content which will allow you to answer questions about that content in the future. Do not provide an answer if no context is provided.`;
export const template = ({ contexts, question }) => {
// Map over contexts to generate the context and score parts
const contextParts = contexts
.map(
({ context, score, normalisedScore }) => `
({ context, normalisedScore }) => `
<context>
${context}
</context>
<context_score>
score: ${score}
normalisedScore: ${normalisedScore}
</context_score>`,
)

7
apps/cf-ai-backend/src/types.ts
View file

@ -1,8 +1,9 @@
import { sourcesZod } from "@repo/shared-types";
import { z } from "zod";
export type Env = {
VECTORIZE_INDEX: VectorizeIndex;
AI: Fetcher;
AI: Ai;
SECURITY_KEY: string;
OPENAI_API_KEY: string;
GOOGLE_AI_API_KEY: string;
@ -37,13 +38,15 @@ export const contentObj = z.object({
export const chatObj = z.object({
chatHistory: z.array(contentObj).optional(),
sources: sourcesZod.optional(),
});
export const vectorObj = z.object({
pageContent: z.string(),
title: z.string().optional(),
description: z.string().optional(),
space: z.string().optional(),
spaces: z.array(z.string()).optional(),
url: z.string(),
user: z.string(),
type: z.string().optional().default("page"),
});

24
apps/cf-ai-backend/src/utils/OpenAIEmbedder.ts
View file

@ -1,3 +1,5 @@
import { z } from "zod";
interface OpenAIEmbeddingsParams {
apiKey: string;
modelName: string;
@ -32,12 +34,22 @@ export class OpenAIEmbeddings {
}),
});
const data = (await response.json()) as {
data: {
embedding: number[];
}[];
};
const data = await response.json();
return data.data[0].embedding;
const zodTypeExpected = z.object({
data: z.array(
z.object({
embedding: z.array(z.number()),
}),
),
});
const json = zodTypeExpected.safeParse(data);
if (!json.success) {
throw new Error("Invalid response from OpenAI: " + json.error.message);
}
return json.data.data[0].embedding;
}
}

3
apps/cf-ai-backend/src/utils/chonker.ts
View file

@ -1,5 +1,8 @@
import nlp from "compromise";
/**
* Split text into chunks of specified max size with some overlap for continuity.
*/
export default function chunkText(
text: string,
maxChunkSize: number,

7
apps/cf-ai-backend/src/utils/seededRandom.ts
View file

@ -1,5 +1,9 @@
import { MersenneTwister19937, integer } from "random-js";
/**
* Hashes a string to a 32-bit integer.
* @param {string} seed - The input string to hash.
*/
function hashString(seed: string) {
let hash = 0;
for (let i = 0; i < seed.length; i++) {
@ -10,6 +14,9 @@ function hashString(seed: string) {
return hash;
}
/**
* returns a funtion that generates same sequence of random numbers for a given seed between 0 and 1.
*/
export function seededRandom(seed: string) {
const seedHash = hashString(seed);
const engine = MersenneTwister19937.seed(seedHash);

3
apps/cf-ai-backend/tsconfig.json
View file

@ -1,6 +1,7 @@
{
"compilerOptions": {
"lib": ["ES2020"],
"types": ["@cloudflare/workers-types"]
"types": ["@cloudflare/workers-types"],
"downlevelIteration": true
}
}

7
apps/cf-ai-backend/wrangler.toml
View file

@ -3,9 +3,14 @@ main = "src/index.ts"
compatibility_date = "2024-02-23"
node_compat = true
# [env.preview]
[[vectorize]]
binding = "VECTORIZE_INDEX"
index_name = "supermem-vector"
index_name = "supermem-vector-dev"
# [[vectorize]]
# binding = "VECTORIZE_INDEX"
# index_name = "supermem-vector-prod"
[ai]
binding = "AI"

37
apps/extension/README.md Normal file
View file

@ -0,0 +1,37 @@
# [projectName]
> This project was bootstrapped using the Extension.js React-TypeScript template.
## Scripts Available
In the project directory, you can run:
### [projectPackageManager] dev
```
// Runs the app in the development mode.
// Will open a new browser instance with your extension loaded.
// The page will reload when you make changes.
[projectPackageManager] dev
```
### [projectPackageManager] start
```
// Runs the app in the production mode.
// Will open a new browser instance with your extension loaded.
// This is how your browser extension will work once published.
[projectPackageManager] start
```
### [projectPackageManager] build
```
// Builds the app for production.
// Bundles your browser extension in production mode for the target browser.
[projectPackageManager] run build
```
## Learn More
You can learn more in the [Extension.js](https://extension.js.org) documentation.

25
apps/extension/background.ts Normal file
View file

@ -0,0 +1,25 @@
chrome.runtime.onInstalled.addListener(function () {
let context = 'selection';
let title = "Supermemory - Save Highlight";
chrome.contextMenus.create({
title: title,
contexts: ['selection'],
id: context,
});
});
chrome.contextMenus.onClicked.addListener(function (info, tab) {
if (info.menuItemId === 'selection') {
// you can add a link to a cf worker or whatever u want
// fetch("", {
// method: "POST",
// headers: { "Content-Type": "application/json" },
// body: JSON.stringify({
// data: info.selectionText,
// }),
// });
//so you first save it and then send the reponse to the screen
chrome.tabs.sendMessage(tab?.id || 1, info.selectionText);
}
});

45
apps/extension/content/ContentApp.tsx Normal file
View file

@ -0,0 +1,45 @@
import React, { useEffect } from "react";
export default function ContentApp() {
const [text, setText] = React.useState("");
const [hover, setHover] = React.useState(false);
useEffect(() => {
const messageListener = (message: any) => {
setText(message);
setTimeout(() => setText(""), 2000);
};
chrome.runtime.onMessage.addListener(messageListener);
document.addEventListener('mousemove', (e)=> {
const percentageX = (e.clientX / window.innerWidth) * 100;
const percentageY = (e.clientY / window.innerHeight) * 100;
if (percentageX > 75 && percentageY > 75){
setHover(true)
} else {
setHover(false)
}
})
return () => {
chrome.runtime.onMessage.removeListener(messageListener);
};
}, []);
return (
<div className="pointer-events-none flex justify-end items-end h-screen w-full absolute z-99999">
<div className="h-[30vh] bg-red-500 absolute flex justify-end items-center">
<div
className={`${hover && "opacity-100 "} transition bg-red-600 opacity-0 h-12 w-12 `}
></div>
</div>
<div
className={`mx-4 my-2 flex flex-col gap-3 rounded-3xl bg-gray-900 text-xl py-4 px-6 overflow-hidden min-w-[20vw] min-h-24 max-w-96 max-h-40 ${text ? "translate-y-0 opacity-100" : "translate-y-[15%] opacity-0"} transition`}
>
<h2 className="text-2xl font-extrabold text-white">Saved!</h2>
<h2 className="text-lg font-medium text-white">{text}</h2>
</div>
</div>
);
}

3
apps/extension/content/base.css Normal file
View file

@ -0,0 +1,3 @@
@tailwind base;
@tailwind components;
@tailwind utilities;

6
apps/extension/content/content.css Normal file
View file

@ -0,0 +1,6 @@
#extension-root {
position: fixed;
bottom: 0;
right: 0;
z-index: 99999;
}

15
apps/extension/content/content.tsx Normal file
View file

@ -0,0 +1,15 @@
import ReactDOM from 'react-dom/client'
import ContentApp from './ContentApp'
import('./base.css')
import('./content.css')
setTimeout(initial, 4000)
function initial() {
const rootDiv = document.createElement('div')
rootDiv.id = 'extension-root'
document.body.appendChild(rootDiv)
const root = ReactDOM.createRoot(rootDiv)
root.render(<><ContentApp /></>)
}

9
apps/extension/extension-env.d.ts vendored Normal file
View file

@ -0,0 +1,9 @@
// Required Extension.js types for TypeScript projects.
// This file is auto-generated and should not be excluded.
// If you need extra types, consider creating a new *.d.ts and
// referencing it in the "include" array in your tsconfig.json file.
// See https://www.typescriptlang.org/tsconfig#include for info.
/// <reference types="@extension-create/develop/dist/types/index.d.ts" />
// Polyfill types for browser.* APIs.
/// <reference types="@extension-create/develop/dist/types/polyfill.d.ts" />

11
apps/extension/index.html Normal file
View file

@ -0,0 +1,11 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>God Bless Vanilla JavaScript!!!</title>
</head>
<body>
<h1>hello World! Follow <a href="https://x.com/supermemoryai">@supermemoryai</a></h1>
</body>
</html>

30
apps/extension/manifest.json Normal file
View file

@ -0,0 +1,30 @@
{
"name": "Supermemory.ai-Extension",
"description": "Uses the chrome.contextMenus API to customize the context menu.",
"version": "0.1",
"permissions": [
"contextMenus"
],
"manifest_version": 3,
"action": {
"default_popup": "index.html"
},
"background": {
"service_worker": "./background.ts"
},
"content_scripts": [
{
"matches": [
"<all_urls>"
],
"js": [
"./content/content.tsx"
]
}
],
"icons": {
"16": "public/icon/logo(16).png",
"48": "public/icon/logo(48).png"
}
}

20
apps/extension/package.json Normal file
View file

@ -0,0 +1,20 @@
{
"devDependencies": {
"@types/react": "^18.0.9",
"@types/react-dom": "^18.0.5",
"react": "^18.1.0",
"react-dom": "^18.1.0",
"tailwindcss": "^3.4.1",
"typescript": "5.3.3",
"extension": "latest"
},
"scripts": {
"dev": "extension dev",
"start": "extension start",
"build": "extension build"
},
"dependencies": {},
"name": "extension",
"private": true,
"version": "0.0.0"
}

BIN
apps/extension/public/icon/logo(16).png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 980 B

BIN
apps/extension/public/icon/logo(48).png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 2 KiB

8
apps/extension/tailwind.config.js Normal file
View file

@ -0,0 +1,8 @@
module.exports = {
content: ['**/*.html', '**/*.tsx'],
theme: {
extend: {}
},
plugins: []
}
module.exports = require("@repo/tailwind-config/tailwind.config");

19
apps/extension/tsconfig.json Normal file
View file

@ -0,0 +1,19 @@
{
"extends": "@repo/typescript-config/nextjs.json",
"compilerOptions": {
"allowJs": true,
"allowSyntheticDefaultImports": true,
"esModuleInterop": true,
"forceConsistentCasingInFileNames": true,
"isolatedModules": false,
"jsx": "react-jsx",
"lib": ["dom", "dom.iterable", "esnext"],
"moduleResolution": "node",
"module": "esnext",
"resolveJsonModule": true,
"strict": true,
"target": "esnext"
},
"include": ["./"],
"exclude": ["node_modules", "dist"]
}

2
apps/web/app/(auth)/auth-buttons.tsx
View file

@ -2,7 +2,7 @@
import { Button } from "@repo/ui/shadcn/button";
import React from "react";
import { signIn } from "../helpers/server/auth";
import { signIn } from "../../server/auth";
function SignIn() {
return (

129
apps/web/app/(canvas)/canvas/[id]/page.tsx Normal file
View file

@ -0,0 +1,129 @@
"use client";
import { Canvas } from "@repo/ui/components/canvas/components/canvas";
import React, { useState } from "react";
import { Panel, PanelGroup, PanelResizeHandle } from "react-resizable-panels";
import { SettingsIcon, DragIcon } from "@repo/ui/icons";
import DraggableComponentsContainer from "@repo/ui/components/canvas/components/draggableComponent";
import { AutocompleteIcon, blockIcon } from "@repo/ui/icons";
import Image from "next/image";
import { Switch } from "@repo/ui/shadcn/switch";
import { Label } from "@repo/ui/shadcn/label";
import { useRouter } from "next/router";
function page() {
const [fullScreen, setFullScreen] = useState(false);
const [visible, setVisible] = useState(true);
const router = useRouter();
router.push("/home");
return (
<div
className={`h-screen w-full ${!fullScreen ? "px-4 py-6" : "bg-[#1F2428]"} transition-all`}
>
<div>
<PanelGroup
onLayout={(l) => {
l[0]! < 20 ? setVisible(false) : setVisible(true);
}}
className={` ${fullScreen ? "w-[calc(100vw-2rem)]" : "w-screen"} transition-all`}
direction="horizontal"
>
<Panel
onExpand={() => {
setTimeout(() => setFullScreen(false), 50);
}}
onCollapse={() => {
setTimeout(() => setFullScreen(true), 50);
}}
defaultSize={30}
collapsible={true}
>
<div
className={`flex transition-all rounded-2xl ${fullScreen ? "h-screen" : "h-[calc(100vh-3rem)]"} w-full flex-col overflow-hidden bg-[#1F2428]`}
>
<div className="flex items-center justify-between bg-[#2C3439] px-4 py-2 text-lg font-medium text-[#989EA4]">
Change Filters
<Image src={SettingsIcon} alt="setting-icon" />
</div>
{visible ? (
<SidePanel />
) : (
<h1 className="text-center py-10 text-xl">
Need more space to show!
</h1>
)}
</div>
</Panel>
<PanelResizeHandle
className={`relative flex items-center transition-all justify-center ${!fullScreen && "px-1"}`}
>
<div
className={`rounded-lg bg-[#2F363B] ${!fullScreen && "px-1"} transition-all py-2`}
>
<Image src={DragIcon} alt="drag-icon" />
</div>
</PanelResizeHandle>
<Panel className="relative" defaultSize={70} minSize={60}>
<div
className={`absolute overflow-hidden transition-all inset-0 ${fullScreen ? "h-screen " : "h-[calc(100vh-3rem)] rounded-2xl"} w-full`}
>
<Canvas />
</div>
</Panel>
</PanelGroup>
</div>
</div>
);
}
function SidePanel() {
const [value, setValue] = useState("");
const [dragAsText, setDragAsText] = useState(false);
return (
<>
<div className="px-3 py-5">
<input
placeholder="search..."
onChange={(e) => {
setValue(e.target.value);
}}
value={value}
// rows={1}
className="w-full resize-none rounded-xl bg-[#151515] px-3 py-4 text-xl text-[#989EA4] outline-none focus:outline-none sm:max-h-52"
/>
</div>
<div className="flex items-center justify-end px-3 py-4">
<Switch
className="bg-[#151515] data-[state=unchecked]:bg-red-400 data-[state=checked]:bg-blue-400"
onCheckedChange={(e) => setDragAsText(e)}
id="drag-text-mode"
/>
<Label htmlFor="drag-text-mode">Drag as Text</Label>
</div>
<DraggableComponentsContainer content={content} />
</>
);
}
export default page;
const content = [
{
content:
"Regional growth patterns diverge, with strong performance in the United States and several emerging markets, contrasted by weaker prospects in many advanced economies, particularly in Europe (World Economic Forum) (OECD). The rapid adoption of artificial intelligence (AI) is expected to drive productivity growth, especially in advanced economies, potentially mitigating labor shortages and boosting income levels in emerging markets (World Economic Forum) (OECD). However, ongoing geopolitical tensions and economic fragmentation are likely to maintain a level of uncertainty and volatility in the global economy (World Economic Forum.",
icon: AutocompleteIcon,
iconAlt: "Autocomplete",
extraInfo:
"Page Url: https://chatgpt.com/c/762cd44e-1752-495b-967a-aa3c23c6024a",
},
{
content:
"As of mid-2024, the global economy is experiencing modest growth with significant regional disparities. Global GDP growth is projected to be around 3.1% in 2024, rising slightly to 3.2% in 2025. This performance, although below the pre-pandemic average, reflects resilience despite various economic pressures, including tight monetary conditions and geopolitical tensions (IMF)(OECD) Inflation is moderating faster than expected, with global headline inflation projected to fall to 5.8% in 2024 and 4.4% in 2025, contributing to improving real incomes and positive trade growth (IMF) (OECD)",
icon: blockIcon,
iconAlt: "Autocomplete",
extraInfo:
"Page Url: https://www.cnbc.com/2024/05/23/nvidia-keeps-hitting-records-can-investors-still-buy-the-stock.html?&qsearchterm=nvidia",
},
];

13
apps/web/app/(canvas)/canvas/layout.tsx Normal file
View file

@ -0,0 +1,13 @@
import "../canvasStyles.css";
export default function RootLayout({
children,
}: {
children: React.ReactNode;
}) {
return (
<div lang="en" className="bg-[#151515]">
<div>{children}</div>
</div>
);
}

9
apps/web/app/(canvas)/canvas/page.tsx Normal file
View file

@ -0,0 +1,9 @@
import { redirect } from "next/navigation";
import React from "react";
function page() {
redirect("/signin");
return <div>page</div>;
}
export default page;

28
apps/web/app/(canvas)/canvasStyles.css Normal file
View file

@ -0,0 +1,28 @@
.tl-background {
background: #1F2428 !important;
}
.tlui-style-panel.tlui-style-panel__wrapper, .tlui-navigation-panel::before ,.tlui-menu-zone, .tlui-toolbar__tools, .tlui-popover__content, .tlui-menu, .tlui-button__help, .tlui-help-menu, .tlui-dialog__content {
background: #2C3439 !important;
border-top: #2C3439 !important;
border-right: #2C3439 !important;
border-bottom: #2C3439 !important;
border-left: #2C3439 !important;
}
.tlui-navigation-panel::before {
border-top: #2C3439 !important;
border-right: #2C3439 !important;
}
.tlui-minimap {
background: #2C3439 !important;
}
.tlui-minimap__canvas {
background: #1F2428 !important;
}
.tlui-dialog__overlay {
position: fixed;
}

48
apps/web/app/(dash)/actions.ts
View file

@ -1,48 +0,0 @@
"use server";
import { cookies, headers } from "next/headers";
import { db } from "../helpers/server/db";
import { sessions, users, space } from "../helpers/server/db/schema";
import { eq } from "drizzle-orm";
import { redirect } from "next/navigation";
export async function ensureAuth() {
const token =
cookies().get("next-auth.session-token")?.value ??
cookies().get("__Secure-authjs.session-token")?.value ??
cookies().get("authjs.session-token")?.value ??
headers().get("Authorization")?.replace("Bearer ", "");
if (!token) {
return undefined;
}
const sessionData = await db
.select()
.from(sessions)
.innerJoin(users, eq(users.id, sessions.userId))
.where(eq(sessions.sessionToken, token));
if (!sessionData || sessionData.length < 0) {
return undefined;
}
return {
user: sessionData[0]!.user,
session: sessionData[0]!,
};
}
export async function getSpaces() {
const data = await ensureAuth();
if (!data) {
redirect("/signin");
}
const sp = await db
.select()
.from(space)
.where(eq(space.user, data.user.email));
return sp;
}

90
apps/web/app/(dash)/chat/CodeBlock.tsx Normal file
View file

@ -0,0 +1,90 @@
import React, { useRef, useState } from "react";
const CodeBlock = ({
lang,
codeChildren,
}: {
lang: string;
codeChildren: React.ReactNode & React.ReactNode[];
}) => {
const codeRef = useRef<HTMLElement>(null);
return (
<div className="bg-black rounded-md">
<CodeBar lang={lang} codeRef={codeRef} />
<div className="p-4 overflow-y-auto">
<code ref={codeRef} className={`!whitespace-pre hljs language-${lang}`}>
{codeChildren}
</code>
</div>
</div>
);
};
const CodeBar = React.memo(
({
lang,
codeRef,
}: {
lang: string;
codeRef: React.RefObject<HTMLElement>;
}) => {
const [isCopied, setIsCopied] = useState<boolean>(false);
return (
<div className="flex items-center relative text-gray-200 bg-gray-800 px-4 py-2 text-xs font-sans">
<span className="">{lang}</span>
<button
className="flex ml-auto gap-2"
aria-label="copy codeblock"
onClick={async () => {
const codeString = codeRef.current?.textContent;
if (codeString)
navigator.clipboard.writeText(codeString).then(() => {
setIsCopied(true);
setTimeout(() => setIsCopied(false), 3000);
});
}}
>
{isCopied ? (
<>
<svg
xmlns="http://www.w3.org/2000/svg"
fill="none"
viewBox="0 0 24 24"
strokeWidth={1.5}
stroke="currentColor"
className="size-4"
>
<path
strokeLinecap="round"
strokeLinejoin="round"
d="M11.35 3.836c-.065.21-.1.433-.1.664 0 .414.336.75.75.75h4.5a.75.75 0 0 0 .75-.75 2.25 2.25 0 0 0-.1-.664m-5.8 0A2.251 2.251 0 0 1 13.5 2.25H15c1.012 0 1.867.668 2.15 1.586m-5.8 0c-.376.023-.75.05-1.124.08C9.095 4.01 8.25 4.973 8.25 6.108V8.25m8.9-4.414c.376.023.75.05 1.124.08 1.131.094 1.976 1.057 1.976 2.192V16.5A2.25 2.25 0 0 1 18 18.75h-2.25m-7.5-10.5H4.875c-.621 0-1.125.504-1.125 1.125v11.25c0 .621.504 1.125 1.125 1.125h9.75c.621 0 1.125-.504 1.125-1.125V18.75m-7.5-10.5h6.375c.621 0 1.125.504 1.125 1.125v9.375m-8.25-3 1.5 1.5 3-3.75"
/>
</svg>
Copied!
</>
) : (
<>
<svg
xmlns="http://www.w3.org/2000/svg"
fill="none"
viewBox="0 0 24 24"
strokeWidth={1.5}
stroke="currentColor"
className="size-4"
>
<path
strokeLinecap="round"
strokeLinejoin="round"
d="M15.75 17.25v3.375c0 .621-.504 1.125-1.125 1.125h-9.75a1.125 1.125 0 0 1-1.125-1.125V7.875c0-.621.504-1.125 1.125-1.125H6.75a9.06 9.06 0 0 1 1.5.124m7.5 10.376h3.375c.621 0 1.125-.504 1.125-1.125V11.25c0-4.46-3.243-8.161-7.5-8.876a9.06 9.06 0 0 0-1.5-.124H9.375c-.621 0-1.125.504-1.125 1.125v3.5m7.5 10.375H9.375a1.125 1.125 0 0 1-1.125-1.125v-9.25m12 6.625v-1.875a3.375 3.375 0 0 0-3.375-3.375h-1.5a1.125 1.125 0 0 1-1.125-1.125v-1.5a3.375 3.375 0 0 0-3.375-3.375H9.75"
/>
</svg>
Copy code
</>
)}
</button>
</div>
);
},
);
export default CodeBlock;

38
apps/web/app/(dash)/chat/[chatid]/page.tsx Normal file
View file

@ -0,0 +1,38 @@
import { getFullChatThread } from "@/app/actions/fetchers";
import { chatSearchParamsCache } from "@/lib/searchParams";
import ChatWindow from "../chatWindow";
async function Page({
params,
searchParams,
}: {
params: { chatid: string };
searchParams: Record<string, string | string[] | undefined>;
}) {
const { firstTime, q, spaces } = chatSearchParamsCache.parse(searchParams);
let chat: Awaited<ReturnType<typeof getFullChatThread>>;
try {
chat = await getFullChatThread(params.chatid);
} catch (e) {
const error = e as Error;
return <div>This page errored out: {error.message}</div>;
}
if (!chat.success || !chat.data) {
console.error(chat.error);
return <div>Chat not found. Check the console for more details.</div>;
}
return (
<ChatWindow
q={q}
spaces={spaces}
initialChat={chat.data.length > 0 ? chat.data : undefined}
threadId={params.chatid}
/>
);
}
export default Page;

1
apps/web/app/(dash)/chat/actions.ts
View file

@ -1 +0,0 @@
"use server";

425
apps/web/app/(dash)/chat/chatWindow.tsx
View file

@ -1,51 +1,438 @@
"use client";
import { AnimatePresence } from "framer-motion";
import React, { useEffect, useState } from "react";
import React, { useEffect, useRef, useState } from "react";
import QueryInput from "../home/queryinput";
import { cn } from "@repo/ui/lib/utils";
import { motion } from "framer-motion";
import { useRouter } from "next/navigation";
import { ChatHistory, sourcesZod } from "@repo/shared-types";
import {
Accordion,
AccordionContent,
AccordionItem,
AccordionTrigger,
} from "@repo/ui/shadcn/accordion";
import Markdown from "react-markdown";
import remarkGfm from "remark-gfm";
import remarkMath from "remark-math";
import rehypeKatex from "rehype-katex";
import rehypeHighlight from "rehype-highlight";
import { code, p } from "./markdownRenderHelpers";
import { codeLanguageSubset } from "@/lib/constants";
import { toast } from "sonner";
import Link from "next/link";
import { createChatObject } from "@/app/actions/doers";
import { ClipboardIcon } from "@heroicons/react/24/outline";
import { SendIcon } from "lucide-react";
function ChatWindow({ q }: { q: string }) {
const [layout, setLayout] = useState<"chat" | "initial">("initial");
function ChatWindow({
q,
spaces,
initialChat = [
{
question: q,
answer: {
parts: [],
sources: [],
},
},
],
threadId,
}: {
q: string;
spaces: { id: string; name: string }[];
initialChat?: ChatHistory[];
threadId: string;
}) {
const [layout, setLayout] = useState<"chat" | "initial">(
initialChat.length > 1 ? "chat" : "initial",
);
const [chatHistory, setChatHistory] = useState<ChatHistory[]>(initialChat);
const removeJustificationFromText = (text: string) => {
// remove everything after the first "<justification>" word
const justificationLine = text.indexOf("<justification>");
if (justificationLine !== -1) {
// Add that justification to the last chat message
const lastChatMessage = chatHistory[chatHistory.length - 1];
if (lastChatMessage) {
lastChatMessage.answer.justification = text.slice(justificationLine);
}
return text.slice(0, justificationLine);
}
return text;
};
const router = useRouter();
const getAnswer = async (query: string, spaces: string[]) => {
const sourcesFetch = await fetch(
`/api/chat?q=${query}&spaces=${spaces}&sourcesOnly=true&threadId=${threadId}`,
{
method: "POST",
body: JSON.stringify({ chatHistory }),
},
);
// TODO: handle this properly
const sources = await sourcesFetch.json();
const sourcesParsed = sourcesZod.safeParse(sources);
if (!sourcesParsed.success) {
console.error(sourcesParsed.error);
toast.error("Something went wrong while getting the sources");
return;
}
window.scrollTo({
top: document.documentElement.scrollHeight,
behavior: "smooth",
});
const updateChatHistoryAndFetch = async () => {
// Step 1: Update chat history with the assistant's response
await new Promise((resolve) => {
setChatHistory((prevChatHistory) => {
const newChatHistory = [...prevChatHistory];
const lastAnswer = newChatHistory[newChatHistory.length - 1];
if (!lastAnswer) {
resolve(undefined);
return prevChatHistory;
}
const filteredSourceUrls = new Set(
sourcesParsed.data.metadata.map((source) => source.url),
);
const uniqueSources = sourcesParsed.data.metadata.filter((source) => {
if (filteredSourceUrls.has(source.url)) {
filteredSourceUrls.delete(source.url);
return true;
}
return false;
});
lastAnswer.answer.sources = uniqueSources.map((source) => ({
title: source.title ?? "Untitled",
type: source.type ?? "page",
source: source.url ?? "https://supermemory.ai",
content: source.description ?? "No content available",
numChunks: sourcesParsed.data.metadata.filter(
(f) => f.url === source.url,
).length,
}));
resolve(newChatHistory);
return newChatHistory;
});
});
// Step 2: Fetch data from the API
const resp = await fetch(
`/api/chat?q=${query}&spaces=${spaces}&threadId=${threadId}`,
{
method: "POST",
body: JSON.stringify({ chatHistory, sources: sourcesParsed.data }),
},
);
// Step 3: Read the response stream and update the chat history
const reader = resp.body?.getReader();
let done = false;
while (!done && reader) {
const { value, done: d } = await reader.read();
if (d) {
setChatHistory((prevChatHistory) => {
createChatObject(threadId, prevChatHistory);
return prevChatHistory;
});
}
done = d;
const txt = new TextDecoder().decode(value);
setChatHistory((prevChatHistory) => {
const newChatHistory = [...prevChatHistory];
const lastAnswer = newChatHistory[newChatHistory.length - 1];
if (!lastAnswer) return prevChatHistory;
window.scrollTo({
top: document.documentElement.scrollHeight,
behavior: "smooth",
});
lastAnswer.answer.parts.push({ text: txt });
return newChatHistory;
});
}
};
updateChatHistoryAndFetch();
};
useEffect(() => {
if (q !== "") {
setTimeout(() => {
setLayout("chat");
}, 300);
if (q.trim().length > 0 || chatHistory.length > 0) {
setLayout("chat");
const lastChat = chatHistory.length > 0 ? chatHistory.length - 1 : 0;
const startGenerating = chatHistory[lastChat]?.answer.parts[0]?.text
? false
: true;
if (startGenerating) {
getAnswer(
q,
spaces.map((s) => `${s}`),
);
}
} else {
router.push("/home");
}
}, [q]);
}, []);
return (
<div>
<div className="h-full">
<AnimatePresence mode="popLayout">
{layout === "initial" ? (
<motion.div
exit={{ opacity: 0 }}
key="initial"
className="max-w-3xl flex mx-auto w-full flex-col"
className="max-w-3xl h-full justify-center items-center flex mx-auto w-full flex-col"
>
<div className="w-full h-96">
<QueryInput initialQuery={q} initialSpaces={[]} disabled />
<QueryInput
handleSubmit={() => {}}
initialQuery={q}
initialSpaces={[]}
disabled
/>
</div>
</motion.div>
) : (
<div
className="max-w-3xl flex mx-auto w-full flex-col mt-8"
className="max-w-3xl z-10 mx-auto relative h-full overflow-y-auto no-scrollbar"
key="chat"
>
<h2
className={cn(
"transition-all transform translate-y-0 opacity-100 duration-500 ease-in-out font-semibold text-2xl",
)}
>
{q}
</h2>
<div className="w-full pt-24 mb-40">
{chatHistory.map((chat, idx) => (
<div key={idx} className="space-y-16">
<div
className={`mt-8 ${idx != chatHistory.length - 1 ? "pb-2 border-b border-b-gray-400" : ""}`}
>
<h2
className={cn(
"text-white transition-all transform translate-y-0 opacity-100 duration-500 ease-in-out font-semibold text-xl",
)}
>
{chat.question}
</h2>
<div className="flex flex-col">
{/* Related memories */}
<div
className={`space-y-4 ${chat.answer.sources.length > 0 || chat.answer.parts.length === 0 ? "flex" : "hidden"}`}
>
<Accordion
defaultValue={
idx === chatHistory.length - 1 ? "memories" : ""
}
type="single"
collapsible
>
<AccordionItem value="memories">
<AccordionTrigger className="text-foreground-menu">
Related Memories
</AccordionTrigger>
{/* TODO: fade out content on the right side, the fade goes away when the user scrolls */}
<AccordionContent
className="flex items-center no-scrollbar overflow-auto gap-4 relative max-w-3xl no-scrollbar"
defaultChecked
>
{/* Loading state */}
{chat.answer.sources.length > 0 ||
(chat.answer.parts.length === 0 && (
<>
{[1, 2, 3, 4].map((_, idx) => (
<div
key={`loadingState-${idx}`}
className="w-[350px] shrink-0 p-4 gap-2 rounded-2xl flex flex-col bg-secondary animate-pulse"
>
<div className="bg-slate-700 h-2 rounded-full w-1/2"></div>
<div className="bg-slate-700 h-2 rounded-full w-full"></div>
</div>
))}
</>
))}
{chat.answer.sources.map((source, idx) => (
<Link
href={source.source}
key={idx}
className="w-[350px] shrink-0 p-4 gap-2 rounded-2xl flex flex-col bg-secondary"
>
<div className="flex justify-between text-foreground-menu text-sm">
<span>{source.type}</span>
{source.numChunks > 1 && (
<span>{source.numChunks} chunks</span>
)}
</div>
<div className="text-base">
{source.title}
</div>
<div className="text-xs line-clamp-2">
{source.content.length > 100
? source.content.slice(0, 100) + "..."
: source.content}
</div>
</Link>
))}
</AccordionContent>
</AccordionItem>
</Accordion>
</div>
{/* Summary */}
<div>
<div className="text-foreground-menu py-2">Summary</div>
<div className="text-base">
{/* Loading state */}
{(chat.answer.parts.length === 0 ||
chat.answer.parts.join("").length === 0) && (
<div className="animate-pulse flex space-x-4">
<div className="flex-1 space-y-3 py-1">
<div className="h-2 bg-slate-700 rounded"></div>
<div className="h-2 bg-slate-700 rounded"></div>
</div>
</div>
)}
<Markdown
remarkPlugins={[remarkGfm, [remarkMath]]}
rehypePlugins={[
rehypeKatex,
[
rehypeHighlight,
{
detect: true,
ignoreMissing: true,
subset: codeLanguageSubset,
},
],
]}
components={{
code: code as any,
p: p as any,
}}
className="flex flex-col gap-2 text-base"
>
{removeJustificationFromText(
chat.answer.parts
.map((part) => part.text)
.join(""),
)}
</Markdown>
<div className="mt-3 relative -left-2 flex items-center gap-1">
{/* TODO: speak response */}
{/* <button className="group h-8 w-8 flex justify-center items-center active:scale-75 duration-200">
<SpeakerWaveIcon className="size-[18px] group-hover:text-primary" />
</button> */}
{/* copy response */}
<button
onClick={() =>
navigator.clipboard.writeText(
chat.answer.parts
.map((part) => part.text)
.join(""),
)
}
className="group h-8 w-8 flex justify-center items-center active:scale-75 duration-200"
>
<ClipboardIcon className="size-[18px] group-hover:text-primary" />
</button>
<button
onClick={async () => {
const isWebShareSupported =
navigator.share !== undefined;
if (isWebShareSupported) {
try {
await navigator.share({
title: "Your Share Title",
text: "Your share text or description",
url: "https://your-url-to-share.com",
});
} catch (e) {
console.error("Error sharing:", e);
}
} else {
console.error("web share is not supported!");
}
}}
className="group h-8 w-8 flex justify-center items-center active:scale-75 duration-200"
>
<SendIcon className="size-[18px] group-hover:text-primary" />
</button>
</div>
</div>
</div>
{/* Justification */}
{chat.answer.justification &&
chat.answer.justification.length && (
<div
className={`${chat.answer.justification && chat.answer.justification.length > 0 ? "flex" : "hidden"}`}
>
<Accordion
defaultValue={""}
type="single"
collapsible
>
<AccordionItem value="justification">
<AccordionTrigger className="text-foreground-menu">
Justification
</AccordionTrigger>
<AccordionContent
className="relative flex gap-2 max-w-3xl overflow-auto no-scrollbar"
defaultChecked
>
{chat.answer.justification.length > 0
? chat.answer.justification
.replaceAll("<justification>", "")
.replaceAll("</justification>", "")
: "No justification provided."}
</AccordionContent>
</AccordionItem>
</Accordion>
</div>
)}
</div>
</div>
</div>
))}
</div>
<div className="fixed bottom-4 w-full max-w-3xl">
<QueryInput
mini
className="w-full shadow-md"
initialQuery={""}
initialSpaces={[]}
handleSubmit={async (q, spaces) => {
setChatHistory((prevChatHistory) => {
return [
...prevChatHistory,
{
question: q,
answer: {
parts: [],
sources: [],
},
},
];
});
await getAnswer(
q,
spaces.map((s) => `${s.id}`),
);
}}
/>
</div>
</div>
)}
</AnimatePresence>

25
apps/web/app/(dash)/chat/markdownRenderHelpers.tsx Normal file
View file

@ -0,0 +1,25 @@
import { DetailedHTMLProps, HTMLAttributes, memo } from "react";
import { ExtraProps } from "react-markdown";
import CodeBlock from "./CodeBlock";
export const code = memo((props: JSX.IntrinsicElements["code"]) => {
const { className, children } = props;
const match = /language-(\w+)/.exec(className || "");
const lang = match && match[1];
return <CodeBlock lang={lang || "text"} codeChildren={children as any} />;
});
export const p = memo(
(
props?: Omit<
DetailedHTMLProps<
HTMLAttributes<HTMLParagraphElement>,
HTMLParagraphElement
>,
"ref"
>,
) => {
return <p className="whitespace-pre-wrap">{props?.children}</p>;
},
);

16
apps/web/app/(dash)/chat/page.tsx
View file

@ -1,16 +0,0 @@
import ChatWindow from "./chatWindow";
import { chatSearchParamsCache } from "../../helpers/lib/searchParams";
function Page({
searchParams,
}: {
searchParams: Record<string, string | string[] | undefined>;
}) {
const { firstTime, q, spaces } = chatSearchParamsCache.parse(searchParams);
console.log(spaces);
return <ChatWindow q={q} />;
}
export default Page;

315
apps/web/app/(dash)/dynamicisland.tsx Normal file
View file

@ -0,0 +1,315 @@
"use client";
import { AddIcon } from "@repo/ui/icons";
import Image from "next/image";
import { AnimatePresence, useMotionValueEvent, useScroll } from "framer-motion";
import { useActionState, useEffect, useRef, useState } from "react";
import { motion } from "framer-motion";
import { Label } from "@repo/ui/shadcn/label";
import { Input } from "@repo/ui/shadcn/input";
import { Textarea } from "@repo/ui/shadcn/textarea";
import { createMemory, createSpace } from "../actions/doers";
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from "@repo/ui/shadcn/select";
import { Space } from "../actions/types";
import { getSpaces } from "../actions/fetchers";
import { toast } from "sonner";
import { useFormStatus } from "react-dom";
export function DynamicIsland() {
const { scrollYProgress } = useScroll();
const [visible, setVisible] = useState(true);
useMotionValueEvent(scrollYProgress, "change", (current) => {
if (typeof current === "number") {
let direction = current! - scrollYProgress.getPrevious()!;
if (direction < 0 || direction === 1) {
setVisible(true);
} else {
setVisible(false);
}
}
});
return (
<div className="">
<AnimatePresence mode="wait">
<motion.div
initial={{
opacity: 1,
y: -150,
}}
animate={{
y: visible ? 0 : -150,
opacity: visible ? 1 : 0,
}}
transition={{
duration: 0.2,
}}
className="flex flex-col items-center"
>
<DynamicIslandContent />
</motion.div>
</AnimatePresence>
</div>
);
}
export default DynamicIsland;
function DynamicIslandContent() {
const [show, setshow] = useState(true);
function cancelfn() {
setshow(true);
}
const lastBtn = useRef<string>();
useEffect(() => {
document.addEventListener("keydown", (e) => {
if (e.key === "Escape") {
setshow(true);
}
if (e.key === "a" && lastBtn.current === "Alt") {
setshow(false);
}
lastBtn.current = e.key;
});
}, []);
return (
<>
{show ? (
<button
onClick={() => setshow(!show)}
className="bg-secondary p-2 text-[#989EA4] rounded-full flex items-center justify-between gap-2 px-4 h-10 pr-5 z-[999] shadow-md"
>
<Image src={AddIcon} alt="add icon" />
Add content
</button>
) : (
<ToolBar cancelfn={cancelfn} />
)}
</>
);
}
const fakeitems = ["page", "spaces"];
function ToolBar({ cancelfn }: { cancelfn: () => void }) {
const [spaces, setSpaces] = useState<Space[]>([]);
const [index, setIndex] = useState(0);
useEffect(() => {
(async () => {
let spaces = await getSpaces();
if (!spaces.success || !spaces.data) {
toast.warning("Unable to get spaces", {
richColors: true,
});
setSpaces([]);
return;
}
setSpaces(spaces.data);
})();
}, []);
return (
<AnimatePresence mode="wait">
<motion.div
initial={{
opacity: 0,
y: 20,
}}
animate={{
y: 0,
opacity: 1,
}}
exit={{
opacity: 0,
y: 20,
}}
transition={{
duration: 0.2,
}}
className="flex flex-col items-center"
>
<div className="bg-secondary py-[.35rem] px-[.6rem] rounded-2xl">
<HoverEffect
items={fakeitems}
index={index}
indexFn={(i) => setIndex(i)}
/>
</div>
{index === 1 ? (
<SpaceForm cancelfn={cancelfn} />
) : (
<PageForm cancelfn={cancelfn} spaces={spaces} />
)}
</motion.div>
</AnimatePresence>
);
}
export const HoverEffect = ({
items,
index,
indexFn,
}: {
items: string[];
index: number;
indexFn: (i: number) => void;
}) => {
return (
<div className={"flex"}>
{items.map((item, idx) => (
<button
key={idx}
className="relative block h-full w-full px-2 py-1"
onClick={() => indexFn(idx)}
>
<AnimatePresence>
{index === idx && (
<motion.span
className="absolute inset-0 block h-full w-full rounded-xl bg-[#2B3237]"
layoutId="hoverBackground"
initial={{ opacity: 0 }}
animate={{
opacity: 1,
transition: { duration: 0.15 },
}}
exit={{
opacity: 0,
transition: { duration: 0.15, delay: 0.2 },
}}
/>
)}
</AnimatePresence>
<h3 className="text-[#858B92] z-50 relative">{item}</h3>
</button>
))}
</div>
);
};
function SpaceForm({ cancelfn }: { cancelfn: () => void }) {
return (
<form
action={createSpace}
className="bg-secondary border border-muted-foreground px-4 py-3 rounded-2xl mt-2 flex flex-col gap-3"
>
<div>
<Label className="text-[#858B92]" htmlFor="name">
Name
</Label>
<Input
className="bg-[#2B3237] focus-visible:ring-0 border-none focus-visible:ring-offset-0"
id="name"
name="name"
/>
</div>
<div className="flex justify-between">
{/* <a className="text-blue-500" href="">
pull from store
</a> */}
{/* <div
onClick={cancelfn}
className="bg-[#2B3237] px-2 py-1 rounded-xl cursor-pointer"
>
cancel
</div> */}
<button
type="submit"
className="bg-[#2B3237] px-2 py-1 rounded-xl cursor-pointer"
>
Submit
</button>
</div>
</form>
);
}
function PageForm({
cancelfn,
spaces,
}: {
cancelfn: () => void;
spaces: Space[];
}) {
const [loading, setLoading] = useState(false);
const { pending } = useFormStatus();
return (
<form
action={async (e: FormData) => {
const content = e.get("content")?.toString();
const space = e.get("space")?.toString();
toast.info("Creating memory...");
if (!content) {
toast.error("Content is required");
return;
}
cancelfn();
const cont = await createMemory({
content: content,
spaces: space ? [space] : undefined,
});
if (cont.success) {
toast.success("Memory created");
} else {
toast.error("Memory creation failed");
}
}}
className="bg-secondary border border-muted-foreground px-4 py-3 rounded-2xl mt-2 flex flex-col gap-3 w-[100vw] md:w-[400px]"
>
<div>
<Label className="text-[#858B92]" htmlFor="space">
Space
</Label>
<Select name="space">
<SelectTrigger>
<SelectValue placeholder="Space" />
</SelectTrigger>
<SelectContent className="bg-secondary text-white">
{spaces.map((space) => (
<SelectItem key={space.id} value={space.id.toString()}>
{space.name}
</SelectItem>
))}
</SelectContent>
</Select>
</div>
<div>
<Label className="text-[#858B92]" htmlFor="name">
Resource (URL or content)
</Label>
<Textarea
className="bg-[#2B3237] focus-visible:ring-0 border-none focus-visible:ring-offset-0"
id="input"
name="content"
placeholder="Start typing a note or paste a URL here. I'll remember it."
/>
</div>
<div className="flex justify-end">
<button
type="submit"
className="bg-[#2B3237] px-2 py-1 rounded-xl cursor-pointer"
>
Submit
</button>
</div>
</form>
);
}

73
apps/web/app/(dash)/header.tsx
View file

@ -3,13 +3,22 @@ import Image from "next/image";
import Link from "next/link";
import Logo from "../../public/logo.svg";
import { AddIcon, ChatIcon } from "@repo/ui/icons";
import { Tabs, TabsContent, TabsList, TabsTrigger } from "@repo/ui/shadcn/tabs";
function Header() {
import DynamicIsland from "./dynamicisland";
import { db } from "@/server/db";
import { getChatHistory } from "../actions/fetchers";
async function Header() {
const chatThreads = await getChatHistory();
if (!chatThreads.success || !chatThreads.data) {
return <div>Error fetching chat threads</div>;
}
return (
<div>
<div className="flex items-center justify-between relative z-10">
<Link href="/">
<div className="p-4 relative z-30 h-16 flex items-center">
<div className="w-full flex items-center justify-between">
<Link className="" href="/home">
<Image
src={Logo}
alt="SuperMemory logo"
@ -17,37 +26,37 @@ function Header() {
/>
</Link>
<Tabs
className="absolute flex flex-col justify-center items-center w-full -z-10 group top-0 transition-transform duration-1000 ease-out"
defaultValue="account"
>
<div className="bg-secondary all-center h-11 rounded-full p-2 min-w-14">
<button className="p-2 group-hover:hidden transition duration-500 ease-in-out">
<Image src={AddIcon} alt="Add icon" />
<div className="fixed z-30 left-1/2 -translate-x-1/2 top-5">
<DynamicIsland />
</div>
<div className="flex items-center gap-2">
<button className="flex duration-200 items-center text-[#7D8994] hover:bg-[#1F2429] text-[13px] gap-2 px-3 py-2 rounded-xl">
<Image src={ChatIcon} alt="Chat icon" className="w-5" />
Start new chat
</button>
<div className="relative group">
<button className="flex duration-200 items-center text-[#7D8994] hover:bg-[#1F2429] text-[13px] gap-2 px-3 py-2 rounded-xl">
History
</button>
<div className="hidden group-hover:flex inset-0 transition-opacity duration-500 ease-in-out">
<TabsList className="p-2">
<TabsTrigger value="account">Account</TabsTrigger>
<TabsTrigger value="password">Password</TabsTrigger>
</TabsList>
<div className="absolute p-4 hidden group-hover:block right-0 w-full md:w-[400px] max-h-[70vh] overflow-auto">
<div className="bg-[#1F2429] rounded-xl p-2 flex flex-col shadow-lg">
{chatThreads.data.map((thread) => (
<Link
prefetch={false}
href={`/chat/${thread.id}`}
key={thread.id}
className="p-2 rounded-md hover:bg-secondary"
>
{thread.firstMessage}
</Link>
))}
</div>
</div>
</div>
<div className="bg-secondary all-center rounded-full p-2 mt-4 min-w-14 hidden group-hover:block">
<TabsContent value="account">
Make changes to your account here.
</TabsContent>
<TabsContent value="password">
Change your password here.
</TabsContent>
</div>
</Tabs>
<button className="flex shrink-0 duration-200 items-center gap-2 px-2 py-1.5 rounded-xl hover:bg-secondary">
<Image src={ChatIcon} alt="Chat icon" />
Start new chat
</button>
</div>
</div>
</div>
);

67
apps/web/app/(dash)/home/page.tsx
View file

@ -1,27 +1,70 @@
import React from "react";
import Menu from "../menu";
import Header from "../header";
import QueryInput from "./queryinput";
import { homeSearchParamsCache } from "@/app/helpers/lib/searchParams";
import { getSpaces } from "../actions";
"use client";
async function Page({
import React, { useEffect, useState } from "react";
import QueryInput from "./queryinput";
import { homeSearchParamsCache } from "@/lib/searchParams";
import { getSpaces } from "@/app/actions/fetchers";
import { useRouter } from "next/navigation";
import { createChatThread, linkTelegramToUser } from "@/app/actions/doers";
import { toast } from "sonner";
import { useSession } from "next-auth/react";
function Page({
searchParams,
}: {
searchParams: Record<string, string | string[] | undefined>;
}) {
// TODO: use this to show a welcome page/modal
const { firstTime } = homeSearchParamsCache.parse(searchParams);
// const { firstTime } = homeSearchParamsCache.parse(searchParams);
const spaces = await getSpaces();
const [telegramUser, setTelegramUser] = useState<string | undefined>(
searchParams.telegramUser as string,
);
const { push } = useRouter();
const [spaces, setSpaces] = useState<{ id: number; name: string }[]>([]);
useEffect(() => {
if (telegramUser) {
const linkTelegram = async () => {
const response = await linkTelegramToUser(telegramUser);
if (response.success) {
toast.success("Your telegram has been linked successfully.");
} else {
toast.error("Failed to link telegram. Please try again.");
}
};
linkTelegram();
}
getSpaces().then((res) => {
if (res.success && res.data) {
setSpaces(res.data);
return;
}
// TODO: HANDLE ERROR
});
}, []);
return (
<div className="max-w-3xl flex mx-auto w-full flex-col">
<div className="max-w-3xl h-full justify-center flex mx-auto w-full flex-col">
{/* all content goes here */}
{/* <div className="">hi {firstTime ? 'first time' : ''}</div> */}
<div className="w-full h-96">
<QueryInput initialSpaces={spaces} />
<div className="w-full pb-20">
<QueryInput
handleSubmit={async (q, spaces) => {
const threadid = await createChatThread(q);
push(
`/chat/${threadid.data}?spaces=${JSON.stringify(spaces)}&q=${q}`,
);
}}
initialSpaces={spaces}
/>
</div>
</div>
);

119
apps/web/app/(dash)/home/queryinput.tsx
View file

@ -2,65 +2,80 @@
import { ArrowRightIcon } from "@repo/ui/icons";
import Image from "next/image";
import React, { useState } from "react";
import React, { useEffect, useMemo, useState } from "react";
import Divider from "@repo/ui/shadcn/divider";
import { MultipleSelector, Option } from "@repo/ui/shadcn/combobox";
import { useRouter } from "next/navigation";
import { getSpaces } from "@/app/actions/fetchers";
function QueryInput({
initialQuery = "",
initialSpaces = [],
disabled = false,
className,
mini = false,
handleSubmit,
}: {
initialQuery?: string;
initialSpaces?: { user: string | null; id: number; name: string }[];
initialSpaces?: {
id: number;
name: string;
}[];
disabled?: boolean;
className?: string;
mini?: boolean;
handleSubmit: (q: string, spaces: { id: number; name: string }[]) => void;
}) {
const [q, setQ] = useState(initialQuery);
const [selectedSpaces, setSelectedSpaces] = useState<number[]>([]);
const { push } = useRouter();
const options = useMemo(
() =>
initialSpaces.map((x) => ({
label: x.name,
value: x.id.toString(),
})),
[initialSpaces],
);
const parseQ = () => {
// preparedSpaces is list of spaces selected by user, with id and name
const preparedSpaces = initialSpaces
.filter((x) => selectedSpaces.includes(x.id))
.map((x) => {
return {
id: x.id,
name: x.name,
};
});
const newQ =
"/chat?q=" +
encodeURI(q) +
(selectedSpaces ? "&spaces=" + JSON.stringify(preparedSpaces) : "");
return newQ;
};
const options = initialSpaces.map((x) => ({
label: x.name,
value: x.id.toString(),
}));
const preparedSpaces = useMemo(
() =>
initialSpaces
.filter((x) => selectedSpaces.includes(x.id))
.map((x) => {
return {
id: x.id,
name: x.name,
};
}),
[selectedSpaces, initialSpaces],
);
return (
<div>
<div className="bg-secondary rounded-t-[24px] w-full mt-40">
<div className={className}>
<div
className={`bg-secondary ${!mini ? "rounded-t-3xl" : "rounded-3xl"}`}
>
{/* input and action button */}
<form action={async () => push(parseQ())} className="flex gap-4 p-3">
<form
action={async () => {
handleSubmit(q, preparedSpaces);
setQ("");
}}
className="flex gap-4 p-3"
>
<textarea
name="q"
cols={30}
rows={4}
className="bg-transparent pt-2.5 text-base text-[#989EA4] focus:text-foreground duration-200 tracking-[3%] outline-none resize-none w-full p-4"
rows={mini ? 2 : 4}
className="bg-transparent pt-2.5 text-base placeholder:text-[#5D6165] text-[#9DA0A4] focus:text-gray-200 duration-200 tracking-[3%] outline-none resize-none w-full p-4"
placeholder="Ask your second brain..."
onKeyDown={(e) => {
if (e.key === "Enter") {
e.preventDefault();
if (!e.shiftKey) push(parseQ());
handleSubmit(q, preparedSpaces);
setQ("");
}
}}
onChange={(e) => setQ(e.target.value)}
@ -70,29 +85,39 @@ function QueryInput({
<button
type="submit"
onClick={(e) => {
e.preventDefault();
handleSubmit(q, preparedSpaces);
}}
disabled={disabled}
className="h-12 w-12 rounded-[14px] bg-[#21303D] all-center shrink-0 hover:brightness-125 duration-200 outline-none focus:outline focus:outline-primary active:scale-90"
>
<Image src={ArrowRightIcon} alt="Right arrow icon" />
</button>
</form>
<Divider />
</div>
{/* selected sources */}
<div className="flex items-center gap-6 p-2 h-auto bg-secondary rounded-b-[24px]">
<MultipleSelector
disabled={disabled}
defaultOptions={options}
onChange={(e) => setSelectedSpaces(e.map((x) => parseInt(x.value)))}
placeholder="Focus on specific spaces..."
emptyIndicator={
<p className="text-center text-lg leading-10 text-gray-600 dark:text-gray-400">
no results found.
</p>
}
/>
</div>
{!mini && (
<>
<Divider />
<div className="flex items-center gap-6 p-2 h-auto bg-secondary rounded-b-3xl">
<MultipleSelector
key={options.length}
disabled={disabled}
defaultOptions={options}
onChange={(e) =>
setSelectedSpaces(e.map((x) => parseInt(x.value)))
}
placeholder="Focus on specific spaces..."
emptyIndicator={
<p className="text-center text-lg leading-10 text-gray-600 dark:text-gray-400">
no results found.
</p>
}
/>
</div>
</>
)}
</div>
);
}

15
apps/web/app/(dash)/layout.tsx
View file

@ -1,22 +1,27 @@
import Header from "./header";
import Menu from "./menu";
import { ensureAuth } from "./actions";
import { redirect } from "next/navigation";
import { auth } from "../../server/auth";
import { Toaster } from "@repo/ui/shadcn/sonner";
async function Layout({ children }: { children: React.ReactNode }) {
const info = await ensureAuth();
const info = await auth();
if (!info) {
return redirect("/signin");
}
return (
<main className="h-screen flex flex-col p-4 relative">
<Header />
<main className="h-screen flex flex-col">
<div className="fixed top-0 left-0 w-full">
<Header />
</div>
<Menu />
{children}
<div className="w-full h-full px-2 md:px-0">{children}</div>
<Toaster />
</main>
);
}

133
apps/web/app/(dash)/memories/page.tsx Normal file
View file

@ -0,0 +1,133 @@
"use client";
import { getAllUserMemoriesAndSpaces } from "@/app/actions/fetchers";
import { Space } from "@/app/actions/types";
import { Content } from "@/server/db/schema";
import { NextIcon, SearchIcon, UrlIcon } from "@repo/ui/icons";
import Image from "next/image";
import React, { useEffect, useState } from "react";
function Page() {
const [filter, setFilter] = useState("All");
const setFilterfn = (i: string) => setFilter(i);
const [search, setSearch] = useState("");
const [memoriesAndSpaces, setMemoriesAndSpaces] = useState<{
memories: Content[];
spaces: Space[];
}>({ memories: [], spaces: [] });
useEffect(() => {
(async () => {
const { success, data } = await getAllUserMemoriesAndSpaces();
if (!success ?? !data) return;
setMemoriesAndSpaces({ memories: data.memories, spaces: data.spaces });
})();
}, []);
return (
<div className="max-w-3xl min-w-3xl py-36 h-full flex mx-auto w-full flex-col gap-12">
<h2 className="text-white w-full font-medium text-2xl text-left">
My Memories
</h2>
<div className="flex flex-col gap-4">
<div className="w-full relative">
<input
type="text"
className=" w-full py-3 rounded-md text-lg pl-8 bg-[#1F2428] outline-none"
placeholder="search here..."
/>
<Image
className="absolute top-1/2 -translate-y-1/2 left-2"
src={SearchIcon}
alt="Search icon"
/>
</div>
<Filters filter={filter} setFilter={setFilterfn} />
</div>
<div>
<div className="text-[#B3BCC5]">Spaces</div>
{memoriesAndSpaces.spaces.map((space) => (
<TabComponent title={space.name} description={space.id.toString()} />
))}
</div>
<div>
<div className="text-[#B3BCC5]">Pages</div>
{memoriesAndSpaces.memories.map((memory) => (
<LinkComponent title={memory.title ?? "No title"} url={memory.url} />
))}
</div>
</div>
);
}
function TabComponent({
title,
description,
}: {
title: string;
description: string;
}) {
return (
<div className="flex items-center my-6">
<div>
<div className="h-12 w-12 bg-[#1F2428] flex justify-center items-center rounded-md">
{title.slice(0, 2).toUpperCase()}
</div>
</div>
<div className="grow px-4">
<div className="text-lg text-[#fff]">{title}</div>
<div>{description}</div>
</div>
<div>
<Image src={NextIcon} alt="Search icon" />
</div>
</div>
);
}
function LinkComponent({ title, url }: { title: string; url: string }) {
return (
<div className="flex items-center my-6">
<div>
<div className="h-12 w-12 bg-[#1F2428] flex justify-center items-center rounded-md">
<Image src={UrlIcon} alt="Url icon" />
</div>
</div>
<div className="grow px-4">
<div className="text-lg text-[#fff]">{title}</div>
<div>{url}</div>
</div>
</div>
);
}
const FilterMethods = ["All", "Spaces", "Pages", "Notes"];
function Filters({
setFilter,
filter,
}: {
setFilter: (i: string) => void;
filter: string;
}) {
return (
<div className="flex gap-4">
{FilterMethods.map((i) => {
return (
<div
onClick={() => setFilter(i)}
className={`transition px-6 py-2 rounded-xl ${i === filter ? "bg-[#21303D] text-[#369DFD]" : "text-[#B3BCC5] bg-[#1F2428] hover:bg-[#1f262d] hover:text-[#76a3cc]"}`}
>
{i}
</div>
);
})}
</div>
);
}
export default Page;

66
apps/web/app/(dash)/menu.tsx
View file

@ -1,48 +1,90 @@
"use client";
import React from "react";
import Image from "next/image";
import { MemoriesIcon, ExploreIcon, HistoryIcon } from "@repo/ui/icons";
import Link from "next/link";
import { MemoriesIcon, ExploreIcon, CanvasIcon } from "@repo/ui/icons";
function Menu() {
const menuItems = [
{
icon: MemoriesIcon,
text: "Memories",
url: "/",
url: "/memories",
disabled: false,
},
{
icon: ExploreIcon,
text: "Explore",
url: "/explore",
disabled: true,
},
{
icon: HistoryIcon,
text: "History",
url: "/history",
icon: CanvasIcon,
text: "Canvas",
url: "/canvas",
disabled: true,
},
];
return (
<div className="absolute h-full p-4 flex items-center top-0 left-0">
<div className="">
<div className="hover:rounded-2x group inline-flex w-14 text-foreground-menu text-[15px] font-medium flex-col items-start gap-6 overflow-hidden rounded-[28px] bg-secondary px-3 py-4 duration-200 hover:w-40">
<>
{/* Desktop Menu */}
<div className="hidden lg:flex fixed h-screen pb-20 w-full p-4 items-center justify-start top-0 left-0 pointer-events-none">
<div className="pointer-events-auto group flex w-14 text-foreground-menu text-[15px] font-medium flex-col items-start gap-6 overflow-hidden rounded-[28px] bg-secondary px-3 py-4 duration-200 hover:w-40">
{menuItems.map((item) => (
<div
<Link
aria-disabled={item.disabled}
href={item.disabled ? "#" : item.url}
key={item.url}
className="flex w-full cursor-pointer items-center gap-3 px-1 duration-200 hover:scale-105 hover:brightness-150 active:scale-90"
className={`flex w-full ${
item.disabled
? "cursor-not-allowed opacity-50"
: "text-[#777E87] brightness-75 hover:brightness-125 cursor-pointer"
} items-center gap-3 px-1 duration-200 hover:scale-105 active:scale-90 justify-start`}
>
<Image
src={item.icon}
alt={`${item.text} icon`}
width={24}
height={24}
className="hover:brightness-125 duration-200"
/>
<p className="opacity-0 duration-200 group-hover:opacity-100">
{item.text}
</p>
</div>
</Link>
))}
</div>
</div>
</div>
{/* Mobile Menu */}
<div className="lg:hidden fixed bottom-0 left-0 w-full p-4 bg-secondary">
<div className="flex justify-around items-center">
{menuItems.map((item) => (
<Link
aria-disabled={item.disabled}
href={item.disabled ? "#" : item.url}
key={item.url}
className={`flex flex-col items-center ${
item.disabled
? "opacity-50 cursor-not-allowed"
: "cursor-pointer"
}`}
onClick={(e) => item.disabled && e.preventDefault()}
>
<Image
src={item.icon}
alt={`${item.text} icon`}
width={24}
height={24}
/>
<p className="text-xs text-foreground-menu mt-2">{item.text}</p>
</Link>
))}
</div>
</div>
</>
);
}

2
apps/web/app/(landing)/twitterLink.tsx
View file

@ -58,7 +58,7 @@ export function HoverBorderGradient({
if (!directions[nextIndex]) {
return directions[0]!;
}
return directions[nextIndex];
return directions[nextIndex]!;
};
const movingMap: Record<Direction, string> = {

383
apps/web/app/actions/doers.ts Normal file
View file

@ -0,0 +1,383 @@
"use server";
import { revalidatePath } from "next/cache";
import { db } from "../../server/db";
import {
chatHistory,
chatThreads,
contentToSpace,
space,
storedContent,
users,
} from "../../server/db/schema";
import { ServerActionReturnType } from "./types";
import { auth } from "../../server/auth";
import { Tweet } from "react-tweet/api";
import { getMetaData } from "@/lib/get-metadata";
import { and, eq, inArray, sql } from "drizzle-orm";
import { LIMITS } from "@/lib/constants";
import { z } from "zod";
import { ChatHistory } from "@repo/shared-types";
import { decipher } from "@/server/encrypt";
import { redirect } from "next/navigation";
export const createSpace = async (
input: string | FormData,
): ServerActionReturnType<number> => {
const data = await auth();
if (!data || !data.user) {
redirect("/signin");
return { error: "Not authenticated", success: false };
}
if (typeof input === "object") {
input = (input as FormData).get("name") as string;
}
try {
const resp = await db
.insert(space)
.values({ name: input, user: data.user.id });
revalidatePath("/home");
return { success: true, data: 1 };
} catch (e: unknown) {
const error = e as Error;
if (
error.message.includes("D1_ERROR: UNIQUE constraint failed: space.name")
) {
return { success: false, data: 0, error: "Space already exists" };
} else {
return {
success: false,
data: 0,
error: "Failed to create space with error: " + error.message,
};
}
}
};
const typeDecider = (content: string) => {
// if the content is a URL, then it's a page. if its a URL with https://x.com/user/status/123, then it's a tweet. else, it's a note.
// do strict checking with regex
if (content.match(/https?:\/\/[\w\.]+\/[\w]+\/[\w]+\/[\d]+/)) {
return "tweet";
} else if (content.match(/https?:\/\/[\w\.]+/)) {
return "page";
} else {
return "note";
}
};
export const limit = async (userId: string, type = "page") => {
const count = await db
.select({
count: sql<number>`count(*)`.mapWith(Number),
})
.from(storedContent)
.where(and(eq(storedContent.userId, userId), eq(storedContent.type, type)));
if (count[0]!.count > LIMITS[type as keyof typeof LIMITS]) {
return false;
}
return true;
};
const getTweetData = async (tweetID: string) => {
const url = `https://cdn.syndication.twimg.com/tweet-result?id=${tweetID}&lang=en&features=tfw_timeline_list%3A%3Btfw_follower_count_sunset%3Atrue%3Btfw_tweet_edit_backend%3Aon%3Btfw_refsrc_session%3Aon%3Btfw_fosnr_soft_interventions_enabled%3Aon%3Btfw_show_birdwatch_pivots_enabled%3Aon%3Btfw_show_business_verified_badge%3Aon%3Btfw_duplicate_scribes_to_settings%3Aon%3Btfw_use_profile_image_shape_enabled%3Aon%3Btfw_show_blue_verified_badge%3Aon%3Btfw_legacy_timeline_sunset%3Atrue%3Btfw_show_gov_verified_badge%3Aon%3Btfw_show_business_affiliate_badge%3Aon%3Btfw_tweet_edit_frontend%3Aon&token=4c2mmul6mnh`;
const resp = await fetch(url, {
headers: {
"User-Agent":
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3",
Accept: "application/json",
"Accept-Language": "en-US,en;q=0.5",
"Accept-Encoding": "gzip, deflate, br",
Connection: "keep-alive",
"Upgrade-Insecure-Requests": "1",
"Cache-Control": "max-age=0",
TE: "Trailers",
},
});
console.log(resp.status);
const data = (await resp.json()) as Tweet;
return data;
};
export const createMemory = async (input: {
content: string;
spaces?: string[];
}): ServerActionReturnType<number> => {
const data = await auth();
if (!data || !data.user || !data.user.id) {
redirect("/signin");
return { error: "Not authenticated", success: false };
}
const type = typeDecider(input.content);
let pageContent = input.content;
let metadata: Awaited<ReturnType<typeof getMetaData>>;
if (!(await limit(data.user.id, type))) {
return {
success: false,
data: 0,
error: `You have exceeded the limit of ${LIMITS[type as keyof typeof LIMITS]} ${type}s.`,
};
}
if (type === "page") {
const response = await fetch("https://md.dhr.wtf/?url=" + input.content, {
headers: {
Authorization: "Bearer " + process.env.BACKEND_SECURITY_KEY,
},
});
pageContent = await response.text();
metadata = await getMetaData(input.content);
} else if (type === "tweet") {
const tweet = await getTweetData(input.content.split("/").pop() as string);
pageContent = JSON.stringify(tweet);
metadata = {
baseUrl: input.content,
description: tweet.text,
image: tweet.user.profile_image_url_https,
title: `Tweet by ${tweet.user.name}`,
};
} else if (type === "note") {
pageContent = input.content;
const noteId = new Date().getTime();
metadata = {
baseUrl: `https://supermemory.ai/note/${noteId}`,
description: `Note created at ${new Date().toLocaleString()}`,
image: "https://supermemory.ai/logo.png",
title: `${pageContent.slice(0, 20)} ${pageContent.length > 20 ? "..." : ""}`,
};
} else {
return {
success: false,
data: 0,
error: "Invalid type",
};
}
let storeToSpaces = input.spaces;
if (!storeToSpaces) {
storeToSpaces = [];
}
const vectorSaveResponse = await fetch(
`${process.env.BACKEND_BASE_URL}/api/add`,
{
method: "POST",
body: JSON.stringify({
pageContent,
title: metadata.title,
description: metadata.description,
url: metadata.baseUrl,
spaces: storeToSpaces,
user: data.user.id,
type,
}),
headers: {
"Content-Type": "application/json",
Authorization: "Bearer " + process.env.BACKEND_SECURITY_KEY,
},
},
);
if (!vectorSaveResponse.ok) {
const errorData = await vectorSaveResponse.text();
console.error(errorData);
return {
success: false,
data: 0,
error: `Failed to save to vector store. Backend returned error: ${errorData}`,
};
}
// Insert into database
const insertResponse = await db
.insert(storedContent)
.values({
content: pageContent,
title: metadata.title,
description: metadata.description,
url: input.content,
baseUrl: metadata.baseUrl,
image: metadata.image,
savedAt: new Date(),
userId: data.user.id,
type,
})
.returning({ id: storedContent.id });
const contentId = insertResponse[0]?.id;
if (!contentId) {
return {
success: false,
data: 0,
error: "Something went wrong while saving the document to the database",
};
}
if (storeToSpaces.length > 0) {
// Adding the many-to-many relationship between content and spaces
const spaceData = await db
.select()
.from(space)
.where(
and(
inArray(
space.id,
storeToSpaces.map((s) => parseInt(s)),
),
eq(space.user, data.user.id),
),
)
.all();
await Promise.all(
spaceData.map(async (space) => {
await db
.insert(contentToSpace)
.values({ contentId: contentId, spaceId: space.id });
}),
);
}
try {
const response = await vectorSaveResponse.json();
const expectedResponse = z.object({ status: z.literal("ok") });
const parsedResponse = expectedResponse.safeParse(response);
if (!parsedResponse.success) {
return {
success: false,
data: 0,
error: `Failed to save to vector store. Backend returned error: ${parsedResponse.error.message}`,
};
}
return {
success: true,
data: 1,
};
} catch (e) {
return {
success: false,
data: 0,
error: `Failed to save to vector store. Backend returned error: ${e}`,
};
}
};
export const createChatThread = async (
firstMessage: string,
): ServerActionReturnType<string> => {
const data = await auth();
if (!data || !data.user || !data.user.id) {
redirect("/signin");
return { error: "Not authenticated", success: false };
}
const thread = await db
.insert(chatThreads)
.values({
firstMessage,
userId: data.user.id,
})
.returning({ id: chatThreads.id })
.execute();
console.log(thread);
if (!thread[0]) {
return {
success: false,
error: "Failed to create chat thread",
};
}
return { success: true, data: thread[0].id };
};
export const createChatObject = async (
threadId: string,
chatHistorySoFar: ChatHistory[],
): ServerActionReturnType<boolean> => {
const data = await auth();
if (!data || !data.user || !data.user.id) {
redirect("/signin");
return { error: "Not authenticated", success: false };
}
const lastChat = chatHistorySoFar[chatHistorySoFar.length - 1];
if (!lastChat) {
return {
success: false,
data: false,
error: "No chat object found",
};
}
console.log("sources: ", lastChat.answer.sources);
const saved = await db.insert(chatHistory).values({
question: lastChat.question,
answer: lastChat.answer.parts.map((part) => part.text).join(""),
answerSources: JSON.stringify(lastChat.answer.sources),
threadId,
});
if (!saved) {
return {
success: false,
data: false,
error: "Failed to save chat object",
};
}
return {
success: true,
data: true,
};
};
export const linkTelegramToUser = async (
telegramUser: string,
): ServerActionReturnType<boolean> => {
const data = await auth();
if (!data || !data.user || !data.user.id) {
redirect("/signin");
return { error: "Not authenticated", success: false };
}
const user = await db
.update(users)
.set({ telegramId: decipher(telegramUser) })
.where(eq(users.id, data.user.id))
.execute();
if (!user) {
return {
success: false,
data: false,
error: "Failed to link telegram to user",
};
}
return {
success: true,
data: true,
};
};

191
apps/web/app/actions/fetchers.ts Normal file
View file

@ -0,0 +1,191 @@
"use server";
import { and, asc, eq, inArray, not, sql } from "drizzle-orm";
import { db } from "../../server/db";
import {
chatHistory,
ChatThread,
chatThreads,
Content,
contentToSpace,
storedContent,
users,
} from "../../server/db/schema";
import { ServerActionReturnType, Space } from "./types";
import { auth } from "../../server/auth";
import { ChatHistory, SourceZod } from "@repo/shared-types";
import { ChatHistory as ChatHistoryType } from "../../server/db/schema";
import { z } from "zod";
import { redirect } from "next/navigation";
export const getSpaces = async (): ServerActionReturnType<Space[]> => {
const data = await auth();
if (!data || !data.user) {
redirect("/signin");
return { error: "Not authenticated", success: false };
}
const spaces = await db.query.space.findMany({
where: eq(users, data.user.id),
});
const spacesWithoutUser = spaces.map((space) => {
return { ...space, user: undefined };
});
return { success: true, data: spacesWithoutUser };
};
export const getAllMemories = async (
freeMemoriesOnly: boolean = false,
): ServerActionReturnType<Content[]> => {
const data = await auth();
if (!data || !data.user) {
redirect("/signin");
return { error: "Not authenticated", success: false };
}
if (!freeMemoriesOnly) {
// Returns all memories, no matter the space.
const memories = await db.query.storedContent.findMany({
where: eq(users, data.user.id),
});
return { success: true, data: memories };
}
// This only returns memories that are not a part of any space.
// This is useful for home page where we want to show a list of spaces and memories.
const contentNotInAnySpace = await db
.select()
.from(storedContent)
.where(
not(
eq(
storedContent.id,
db
.select({ contentId: contentToSpace.contentId })
.from(contentToSpace),
),
),
)
.execute();
return { success: true, data: contentNotInAnySpace };
};
export const getAllUserMemoriesAndSpaces = async (): ServerActionReturnType<{
spaces: Space[];
memories: Content[];
}> => {
const data = await auth();
if (!data || !data.user) {
redirect("/signin");
return { error: "Not authenticated", success: false };
}
const spaces = await db.query.space.findMany({
where: eq(users, data.user.id),
});
const memories = await db.query.storedContent.findMany({
where: eq(users, data.user.id),
});
return {
success: true,
data: { spaces: spaces, memories: memories },
};
};
export const getFullChatThread = async (
threadId: string,
): ServerActionReturnType<ChatHistory[]> => {
const data = await auth();
if (!data || !data.user || !data.user.id) {
redirect("/signin");
return { error: "Not authenticated", success: false };
}
const thread = await db.query.chatThreads.findFirst({
where: and(
eq(chatThreads.id, threadId),
eq(chatThreads.userId, data.user.id),
),
});
if (!thread) {
return { error: "Thread not found", success: false };
}
const allChatsInThisThread = await db.query.chatHistory
.findMany({
where: and(eq(chatHistory.threadId, threadId)),
orderBy: asc(chatHistory.id),
})
.execute();
const accumulatedChatHistory: ChatHistory[] = allChatsInThisThread.map(
(chat) => {
console.log("answer sources", chat.answerSources);
const sourceCheck = z
.array(SourceZod)
.safeParse(JSON.parse(chat.answerSources ?? "[]"));
if (!sourceCheck.success || !sourceCheck.data) {
console.error("sourceCheck.error", sourceCheck.error);
throw new Error("Invalid source data");
}
const sources = sourceCheck.data;
return {
question: chat.question,
answer: {
parts: [
{
text: chat.answer ?? undefined,
},
],
sources: sources ?? [],
},
};
},
);
return {
success: true,
data: accumulatedChatHistory,
};
};
export const getChatHistory = async (): ServerActionReturnType<
ChatThread[]
> => {
const data = await auth();
if (!data || !data.user || !data.user.id) {
redirect("/signin");
return { error: "Not authenticated", success: false };
}
try {
const chatHistorys = await db.query.chatThreads.findMany({
where: eq(chatThreads.userId, data.user.id),
});
return {
success: true,
data: chatHistorys,
};
} catch (e) {
return {
success: false,
error: (e as Error).message,
};
}
};

11
apps/web/app/actions/types.ts Normal file
View file

@ -0,0 +1,11 @@
export type Space = {
id: number;
name: string;
numberOfMemories?: number;
};
export type ServerActionReturnType<T> = Promise<{
error?: string;
success: boolean;
data?: T;
}>;

2
apps/web/app/api/[...nextauth]/route.ts
View file

@ -1,2 +1,2 @@
export { GET, POST } from "../../helpers/server/auth";
export { GET, POST } from "../../../server/auth";
export const runtime = "edge";

94
apps/web/app/api/chat/route.ts
View file

@ -1,6 +1,12 @@
import { type NextRequest } from "next/server";
import { ChatHistory } from "@repo/shared-types";
import {
ChatHistory,
ChatHistoryZod,
convertChatHistoryList,
SourcesFromApi,
} from "@repo/shared-types";
import { ensureAuth } from "../ensureAuth";
import { z } from "zod";
export const runtime = "edge";
@ -15,59 +21,67 @@ export async function POST(req: NextRequest) {
return new Response("Missing BACKEND_SECURITY_KEY", { status: 500 });
}
const query = new URL(req.url).searchParams.get("q");
const spaces = new URL(req.url).searchParams.get("spaces");
const url = new URL(req.url);
const sourcesOnly =
new URL(req.url).searchParams.get("sourcesOnly") ?? "false";
const query = url.searchParams.get("q");
const spaces = url.searchParams.get("spaces");
const chatHistory = (await req.json()) as {
const sourcesOnly = url.searchParams.get("sourcesOnly") ?? "false";
const jsonRequest = (await req.json()) as {
chatHistory: ChatHistory[];
sources: SourcesFromApi[] | undefined;
};
const { chatHistory, sources } = jsonRequest;
console.log("CHathistory", chatHistory);
if (!query) {
if (!query || query.trim.length < 0) {
return new Response(JSON.stringify({ message: "Invalid query" }), {
status: 400,
});
}
try {
const resp = await fetch(
`https://cf-ai-backend.dhravya.workers.dev/chat?q=${query}&user=${session.user.email ?? session.user.name}&sourcesOnly=${sourcesOnly}&spaces=${spaces}`,
{
headers: {
"X-Custom-Auth-Key": process.env.BACKEND_SECURITY_KEY!,
},
method: "POST",
body: JSON.stringify({
chatHistory: chatHistory.chatHistory ?? [],
}),
},
const validated = z.array(ChatHistoryZod).safeParse(chatHistory ?? []);
if (!validated.success) {
return new Response(
JSON.stringify({
message: "Invalid chat history",
error: validated.error,
}),
{ status: 400 },
);
}
console.log("sourcesOnly", sourcesOnly);
const modelCompatible = await convertChatHistoryList(validated.data);
if (sourcesOnly == "true") {
const data = await resp.json();
console.log("data", data);
return new Response(JSON.stringify(data), { status: 200 });
}
const resp = await fetch(
`${process.env.BACKEND_BASE_URL}/api/chat?query=${query}&user=${session.user.id}&sourcesOnly=${sourcesOnly}&spaces=${spaces}`,
{
headers: {
Authorization: `Bearer ${process.env.BACKEND_SECURITY_KEY}`,
"Content-Type": "application/json",
},
method: "POST",
body: JSON.stringify({
chatHistory: modelCompatible,
sources,
}),
},
);
if (resp.status !== 200 || !resp.ok) {
const errorData = await resp.json();
console.log(errorData);
return new Response(
JSON.stringify({ message: "Error in CF function", error: errorData }),
{ status: resp.status },
);
}
if (sourcesOnly == "true") {
const data = (await resp.json()) as SourcesFromApi;
return new Response(JSON.stringify(data), { status: 200 });
}
// Stream the response back to the client
const { readable, writable } = new TransformStream();
resp && resp.body!.pipeTo(writable);
if (resp.status !== 200 || !resp.ok) {
const errorData = await resp.text();
console.log(errorData);
return new Response(
JSON.stringify({ message: "Error in CF function", error: errorData }),
{ status: resp.status },
);
}
return new Response(readable, { status: 200 });
} catch {}
return new Response(resp.body, { status: 200 });
}

30
apps/web/app/api/editorai/route.ts Normal file
View file

@ -0,0 +1,30 @@
import type { NextRequest } from "next/server";
import { ensureAuth } from "../ensureAuth";
export const runtime = "edge";
// ERROR #2 - This the the next function that calls the backend, I sometimes think this is redundency, but whatever
// I have commented the auth code, It should not work in development, but it still does sometimes
export async function POST(request: NextRequest) {
// const d = await ensureAuth(request);
// if (!d) {
// return new Response("Unauthorized", { status: 401 });
// }
const res : {context: string, request: string} = await request.json()
try {
const resp = await fetch(`${process.env.BACKEND_BASE_URL}/api/editorai?context=${res.context}&request=${res.request}`);
// this just checks if there are erros I am keeping it commented for you to better understand the important pieces
// if (resp.status !== 200 || !resp.ok) {
// const errorData = await resp.text();
// console.log(errorData);
// return new Response(
// JSON.stringify({ message: "Error in CF function", error: errorData }),
// { status: resp.status },
// );
// }
return new Response(resp.body, { status: 200 });
} catch (error) {
return new Response(`Error, ${error}`)
}
}

4
apps/web/app/api/ensureAuth.ts
View file

@ -1,6 +1,6 @@
import { NextRequest } from "next/server";
import { db } from "../helpers/server/db";
import { sessions, users } from "../helpers/server/db/schema";
import { db } from "../../server/db";
import { sessions, users } from "../../server/db/schema";
import { eq } from "drizzle-orm";
export async function ensureAuth(req: NextRequest) {

8
apps/web/app/api/getCount/route.ts
View file

@ -1,6 +1,6 @@
import { db } from "@/app/helpers/server/db";
import { db } from "@/server/db";
import { and, eq, ne, sql } from "drizzle-orm";
import { sessions, storedContent, users } from "@/app/helpers/server/db/schema";
import { sessions, storedContent, users } from "@/server/db/schema";
import { type NextRequest, NextResponse } from "next/server";
import { ensureAuth } from "../ensureAuth";
@ -20,7 +20,7 @@ export async function GET(req: NextRequest) {
.from(storedContent)
.where(
and(
eq(storedContent.user, session.user.id),
eq(storedContent.userId, session.user.id),
eq(storedContent.type, "twitter-bookmark"),
),
);
@ -32,7 +32,7 @@ export async function GET(req: NextRequest) {
.from(storedContent)
.where(
and(
eq(storedContent.user, session.user.id),
eq(storedContent.userId, session.user.id),
ne(storedContent.type, "twitter-bookmark"),
),
);

4
apps/web/app/api/me/route.ts
View file

@ -1,6 +1,6 @@
import { db } from "@/app/helpers/server/db";
import { db } from "@/server/db";
import { eq } from "drizzle-orm";
import { sessions, users } from "@/app/helpers/server/db/schema";
import { sessions, users } from "@/server/db/schema";
import { type NextRequest, NextResponse } from "next/server";
export const runtime = "edge";

4
apps/web/app/api/spaces/route.ts
View file

@ -1,5 +1,5 @@
import { db } from "@/app/helpers/server/db";
import { sessions, space, users } from "@/app/helpers/server/db/schema";
import { db } from "@/server/db";
import { sessions, space, users } from "@/server/db/schema";
import { eq } from "drizzle-orm";
import { NextRequest, NextResponse } from "next/server";
import { ensureAuth } from "../ensureAuth";

121
apps/web/app/api/store/route.ts
View file

@ -1,121 +0,0 @@
import { db } from "@/app/helpers/server/db";
import { and, eq, sql, inArray } from "drizzle-orm";
import {
contentToSpace,
sessions,
storedContent,
users,
space,
} from "@/app/helpers/server/db/schema";
import { type NextRequest, NextResponse } from "next/server";
import { getMetaData } from "@/app/helpers/lib/get-metadata";
import { ensureAuth } from "../ensureAuth";
export const runtime = "edge";
export async function POST(req: NextRequest) {
const session = await ensureAuth(req);
if (!session) {
return new Response("Unauthorized", { status: 401 });
}
const data = (await req.json()) as {
pageContent: string;
url: string;
spaces?: string[];
};
const metadata = await getMetaData(data.url);
let storeToSpaces = data.spaces;
if (!storeToSpaces) {
storeToSpaces = [];
}
const count = await db
.select({
count: sql<number>`count(*)`.mapWith(Number),
})
.from(storedContent)
.where(
and(
eq(storedContent.user, session.user.id),
eq(storedContent.type, "page"),
),
);
if (count[0]!.count > 100) {
return NextResponse.json(
{ message: "Error", error: "Limit exceeded" },
{ status: 499 },
);
}
const rep = await db
.insert(storedContent)
.values({
content: data.pageContent,
title: metadata.title,
description: metadata.description,
url: data.url,
baseUrl: metadata.baseUrl,
image: metadata.image,
savedAt: new Date(),
user: session.user.id,
})
.returning({ id: storedContent.id });
const id = rep[0]?.id;
if (!id) {
return NextResponse.json(
{ message: "Error", error: "Error in CF function" },
{ status: 500 },
);
}
if (storeToSpaces.length > 0) {
const spaceData = await db
.select()
.from(space)
.where(
and(
inArray(space.name, storeToSpaces ?? []),
eq(space.user, session.user.id),
),
)
.all();
await Promise.all([
spaceData.forEach(async (space) => {
await db
.insert(contentToSpace)
.values({ contentId: id, spaceId: space.id });
}),
]);
}
const res = (await Promise.race([
fetch("https://cf-ai-backend.dhravya.workers.dev/add", {
method: "POST",
headers: {
"X-Custom-Auth-Key": process.env.BACKEND_SECURITY_KEY,
},
body: JSON.stringify({ ...data, user: session.user.email }),
}),
new Promise((_, reject) =>
setTimeout(() => reject(new Error("Request timed out")), 40000),
),
])) as Response;
if (res.status !== 200) {
console.log(res.status, res.statusText);
return NextResponse.json(
{ message: "Error", error: "Error in CF function" },
{ status: 500 },
);
}
return NextResponse.json({ message: "OK", data: "Success" }, { status: 200 });
}

55
apps/web/app/api/telegram/readme.md Normal file
View file

@ -0,0 +1,55 @@
## how telegram bot stuff works
### Let's start with the important bit: authentication.
We wanted to find a good and secure way to authenticate users, or "link their supermemory account" to their telegram account. This was kinda challenging - because the requirements were tight and privacy was a big concern.
1. No personally identifiable information should be stored, except the user's telegram ID and supermemory email.
2. The link should be as simple as a click of a button
3. it should work two-ways: If the user signs in to the website first, or uses the telegram bot first.
4. The user should be able to unlink their account at any time.
5. Should be very, very easy to host the telegram bot.
We started out by trying to mingle with next-auth credentials provider - but that was a dead end. It would _work_, but would be too hard for us to implement and maintain, and would be a very bad user experience (get the token, copy it, paste it, etc).
So we decided to go with a simple, yet secure, way of doing it.
### the solution
Well, the solution is simple af, surprisingly. To meet all these requirements,
First off, we used the `grammy` library to create a telegram bot that works using websockets. (so, it's hosted with the website, and doesn't need a separate server)
Now, let's examine both the flows:
1. User signs in to the website first
2. Saves a bunch of stuff
3. wants to link their telegram account
and...
1. User uses the telegram bot first
2. Saves a bunch of stuff
3. wants to see their stuff in the supermemory account.
What we ended up doing is creating a simple, yet secure way - always require signin through supermemory.ai website.
And if the user comes from the telegram bot, we just redirect them to the website with a token in the URL.
The token.
The token is literally just their telegram ID, but encrypted. We use a simple encryption algorithm to encrypt the telegram ID, and then decrypt it on the website.
Why encryption? Because we don't want any random person to link any telegram account with their user id. The encryption is also interesting, done using an algorithm called [hushh](https://github.com/dhravya/hushh) that I made a while ago. It's simple and secure and all that's really needed is a secret key.
Once the user signs in, we take the decrypted token and link it to their account. And that's it. The user can now use the telegram bot to access their stuff. Because it's on the same codebase on the server side, it's very easy to make database calls and also calls to the cf-ai-backend to generate stuff.
### Natural language generation
I wanted to add this: the bot actually does both - adding content and talking to the user - at the same time.
How tho?
We use function calling in the backend repo smartly to decide what the user's intent would be. So, i can literally send the message "yo, can you remember this? (with anything else, can even be a URL!)" and the bot will understand that it's a command to add content.
orr, i can send "hey, can you tell me about the time i went to the beach?" and the bot will understand that it's a command to get content.
it's pretty cool. function calling using a cheap model works very well.

113
apps/web/app/api/telegram/route.ts Normal file
View file

@ -0,0 +1,113 @@
import { db } from "@/server/db";
import { storedContent, users } from "@/server/db/schema";
import { cipher } from "@/server/encrypt";
import { eq } from "drizzle-orm";
import { Bot, webhookCallback } from "grammy";
import { User } from "grammy/types";
export const runtime = "edge";
if (!process.env.TELEGRAM_BOT_TOKEN) {
throw new Error("TELEGRAM_BOT_TOKEN is not defined");
}
console.log("Telegram bot activated");
const token = process.env.TELEGRAM_BOT_TOKEN;
const bot = new Bot(token);
bot.command("start", async (ctx) => {
const user: User = (await ctx.getAuthor()).user;
const cipherd = cipher(user.id.toString());
await ctx.reply(
`Welcome to Supermemory bot. I am here to help you remember things better. Click here to create and link your account: https://beta.supermemory.ai/signin?telegramUser=${cipherd}`,
);
});
bot.on("message", async (ctx) => {
const user: User = (await ctx.getAuthor()).user;
const cipherd = cipher(user.id.toString());
const dbUser = await db.query.users
.findFirst({
where: eq(users.telegramId, user.id.toString()),
})
.execute();
if (!dbUser) {
await ctx.reply(
`Welcome to Supermemory bot. I am here to help you remember things better. Click here to create and link your account: https://beta.supermemory.ai/signin?telegramUser=${cipherd}`,
);
return;
}
const message = await ctx.reply("I'm thinking...");
const response = await fetch(
`${process.env.BACKEND_BASE_URL}/api/autoChatOrAdd?query=${ctx.message.text}&user=${dbUser.id}`,
{
method: "POST",
headers: {
Authorization: "Bearer " + process.env.BACKEND_SECURITY_KEY,
"Content-Type": "application/json",
},
body: JSON.stringify({
// TODO: we can use the conversations API to get the last 5 messages
// get chatHistory from this conversation.
// Basically the last 5 messages between the user and the assistant.
// In ths form of [{role: 'user' | 'assistant', content: string}]
// https://grammy.dev/plugins/conversations
chatHistory: [],
}),
},
);
if (response.status !== 200) {
console.log("Failed to get response from backend");
console.log(response.status);
console.log(await response.text());
await ctx.reply(
"Sorry, I am not able to process your request at the moment.",
);
return;
}
const data = (await response.json()) as {
status: string;
response: string;
contentAdded: {
type: string;
content: string;
url: string;
};
};
// TODO: we might want to enrich this data with more information
if (data.contentAdded) {
await db
.insert(storedContent)
.values({
content: data.contentAdded.content,
title: `${data.contentAdded.content.slice(0, 30)}... (Added from chatbot)`,
description: "",
url: data.contentAdded.url,
baseUrl: data.contentAdded.url,
image: "",
savedAt: new Date(),
userId: dbUser.id,
type: data.contentAdded.type,
})
.returning({ id: storedContent.id });
}
await ctx.api.editMessageText(ctx.chat.id, message.message_id, data.response);
});
export const POST = webhookCallback(bot, "std/http");
export const GET = async () => {
return new Response("OK", { status: 200 });
};

156
apps/web/app/api/unfirlsite/route.ts Normal file
View file

@ -0,0 +1,156 @@
import { load } from "cheerio";
import { AwsClient } from "aws4fetch";
import type { NextRequest } from "next/server";
import { ensureAuth } from "../ensureAuth";
export const runtime = "edge";
export async function POST(request: NextRequest) {
const r2 = new AwsClient({
accessKeyId: process.env.R2_ACCESS_KEY_ID,
secretAccessKey: process.env.R2_SECRET_ACCESS_KEY,
});
async function unfurl(url: string) {
const response = await fetch(url);
if (response.status >= 400) {
throw new Error(`Error fetching url: ${response.status}`);
}
const contentType = response.headers.get("content-type");
if (!contentType?.includes("text/html")) {
throw new Error(`Content-type not right: ${contentType}`);
}
const content = await response.text();
const $ = load(content);
const og: { [key: string]: string | undefined } = {};
const twitter: { [key: string]: string | undefined } = {};
$("meta[property^=og:]").each(
// @ts-ignore, it just works so why care of type safety if someone has better way go ahead
(_, el) => (og[$(el).attr("property")!] = $(el).attr("content")),
);
$("meta[name^=twitter:]").each(
// @ts-ignore
(_, el) => (twitter[$(el).attr("name")!] = $(el).attr("content")),
);
const title =
og["og:title"] ??
twitter["twitter:title"] ??
$("title").text() ??
undefined;
const description =
og["og:description"] ??
twitter["twitter:description"] ??
$('meta[name="description"]').attr("content") ??
undefined;
const image =
og["og:image:secure_url"] ??
og["og:image"] ??
twitter["twitter:image"] ??
undefined;
return {
title,
description,
image,
};
}
const d = await ensureAuth(request);
if (!d) {
return new Response("Unauthorized", { status: 401 });
}
if (
!process.env.R2_ACCESS_KEY_ID ||
!process.env.R2_ACCOUNT_ID ||
!process.env.R2_SECRET_ACCESS_KEY ||
!process.env.R2_BUCKET_NAME
) {
return new Response(
"Missing one or more R2 env variables: R2_ENDPOINT, R2_ACCESS_ID, R2_SECRET_KEY, R2_BUCKET_NAME. To get them, go to the R2 console, create and paste keys in a `.dev.vars` file in the root of this project.",
{ status: 500 },
);
}
const website = new URL(request.url).searchParams.get("website");
if (!website) {
return new Response("Missing website", { status: 400 });
}
const salt = () => Math.floor(Math.random() * 11);
const encodeWebsite = `${encodeURIComponent(website)}${salt()}`;
try {
// this returns the og image, description and title of website
const response = await unfurl(website);
if (!response.image) {
return new Response(JSON.stringify(response));
}
if (!process.env.DEV_IMAGES) {
return new Response("Missing DEV_IMAGES namespace.", { status: 500 });
}
const imageUrl = await process.env.DEV_IMAGES!.get(encodeWebsite);
if (imageUrl) {
return new Response(
JSON.stringify({
image: imageUrl,
title: response.title,
description: response.description,
}),
);
}
const res = await fetch(`${response.image}`);
const image = await res.blob();
const url = new URL(
`https://${process.env.R2_BUCKET_NAME}.${process.env.R2_ACCOUNT_ID}.r2.cloudflarestorage.com`,
);
url.pathname = encodeWebsite;
url.searchParams.set("X-Amz-Expires", "3600");
const signedPuturl = await r2.sign(
new Request(url, {
method: "PUT",
}),
{
aws: { signQuery: true },
},
);
await fetch(signedPuturl.url, {
method: "PUT",
body: image,
});
await process.env.DEV_IMAGES.put(
encodeWebsite,
`${process.env.R2_PUBLIC_BUCKET_ADDRESS}/${encodeWebsite}`,
);
return new Response(
JSON.stringify({
image: `${process.env.R2_PUBLIC_BUCKET_ADDRESS}/${encodeWebsite}`,
title: response.title,
description: response.description,
}),
);
} catch (error) {
console.log(error);
return new Response(
JSON.stringify({
status: 500,
error: error,
}),
);
}
}

143
apps/web/app/helpers/server/db/schema.ts
View file

@ -1,143 +0,0 @@
import { relations, sql } from "drizzle-orm";
import {
index,
int,
primaryKey,
sqliteTableCreator,
text,
integer,
} from "drizzle-orm/sqlite-core";
export const createTable = sqliteTableCreator((name) => `${name}`);
export const users = createTable("user", {
id: text("id", { length: 255 }).notNull().primaryKey(),
name: text("name", { length: 255 }),
email: text("email", { length: 255 }).notNull(),
emailVerified: int("emailVerified", { mode: "timestamp" }).default(
sql`CURRENT_TIMESTAMP`,
),
image: text("image", { length: 255 }),
});
export type User = typeof users.$inferSelect;
export const usersRelations = relations(users, ({ many }) => ({
accounts: many(accounts),
sessions: many(sessions),
}));
export const accounts = createTable(
"account",
{
id: integer("id").notNull().primaryKey({ autoIncrement: true }),
userId: text("userId", { length: 255 })
.notNull()
.references(() => users.id, { onDelete: "cascade" }),
type: text("type", { length: 255 }).notNull(),
provider: text("provider", { length: 255 }).notNull(),
providerAccountId: text("providerAccountId", { length: 255 }).notNull(),
refresh_token: text("refresh_token"),
access_token: text("access_token"),
expires_at: int("expires_at"),
token_type: text("token_type", { length: 255 }),
scope: text("scope", { length: 255 }),
id_token: text("id_token"),
session_state: text("session_state", { length: 255 }),
oauth_token_secret: text("oauth_token_secret"),
oauth_token: text("oauth_token"),
},
(account) => ({
userIdIdx: index("account_userId_idx").on(account.userId),
}),
);
export const sessions = createTable(
"session",
{
id: integer("id").notNull().primaryKey({ autoIncrement: true }),
sessionToken: text("sessionToken", { length: 255 }).notNull(),
userId: text("userId", { length: 255 })
.notNull()
.references(() => users.id, { onDelete: "cascade" }),
expires: int("expires", { mode: "timestamp" }).notNull(),
},
(session) => ({
userIdIdx: index("session_userId_idx").on(session.userId),
}),
);
export const verificationTokens = createTable(
"verificationToken",
{
identifier: text("identifier", { length: 255 }).notNull(),
token: text("token", { length: 255 }).notNull(),
expires: int("expires", { mode: "timestamp" }).notNull(),
},
(vt) => ({
compoundKey: primaryKey({ columns: [vt.identifier, vt.token] }),
}),
);
export const storedContent = createTable(
"storedContent",
{
id: integer("id").notNull().primaryKey({ autoIncrement: true }),
content: text("content").notNull(),
title: text("title", { length: 255 }),
description: text("description", { length: 255 }),
url: text("url").notNull(),
savedAt: int("savedAt", { mode: "timestamp" }).notNull(),
baseUrl: text("baseUrl", { length: 255 }),
ogImage: text("ogImage", { length: 255 }),
type: text("type", { enum: ["note", "page", "twitter-bookmark"] }).default(
"page",
),
image: text("image", { length: 255 }),
user: text("user", { length: 255 }).references(() => users.id, {
onDelete: "cascade",
}),
},
(sc) => ({
urlIdx: index("storedContent_url_idx").on(sc.url),
savedAtIdx: index("storedContent_savedAt_idx").on(sc.savedAt),
titleInx: index("storedContent_title_idx").on(sc.title),
userIdx: index("storedContent_user_idx").on(sc.user),
}),
);
export const contentToSpace = createTable(
"contentToSpace",
{
contentId: integer("contentId")
.notNull()
.references(() => storedContent.id, { onDelete: "cascade" }),
spaceId: integer("spaceId")
.notNull()
.references(() => space.id, { onDelete: "cascade" }),
},
(cts) => ({
compoundKey: primaryKey({ columns: [cts.contentId, cts.spaceId] }),
}),
);
export const space = createTable(
"space",
{
id: integer("id").notNull().primaryKey({ autoIncrement: true }),
name: text("name").notNull().unique().default("none"),
user: text("user", { length: 255 }).references(() => users.id, {
onDelete: "cascade",
}),
},
(space) => ({
nameIdx: index("spaces_name_idx").on(space.name),
userIdx: index("spaces_user_idx").on(space.user),
}),
);
export type StoredContent = Omit<typeof storedContent.$inferSelect, "user">;
export type StoredSpace = typeof space.$inferSelect;
export type ChachedSpaceContent = StoredContent & {
space: number;
};

4
apps/web/app/layout.tsx
View file

@ -6,6 +6,7 @@ import { GeistSans } from "geist/font/sans";
import { GeistMono } from "geist/font/mono";
import { cn } from "@repo/ui/lib/utils";
import BackgroundPlus from "./(landing)/GridPatterns/PlusGrid";
import { Toaster } from "@repo/ui/shadcn/toaster";
const inter = Inter({ subsets: ["latin"] });
export const runtime = "edge";
@ -77,6 +78,9 @@ export default function RootLayout({
)}
>
{children}
<body className={`${inter.className} dark`}>
{children}
<Toaster />
</body>
</html>
);

8
apps/web/app/ref/page.tsx
View file

@ -1,9 +1,9 @@
import { Button } from "@repo/ui/shadcn/button";
import { auth, signIn, signOut } from "../helpers/server/auth";
import { db } from "../helpers/server/db";
import { auth, signIn, signOut } from "../../server/auth";
import { db } from "../../server/db";
import { sql } from "drizzle-orm";
import { users } from "../helpers/server/db/schema";
import { getThemeToggler } from "../helpers/lib/get-theme-button";
import { users } from "../../server/db/schema";
import { getThemeToggler } from "../../lib/get-theme-button";
export const runtime = "edge";

13
apps/web/cf-env.d.ts vendored
View file

@ -1,6 +1,17 @@
declare global {
namespace NodeJS {
interface ProcessEnv extends CloudflareEnv {}
interface ProcessEnv extends CloudflareEnv {
GOOGLE_CLIENT_ID: string;
GOOGLE_CLIENT_SECRET: string;
AUTH_SECRET: string;
R2_ENDPOINT: string;
R2_ACCESS_KEY_ID: string;
R2_SECRET_ACCESS_KEY: string;
R2_PUBLIC_BUCKET_ADDRESS: string;
R2_BUCKET_NAME: string;
BACKEND_SECURITY_KEY: string;
BACKEND_BASE_URL: string;
}
}
}

2
apps/web/drizzle.config.ts
View file

@ -1,7 +1,7 @@
import { type Config } from "drizzle-kit";
export default {
schema: "./app/helpers/server/db/schema.ts",
schema: "./server/db/schema.ts",
dialect: "sqlite",
driver: "d1",
dbCredentials: {

9
apps/web/env.d.ts vendored
View file

@ -2,14 +2,7 @@
// by running `wrangler types --env-interface CloudflareEnv env.d.ts`
interface CloudflareEnv {
GOOGLE_CLIENT_ID: string;
GOOGLE_CLIENT_SECRET: string;
AUTH_SECRET: string;
R2_ENDPOINT: string;
R2_ACCESS_ID: string;
R2_SECRET_KEY: string;
R2_BUCKET_NAME: string;
BACKEND_SECURITY_KEY: string;
STORAGE: R2Bucket;
DATABASE: D1Database;
DEV_IMAGES: R2Bucket;
}

43
apps/web/lib/constants.ts Normal file
View file

@ -0,0 +1,43 @@
export const LIMITS = {
page: 100,
tweet: 1000,
note: 1000,
};
export const codeLanguageSubset = [
"python",
"javascript",
"java",
"go",
"bash",
"c",
"cpp",
"csharp",
"css",
"diff",
"graphql",
"json",
"kotlin",
"less",
"lua",
"makefile",
"markdown",
"objectivec",
"perl",
"php",
"php-template",
"plaintext",
"python-repl",
"r",
"ruby",
"rust",
"scss",
"shell",
"sql",
"swift",
"typescript",
"vbnet",
"wasm",
"xml",
"yaml",
];

0
apps/web/app/helpers/lib/get-metadata.ts → apps/web/lib/get-metadata.ts
View file

0
apps/web/app/helpers/lib/get-theme-button.tsx → apps/web/lib/get-theme-button.tsx
View file

0
apps/web/app/helpers/lib/handle-errors.ts → apps/web/lib/handle-errors.ts
View file

20
apps/web/app/helpers/lib/searchParams.ts → apps/web/lib/searchParams.ts
View file

@ -16,11 +16,19 @@ export const chatSearchParamsCache = createSearchParamsCache({
firstTime: parseAsBoolean.withDefault(false),
q: parseAsString.withDefault(""),
spaces: parseAsArrayOf(
parseAsJson(() =>
z.object({
id: z.string(),
name: z.string(),
}),
),
parseAsJson((c) => {
const valid = z
.object({
id: z.string(),
name: z.string(),
})
.safeParse(c);
if (!valid.success) {
return null;
}
return valid.data;
}),
).withDefault([]),
});

4
apps/web/migrations/0001_remarkable_avengers.sql Normal file
View file

@ -0,0 +1,4 @@
ALTER TABLE `user` ADD `telegramId` text;--> statement-breakpoint
CREATE INDEX `users_email_idx` ON `user` (`email`);--> statement-breakpoint
CREATE INDEX `users_telegram_idx` ON `user` (`telegramId`);--> statement-breakpoint
CREATE INDEX `users_id_idx` ON `user` (`id`);

74
apps/web/migrations/000_setup.sql
View file

@ -1,18 +1,46 @@
CREATE TABLE `account` (
`id` integer PRIMARY KEY AUTOINCREMENT NOT NULL,
`userId` text(255) NOT NULL,
`type` text(255) NOT NULL,
`provider` text(255) NOT NULL,
`providerAccountId` text(255) NOT NULL,
`userId` text NOT NULL,
`type` text NOT NULL,
`provider` text NOT NULL,
`providerAccountId` text NOT NULL,
`refresh_token` text,
`access_token` text,
`expires_at` integer,
`token_type` text(255),
`scope` text(255),
`token_type` text,
`scope` text,
`id_token` text,
`session_state` text(255),
`oauth_token_secret` text,
`oauth_token` text,
`session_state` text,
PRIMARY KEY(`provider`, `providerAccountId`),
FOREIGN KEY (`userId`) REFERENCES `user`(`id`) ON UPDATE no action ON DELETE cascade
);
--> statement-breakpoint
CREATE TABLE `authenticator` (
`credentialID` text NOT NULL,
`userId` text NOT NULL,
`providerAccountId` text NOT NULL,
`credentialPublicKey` text NOT NULL,
`counter` integer NOT NULL,
`credentialDeviceType` text NOT NULL,
`credentialBackedUp` integer NOT NULL,
`transports` text,
PRIMARY KEY(`credentialID`, `userId`),
FOREIGN KEY (`userId`) REFERENCES `user`(`id`) ON UPDATE no action ON DELETE cascade
);
--> statement-breakpoint
CREATE TABLE `chatHistory` (
`id` integer PRIMARY KEY AUTOINCREMENT NOT NULL,
`threadId` text NOT NULL,
`question` text NOT NULL,
`answerParts` text,
`answerSources` text,
`answerJustification` text,
FOREIGN KEY (`threadId`) REFERENCES `chatThread`(`id`) ON UPDATE no action ON DELETE cascade
);
--> statement-breakpoint
CREATE TABLE `chatThread` (
`id` text PRIMARY KEY NOT NULL,
`firstMessage` text NOT NULL,
`userId` text NOT NULL,
FOREIGN KEY (`userId`) REFERENCES `user`(`id`) ON UPDATE no action ON DELETE cascade
);
--> statement-breakpoint
@ -25,9 +53,8 @@ CREATE TABLE `contentToSpace` (
);
--> statement-breakpoint
CREATE TABLE `session` (
`id` integer PRIMARY KEY AUTOINCREMENT NOT NULL,
`sessionToken` text(255) NOT NULL,
`userId` text(255) NOT NULL,
`sessionToken` text PRIMARY KEY NOT NULL,
`userId` text NOT NULL,
`expires` integer NOT NULL,
FOREIGN KEY (`userId`) REFERENCES `user`(`id`) ON UPDATE no action ON DELETE cascade
);
@ -50,27 +77,28 @@ CREATE TABLE `storedContent` (
`ogImage` text(255),
`type` text DEFAULT 'page',
`image` text(255),
`user` text(255),
`user` text,
FOREIGN KEY (`user`) REFERENCES `user`(`id`) ON UPDATE no action ON DELETE cascade
);
--> statement-breakpoint
CREATE TABLE `user` (
`id` text(255) PRIMARY KEY NOT NULL,
`name` text(255),
`email` text(255) NOT NULL,
`emailVerified` integer DEFAULT CURRENT_TIMESTAMP,
`image` text(255)
`id` text PRIMARY KEY NOT NULL,
`name` text,
`email` text NOT NULL,
`emailVerified` integer,
`image` text
);
--> statement-breakpoint
CREATE TABLE `verificationToken` (
`identifier` text(255) NOT NULL,
`token` text(255) NOT NULL,
`identifier` text NOT NULL,
`token` text NOT NULL,
`expires` integer NOT NULL,
PRIMARY KEY(`identifier`, `token`)
);
--> statement-breakpoint
CREATE INDEX `account_userId_idx` ON `account` (`userId`);--> statement-breakpoint
CREATE INDEX `session_userId_idx` ON `session` (`userId`);--> statement-breakpoint
CREATE UNIQUE INDEX `authenticator_credentialID_unique` ON `authenticator` (`credentialID`);--> statement-breakpoint
CREATE INDEX `chatHistory_thread_idx` ON `chatHistory` (`threadId`);--> statement-breakpoint
CREATE INDEX `chatThread_user_idx` ON `chatThread` (`userId`);--> statement-breakpoint
CREATE UNIQUE INDEX `space_name_unique` ON `space` (`name`);--> statement-breakpoint
CREATE INDEX `spaces_name_idx` ON `space` (`name`);--> statement-breakpoint
CREATE INDEX `spaces_user_idx` ON `space` (`user`);--> statement-breakpoint

271
apps/web/migrations/meta/0000_snapshot.json
View file

@ -1,43 +1,36 @@
{
"version": "6",
"dialect": "sqlite",
"id": "409cec60-0c4b-4cda-8751-3e70768bbb6c",
"id": "349eea0d-f26e-4579-9c65-3982816b0c6c",
"prevId": "00000000-0000-0000-0000-000000000000",
"tables": {
"account": {
"name": "account",
"columns": {
"id": {
"name": "id",
"type": "integer",
"primaryKey": true,
"notNull": true,
"autoincrement": true
},
"userId": {
"name": "userId",
"type": "text(255)",
"type": "text",
"primaryKey": false,
"notNull": true,
"autoincrement": false
},
"type": {
"name": "type",
"type": "text(255)",
"type": "text",
"primaryKey": false,
"notNull": true,
"autoincrement": false
},
"provider": {
"name": "provider",
"type": "text(255)",
"type": "text",
"primaryKey": false,
"notNull": true,
"autoincrement": false
},
"providerAccountId": {
"name": "providerAccountId",
"type": "text(255)",
"type": "text",
"primaryKey": false,
"notNull": true,
"autoincrement": false
@ -65,14 +58,14 @@
},
"token_type": {
"name": "token_type",
"type": "text(255)",
"type": "text",
"primaryKey": false,
"notNull": false,
"autoincrement": false
},
"scope": {
"name": "scope",
"type": "text(255)",
"type": "text",
"primaryKey": false,
"notNull": false,
"autoincrement": false
@ -86,20 +79,86 @@
},
"session_state": {
"name": "session_state",
"type": "text(255)",
"primaryKey": false,
"notNull": false,
"autoincrement": false
},
"oauth_token_secret": {
"name": "oauth_token_secret",
"type": "text",
"primaryKey": false,
"notNull": false,
"autoincrement": false
}
},
"indexes": {},
"foreignKeys": {
"account_userId_user_id_fk": {
"name": "account_userId_user_id_fk",
"tableFrom": "account",
"tableTo": "user",
"columnsFrom": ["userId"],
"columnsTo": ["id"],
"onDelete": "cascade",
"onUpdate": "no action"
}
},
"compositePrimaryKeys": {
"account_provider_providerAccountId_pk": {
"columns": ["provider", "providerAccountId"],
"name": "account_provider_providerAccountId_pk"
}
},
"uniqueConstraints": {}
},
"authenticator": {
"name": "authenticator",
"columns": {
"credentialID": {
"name": "credentialID",
"type": "text",
"primaryKey": false,
"notNull": true,
"autoincrement": false
},
"oauth_token": {
"name": "oauth_token",
"userId": {
"name": "userId",
"type": "text",
"primaryKey": false,
"notNull": true,
"autoincrement": false
},
"providerAccountId": {
"name": "providerAccountId",
"type": "text",
"primaryKey": false,
"notNull": true,
"autoincrement": false
},
"credentialPublicKey": {
"name": "credentialPublicKey",
"type": "text",
"primaryKey": false,
"notNull": true,
"autoincrement": false
},
"counter": {
"name": "counter",
"type": "integer",
"primaryKey": false,
"notNull": true,
"autoincrement": false
},
"credentialDeviceType": {
"name": "credentialDeviceType",
"type": "text",
"primaryKey": false,
"notNull": true,
"autoincrement": false
},
"credentialBackedUp": {
"name": "credentialBackedUp",
"type": "integer",
"primaryKey": false,
"notNull": true,
"autoincrement": false
},
"transports": {
"name": "transports",
"type": "text",
"primaryKey": false,
"notNull": false,
@ -107,16 +166,134 @@
}
},
"indexes": {
"account_userId_idx": {
"name": "account_userId_idx",
"authenticator_credentialID_unique": {
"name": "authenticator_credentialID_unique",
"columns": ["credentialID"],
"isUnique": true
}
},
"foreignKeys": {
"authenticator_userId_user_id_fk": {
"name": "authenticator_userId_user_id_fk",
"tableFrom": "authenticator",
"tableTo": "user",
"columnsFrom": ["userId"],
"columnsTo": ["id"],
"onDelete": "cascade",
"onUpdate": "no action"
}
},
"compositePrimaryKeys": {
"authenticator_userId_credentialID_pk": {
"columns": ["credentialID", "userId"],
"name": "authenticator_userId_credentialID_pk"
}
},
"uniqueConstraints": {}
},
"chatHistory": {
"name": "chatHistory",
"columns": {
"id": {
"name": "id",
"type": "integer",
"primaryKey": true,
"notNull": true,
"autoincrement": true
},
"threadId": {
"name": "threadId",
"type": "text",
"primaryKey": false,
"notNull": true,
"autoincrement": false
},
"question": {
"name": "question",
"type": "text",
"primaryKey": false,
"notNull": true,
"autoincrement": false
},
"answerParts": {
"name": "answerParts",
"type": "text",
"primaryKey": false,
"notNull": false,
"autoincrement": false
},
"answerSources": {
"name": "answerSources",
"type": "text",
"primaryKey": false,
"notNull": false,
"autoincrement": false
},
"answerJustification": {
"name": "answerJustification",
"type": "text",
"primaryKey": false,
"notNull": false,
"autoincrement": false
}
},
"indexes": {
"chatHistory_thread_idx": {
"name": "chatHistory_thread_idx",
"columns": ["threadId"],
"isUnique": false
}
},
"foreignKeys": {
"chatHistory_threadId_chatThread_id_fk": {
"name": "chatHistory_threadId_chatThread_id_fk",
"tableFrom": "chatHistory",
"tableTo": "chatThread",
"columnsFrom": ["threadId"],
"columnsTo": ["id"],
"onDelete": "cascade",
"onUpdate": "no action"
}
},
"compositePrimaryKeys": {},
"uniqueConstraints": {}
},
"chatThread": {
"name": "chatThread",
"columns": {
"id": {
"name": "id",
"type": "text",
"primaryKey": true,
"notNull": true,
"autoincrement": false
},
"firstMessage": {
"name": "firstMessage",
"type": "text",
"primaryKey": false,
"notNull": true,
"autoincrement": false
},
"userId": {
"name": "userId",
"type": "text",
"primaryKey": false,
"notNull": true,
"autoincrement": false
}
},
"indexes": {
"chatThread_user_idx": {
"name": "chatThread_user_idx",
"columns": ["userId"],
"isUnique": false
}
},
"foreignKeys": {
"account_userId_user_id_fk": {
"name": "account_userId_user_id_fk",
"tableFrom": "account",
"chatThread_userId_user_id_fk": {
"name": "chatThread_userId_user_id_fk",
"tableFrom": "chatThread",
"tableTo": "user",
"columnsFrom": ["userId"],
"columnsTo": ["id"],
@ -177,23 +354,16 @@
"session": {
"name": "session",
"columns": {
"id": {
"name": "id",
"type": "integer",
"primaryKey": true,
"notNull": true,
"autoincrement": true
},
"sessionToken": {
"name": "sessionToken",
"type": "text(255)",
"primaryKey": false,
"type": "text",
"primaryKey": true,
"notNull": true,
"autoincrement": false
},
"userId": {
"name": "userId",
"type": "text(255)",
"type": "text",
"primaryKey": false,
"notNull": true,
"autoincrement": false
@ -206,13 +376,7 @@
"autoincrement": false
}
},
"indexes": {
"session_userId_idx": {
"name": "session_userId_idx",
"columns": ["userId"],
"isUnique": false
}
},
"indexes": {},
"foreignKeys": {
"session_userId_user_id_fk": {
"name": "session_userId_user_id_fk",
@ -360,7 +524,7 @@
},
"user": {
"name": "user",
"type": "text(255)",
"type": "text",
"primaryKey": false,
"notNull": false,
"autoincrement": false
@ -407,21 +571,21 @@
"columns": {
"id": {
"name": "id",
"type": "text(255)",
"type": "text",
"primaryKey": true,
"notNull": true,
"autoincrement": false
},
"name": {
"name": "name",
"type": "text(255)",
"type": "text",
"primaryKey": false,
"notNull": false,
"autoincrement": false
},
"email": {
"name": "email",
"type": "text(255)",
"type": "text",
"primaryKey": false,
"notNull": true,
"autoincrement": false
@ -431,12 +595,11 @@
"type": "integer",
"primaryKey": false,
"notNull": false,
"autoincrement": false,
"default": "CURRENT_TIMESTAMP"
"autoincrement": false
},
"image": {
"name": "image",
"type": "text(255)",
"type": "text",
"primaryKey": false,
"notNull": false,
"autoincrement": false
@ -452,14 +615,14 @@
"columns": {
"identifier": {
"name": "identifier",
"type": "text(255)",
"type": "text",
"primaryKey": false,
"notNull": true,
"autoincrement": false
},
"token": {
"name": "token",
"type": "text(255)",
"type": "text",
"primaryKey": false,
"notNull": true,
"autoincrement": false

678
apps/web/migrations/meta/0001_snapshot.json Normal file
View file

@ -0,0 +1,678 @@
{
"version": "6",
"dialect": "sqlite",
"id": "d1588d98-1bac-46c3-a646-868cd56461b1",
"prevId": "349eea0d-f26e-4579-9c65-3982816b0c6c",
"tables": {
"account": {
"name": "account",
"columns": {
"userId": {
"name": "userId",
"type": "text",
"primaryKey": false,
"notNull": true,
"autoincrement": false
},
"type": {
"name": "type",
"type": "text",
"primaryKey": false,
"notNull": true,
"autoincrement": false
},
"provider": {
"name": "provider",
"type": "text",
"primaryKey": false,
"notNull": true,
"autoincrement": false
},
"providerAccountId": {
"name": "providerAccountId",
"type": "text",
"primaryKey": false,
"notNull": true,
"autoincrement": false
},
"refresh_token": {
"name": "refresh_token",
"type": "text",
"primaryKey": false,
"notNull": false,
"autoincrement": false
},
"access_token": {
"name": "access_token",
"type": "text",
"primaryKey": false,
"notNull": false,
"autoincrement": false
},
"expires_at": {
"name": "expires_at",
"type": "integer",
"primaryKey": false,
"notNull": false,
"autoincrement": false
},
"token_type": {
"name": "token_type",
"type": "text",
"primaryKey": false,
"notNull": false,
"autoincrement": false
},
"scope": {
"name": "scope",
"type": "text",
"primaryKey": false,
"notNull": false,
"autoincrement": false
},
"id_token": {
"name": "id_token",
"type": "text",
"primaryKey": false,
"notNull": false,
"autoincrement": false
},
"session_state": {
"name": "session_state",
"type": "text",
"primaryKey": false,
"notNull": false,
"autoincrement": false
}
},
"indexes": {},
"foreignKeys": {
"account_userId_user_id_fk": {
"name": "account_userId_user_id_fk",
"tableFrom": "account",
"tableTo": "user",
"columnsFrom": ["userId"],
"columnsTo": ["id"],
"onDelete": "cascade",
"onUpdate": "no action"
}
},
"compositePrimaryKeys": {
"account_provider_providerAccountId_pk": {
"columns": ["provider", "providerAccountId"],
"name": "account_provider_providerAccountId_pk"
}
},
"uniqueConstraints": {}
},
"authenticator": {
"name": "authenticator",
"columns": {
"credentialID": {
"name": "credentialID",
"type": "text",
"primaryKey": false,
"notNull": true,
"autoincrement": false
},
"userId": {
"name": "userId",
"type": "text",
"primaryKey": false,
"notNull": true,
"autoincrement": false
},
"providerAccountId": {
"name": "providerAccountId",
"type": "text",
"primaryKey": false,
"notNull": true,
"autoincrement": false
},
"credentialPublicKey": {
"name": "credentialPublicKey",
"type": "text",
"primaryKey": false,
"notNull": true,
"autoincrement": false
},
"counter": {
"name": "counter",
"type": "integer",
"primaryKey": false,
"notNull": true,
"autoincrement": false
},
"credentialDeviceType": {
"name": "credentialDeviceType",
"type": "text",
"primaryKey": false,
"notNull": true,
"autoincrement": false
},
"credentialBackedUp": {
"name": "credentialBackedUp",
"type": "integer",
"primaryKey": false,
"notNull": true,
"autoincrement": false
},
"transports": {
"name": "transports",
"type": "text",
"primaryKey": false,
"notNull": false,
"autoincrement": false
}
},
"indexes": {
"authenticator_credentialID_unique": {
"name": "authenticator_credentialID_unique",
"columns": ["credentialID"],
"isUnique": true
}
},
"foreignKeys": {
"authenticator_userId_user_id_fk": {
"name": "authenticator_userId_user_id_fk",
"tableFrom": "authenticator",
"tableTo": "user",
"columnsFrom": ["userId"],
"columnsTo": ["id"],
"onDelete": "cascade",
"onUpdate": "no action"
}
},
"compositePrimaryKeys": {
"authenticator_userId_credentialID_pk": {
"columns": ["credentialID", "userId"],
"name": "authenticator_userId_credentialID_pk"
}
},
"uniqueConstraints": {}
},
"chatHistory": {
"name": "chatHistory",
"columns": {
"id": {
"name": "id",
"type": "integer",
"primaryKey": true,
"notNull": true,
"autoincrement": true
},
"threadId": {
"name": "threadId",
"type": "text",
"primaryKey": false,
"notNull": true,
"autoincrement": false
},
"question": {
"name": "question",
"type": "text",
"primaryKey": false,
"notNull": true,
"autoincrement": false
},
"answerParts": {
"name": "answerParts",
"type": "text",
"primaryKey": false,
"notNull": false,
"autoincrement": false
},
"answerSources": {
"name": "answerSources",
"type": "text",
"primaryKey": false,
"notNull": false,
"autoincrement": false
},
"answerJustification": {
"name": "answerJustification",
"type": "text",
"primaryKey": false,
"notNull": false,
"autoincrement": false
}
},
"indexes": {
"chatHistory_thread_idx": {
"name": "chatHistory_thread_idx",
"columns": ["threadId"],
"isUnique": false
}
},
"foreignKeys": {
"chatHistory_threadId_chatThread_id_fk": {
"name": "chatHistory_threadId_chatThread_id_fk",
"tableFrom": "chatHistory",
"tableTo": "chatThread",
"columnsFrom": ["threadId"],
"columnsTo": ["id"],
"onDelete": "cascade",
"onUpdate": "no action"
}
},
"compositePrimaryKeys": {},
"uniqueConstraints": {}
},
"chatThread": {
"name": "chatThread",
"columns": {
"id": {
"name": "id",
"type": "text",
"primaryKey": true,
"notNull": true,
"autoincrement": false
},
"firstMessage": {
"name": "firstMessage",
"type": "text",
"primaryKey": false,
"notNull": true,
"autoincrement": false
},
"userId": {
"name": "userId",
"type": "text",
"primaryKey": false,
"notNull": true,
"autoincrement": false
}
},
"indexes": {
"chatThread_user_idx": {
"name": "chatThread_user_idx",
"columns": ["userId"],
"isUnique": false
}
},
"foreignKeys": {
"chatThread_userId_user_id_fk": {
"name": "chatThread_userId_user_id_fk",
"tableFrom": "chatThread",
"tableTo": "user",
"columnsFrom": ["userId"],
"columnsTo": ["id"],
"onDelete": "cascade",
"onUpdate": "no action"
}
},
"compositePrimaryKeys": {},
"uniqueConstraints": {}
},
"contentToSpace": {
"name": "contentToSpace",
"columns": {
"contentId": {
"name": "contentId",
"type": "integer",
"primaryKey": false,
"notNull": true,
"autoincrement": false
},
"spaceId": {
"name": "spaceId",
"type": "integer",
"primaryKey": false,
"notNull": true,
"autoincrement": false
}
},
"indexes": {},
"foreignKeys": {
"contentToSpace_contentId_storedContent_id_fk": {
"name": "contentToSpace_contentId_storedContent_id_fk",
"tableFrom": "contentToSpace",
"tableTo": "storedContent",
"columnsFrom": ["contentId"],
"columnsTo": ["id"],
"onDelete": "cascade",
"onUpdate": "no action"
},
"contentToSpace_spaceId_space_id_fk": {
"name": "contentToSpace_spaceId_space_id_fk",
"tableFrom": "contentToSpace",
"tableTo": "space",
"columnsFrom": ["spaceId"],
"columnsTo": ["id"],
"onDelete": "cascade",
"onUpdate": "no action"
}
},
"compositePrimaryKeys": {
"contentToSpace_contentId_spaceId_pk": {
"columns": ["contentId", "spaceId"],
"name": "contentToSpace_contentId_spaceId_pk"
}
},
"uniqueConstraints": {}
},
"session": {
"name": "session",
"columns": {
"sessionToken": {
"name": "sessionToken",
"type": "text",
"primaryKey": true,
"notNull": true,
"autoincrement": false
},
"userId": {
"name": "userId",
"type": "text",
"primaryKey": false,
"notNull": true,
"autoincrement": false
},
"expires": {
"name": "expires",
"type": "integer",
"primaryKey": false,
"notNull": true,
"autoincrement": false
}
},
"indexes": {},
"foreignKeys": {
"session_userId_user_id_fk": {
"name": "session_userId_user_id_fk",
"tableFrom": "session",
"tableTo": "user",
"columnsFrom": ["userId"],
"columnsTo": ["id"],
"onDelete": "cascade",
"onUpdate": "no action"
}
},
"compositePrimaryKeys": {},
"uniqueConstraints": {}
},
"space": {
"name": "space",
"columns": {
"id": {
"name": "id",
"type": "integer",
"primaryKey": true,
"notNull": true,
"autoincrement": true
},
"name": {
"name": "name",
"type": "text",
"primaryKey": false,
"notNull": true,
"autoincrement": false,
"default": "'none'"
},
"user": {
"name": "user",
"type": "text(255)",
"primaryKey": false,
"notNull": false,
"autoincrement": false
}
},
"indexes": {
"space_name_unique": {
"name": "space_name_unique",
"columns": ["name"],
"isUnique": true
},
"spaces_name_idx": {
"name": "spaces_name_idx",
"columns": ["name"],
"isUnique": false
},
"spaces_user_idx": {
"name": "spaces_user_idx",
"columns": ["user"],
"isUnique": false
}
},
"foreignKeys": {
"space_user_user_id_fk": {
"name": "space_user_user_id_fk",
"tableFrom": "space",
"tableTo": "user",
"columnsFrom": ["user"],
"columnsTo": ["id"],
"onDelete": "cascade",
"onUpdate": "no action"
}
},
"compositePrimaryKeys": {},
"uniqueConstraints": {}
},
"storedContent": {
"name": "storedContent",
"columns": {
"id": {
"name": "id",
"type": "integer",
"primaryKey": true,
"notNull": true,
"autoincrement": true
},
"content": {
"name": "content",
"type": "text",
"primaryKey": false,
"notNull": true,
"autoincrement": false
},
"title": {
"name": "title",
"type": "text(255)",
"primaryKey": false,
"notNull": false,
"autoincrement": false
},
"description": {
"name": "description",
"type": "text(255)",
"primaryKey": false,
"notNull": false,
"autoincrement": false
},
"url": {
"name": "url",
"type": "text",
"primaryKey": false,
"notNull": true,
"autoincrement": false
},
"savedAt": {
"name": "savedAt",
"type": "integer",
"primaryKey": false,
"notNull": true,
"autoincrement": false
},
"baseUrl": {
"name": "baseUrl",
"type": "text(255)",
"primaryKey": false,
"notNull": false,
"autoincrement": false
},
"ogImage": {
"name": "ogImage",
"type": "text(255)",
"primaryKey": false,
"notNull": false,
"autoincrement": false
},
"type": {
"name": "type",
"type": "text",
"primaryKey": false,
"notNull": false,
"autoincrement": false,
"default": "'page'"
},
"image": {
"name": "image",
"type": "text(255)",
"primaryKey": false,
"notNull": false,
"autoincrement": false
},
"user": {
"name": "user",
"type": "text",
"primaryKey": false,
"notNull": false,
"autoincrement": false
}
},
"indexes": {
"storedContent_url_idx": {
"name": "storedContent_url_idx",
"columns": ["url"],
"isUnique": false
},
"storedContent_savedAt_idx": {
"name": "storedContent_savedAt_idx",
"columns": ["savedAt"],
"isUnique": false
},
"storedContent_title_idx": {
"name": "storedContent_title_idx",
"columns": ["title"],
"isUnique": false
},
"storedContent_user_idx": {
"name": "storedContent_user_idx",
"columns": ["user"],
"isUnique": false
}
},
"foreignKeys": {
"storedContent_user_user_id_fk": {
"name": "storedContent_user_user_id_fk",
"tableFrom": "storedContent",
"tableTo": "user",
"columnsFrom": ["user"],
"columnsTo": ["id"],
"onDelete": "cascade",
"onUpdate": "no action"
}
},
"compositePrimaryKeys": {},
"uniqueConstraints": {}
},
"user": {
"name": "user",
"columns": {
"id": {
"name": "id",
"type": "text",
"primaryKey": true,
"notNull": true,
"autoincrement": false
},
"name": {
"name": "name",
"type": "text",
"primaryKey": false,
"notNull": false,
"autoincrement": false
},
"email": {
"name": "email",
"type": "text",
"primaryKey": false,
"notNull": true,
"autoincrement": false
},
"emailVerified": {
"name": "emailVerified",
"type": "integer",
"primaryKey": false,
"notNull": false,
"autoincrement": false
},
"image": {
"name": "image",
"type": "text",
"primaryKey": false,
"notNull": false,
"autoincrement": false
},
"telegramId": {
"name": "telegramId",
"type": "text",
"primaryKey": false,
"notNull": false,
"autoincrement": false
}
},
"indexes": {
"users_email_idx": {
"name": "users_email_idx",
"columns": ["email"],
"isUnique": false
},
"users_telegram_idx": {
"name": "users_telegram_idx",
"columns": ["telegramId"],
"isUnique": false
},
"users_id_idx": {
"name": "users_id_idx",
"columns": ["id"],
"isUnique": false
}
},
"foreignKeys": {},
"compositePrimaryKeys": {},
"uniqueConstraints": {}
},
"verificationToken": {
"name": "verificationToken",
"columns": {
"identifier": {
"name": "identifier",
"type": "text",
"primaryKey": false,
"notNull": true,
"autoincrement": false
},
"token": {
"name": "token",
"type": "text",
"primaryKey": false,
"notNull": true,
"autoincrement": false
},
"expires": {
"name": "expires",
"type": "integer",
"primaryKey": false,
"notNull": true,
"autoincrement": false
}
},
"indexes": {},
"foreignKeys": {},
"compositePrimaryKeys": {
"verificationToken_identifier_token_pk": {
"columns": ["identifier", "token"],
"name": "verificationToken_identifier_token_pk"
}
},
"uniqueConstraints": {}
}
},
"enums": {},
"_meta": {
"schemas": {},
"tables": {},
"columns": {}
}
}

11
apps/web/migrations/meta/_journal.json
View file

@ -5,8 +5,15 @@
{
"idx": 0,
"version": "6",
"when": 1716677954608,
"tag": "0000_calm_monster_badoon",
"when": 1719075265633,
"tag": "0000_conscious_arachne",
"breakpoints": true
},
{
"idx": 1,
"version": "6",
"when": 1719181427523,
"tag": "0001_remarkable_avengers",
"breakpoints": true
}
]

12
apps/web/next.config.mjs
View file

@ -4,14 +4,10 @@ import { setupDevPlatform } from "@cloudflare/next-on-pages/next-dev";
/** @type {import('next').NextConfig} */
const nextConfig = {
transpilePackages: ["@repo/ui"],
// images: {
// remotePatterns: [
// {
// protocol: "https",
// hostname: "github.com",
// },
// ],
// },
reactStrictMode: false,
env: {
TELEGRAM_BOT_TOKEN: process.env.TELEGRAM_BOT_TOKEN,
},
};
export default MillionLint.next({
rsc: true,

15
apps/web/package.json
View file

@ -2,26 +2,33 @@
"name": "@repo/web",
"version": "1.0.0",
"private": true,
"packageManager": "yarn@1.22.21",
"scripts": {
"dev": "next dev",
"build": "next build",
"start": "next start",
"lint": "eslint . --max-warnings 0",
"cf-typegen": "wrangler types --env-interface CloudflareEnv env.d.ts",
"pages:build": "bunx @cloudflare/next-on-pages",
"preview": "bun pages:build && wrangler pages dev",
"deploy": "bun pages:build && wrangler pages deploy"
"pages:build": "npx @cloudflare/next-on-pages",
"preview": "npm run pages:build && wrangler pages dev",
"deploy": "npm run pages:build && wrangler pages deploy",
"schema-update": "bunx drizzle-kit generate sqlite",
"update-local-db": "bunx wrangler d1 execute dev-d1-anycontext --local"
},
"dependencies": {
"@million/lint": "^1.0.0-rc.11",
"@radix-ui/react-dialog": "^1.0.5",
"@radix-ui/react-popover": "^1.0.7",
"cmdk": "^1.0.0",
"lowlight": "^3.1.0",
"million": "^3.1.6",
"next": "^14.1.1",
"novel": "^0.4.2",
"nuqs": "^1.17.4",
"react": "^18.2.0",
"react-dom": "^18.2.0"
"react-dom": "^18.2.0",
"react-resizable-panels": "^2.0.19",
"use-debounce": "^10.0.1"
},
"devDependencies": {
"@next/eslint-plugin-next": "^14.1.1",

BIN
apps/web/public/embed-icons/codepen.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 2.1 KiB

BIN
apps/web/public/embed-icons/codesandbox.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 237 B

BIN
apps/web/public/embed-icons/desmos.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 1.4 KiB

BIN
apps/web/public/embed-icons/excalidraw.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 846 B

BIN
apps/web/public/embed-icons/felt.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 977 B

BIN
apps/web/public/embed-icons/figma.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 1.3 KiB

BIN
apps/web/public/embed-icons/github_gist.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 1.4 KiB

BIN
apps/web/public/embed-icons/google_calendar.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 962 B

BIN
apps/web/public/embed-icons/google_maps.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 1.9 KiB

BIN
apps/web/public/embed-icons/google_slides.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 1.4 KiB

BIN
apps/web/public/embed-icons/observable.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 769 B

BIN
apps/web/public/embed-icons/replit.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 526 B

BIN
apps/web/public/embed-icons/scratch.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 3.6 KiB

BIN
apps/web/public/embed-icons/spotify.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 2.2 KiB

BIN
apps/web/public/embed-icons/tldraw.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 625 B

BIN
apps/web/public/embed-icons/val_town.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 540 B

BIN
apps/web/public/embed-icons/vimeo.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 864 B

BIN
apps/web/public/embed-icons/youtube.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 846 B

Some files were not shown because too many files have changed in this diff Show more