Commit graph

32 commits

Author SHA1 Message Date
Mouez
443304642b
feat: add Nextcloud connector (#19)
Some checks failed
Build Docker Image / build (linux/amd64) (push) Has been cancelled
Build Docker Image / build (linux/arm64) (push) Has been cancelled
Build Docker Image / merge (push) Has been cancelled
* feat: add Nextcloud connector

* refactor: extract internal WebDAV client

* fix: guard recursive WebDAV walk
2026-06-12 16:22:47 +01:00
Tim Baek
bebc3a2b59
Merge pull request #17 from expilu/patch-1
Change deprecated JIRA API endpoint to the current one for JQL
2026-06-12 10:22:02 -05:00
Daniel López Lacalle
fb7c13ec6a
Change JIRA API endpoint to use JQL
/rest/api/3/search was deprecated, changes to the current /rest/api/3/search/jql
2026-06-03 12:21:03 +01:00
Timothy Jaeryang Baek
dcf241da7f feat: SharePoint certificate-based authentication (#3)
Add SHAREPOINT_CERTIFICATE_PATH as a secure alternative to
SHAREPOINT_CLIENT_SECRET for authenticating with SharePoint via
Microsoft Graph API.

- JWT assertion flow using cryptography + PyJWT
- SHAREPOINT_CERTIFICATE_PASSWORD for encrypted PEM keys
- Mutual exclusivity validation (secret vs cert)
- New optional extra: pip install oikb[sharepoint-cert]

Closes #3
2026-05-28 16:24:57 -05:00
Timothy Jaeryang Baek
04ccb30d72 0.3.5: minimal status dashboard at GET /
2KB inline HTML page, monospace, dark background. Polls /health
every 10s and renders source statuses with colored dots, last
sync time, duration, file counts. Zero dependencies.
2026-05-21 13:53:05 +04:00
Timothy Jaeryang Baek
597bb6c9d9 0.3.4: validate --deep, fix stale docker-compose.yaml
validate --deep pings Open WebUI API to verify connectivity, API key
validity, and that each KB ID exists. Catches config errors before
deployment. Also applies defaults and env var interpolation in
validate (was missing).

docker-compose.yaml updated from stale watch mode to production
daemon mode with healthcheck, LOG_FORMAT, and OIKB_API_KEY.
2026-05-21 13:47:28 +04:00
Timothy Jaeryang Baek
fe3a94961f docs: comprehensive guide with tutorial, deployment, and reference
docs/guide.md covers installation, basic usage, config file,
all 44 connectors, filtering, daemon mode, scheduling, API,
enterprise features (metrics, logging, notifications, locking),
deployment (Docker, Compose, GHA, k8s), CLI reference, and
troubleshooting. Linked from README.
2026-05-21 13:35:46 +04:00
Timothy Jaeryang Baek
72d85d1eb7 readme: split quick start into simple (one-liner) and advanced (yaml/daemon)
Simple path: pip install, export env vars, oikb sync ./docs --kb-id
Advanced path: oikb init → oikb daemon. No yaml needed for basic use.
2026-05-21 13:30:48 +04:00
Timothy Jaeryang Baek
9d0a6ae0c0 0.3.3: oikb init interactive wizard
Generates .oikb.yaml by prompting for source, KB ID, name, and
interval. Outputs next-step commands (validate, sync, daemon).
Updated README Quick Start to lead with init.
2026-05-21 13:30:14 +04:00
Timothy Jaeryang Baek
878e797b5f 0.3.2: global defaults key, Docker usage docs
defaults: key in .oikb.yaml sets interval, concurrency, filter,
notify, etc. once for all entries — per-entry values override.
Deep merges nested dicts. Added standalone docker run example
and LOG_FORMAT=json to Docker Compose.
2026-05-21 13:17:23 +04:00
Timothy Jaeryang Baek
4e047b3307 0.3.1: sync locking, dry-run API, env var interpolation
- Per-KB asyncio.Lock prevents overlapping syncs from scheduler,
  webhook, and API triggers running simultaneously
- POST /sync/{id}?dry_run=true previews changes without uploading
- ${VAR} and ${VAR:-default} interpolation in all .oikb.yaml strings
  for GitOps workflows where secrets come from the runtime
2026-05-21 13:11:52 +04:00
Timothy Jaeryang Baek
3a8396e2c0 0.3.0: webhook failure notifications
notify key in .oikb.yaml entries POSTs JSON to any HTTP endpoint on
sync error. Includes a text field for native Slack incoming webhook
support. Configurable: on: error (default) or on: always.
2026-05-21 13:02:19 +04:00
Timothy Jaeryang Baek
e1dd791801 0.2.9: max-size filter to skip oversized files
filter.max-size in .oikb.yaml (or --max-file-size CLI flag) skips
files exceeding the limit before they reach the diff endpoint.
Supports human-readable sizes: 50mb, 1gb, 500kb. Warns per file.
2026-05-21 12:59:08 +04:00
Timothy Jaeryang Baek
beb5f4ec43 0.2.8: structured JSON logging for production deployments
oikb daemon --log-format json (or LOG_FORMAT=json env var) outputs
one JSON object per line with ts, level, logger, msg, and optional
sync-specific fields. Compatible with Datadog, Splunk, ELK,
CloudWatch, and Loki.
2026-05-21 12:53:28 +04:00
Timothy Jaeryang Baek
2aa721147b 0.2.7: cron expression support for daemon scheduling
interval key now accepts both simple intervals (30m, 1h) and
standard cron expressions (0 6 * * 1-5). Auto-detected based on
field count — no config flag needed. Adds croniter>=2.0 dep.
2026-05-21 12:48:58 +04:00
Timothy Jaeryang Baek
6efcee9af8 0.2.6: Prometheus metrics endpoint for daemon observability
GET /metrics exposes sync counters (total, errors), duration
histogram, and file upload/delete counters — all labeled by source.
Includes oikb_info with build version. Adds prometheus-client dep.
2026-05-21 12:43:19 +04:00
Timothy Jaeryang Baek
dae21fa4de 0.2.5: rich progress bars for sync operations
Spinner during scan/diff phases, progress bar with file count and
elapsed time during uploads. Output goes to stderr, respects --quiet
(daemon) and --verbose modes. Adds rich>=13.0 as core dependency.
2026-05-21 12:39:33 +04:00
Timothy Jaeryang Baek
3e78291231 0.2.4: validate command, concurrent uploads, retry, daemon filters, cleanup
- oikb validate: check .oikb.yaml without running syncs
- --concurrency flag for opt-in parallel uploads (yaml: concurrency key)
- 3x retry with exponential backoff on 5xx upload failures
- Daemon now applies filter.include/filter.exclude (was silently ignored)
- Connectors properly closed after sync via BaseConnector.close()
- GitHub connector caches default branch (was hitting API per file)
- Removed 30+ no-op optional deps that just re-declared httpx
2026-05-21 12:09:47 +04:00
Timothy Jaeryang Baek
7c3d36f1b5 0.2.3: remove routes, simplify config to one entry = one KB
Drop the ambiguous 'routes' key from .oikb.yaml. Multi-KB from a
single source is now done with multiple entries + filter.include.
Every entry is strictly source + kb-id, no branching logic.
2026-05-21 12:01:12 +04:00
Timothy Jaeryang Baek
3c8edaf93f doc: readme 2026-05-21 10:25:27 +04:00
Timothy Jaeryang Baek
e450c6ede4 chore: bump v0.2.2 — named entries, /sync/{identifier} endpoint 2026-05-21 10:15:37 +04:00
Timothy Jaeryang Baek
f588d8a63f chore: bump v0.2.1 2026-05-20 23:07:22 +04:00
Timothy Jaeryang Baek
86418bac28 refactor: mirror open-terminal auth pattern
- Add env.py with _resolve_file_env for Docker secrets _FILE support
- HTTPBearer + Depends(verify_api_key) per route instead of middleware
- hmac.compare_digest for timing-safe key comparison
- Health endpoints public, /history and /sync require auth
2026-05-20 23:04:59 +04:00
Timothy Jaeryang Baek
af83f4b4c8 refactor: rename .oikb.yaml top-level key from sync: to sources:
Backward compatible — both sync: and sources: are accepted,
sources: takes precedence.
2026-05-20 22:58:14 +04:00
Timothy Jaeryang Baek
f3ca110358 enhance: configurable fields, format, JQL for Jira connector
Match ServiceNow's configurability: custom JQL, field selection,
markdown/json format, and limit. Supports query string params:
  jira:PROJ?jql=status=open&fields=summary,status&format=json&limit=50
2026-05-20 22:55:10 +04:00
Timothy Jaeryang Baek
ff37b83b14 feat: v0.2.0 — daemon, webhooks, history, 44 connectors, routing, filters, OpenAPI tool server
- oikb daemon: FastAPI scheduler with /health, /history, /sync endpoints
- oikb history: local SQLite sync log with --json output
- Webhooks: /webhooks/github, /webhooks/gitlab, /webhooks/slack, /webhooks/confluence
- 13 new connectors: Document360, Slab, Outline, Google Sites, Egnyte, Oracle Cloud Storage, ProductBoard, XenForo, Zulip, Gong, Fireflies, DokuWiki, ServiceNow
- Multi-KB routing via routes key in .oikb.yaml
- Selective sync filters via filter.include / filter.exclude
- Daemon doubles as OpenAPI tool server for Open WebUI
- ServiceNow: configurable table, fields, format, query, limit
- FastAPI + uvicorn now core dependencies
2026-05-20 22:53:44 +04:00
Timothy Jaeryang Baek
3ed5eade84 fix: align .oikb.yaml key to kb-id (v0.1.3) 2026-05-20 22:13:35 +04:00
Timothy Jaeryang Baek
2fd9319ca5 fix: split messaging connectors by day for incremental sync (v0.1.2) 2026-05-20 13:59:42 +04:00
Timothy Jaeryang Baek
08a1bd1859 doc: readme 2026-05-20 13:50:01 +04:00
Timothy Jaeryang Baek
24ee149a4a feat: add 28 new connectors, rename --kb to --kb-id (v0.1.1)
- 31 total connectors: GitLab, Bitbucket, GDrive, GCS, Azure Blob,
  Dropbox, R2, Confluence, Notion, BookStack, Discourse, GitBook,
  Guru, Slack, Discord, Teams, Gmail, Jira, Linear, Zendesk,
  Freshdesk, Asana, ClickUp, Airtable, Salesforce, HubSpot,
  SharePoint, Web Crawler
- Renamed --kb to --kb-id for clarity
- Updated README with 30+ connector showcase
- Added Open WebUI 0.9.6+ version requirement
2026-05-20 13:42:23 +04:00
Timothy Jaeryang Baek
1996359a53 doc: readme 2026-05-20 13:20:40 +04:00
Timothy Jaeryang Baek
e3c59f1b03 init 2026-05-20 13:19:16 +04:00