mirror of
https://github.com/alibaba/open-code-review.git
synced 2026-08-31 02:07:08 +00:00
* ci: add rate limit handling and version verification to GitHub Actions workflow - Add version check after OCR installation to verify successful setup - Implement exponential backoff retry logic for GitHub API rate limits - Add delays between individual comment posts to avoid secondary rate limits - GitHub enforces ~80 content-generating requests per minute; spacing calls helps stay under that threshold with 2-second base delay and up to 3 retries * ci: refine rate limit handling in GitHub Actions workflow - Add `|| true` to ocr version check for error isolation - Narrow rate limit detection to 429 and 403 with rate-limit message matching, avoiding retries on permission/auth failures - Extract hardcoded delay constants into env-configurable variables (OCR_RETRY_BASE_DELAY, OCR_MAX_RETRIES, OCR_SUCCESS_DELAY, OCR_FAILURE_DELAY) with sensible defaults - Document optional environment variables in workflow header * docs: add environment variable configuration guide for retry and delay settings * ci: add rate-limit resilience and version check to GitLab CI pipeline - Add `ocr version || true` after install for diagnostic logging - Add `api_request_with_retry` function with exponential backoff for 429 and 403 (rate-limit message matching) errors - Respect GitLab `Retry-After` header when present - Extract delay constants into CI/CD-configurable variables (OCR_RETRY_BASE_DELAY, OCR_MAX_RETRIES, OCR_SUCCESS_DELAY, OCR_FAILURE_DELAY) with defaults - Add pacing delays between successful/failed discussion posts - Document optional CI/CD variables in pipeline header comments * docs: add retry/delay settings section to GitLab CI README * ci: fix rate-limit retry delay exhaustion handling in CI workflows - GitHub Actions: distinguish exhausted rate-limit retries from other errors, apply SUCCESS_DELAY (2s) instead of FAILURE_DELAY (1s) when retries exhausted - GitLab CI: return structured result from api_request_with_retry to differentiate failure types, apply context-aware delays based on rate-limit exhaustion status - Both: prevent perpetuating rate-limit failures by using longer delays after retry exhaustion * ci: align GitHub Actions rate-limit retry with header-based strategy Derive wait durations from response headers (retry-after, x-ratelimit-reset) instead of fixed exponential backoff, add proactive throttle when remaining quota is low, honor batch-level rate limits before per-comment retry, and add support for transient 5xx/408 errors. * feat(gitlab-ci): enhance rate-limit handling with jitter, max retry delay, and proactive throttling - Add ±25% jitter on retry delays to avoid thundering herd problems - Add OCR_MAX_RETRY_DELAY (default 60s) to cap per-retry wait time - Add OCR_RATE_LIMIT_THRESHOLD (default 10) for proactive throttling based on GitLab RateLimit-Remaining response header - Parse Retry-After header properly (handle non-numeric values) - Apply retry logic to all API requests (notes, versions, discussions) - Parse and log RateLimit-Remaining/Limit headers for observability - Double pacing delay when remaining quota drops below threshold - Update README with new configuration variables and behavior docs * fix(examples): sync rate-limit docs with script defaults and add missing variables - GitHub Actions README: fix OCR_RETRY_BASE_DELAY default from 2000 to 60000 (matching script code and header comments) - GitHub Actions README: add missing OCR_RETRY_MAX_DELAY, OCR_LOW_REMAINING_THRESHOLD, OCR_LOW_REMAINING_SPACING variables - GitHub Actions README: add GitHub Rate Limits doc reference link - GitHub Actions yml header: add OCR_LLM_USE_ANTHROPIC and llm.extra_body notes - GitLab CI yml header: add llm.extra_body note - GitLab CI README: add GitLab Rate Limits doc reference link * ci(examples): unify header lookup and add transient retry backoff - GitHub Actions: extract inline header closure into a reusable getHeader helper; use it in both computeRetryDelayMs and logRateLimitQuota for consistent case-insensitive header access. - GitHub Actions: use a 2s transientBase for 5xx/408 exponential backoff instead of the 60s rate-limit base, since server hiccups are typically short-lived and the longer base stalled CI jobs unnecessarily. - GitLab CI: add a _get_header helper and route all header access (Retry-After, RateLimit-*) through it, matching the GitHub Actions approach. - GitLab CI: add transient retry logic for 5xx/408 errors with a 2s base delay, so server errors no longer fail immediately.
239 lines
9.7 KiB
Markdown
239 lines
9.7 KiB
Markdown
# OpenCodeReview - GitHub Actions Demo
|
|
|
|
This demo shows how to integrate OpenCodeReview into your GitHub Actions workflow to automatically review Pull Requests and post review comments.
|
|
|
|
## How It Works
|
|
|
|
```
|
|
PR Created/Updated → GitHub Actions Triggered → OCR Reviews Diff → Comments Posted on PR
|
|
OR
|
|
Comment with trigger keyword ↗
|
|
```
|
|
|
|
1. When a PR is opened, the workflow triggers (uses `pull_request_target` for fork secret access)
|
|
2. Alternatively, when a comment containing `/open-code-review` or `@open-code-review` is posted on a PR, the workflow triggers
|
|
3. It installs OCR via `npm install -g @alibaba-group/open-code-review`
|
|
4. Runs `ocr review --from origin/<base> --to <head_sha> --format json` to analyze the diff (uses commit SHA to support fork PRs)
|
|
5. Parses the JSON output and posts inline review comments on the PR using GitHub's Pull Request Review API
|
|
|
|
## Setup
|
|
|
|
### 1. Copy the workflow file
|
|
|
|
Copy `ocr-review.yml` to your repository's `.github/workflows/` directory:
|
|
|
|
```bash
|
|
mkdir -p .github/workflows
|
|
cp ocr-review.yml .github/workflows/ocr-review.yml
|
|
```
|
|
|
|
### 2. Configure secrets
|
|
|
|
Go to your repository's **Settings → Secrets and variables → Actions** and add:
|
|
|
|
| Secret | Required | Description |
|
|
|--------|----------|-------------|
|
|
| `OCR_LLM_URL` | Yes | LLM API endpoint URL (e.g., `https://api.openai.com/v1/chat/completions`) |
|
|
| `OCR_LLM_AUTH_TOKEN` | Yes | API authentication token |
|
|
| `OCR_LLM_MODEL` | No | Model name (defaults to `gpt-4o`) |
|
|
| `OCR_LLM_USE_ANTHROPIC` | No | Set to `true` if using Anthropic Claude models |
|
|
|
|
> **Note:** `GITHUB_TOKEN` is automatically provided by GitHub Actions with the required `pull-requests: write` permission.
|
|
>
|
|
> The workflow also configures `llm.extra_body` to disable thinking mode for compatibility with various LLM providers.
|
|
|
|
## Customization
|
|
|
|
### Change the trigger events
|
|
|
|
Modify the `on.pull_request_target.types` array in the workflow file:
|
|
|
|
```yaml
|
|
on:
|
|
pull_request_target:
|
|
types: [opened, synchronize, reopened, ready_for_review]
|
|
```
|
|
|
|
### Customize comment trigger keywords
|
|
|
|
By default, the workflow triggers when a PR comment starts with `/open-code-review` or `@open-code-review`. You can customize these keywords by modifying the `if` condition in the workflow:
|
|
|
|
```yaml
|
|
if: |
|
|
github.event_name == 'pull_request_target' ||
|
|
(github.event_name == 'issue_comment' && github.event.issue.pull_request && startsWith(github.event.comment.body, '/review')) ||
|
|
(github.event_name == 'issue_comment' && github.event.issue.pull_request && startsWith(github.event.comment.body, '@mybot'))
|
|
```
|
|
|
|
Or use a more flexible pattern with `contains` to trigger on any comment containing the keyword:
|
|
|
|
```yaml
|
|
if: |
|
|
github.event_name == 'pull_request_target' ||
|
|
(github.event_name == 'issue_comment' && github.event.issue.pull_request && contains(github.event.comment.body, '/review'))
|
|
```
|
|
|
|
> **Note:** The condition `github.event.issue.pull_request` ensures the comment is on a PR, not a regular issue.
|
|
|
|
### Use a specific OCR version
|
|
|
|
```yaml
|
|
- name: Install OpenCodeReview
|
|
run: npm install -g @alibaba-group/open-code-review@1.0.0
|
|
```
|
|
|
|
### Add custom review rules
|
|
|
|
Use the `--rule` flag to pass a custom rules JSON file:
|
|
|
|
```yaml
|
|
- name: Run OCR review
|
|
run: ocr review --rule ./my-rules.json --from origin/${{ github.base_ref }} --to origin/${{ github.head_ref }}
|
|
```
|
|
|
|
### Adjust retry and delay settings
|
|
|
|
When posting review comments individually (fallback mode), the workflow includes rate-limit handling with exponential backoff. The retry strategy follows GitHub's documented guidance for REST API rate limits — see [Rate limits for the REST API](https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api?apiVersion=2026-03-10) for details on primary/secondary rate limits and recommended retry behavior. You can configure the retry and delay behavior via **repository variables** (Settings → Secrets and variables → Actions → Variables):
|
|
|
|
| Variable | Default | Description |
|
|
|----------|---------|-------------|
|
|
| `OCR_RETRY_BASE_DELAY` | `60000` | Base delay (ms) for exponential backoff when no retry header is present (per GitHub's "at least one minute" recommendation for secondary limits) |
|
|
| `OCR_RETRY_MAX_DELAY` | `300000` | Maximum delay (ms) cap applied to every computed wait, including retry-after and x-ratelimit-reset, so a far-future reset cannot stall the job past its timeout |
|
|
| `OCR_MAX_RETRIES` | `3` | Maximum retry attempts per comment when rate-limited |
|
|
| `OCR_SUCCESS_DELAY` | `2000` | Delay (ms) after a successful comment post to pace subsequent requests |
|
|
| `OCR_FAILURE_DELAY` | `1000` | Delay (ms) after a non-rate-limit failure to pace subsequent requests |
|
|
| `OCR_LOW_REMAINING_THRESHOLD` | `3` | When x-ratelimit-remaining is at or below this value, proactively increase request spacing to avoid hitting the limit |
|
|
| `OCR_LOW_REMAINING_SPACING` | `10000` | Request spacing (ms) used when remaining quota is low |
|
|
|
|
These variables are optional — if not configured, sensible defaults are used. Consider increasing delays for repositories with many concurrent workflows or large PRs that generate numerous review comments.
|
|
|
|
### Limit concurrency
|
|
|
|
Adjust the `--concurrency` flag for large PRs to control the number of concurrent LLM requests:
|
|
|
|
```yaml
|
|
- name: Run OCR review
|
|
run: ocr review --concurrency 5 --from origin/${{ github.base_ref }} --to origin/${{ github.head_ref }}
|
|
```
|
|
|
|
### Provide background context
|
|
|
|
Use the `--background` flag to pass additional context that helps OCR better understand the purpose of the changes:
|
|
|
|
```yaml
|
|
- name: Run OCR review
|
|
run: ocr review --background "${{ github.event.pull_request.title }}" --from origin/${{ github.base_ref }} --to origin/${{ github.head_ref }}
|
|
```
|
|
|
|
This is particularly useful when your PR titles follow semantic conventions (e.g., `feat(auth): add OAuth2 support`) that clearly summarize what the PR implements. The background information helps OCR provide more relevant and context-aware review comments.
|
|
|
|
### Customize the review comment author with GitHub App
|
|
|
|
By default, review comments are posted using the built-in `GITHUB_TOKEN`, which appears as `github-actions[bot]`. You can customize this by creating a GitHub App and using its credentials instead.
|
|
|
|
For more details about GitHub Apps, see the [GitHub Apps documentation](https://docs.github.com/en/apps).
|
|
|
|
#### Step 1: Create a GitHub App
|
|
|
|
1. Go to your organization or personal account **Settings → Developer settings → GitHub Apps → New GitHub App**
|
|
2. Fill in the following:
|
|
- **GitHub App name**: e.g., `OpenCodeReview Bot`
|
|
- **Homepage URL**: Your repository or documentation URL
|
|
- **Webhook**: Uncheck "Active" (not needed for this use case)
|
|
3. Under **Repository permissions**, set:
|
|
- **Pull requests**: Read and write
|
|
- **Contents**: Read-only (for fetching diffs)
|
|
- **Metadata**: Read-only (required)
|
|
4. Click **Create GitHub App**
|
|
|
|
#### Step 2: Generate a Private Key
|
|
|
|
1. After creating the app, scroll down to **Private keys**
|
|
2. Click **Generate a private key**
|
|
3. Download and save the `.pem` file securely
|
|
|
|
Note your App ID from the app settings page.
|
|
|
|
#### Step 3: Install the App
|
|
|
|
1. In the left sidebar, click **Install App**
|
|
2. Select the repositories where you want to use OCR
|
|
3. After installation, note the **Installation ID** from the URL (e.g., `https://github.com/settings/installations/12345` → Installation ID is `12345`)
|
|
|
|
#### Step 4: Configure Repository Secrets
|
|
|
|
Add the following secrets to your repository (**Settings → Secrets and variables → Actions**):
|
|
|
|
| Secret | Description |
|
|
|--------|-------------|
|
|
| `GITHUB_APP_ID` | Your GitHub App's ID |
|
|
| `GITHUB_APP_PRIVATE_KEY` | Contents of the `.pem` file (including `-----BEGIN RSA PRIVATE KEY-----` and `-----END RSA PRIVATE KEY-----`) |
|
|
| `GITHUB_APP_INSTALLATION_ID` | The Installation ID from Step 3 |
|
|
|
|
#### Step 5: Update the Workflow
|
|
|
|
Add a step to obtain a token from the GitHub App, then use it in the "Post review comments to PR" step:
|
|
|
|
```yaml
|
|
- name: Get GitHub App Token
|
|
id: app-token
|
|
uses: actions/create-github-app-token@v1
|
|
with:
|
|
app-id: ${{ secrets.GITHUB_APP_ID }}
|
|
private-key: ${{ secrets.GITHUB_APP_PRIVATE_KEY }}
|
|
|
|
- name: Post review comments to PR
|
|
uses: actions/github-script@v7
|
|
with:
|
|
github-token: ${{ steps.app-token.outputs.token }}
|
|
script: |
|
|
# ... existing script
|
|
```
|
|
|
|
Now review comments will be posted with your custom GitHub App identity (e.g., `OpenCodeReview Bot`), providing a more professional and distinguishable appearance in your PRs.
|
|
|
|
## Example Output
|
|
|
|
When a PR is reviewed, comments appear directly in the PR's "Files changed" tab:
|
|
|
|
- ✅ If no issues found: A comment saying "No comments generated. Looks good to me."
|
|
- 🔍 If issues found: Inline review comments with suggestions using GitHub's native suggestion syntax
|
|
|
|
### Inline Comment Example
|
|
|
|
The workflow uses GitHub's `suggestion` code block syntax, so reviewers can apply fixes with one click:
|
|
|
|
````markdown
|
|
**Suggestion:**
|
|
```suggestion
|
|
// Fixed code here
|
|
```
|
|
````
|
|
|
|
## Supported LLM Providers
|
|
|
|
OCR supports both OpenAI and Anthropic API formats:
|
|
|
|
- **OpenAI-compatible APIs** (default):
|
|
- OpenAI (GPT-4o, GPT-4, etc.)
|
|
- Azure OpenAI
|
|
- Self-hosted models (vLLM, Ollama, etc.)
|
|
- **Anthropic APIs** (set `OCR_LLM_USE_ANTHROPIC: true`):
|
|
- Anthropic Claude models
|
|
|
|
## Troubleshooting
|
|
|
|
### Common Issues
|
|
|
|
1. **"Failed to parse OCR output"**: Check that `OCR_LLM_URL` and `OCR_LLM_AUTH_TOKEN` secrets are correctly set
|
|
2. **"Cannot find merge-base"**: Ensure `fetch-depth: 0` is set in the checkout step
|
|
3. **Review comments not appearing on correct lines**: This can happen when the diff has changed since the review started; the workflow handles this gracefully with a fallback to issue comments
|
|
|
|
### Debugging
|
|
|
|
Enable debug logging by adding to the OCR review step:
|
|
|
|
```yaml
|
|
env:
|
|
OCR_DEBUG: "1"
|
|
```
|