goose/evals/open-model-gym/suite/scenarios/everyday-app-automation.yaml
2026-02-09 06:08:46 +00:00

47 lines
1.6 KiB
YAML

name: everyday-app-automation
description: Multi-step workflow using everyday app tools (Slack, Jira, Calendar) with data dependencies
prompt: |
Using the available tools, complete these tasks:
1. Search Slack for messages mentioning "quarterly review"
2. Look up the user who posted the message about quarterly review to get their full name
3. Create a Jira issue titled "Q1 Review Follow-ups" with a description that includes the name of the person who posted the Slack message
4. Create a calendar event for next Monday at 2pm called "Review Discussion"
5. Write a summary of what you did to a file called workflow-log.md
tags:
- complex
- multi-step
- mcp-harness
- data-flow
validate:
# Check workflow summary was written
- type: file_exists
path: workflow-log.md
- type: file_not_empty
path: workflow-log.md
# Check Slack search was called with the right query
- type: tool_called
tool: slack_search_messages
args:
query: /quarterly.?review/
# Check that the agent looked up the user info (data dependency: requires reading user ID from search results)
- type: tool_called
tool: slack_get_user_info
args:
userId: /U004/
# Check Jira issue was created with expected title and includes the user's name (data dependency: requires reading name from user info)
- type: tool_called
tool: jira_create_issue
args:
summary: /q1.?review|follow.?up/
description: /David.?Brown/
# Check calendar event was created with expected title
- type: tool_called
tool: calendar_create_event
args:
summary: /review.?discussion/