Skip to content

Get Quotas Command

The get-quotas command displays your remaining submission quotas for each dataset subset and split combination.

Usage

sb-cli get-quotas

Example output: 

> sb-cli get-quotas
        Remaining Submission Quotas
┏━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━┓
┃ Subset         ┃ Split ┃ Remaining Runs ┃
┡━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━┩
│ swe-bench-m    │ test  │              1 │
│ swe-bench-m    │ dev   │            997 │
│ swe-bench_lite │ test  │              1 │
│ swe-bench_lite │ dev   │            976 │
└────────────────┴───────┴────────────────┘

Options

  • --api_key: API key to use (defaults to SWEBENCH_API_KEY environment variable)

Output Format

The command displays a table with three columns: - Subset: Dataset subset (swe-bench-m or swe-bench_lite) - Split: Dataset split (dev or test) - Remaining Runs: Number of submissions remaining for this subset/split combination

This command will display the remaining submissions you can make for each dataset subset and split combination.

Notes

  • Quotas are refreshed periodically according to your subscription level