Installation

SWE-bench is designed to be easy to install and run on most systems with Docker support.

Prerequisites

Before installing SWE-bench, make sure you have the following prerequisites:

For most users, the standard installation is the best option:

# Clone the repository
git clone https://github.com/princeton-nlp/SWE-bench.git
cd SWE-bench

# Install the package
pip install -e .

This will install the package in development mode, allowing you to make changes to the code if needed.

To install the dependencies for dataset generation, you can run the following command:

pip install -e ".[make_datasets]"

To install the dependencies for inference and dataset generation, you can run the following command:

pip install -e ".[inference]"

SWE-bench relies heavily on Docker for its evaluation environment. Make sure Docker is correctly installed and running:

# Test that Docker is installed correctly
docker --version
docker run hello-world

Depending on your use case, you might want to install additional dependencies:

# For using SWE-Llama models locally
pip install -e ".[llama]"

# For development and testing
pip install -e ".[dev]"

For running evaluations in the cloud using Modal:

pip install modal
modal setup  # First-time setup only
pip install -e ".[modal]"

If you encounter any issues during installation:

Docker permission issues: You might need to add your user to the Docker group
Python version conflicts: Make sure you're using Python 3.9+
Package conflicts: Consider using a virtual environment

For more detailed troubleshooting, please refer to our FAQ page or open an issue on GitHub.