Quickstart

Library usage (recommended)

The recommended way to use ProbeLLM is as a Python library:

from probellm import VulnerabilityPipelineAsync

pipeline = VulnerabilityPipelineAsync(
    model_name="gpt-5.2",
    test_model="gpt-4o-mini",
    judge_model="gpt-5.2",
    max_depth=3,
    num_simulations=100,
    num_samples=5,
)

pipeline.add_datasets_batch(["mbpp", "mmlu"])
pipeline.run()

CLI usage

You can also use the CLI module wrappers:

python -m probellm.search

Outputs

Results are written under results/:

results/run_<timestamp>_sim<N>_samples<M>/
├── metadata.json
├── <dataset_id>/
│   ├── results_*.json
│   └── checkpoints/
└── enhanced_analysis/