Prevents training hangs by validating `num_key_value_heads %
ulysses_sequence_parallel_size == 0` before training.

## What does this PR do?

Add document for using vLLM 0.8 in verl

## Who can review?

@eric-haibin-lin

Adding Openmanus-RL: a llm agent rl tunning repo with verl

Add `verl` as the `framework` parameter to the SwanLab config table, so
more developers can see that this training comes from `verl`.

https://github.com/volcengine/verl/issues/680

Changes:
- Move math-verify to the optional dependencies. Now it can be installed
via `cd verl && pip install -e .[math]`
- Revert using naive verifier for math dataset. Users can switch to
math-verify or custom a new `compute_score` function.

We propose a more accurate description of DeepRetrieval.
Thanks for your awesome work!

Use ray actor instead of task to run main_task
- Ray task is retried in system error(oom/segmentfault), which may cause
unexpectedly behavior
- Actor is more trackable in ray dashboard, e.g
logging/stacktrace/profile

close #539

Commit c3420692 Rebase caused error. Try to revert and add an assertion
check.

This PR adds **DeepEnlighten** to the "Awesome Work Using Verl" section.

Co-authored-by: yu_wang <yuwang@astri.com>
Co-authored-by: Chi Zhang <zhangchi.usc1992@bytedance.com>

- As titled

#22 . WIP, will add more details tomorrow :)

---------

Co-authored-by: zhaochenyang20 <zhaochen20@outlook.com>

add MetaSpatial in Awesome Work using EasyR1

## What does this PR do?

Use metric_utils to maintain the logic of computing metrics, avoiding
too many lines in ppo trainer

## Who can review?

@vermouth1992 @PeterSH6

Support for GRPO with Megatron backend and fix a configuration bug when
not using virtual pipeline.

Calibrated with FSDP backend.

This PR adds the `lr_warmup_steps` configuration.

Note the `num_warmup_steps` is prior to `lr_warmup_steps_ratio`.

## Summary

Providing an option in the config to turn off the `torch.compile` used
in `dp_actor.py`

## Usage

Adding the following line to the driver or cli scripts to turn off
`torch.compile`.
```python
+actor_rollout_ref.actor.use_torch_compile=False
```
Otherwise, `torch.compile` will be used by default

## Related Issue

#354 #245

---------

Signed-off-by: Hongpeng Guo <hpguo@anyscale.com>