TODO

feat: add openrlhf.diff file for pending modifications

- Added openrlhf.diff file to track pending modifications to the openrlhf project.

- Reorganized project structure for better modularity and maintainability.
- Note: This commit is a work in progress and is not yet functional. Further adjustments to imports and code logic are required to make the project runnable.

utils: finish todo: refactor vllm_chatcomplete and vllm_score, and other functions in step1. Don't save results in function.

TODO: refactor vllm_chatcomplete

Modifications:
- Do not mask the first turn in the chat history when `mask_history`.
- Preserve the original order of messages (do not reverse the list).

- Addresses occasional deadlocks in multi-threaded model server requests.
- Implemented a single-threaded, single-server setup for testing purposes.
- Observed that the Reward model's performance is acceptable and can be used.
- Temporarily adopting this single-threaded version