wangtao
/
autogen
mirror of https://github.com/microsoft/autogen.git
1
0
Fork 1
Code Issues Packages Projects Releases Wiki Activity
autogen/python/packages/agbench/benchmarks
History
Eric Zhu 7615c7b83b
Rename to use BaseChatMessage and BaseAgentEvent. Bring back union types. (#6144) 
Rename the `ChatMessage` and `AgentEvent` base classes to `BaseChatMessage` and `BaseAgentEvent`. 

Bring back the `ChatMessage` and `AgentEvent` as union of built-in concrete types to avoid breaking existing applications that depends on Pydantic serialization. 

Why?

Many existing code uses containers like this:

```python
class AppMessage(BaseModel):
   name: str
   message: ChatMessage 

# Serialization is this:
m = AppMessage(...)
m.model_dump_json()

# Fields like HandoffMessage.target will be lost because it is now treated as a base class without content or target fields.
```

The assumption on `ChatMessage` or `AgentEvent` to be a union of concrete types could be in many existing code bases. So this PR brings back the union types, while keep method type hints such as those on `on_messages` to use the `BaseChatMessage` and `BaseAgentEvent` base classes for flexibility.
 		2025-03-30 09:34:40 -07:00
..
GAIA Rename to use BaseChatMessage and BaseAgentEvent. Bring back union types. (#6144) 2025-03-30 09:34:40 -07:00
HumanEval Improvements to agbench (#5776) 2025-03-16 09:13:12 -07:00
.gitignore Adding Benchmarks to agbench (#3803) 2024-10-18 06:33:33 +02:00
README.md Adding Benchmarks to agbench (#3803) 2024-10-18 06:33:33 +02:00
process_logs.py Adding Benchmarks to agbench (#3803) 2024-10-18 06:33:33 +02:00

README.md

Benchmarking Agents

This directory provides ability to benchmarks agents (e.g., built using Autogen) using AgBench. Use the instructions below to prepare your environment for benchmarking. Once done, proceed to relevant benchmarks directory (e.g., benchmarks/GAIA) for further scenario-specific instructions.

Setup on WSL

  1. Install Docker Desktop. After installation, restart is needed, then open Docker Desktop, in Settings, Ressources, WSL Integration, Enable integration with additional distros – Ubuntu

  2. Clone autogen and export AUTOGEN_REPO_BASE. This environment variable enables the Docker containers to use the correct version agents.

    git clone git@github.com:microsoft/autogen.git
export AUTOGEN_REPO_BASE=<path_to_autogen>

  3. Install agbench. AgBench is currently a tool in the Autogen repo.

    cd autogen/python/packages/agbench
pip install -e .