The benchmarking tools allow measuring the performance of an agent service realised on the Open Autonomy framework.
How to use benchmarking tools
Setup agent runtime environment in dev mode using
autonomy deploy build KEYS --dev.
Run agents for
1period and wait until the round where the skill is configured to call the
BenchmarkTool.savemethod. (You can also run agents for
Nperiods if you want more data).
Point the aggregation script to the directory containing the benchmark data. This will generate a
benchmarks.htmlfile containing benchmark stats in your current directory.
oracle/price_estimation service as explained here.
By default this will create a
4 agent runtime where you can wait until all
4 agents are at the end of the first period (you can wait for more periods if you want) and then you can stop the runtime. The data will be stored in the
abci_build/persistent_data/benchmarks folder. You can use the following script to aggregate this data:
autonomy analyse benchmarks abci_build/persistent_data/benchmarks
By default the script will generate output for all periods but you can specify which period to generate output for. Similarly, block types aggregation is configurable as well.