Compare

The compare script, compare a list of system using the aggregated balanced accuracy, which is a cross-dataset, evaluation protocol. An example of this evaluation protocol can be seen on the following figure.

The aggreagted method is the weighted average, where the weights are the number of 30s-window (name epoch in the package) in the subset.

You may issue sleepless compare --help for a help message containing more detailed instructions.

To compare systems A and B, use the sub-command compare to generate table and plot to compare both systems:

sleepless compare -vv <A> <path/to/A/metric_table.csv> <B> <path/to/B/metric_table.csv> -o <path/to/output_folder>

Replace <A> and <B> by the names you want to give to each system, and <path/to/A/metric_table.csv> and <path/to/B/metric_table.csv> by the respective paths to the csv metrics table of each system.

You can compare more than 2 systems by following the same pattern (name + path).