iicd/xautodl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

re-organize NATS-Bench

Browse Source
This commit is contained in:
D-X-Y
2020-08-18 06:29:30 +00:00
parent ffd23a6cbd
commit 3529b993ff
10 changed files with 102 additions and 153 deletions
Download Patch File Download Diff File Expand all files Collapse all files

13
docs/NATS-Bench.md
View File

@@ -7,11 +7,23 @@ We analyze the validity of our benchmark in terms of various criteria and perfor
We also show the versatility of NATS-Bench by benchmarking 13 recent state-of-the-art NAS algorithms on it. All logs and diagnostic information trained using the same setup for each candidate are provided.
This facilitates a much larger community of researchers to focus on developing better NAS algorithms in a more comparable and computationally effective environment.
**coming soon!**
## How to Use NATS-Bench
## The Procedure of Creating NATS-Bench
1, train all architecture candidate in the size search space with 90 epochs and use the random seed of `777`.
```
bash ./scripts/NATS-Bench/train-shapes.sh 00000-32767 90 777
```
The checkpoint of all candidates are located at `output/NATS-Bench-size` by default
## To Reproduce 13 Baseline NAS Algorithms in NAS-Bench-201
### Reproduce NAS methods on the topology search space
@@ -50,6 +62,7 @@ python ./exps/algos-v2/search-cell.py --dataset ImageNet16-120 --data_path $TORC
### Reproduce NAS methods on the size search space
### Final Discovered Architectures for Each Algorithm
The architecture index can be found by use `api.query_index_by_arch(architecture_string)`.