Searchable Objects¶
When defining custom Python objects such as network architectures, or specialized optimizers, it may be hard to decide what values to set for all of their attributes. AutoGluon provides an API that allows you to instead specify a search space of possible values to consider for such attributes, within which the optimal value will be automatically searched for at runtime. This tutorial demonstrates how easy this is to do, without having to modify your existing code at all!
Example for Constructing a Network¶
This tutorial covers an example of selecting a neural network’s
architecture as a hyperparameter optimization (HPO) task. If you are
interested in efficient neural architecture search (NAS), please refer
to this other tutorial instead: sec_proxyless_ .
CIFAR ResNet in GluonCV¶
GluonCV provides CIFARResNet, which allow user to specify how many layers at each stage. For example, we can construct a CIFAR ResNet with only 1 layer per stage:
from gluoncv.model_zoo.cifarresnet import CIFARResNetV1, CIFARBasicBlockV1
layers = [1, 1, 1]
channels = [16, 16, 32, 64]
net = CIFARResNetV1(CIFARBasicBlockV1, layers, channels)
We can visualize the network:
import autogluon.core as ag
from autogluon.vision.utils import plot_network
plot_network(net, (1, 3, 32, 32))
Searchable Network Architecture Using AutoGluon Object¶
autogluon.obj() enables customized search space to any user
defined class. It can also be used within autogluon.Categorical() if
you have multiple networks to choose from.
@ag.obj(
nstage1=ag.space.Int(2, 4),
nstage2=ag.space.Int(2, 4),
)
class MyCifarResNet(CIFARResNetV1):
def __init__(self, nstage1, nstage2):
nstage3 = 9 - nstage1 - nstage2
layers = [nstage1, nstage2, nstage3]
channels = [16, 16, 32, 64]
super().__init__(CIFARBasicBlockV1, layers=layers, channels=channels)
Create one network instance and print the configuration space:
mynet=MyCifarResNet()
print(mynet.cs)
Configuration space object:
Hyperparameters:
nstage1, Type: UniformInteger, Range: [2, 4], Default: 3
nstage2, Type: UniformInteger, Range: [2, 4], Default: 3
We can also overwrite existing search spaces:
mynet1 = MyCifarResNet(nstage1=1,
nstage2=ag.space.Int(5, 10))
print(mynet1.cs)
Configuration space object:
Hyperparameters:
nstage2, Type: UniformInteger, Range: [5, 10], Default: 8
Decorate Existing Class¶
We can also use autogluon.obj() to easily decorate any existing
classes. For example, if we want to search learning rate and weight
decay for Adam optimizer, we only need to add a decorator:
from mxnet import optimizer as optim
@ag.obj()
class Adam(optim.Adam):
pass
Then we can create an instance:
myoptim = Adam(learning_rate=ag.Real(1e-2, 1e-1, log=True), wd=ag.Real(1e-5, 1e-3, log=True))
print(myoptim.cs)
Configuration space object:
Hyperparameters:
learning_rate, Type: UniformFloat, Range: [0.01, 0.1], Default: 0.0316227766, on log-scale
wd, Type: UniformFloat, Range: [1e-05, 0.001], Default: 0.0001, on log-scale
Launch Experiments Using AutoGluon Object¶
AutoGluon Object is compatible with Fit API in AutoGluon tasks, and also
works with user-defined training scripts using
autogluon.autogluon_register_args(). We can start fitting:
from autogluon.vision import ImagePredictor
classifier = ImagePredictor().fit('cifar10', hyperparameters={'net': mynet, 'optimizer': myoptim, 'epochs': 1}, ngpus_per_trial=1)
INFO:root:time_limit=auto set to time_limit=7200. INFO:gluoncv.auto.tasks.image_classification:Starting fit without HPO INFO:ImageClassificationEstimator:modified configs(<old> != <new>): { INFO:ImageClassificationEstimator:root.valid.batch_size 128 != 16 INFO:ImageClassificationEstimator:root.valid.num_workers 4 != 8 INFO:ImageClassificationEstimator:root.img_cls.model resnet50_v1 != resnet50_v1b INFO:ImageClassificationEstimator:root.train.epochs 10 != 1 INFO:ImageClassificationEstimator:root.train.num_workers 4 != 8 INFO:ImageClassificationEstimator:root.train.rec_val_idx ~/.mxnet/datasets/imagenet/rec/val.idx != auto INFO:ImageClassificationEstimator:root.train.rec_train_idx ~/.mxnet/datasets/imagenet/rec/train.idx != auto INFO:ImageClassificationEstimator:root.train.rec_train ~/.mxnet/datasets/imagenet/rec/train.rec != auto INFO:ImageClassificationEstimator:root.train.lr 0.1 != 0.01 INFO:ImageClassificationEstimator:root.train.rec_val ~/.mxnet/datasets/imagenet/rec/val.rec != auto INFO:ImageClassificationEstimator:root.train.data_dir ~/.mxnet/datasets/imagenet != auto INFO:ImageClassificationEstimator:root.train.batch_size 128 != 16 INFO:ImageClassificationEstimator:root.train.num_training_samples 1281167 != -1 INFO:ImageClassificationEstimator:} INFO:ImageClassificationEstimator:Saved config to /var/lib/jenkins/workspace/workspace/autogluon-tutorial-course-v3/docs/_build/eval/tutorials/course/a0f93843/.trial_0/config.yaml INFO:ImageClassificationEstimator:Start training from [Epoch 0] INFO:ImageClassificationEstimator:Epoch[0] Batch [49] Speed: 102.239337 samples/sec accuracy=0.166250 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [99] Speed: 104.409231 samples/sec accuracy=0.251250 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [149] Speed: 104.085371 samples/sec accuracy=0.291250 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [199] Speed: 103.474138 samples/sec accuracy=0.324375 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [249] Speed: 103.065959 samples/sec accuracy=0.348750 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [299] Speed: 102.695133 samples/sec accuracy=0.367292 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [349] Speed: 102.296427 samples/sec accuracy=0.389464 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [399] Speed: 101.924311 samples/sec accuracy=0.406875 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [449] Speed: 101.762614 samples/sec accuracy=0.416528 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [499] Speed: 101.344870 samples/sec accuracy=0.431750 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [549] Speed: 100.894814 samples/sec accuracy=0.441818 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [599] Speed: 100.554430 samples/sec accuracy=0.454688 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [649] Speed: 100.424913 samples/sec accuracy=0.466250 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [699] Speed: 99.935436 samples/sec accuracy=0.475804 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [749] Speed: 99.428443 samples/sec accuracy=0.483083 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [799] Speed: 99.239016 samples/sec accuracy=0.491563 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [849] Speed: 98.722604 samples/sec accuracy=0.500147 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [899] Speed: 98.356990 samples/sec accuracy=0.504861 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [949] Speed: 98.000771 samples/sec accuracy=0.510066 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [999] Speed: 97.264786 samples/sec accuracy=0.515938 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [1049] Speed: 97.085536 samples/sec accuracy=0.521250 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [1099] Speed: 96.583845 samples/sec accuracy=0.525000 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [1149] Speed: 96.462277 samples/sec accuracy=0.529728 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [1199] Speed: 95.955294 samples/sec accuracy=0.532969 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [1249] Speed: 96.258887 samples/sec accuracy=0.538000 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [1299] Speed: 95.991320 samples/sec accuracy=0.543125 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [1349] Speed: 95.739958 samples/sec accuracy=0.547037 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [1399] Speed: 94.593987 samples/sec accuracy=0.550580 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [1449] Speed: 95.624894 samples/sec accuracy=0.554655 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [1499] Speed: 95.660953 samples/sec accuracy=0.558000 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [1549] Speed: 95.315118 samples/sec accuracy=0.561048 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [1599] Speed: 95.297005 samples/sec accuracy=0.563516 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [1649] Speed: 95.321024 samples/sec accuracy=0.566856 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [1699] Speed: 94.723174 samples/sec accuracy=0.570331 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [1749] Speed: 94.696214 samples/sec accuracy=0.572786 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [1799] Speed: 94.407597 samples/sec accuracy=0.575382 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [1849] Speed: 94.396269 samples/sec accuracy=0.578209 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [1899] Speed: 93.908379 samples/sec accuracy=0.581053 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [1949] Speed: 94.307603 samples/sec accuracy=0.583590 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [1999] Speed: 93.860615 samples/sec accuracy=0.586094 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [2049] Speed: 94.086335 samples/sec accuracy=0.588567 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [2099] Speed: 93.954787 samples/sec accuracy=0.590655 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [2149] Speed: 93.705090 samples/sec accuracy=0.592645 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [2199] Speed: 93.967559 samples/sec accuracy=0.595256 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [2249] Speed: 93.808962 samples/sec accuracy=0.596861 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [2299] Speed: 93.777441 samples/sec accuracy=0.599321 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [2349] Speed: 93.850631 samples/sec accuracy=0.601144 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [2399] Speed: 93.811884 samples/sec accuracy=0.603516 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [2449] Speed: 93.600883 samples/sec accuracy=0.605612 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [2499] Speed: 93.800051 samples/sec accuracy=0.607050 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [2549] Speed: 93.449072 samples/sec accuracy=0.608775 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [2599] Speed: 93.702326 samples/sec accuracy=0.610385 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [2649] Speed: 93.664112 samples/sec accuracy=0.612193 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [2699] Speed: 93.586451 samples/sec accuracy=0.613935 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [2749] Speed: 93.733507 samples/sec accuracy=0.615636 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [2799] Speed: 93.594619 samples/sec accuracy=0.617522 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [2849] Speed: 93.641238 samples/sec accuracy=0.619430 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [2899] Speed: 93.744992 samples/sec accuracy=0.621034 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [2949] Speed: 93.570871 samples/sec accuracy=0.622669 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [2999] Speed: 93.595249 samples/sec accuracy=0.624250 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [3049] Speed: 93.500899 samples/sec accuracy=0.626107 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [3099] Speed: 93.599946 samples/sec accuracy=0.627762 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [3149] Speed: 93.387861 samples/sec accuracy=0.629405 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [3199] Speed: 93.452239 samples/sec accuracy=0.630488 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [3249] Speed: 93.668604 samples/sec accuracy=0.631481 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [3299] Speed: 93.378944 samples/sec accuracy=0.632879 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [3349] Speed: 93.586893 samples/sec accuracy=0.634104 lr=0.010000 INFO:ImageClassificationEstimator:[Epoch 0] training: accuracy=0.634722 INFO:ImageClassificationEstimator:[Epoch 0] speed: 96 samples/sec time cost: 583.483227 INFO:ImageClassificationEstimator:[Epoch 0] validation: top1=0.906167 top5=0.997000 INFO:ImageClassificationEstimator:[Epoch 0] Current best top-1: 0.906167 vs previous 0.000000, saved to /var/lib/jenkins/workspace/workspace/autogluon-tutorial-course-v3/docs/_build/eval/tutorials/course/a0f93843/.trial_0/best_checkpoint.pkl INFO:ImageClassificationEstimator:Pickled to /var/lib/jenkins/workspace/workspace/autogluon-tutorial-course-v3/docs/_build/eval/tutorials/course/a0f93843/.trial_0/best_checkpoint.pkl INFO:gluoncv.auto.tasks.image_classification:Finished, total runtime is 592.73 s INFO:gluoncv.auto.tasks.image_classification:{ 'best_config': { 'batch_size': 16, 'custom_net': MyCifarResNet( (features): HybridSequential( (0): Conv2D(None -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): HybridSequential( (0): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(16 -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(16 -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) (1): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(16 -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(16 -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) (2): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(16 -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(16 -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) ) (3): HybridSequential( (0): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(16 -> 32, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) (downsample): HybridSequential( (0): Conv2D(16 -> 32, kernel_size=(1, 1), stride=(2, 2), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) (1): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) (2): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) (3): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) (4): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) (5): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) (6): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) (7): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) ) (4): HybridSequential( (0): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(32 -> 64, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(64 -> 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) (downsample): HybridSequential( (0): Conv2D(32 -> 64, kernel_size=(1, 1), stride=(2, 2), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) ) (5): GlobalAvgPool2D(size=(1, 1), stride=(1, 1), padding=(0, 0), ceil_mode=True, global_pool=True, pool_type=avg, layout=NCHW) ) (output): Dense(64 -> 10, linear) ), 'custom_optimizer': <__main__.Adam object at 0x7f4e1d920280>, 'dist_ip_addrs': None, 'epochs': 1, 'estimator': <class 'gluoncv.auto.estimators.image_classification.image_classification.ImageClassificationEstimator'>, 'final_fit': False, 'gpus': [0], 'log_dir': '/var/lib/jenkins/workspace/workspace/autogluon-tutorial-course-v3/docs/_build/eval/tutorials/course/a0f93843', 'lr': 0.01, 'model': 'resnet50_v1b', 'ngpus_per_trial': 1, 'nthreads_per_trial': 128, 'num_trials': 1, 'num_workers': 8, 'search_strategy': 'random', 'seed': 551, 'time_limits': 7200, 'wall_clock_tick': 1614115804.920842}, 'total_time': 575.0971519947052, 'train_acc': 0.6347222222222222, 'valid_acc': 0.9061666666666667}
print(classifier.fit_summary())
{'train_acc': 0.6347222222222222, 'valid_acc': 0.9061666666666667, 'total_time': 575.0971519947052, 'best_config': {'model': 'resnet50_v1b', 'lr': 0.01, 'num_trials': 1, 'epochs': 1, 'batch_size': 16, 'nthreads_per_trial': 128, 'ngpus_per_trial': 1, 'time_limits': 7200, 'search_strategy': 'random', 'dist_ip_addrs': None, 'log_dir': '/var/lib/jenkins/workspace/workspace/autogluon-tutorial-course-v3/docs/_build/eval/tutorials/course/a0f93843', 'custom_net': MyCifarResNet(
(features): HybridSequential(
(0): Conv2D(None -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
(2): HybridSequential(
(0): CIFARBasicBlockV1(
(body): HybridSequential(
(0): Conv2D(16 -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
(2): Activation(relu)
(3): Conv2D(16 -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
)
)
(1): CIFARBasicBlockV1(
(body): HybridSequential(
(0): Conv2D(16 -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
(2): Activation(relu)
(3): Conv2D(16 -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
)
)
(2): CIFARBasicBlockV1(
(body): HybridSequential(
(0): Conv2D(16 -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
(2): Activation(relu)
(3): Conv2D(16 -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
)
)
)
(3): HybridSequential(
(0): CIFARBasicBlockV1(
(body): HybridSequential(
(0): Conv2D(16 -> 32, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
(2): Activation(relu)
(3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
)
(downsample): HybridSequential(
(0): Conv2D(16 -> 32, kernel_size=(1, 1), stride=(2, 2), bias=False)
(1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
)
)
(1): CIFARBasicBlockV1(
(body): HybridSequential(
(0): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
(2): Activation(relu)
(3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
)
)
(2): CIFARBasicBlockV1(
(body): HybridSequential(
(0): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
(2): Activation(relu)
(3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
)
)
(3): CIFARBasicBlockV1(
(body): HybridSequential(
(0): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
(2): Activation(relu)
(3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
)
)
(4): CIFARBasicBlockV1(
(body): HybridSequential(
(0): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
(2): Activation(relu)
(3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
)
)
(5): CIFARBasicBlockV1(
(body): HybridSequential(
(0): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
(2): Activation(relu)
(3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
)
)
(6): CIFARBasicBlockV1(
(body): HybridSequential(
(0): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
(2): Activation(relu)
(3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
)
)
(7): CIFARBasicBlockV1(
(body): HybridSequential(
(0): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
(2): Activation(relu)
(3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
)
)
)
(4): HybridSequential(
(0): CIFARBasicBlockV1(
(body): HybridSequential(
(0): Conv2D(32 -> 64, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
(2): Activation(relu)
(3): Conv2D(64 -> 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
)
(downsample): HybridSequential(
(0): Conv2D(32 -> 64, kernel_size=(1, 1), stride=(2, 2), bias=False)
(1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
)
)
)
(5): GlobalAvgPool2D(size=(1, 1), stride=(1, 1), padding=(0, 0), ceil_mode=True, global_pool=True, pool_type=avg, layout=NCHW)
)
(output): Dense(64 -> 10, linear)
), 'custom_optimizer': <__main__.Adam object at 0x7f4e1d920280>, 'num_workers': 8, 'gpus': [0], 'seed': 551, 'final_fit': False, 'estimator': <class 'gluoncv.auto.estimators.image_classification.image_classification.ImageClassificationEstimator'>, 'wall_clock_tick': 1614115804.920842}, 'fit_history': {'train_acc': 0.6347222222222222, 'valid_acc': 0.9061666666666667, 'total_time': 575.0971519947052, 'best_config': {'model': 'resnet50_v1b', 'lr': 0.01, 'num_trials': 1, 'epochs': 1, 'batch_size': 16, 'nthreads_per_trial': 128, 'ngpus_per_trial': 1, 'time_limits': 7200, 'search_strategy': 'random', 'dist_ip_addrs': None, 'log_dir': '/var/lib/jenkins/workspace/workspace/autogluon-tutorial-course-v3/docs/_build/eval/tutorials/course/a0f93843', 'custom_net': MyCifarResNet(
(features): HybridSequential(
(0): Conv2D(None -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
(2): HybridSequential(
(0): CIFARBasicBlockV1(
(body): HybridSequential(
(0): Conv2D(16 -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
(2): Activation(relu)
(3): Conv2D(16 -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
)
)
(1): CIFARBasicBlockV1(
(body): HybridSequential(
(0): Conv2D(16 -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
(2): Activation(relu)
(3): Conv2D(16 -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
)
)
(2): CIFARBasicBlockV1(
(body): HybridSequential(
(0): Conv2D(16 -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
(2): Activation(relu)
(3): Conv2D(16 -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
)
)
)
(3): HybridSequential(
(0): CIFARBasicBlockV1(
(body): HybridSequential(
(0): Conv2D(16 -> 32, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
(2): Activation(relu)
(3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
)
(downsample): HybridSequential(
(0): Conv2D(16 -> 32, kernel_size=(1, 1), stride=(2, 2), bias=False)
(1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
)
)
(1): CIFARBasicBlockV1(
(body): HybridSequential(
(0): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
(2): Activation(relu)
(3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
)
)
(2): CIFARBasicBlockV1(
(body): HybridSequential(
(0): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
(2): Activation(relu)
(3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
)
)
(3): CIFARBasicBlockV1(
(body): HybridSequential(
(0): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
(2): Activation(relu)
(3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
)
)
(4): CIFARBasicBlockV1(
(body): HybridSequential(
(0): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
(2): Activation(relu)
(3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
)
)
(5): CIFARBasicBlockV1(
(body): HybridSequential(
(0): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
(2): Activation(relu)
(3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
)
)
(6): CIFARBasicBlockV1(
(body): HybridSequential(
(0): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
(2): Activation(relu)
(3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
)
)
(7): CIFARBasicBlockV1(
(body): HybridSequential(
(0): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
(2): Activation(relu)
(3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
)
)
)
(4): HybridSequential(
(0): CIFARBasicBlockV1(
(body): HybridSequential(
(0): Conv2D(32 -> 64, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
(2): Activation(relu)
(3): Conv2D(64 -> 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
)
(downsample): HybridSequential(
(0): Conv2D(32 -> 64, kernel_size=(1, 1), stride=(2, 2), bias=False)
(1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None)
)
)
)
(5): GlobalAvgPool2D(size=(1, 1), stride=(1, 1), padding=(0, 0), ceil_mode=True, global_pool=True, pool_type=avg, layout=NCHW)
)
(output): Dense(64 -> 10, linear)
), 'custom_optimizer': <__main__.Adam object at 0x7f4e1d920280>, 'num_workers': 8, 'gpus': [0], 'seed': 551, 'final_fit': False, 'estimator': <class 'gluoncv.auto.estimators.image_classification.image_classification.ImageClassificationEstimator'>, 'wall_clock_tick': 1614115804.920842}}}