Object Detection - Quick Start¶
Object detection is the process of identifying and localizing objects in an image and is an important task in computer vision. Follow this tutorial to learn how to use AutoGluon for object detection.
Tip: If you are new to AutoGluon, review Image Prediction - Quick Start first to learn the basics of the AutoGluon API.
Our goal is to detect motorbike in images by YOLOv3 model. A tiny dataset is collected from VOC dataset, which only contains the motorbike category. The model pretrained on the COCO dataset is used to fine-tune our small dataset. With the help of AutoGluon, we are able to try many models with different hyperparameters automatically, and return the best one as our final model.
To start, import autogluon.vision and ObjectDetector:
import autogluon.core as ag
from autogluon.vision import ObjectDetector
Tiny_motorbike Dataset¶
We collect a toy dataset for detecting motorbikes in images. From the VOC dataset, images are randomly selected for training, validation, and testing - 120 images for training, 50 images for validation, and 50 for testing. This tiny dataset follows the same format as VOC.
Using the commands below, we can download this dataset, which is only
23M. The name of unzipped folder is called tiny_motorbike. Anyway,
the task dataset helper can perform the download and extraction
automatically, and load the dataset according to the detection formats.
url = 'https://autogluon.s3.amazonaws.com/datasets/tiny_motorbike.zip'
dataset_train = ObjectDetector.Dataset.from_voc(url, splits='trainval')
tiny_motorbike/
├── Annotations/
├── ImageSets/
└── JPEGImages/
Fit Models by AutoGluon¶
In this section, we demonstrate how to apply AutoGluon to fit our detection models. We use mobilenet as the backbone for the YOLOv3 model. Two different learning rates are used to fine-tune the network. The best model is the one that obtains the best performance on the validation dataset. You can also try using more networks and hyperparameters to create a larger searching space.
We fit a classifier using AutoGluon as follows. In each experiment
(one trial in our searching space), we train the model for 5 epochs to
avoid bursting our tutorial runtime.
time_limit = 60*30 # at most 0.5 hour
detector = ObjectDetector()
hyperparameters = {'epochs': 5, 'batch_size': 8}
hyperparamter_tune_kwargs={'num_trials': 2}
detector.fit(dataset_train, time_limit=time_limit, hyperparameters=hyperparameters, hyperparamter_tune_kwargs=hyperparamter_tune_kwargs)
WARNING:gluoncv.auto.tasks.object_detection:The number of requested GPUs is greater than the number of available GPUs.Reduce the number to 1
INFO:gluoncv.auto.tasks.object_detection:Randomly split train_data into train[158]/validation[12] splits.
INFO:gluoncv.auto.tasks.object_detection:Starting fit without HPO
INFO:SSDEstimator:modified configs(<old> != <new>): {
INFO:SSDEstimator:root.dataset voc_tiny != auto
INFO:SSDEstimator:root.valid.batch_size 16 != 8
INFO:SSDEstimator:root.train.seed 233 != 363
INFO:SSDEstimator:root.train.batch_size 16 != 8
INFO:SSDEstimator:root.train.epochs 20 != 5
INFO:SSDEstimator:root.ssd.base_network vgg16_atrous != resnet50_v1
INFO:SSDEstimator:root.ssd.data_shape 300 != 512
INFO:SSDEstimator:root.gpus (0, 1, 2, 3) != (0,)
INFO:SSDEstimator:root.num_workers 4 != 8
INFO:SSDEstimator:root.dataset_root ~/.mxnet/datasets/ != auto
INFO:SSDEstimator:}
INFO:SSDEstimator:Saved config to /var/lib/jenkins/workspace/workspace/autogluon-tutorial-object-detection-v3/docs/_build/eval/tutorials/object_detection/2b83e566/.trial_0/config.yaml
INFO:SSDEstimator:Using transfer learning from ssd_512_resnet50_v1_coco, the other network parameters are ignored.
INFO:SSDEstimator:Start training from [Epoch 0]
INFO:SSDEstimator:[Epoch 0] Training cost: 10.248370, CrossEntropy=3.554196, SmoothL1=1.035837
INFO:SSDEstimator:[Epoch 0] Validation:
cow=0.4669284204167926
chair=0.0
motorbike=0.7602732155819976
person=0.7040791760112008
boat=0.5000000000000001
bus=0.5
bicycle=0.010008340283569643
dog=1.0000000000000002
pottedplant=0.07332251082251082
car=0.5092859417789389
mAP=0.452389760489501
INFO:SSDEstimator:[Epoch 0] Current best map: 0.452390 vs previous 0.000000, saved to /var/lib/jenkins/workspace/workspace/autogluon-tutorial-object-detection-v3/docs/_build/eval/tutorials/object_detection/2b83e566/.trial_0/best_checkpoint.pkl
INFO:SSDEstimator:Pickled to /var/lib/jenkins/workspace/workspace/autogluon-tutorial-object-detection-v3/docs/_build/eval/tutorials/object_detection/2b83e566/.trial_0/best_checkpoint.pkl
INFO:SSDEstimator:[Epoch 1] Training cost: 9.167524, CrossEntropy=2.630478, SmoothL1=1.130499
INFO:SSDEstimator:[Epoch 1] Validation:
cow=1.0000000000000002
chair=0.33333333333333326
motorbike=0.8520073104615397
person=0.7750308369839314
boat=1.0000000000000002
bus=1.0000000000000002
bicycle=0.007083825265643448
dog=1.0000000000000002
pottedplant=0.009469696969696968
car=0.6601000544078347
mAP=0.663702505742198
INFO:SSDEstimator:[Epoch 1] Current best map: 0.663703 vs previous 0.452390, saved to /var/lib/jenkins/workspace/workspace/autogluon-tutorial-object-detection-v3/docs/_build/eval/tutorials/object_detection/2b83e566/.trial_0/best_checkpoint.pkl
INFO:SSDEstimator:Pickled to /var/lib/jenkins/workspace/workspace/autogluon-tutorial-object-detection-v3/docs/_build/eval/tutorials/object_detection/2b83e566/.trial_0/best_checkpoint.pkl
INFO:SSDEstimator:[Epoch 2] Training cost: 8.933857, CrossEntropy=2.598043, SmoothL1=1.252567
INFO:SSDEstimator:[Epoch 2] Validation:
cow=0.6363636363636365
chair=0.058823529411764705
motorbike=0.8541683361934911
person=0.8026547288553467
boat=1.0000000000000002
bus=1.0000000000000002
bicycle=0.2924726962668786
dog=0.5000000000000001
pottedplant=0.024793388429752067
car=0.7834469915379487
mAP=0.5952723307058818
INFO:SSDEstimator:[Epoch 3] Training cost: 9.334927, CrossEntropy=2.380490, SmoothL1=1.114254
INFO:SSDEstimator:[Epoch 3] Validation:
cow=0.653958944281525
chair=0.0
motorbike=0.8477194840952025
person=0.8084021120514867
boat=1.0000000000000002
bus=1.0000000000000002
bicycle=0.04609714031378658
dog=1.0000000000000002
pottedplant=0.005244755244755245
car=0.7806170891186881
mAP=0.6142039525105445
INFO:SSDEstimator:[Epoch 4] Training cost: 9.253709, CrossEntropy=2.342552, SmoothL1=1.029879
INFO:SSDEstimator:[Epoch 4] Validation:
cow=1.0000000000000002
chair=0.5000000000000001
motorbike=0.8785846402607892
person=0.8090508956545207
boat=1.0000000000000002
bus=1.0000000000000002
bicycle=0.20733652312599676
dog=1.0000000000000002
pottedplant=0.026136363636363638
car=0.8646813284523287
mAP=0.7285789751130001
INFO:SSDEstimator:[Epoch 4] Current best map: 0.728579 vs previous 0.663703, saved to /var/lib/jenkins/workspace/workspace/autogluon-tutorial-object-detection-v3/docs/_build/eval/tutorials/object_detection/2b83e566/.trial_0/best_checkpoint.pkl
INFO:SSDEstimator:Pickled to /var/lib/jenkins/workspace/workspace/autogluon-tutorial-object-detection-v3/docs/_build/eval/tutorials/object_detection/2b83e566/.trial_0/best_checkpoint.pkl
INFO:gluoncv.auto.tasks.object_detection:Finished, total runtime is 81.86 s
INFO:gluoncv.auto.tasks.object_detection:{ 'best_config': { 'batch_size': 8,
'dist_ip_addrs': None,
'epochs': 5,
'final_fit': False,
'gpus': [0],
'log_dir': '/var/lib/jenkins/workspace/workspace/autogluon-tutorial-object-detection-v3/docs/_build/eval/tutorials/object_detection/2b83e566',
'lr': 0.001,
'ngpus_per_trial': 8,
'nthreads_per_trial': 128,
'num_trials': 1,
'num_workers': 8,
'search_strategy': 'random',
'seed': 363,
'time_limits': 1800,
'transfer': 'ssd_512_resnet50_v1_coco',
'wall_clock_tick': 1614914760.696897},
'total_time': 67.42109441757202,
'train_map': 0.7285789751130001,
'valid_map': 0.7285789751130001}
<autogluon.vision.detector.detector.ObjectDetector at 0x7f7da7c5c6d0>
Note that num_trials=2 above is only used to speed up the tutorial.
In normal practice, it is common to only use time_limit and drop
num_trials. Also note that hyperparameter tuning defaults to random
search. Model-based variants, such as search_strategy='bayesopt' or
search_strategy='bayesopt_hyperband' can be a lot more
sample-efficient.
After fitting, AutoGluon automatically returns the best model among all models in the searching space. From the output, we know the best model is the one trained with the second learning rate. To see how well the returned model performed on test dataset, call detector.evaluate().
dataset_test = ObjectDetector.Dataset.from_voc(url, splits='test')
test_map = detector.evaluate(dataset_test)
print("mAP on test dataset: {}".format(test_map[1][-1]))
tiny_motorbike/
├── Annotations/
├── ImageSets/
└── JPEGImages/
mAP on test dataset: 0.06417902544299443
Below, we randomly select an image from test dataset and show the
predicted class, box and probability over the origin image, stored in
predict_class, predict_rois and predict_score columns,
respectively. You can interpret predict_rois as a dict of (xmin,
ymin, xmax, ymax) proportional to original image size.
image_path = dataset_test.iloc[0]['image']
result = detector.predict(image_path)
print(result)
INFO:numexpr.utils:NumExpr defaulting to 8 threads.
predict_class predict_score 0 person 0.969693
1 motorbike 0.958952
2 car 0.318350
3 person 0.254405
4 person 0.186449
.. ... ...
75 person 0.028974
76 person 0.028904
77 person 0.028717
78 person 0.028642
79 car 0.028426
predict_rois
0 {'xmin': 0.387190580368042, 'ymin': 0.29214894...
1 {'xmin': 0.3211327791213989, 'ymin': 0.4409473...
2 {'xmin': 0.00846653338521719, 'ymin': 0.624620...
3 {'xmin': 0.4059852659702301, 'ymin': 0.2877930...
4 {'xmin': 0.9953566193580627, 'ymin': 0.4539755...
.. ...
75 {'xmin': 0.6291181445121765, 'ymin': 0.2445436...
76 {'xmin': 0.9616981744766235, 'ymin': 0.3743298...
77 {'xmin': 0.9612432718276978, 'ymin': 0.1836769...
78 {'xmin': 0.9492825269699097, 'ymin': 0.0447137...
79 {'xmin': 0.20784150063991547, 'ymin': 0.146993...
[80 rows x 3 columns]
Prediction with multiple images is permitted:
bulk_result = detector.predict(dataset_test)
print(bulk_result)
predict_class predict_score 0 person 0.969693
1 motorbike 0.958952
2 car 0.318350
3 person 0.254405
4 person 0.186449
... ... ...
3927 person 0.034960
3928 car 0.034631
3929 person 0.034552
3930 car 0.034159
3931 motorbike 0.033999
predict_rois 0 {'xmin': 0.387190580368042, 'ymin': 0.29214894...
1 {'xmin': 0.3211327791213989, 'ymin': 0.4409473...
2 {'xmin': 0.00846653338521719, 'ymin': 0.624620...
3 {'xmin': 0.4059852659702301, 'ymin': 0.2877930...
4 {'xmin': 0.9953566193580627, 'ymin': 0.4539755...
... ...
3927 {'xmin': 0.1466965526342392, 'ymin': 0.6714547...
3928 {'xmin': 0.8387094736099243, 'ymin': 0.6668641...
3929 {'xmin': 0.09305252134799957, 'ymin': 0.456205...
3930 {'xmin': 0.05045142024755478, 'ymin': 0.690966...
3931 {'xmin': 0.21304212510585785, 'ymin': 0.634236...
image
0 /var/lib/jenkins/.gluoncv/datasets/tiny_motorb...
1 /var/lib/jenkins/.gluoncv/datasets/tiny_motorb...
2 /var/lib/jenkins/.gluoncv/datasets/tiny_motorb...
3 /var/lib/jenkins/.gluoncv/datasets/tiny_motorb...
4 /var/lib/jenkins/.gluoncv/datasets/tiny_motorb...
... ...
3927 /var/lib/jenkins/.gluoncv/datasets/tiny_motorb...
3928 /var/lib/jenkins/.gluoncv/datasets/tiny_motorb...
3929 /var/lib/jenkins/.gluoncv/datasets/tiny_motorb...
3930 /var/lib/jenkins/.gluoncv/datasets/tiny_motorb...
3931 /var/lib/jenkins/.gluoncv/datasets/tiny_motorb...
[3932 rows x 4 columns]
We can also save the trained model, and use it later.
savefile = 'detector.ag'
detector.save(savefile)
new_detector = ObjectDetector.load(savefile)
/var/lib/jenkins/workspace/workspace/autogluon-tutorial-object-detection-v3/venv/lib/python3.7/site-packages/mxnet/gluon/block.py:1512: UserWarning: Cannot decide type for the following arguments. Consider providing them as input:
data: None
input_sym_arg_type = in_param.infer_type()[0]