AutoMM Detection - Finetune on COCO Format Dataset with Customized Settings¶
In this section, our goal is to fast finetune and evaluate a pretrained model
on Pothole dataset in COCO format with customized setting.
Pothole is a single object, i.e. pothole, detection dataset, containing 665 images with bounding box annotations
for the creation of detection models and can work as POC/POV for the maintenance of roads.
See AutoMM Detection - Prepare Pothole Dataset for how to prepare Pothole dataset.
To start, let’s import MultiModalPredictor and make sure mmcv and mmdet are installed:
#!pip install torch==2.1.0 torchvision==0.16.0 torchaudio==2.1.0 # To use object detection, downgrade the torch version if it's >=2.2
!mim install "mmcv==2.1.0" # For Google Colab, use the line below instead to install mmcv
#!pip install "mmcv==2.1.0" -f https://download.openmmlab.com/mmcv/dist/cu121/torch2.1.0/index.html
!pip install "mmdet==3.2.0"
Show code cell output
Looking in links: https://download.openmmlab.com/mmcv/dist/cu124/torch2.5.0/index.html
Requirement already satisfied: mmcv==2.1.0 in /home/ci/opt/venv/lib/python3.11/site-packages (2.1.0)
Requirement already satisfied: addict in /home/ci/opt/venv/lib/python3.11/site-packages (from mmcv==2.1.0) (2.4.0)
Requirement already satisfied: mmengine>=0.3.0 in /home/ci/opt/venv/lib/python3.11/site-packages (from mmcv==2.1.0) (0.10.5)
Requirement already satisfied: numpy in /home/ci/opt/venv/lib/python3.11/site-packages (from mmcv==2.1.0) (1.26.4)
Requirement already satisfied: packaging in /home/ci/opt/venv/lib/python3.11/site-packages (from mmcv==2.1.0) (24.2)
Requirement already satisfied: Pillow in /home/ci/opt/venv/lib/python3.11/site-packages (from mmcv==2.1.0) (11.0.0)
Requirement already satisfied: pyyaml in /home/ci/opt/venv/lib/python3.11/site-packages (from mmcv==2.1.0) (6.0.2)
Requirement already satisfied: yapf in /home/ci/opt/venv/lib/python3.11/site-packages (from mmcv==2.1.0) (0.43.0)
Requirement already satisfied: matplotlib in /home/ci/opt/venv/lib/python3.11/site-packages (from mmengine>=0.3.0->mmcv==2.1.0) (3.9.4)
Requirement already satisfied: rich in /home/ci/opt/venv/lib/python3.11/site-packages (from mmengine>=0.3.0->mmcv==2.1.0) (13.9.4)
Requirement already satisfied: termcolor in /home/ci/opt/venv/lib/python3.11/site-packages (from mmengine>=0.3.0->mmcv==2.1.0) (2.5.0)
Requirement already satisfied: opencv-python>=3 in /home/ci/opt/venv/lib/python3.11/site-packages (from mmengine>=0.3.0->mmcv==2.1.0) (4.10.0.84)
Requirement already satisfied: platformdirs>=3.5.1 in /home/ci/opt/venv/lib/python3.11/site-packages (from yapf->mmcv==2.1.0) (4.3.6)
Requirement already satisfied: contourpy>=1.0.1 in /home/ci/opt/venv/lib/python3.11/site-packages (from matplotlib->mmengine>=0.3.0->mmcv==2.1.0) (1.3.1)
Requirement already satisfied: cycler>=0.10 in /home/ci/opt/venv/lib/python3.11/site-packages (from matplotlib->mmengine>=0.3.0->mmcv==2.1.0) (0.12.1)
Requirement already satisfied: fonttools>=4.22.0 in /home/ci/opt/venv/lib/python3.11/site-packages (from matplotlib->mmengine>=0.3.0->mmcv==2.1.0) (4.55.3)
Requirement already satisfied: kiwisolver>=1.3.1 in /home/ci/opt/venv/lib/python3.11/site-packages (from matplotlib->mmengine>=0.3.0->mmcv==2.1.0) (1.4.7)
Requirement already satisfied: pyparsing>=2.3.1 in /home/ci/opt/venv/lib/python3.11/site-packages (from matplotlib->mmengine>=0.3.0->mmcv==2.1.0) (3.2.0)
Requirement already satisfied: python-dateutil>=2.7 in /home/ci/opt/venv/lib/python3.11/site-packages (from matplotlib->mmengine>=0.3.0->mmcv==2.1.0) (2.9.0.post0)
Requirement already satisfied: markdown-it-py>=2.2.0 in /home/ci/opt/venv/lib/python3.11/site-packages (from rich->mmengine>=0.3.0->mmcv==2.1.0) (3.0.0)
Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /home/ci/opt/venv/lib/python3.11/site-packages (from rich->mmengine>=0.3.0->mmcv==2.1.0) (2.18.0)
Requirement already satisfied: mdurl~=0.1 in /home/ci/opt/venv/lib/python3.11/site-packages (from markdown-it-py>=2.2.0->rich->mmengine>=0.3.0->mmcv==2.1.0) (0.1.2)
Requirement already satisfied: six>=1.5 in /home/ci/opt/venv/lib/python3.11/site-packages (from python-dateutil>=2.7->matplotlib->mmengine>=0.3.0->mmcv==2.1.0) (1.17.0)
Requirement already satisfied: mmdet==3.2.0 in /home/ci/opt/venv/lib/python3.11/site-packages (3.2.0)
Requirement already satisfied: matplotlib in /home/ci/opt/venv/lib/python3.11/site-packages (from mmdet==3.2.0) (3.9.4)
Requirement already satisfied: numpy in /home/ci/opt/venv/lib/python3.11/site-packages (from mmdet==3.2.0) (1.26.4)
Requirement already satisfied: pycocotools in /home/ci/opt/venv/lib/python3.11/site-packages (from mmdet==3.2.0) (2.0.8)
Requirement already satisfied: scipy in /home/ci/opt/venv/lib/python3.11/site-packages (from mmdet==3.2.0) (1.14.1)
Requirement already satisfied: shapely in /home/ci/opt/venv/lib/python3.11/site-packages (from mmdet==3.2.0) (2.0.6)
Requirement already satisfied: six in /home/ci/opt/venv/lib/python3.11/site-packages (from mmdet==3.2.0) (1.17.0)
Requirement already satisfied: terminaltables in /home/ci/opt/venv/lib/python3.11/site-packages (from mmdet==3.2.0) (3.1.10)
Requirement already satisfied: tqdm in /home/ci/opt/venv/lib/python3.11/site-packages (from mmdet==3.2.0) (4.67.1)
Requirement already satisfied: contourpy>=1.0.1 in /home/ci/opt/venv/lib/python3.11/site-packages (from matplotlib->mmdet==3.2.0) (1.3.1)
Requirement already satisfied: cycler>=0.10 in /home/ci/opt/venv/lib/python3.11/site-packages (from matplotlib->mmdet==3.2.0) (0.12.1)
Requirement already satisfied: fonttools>=4.22.0 in /home/ci/opt/venv/lib/python3.11/site-packages (from matplotlib->mmdet==3.2.0) (4.55.3)
Requirement already satisfied: kiwisolver>=1.3.1 in /home/ci/opt/venv/lib/python3.11/site-packages (from matplotlib->mmdet==3.2.0) (1.4.7)
Requirement already satisfied: packaging>=20.0 in /home/ci/opt/venv/lib/python3.11/site-packages (from matplotlib->mmdet==3.2.0) (24.2)
Requirement already satisfied: pillow>=8 in /home/ci/opt/venv/lib/python3.11/site-packages (from matplotlib->mmdet==3.2.0) (11.0.0)
Requirement already satisfied: pyparsing>=2.3.1 in /home/ci/opt/venv/lib/python3.11/site-packages (from matplotlib->mmdet==3.2.0) (3.2.0)
Requirement already satisfied: python-dateutil>=2.7 in /home/ci/opt/venv/lib/python3.11/site-packages (from matplotlib->mmdet==3.2.0) (2.9.0.post0)
from autogluon.multimodal import MultiModalPredictor
/home/ci/opt/venv/lib/python3.11/site-packages/mmengine/optim/optimizer/zero_optimizer.py:11: DeprecationWarning: `TorchScript` support for functional optimizers is deprecated and will be removed in a future PyTorch release. Consider using the `torch.compile` optimizer instead.
from torch.distributed.optim import \
And also import some other packages that will be used in this tutorial:
import os
from autogluon.core.utils.loaders import load_zip
We have the sample dataset ready in the cloud. Let’s download it and store the paths for each data split:
zip_file = "https://automl-mm-bench.s3.amazonaws.com/object_detection/dataset/pothole.zip"
download_dir = "./pothole"
load_zip.unzip(zip_file, unzip_dir=download_dir)
data_dir = os.path.join(download_dir, "pothole")
train_path = os.path.join(data_dir, "Annotations", "usersplit_train_cocoformat.json")
val_path = os.path.join(data_dir, "Annotations", "usersplit_val_cocoformat.json")
test_path = os.path.join(data_dir, "Annotations", "usersplit_test_cocoformat.json")
Downloading ./pothole/file.zip from https://automl-mm-bench.s3.amazonaws.com/object_detection/dataset/pothole.zip...
0%| | 0.00/351M [00:00<?, ?iB/s]
3%|▎ | 10.6M/351M [00:00<00:03, 106MiB/s]
6%|▌ | 21.1M/351M [00:00<00:06, 53.0MiB/s]
9%|▉ | 31.9M/351M [00:00<00:04, 69.2MiB/s]
12%|█▏ | 40.4M/351M [00:01<00:10, 30.4MiB/s]
14%|█▍ | 48.5M/351M [00:01<00:08, 33.7MiB/s]
15%|█▌ | 53.7M/351M [00:01<00:09, 31.4MiB/s]
17%|█▋ | 58.1M/351M [00:02<00:15, 18.6MiB/s]
17%|█▋ | 61.4M/351M [00:02<00:14, 20.2MiB/s]
20%|█▉ | 68.7M/351M [00:02<00:12, 23.5MiB/s]
23%|██▎ | 79.3M/351M [00:02<00:07, 35.5MiB/s]
25%|██▍ | 86.2M/351M [00:02<00:06, 41.3MiB/s]
28%|██▊ | 97.2M/351M [00:02<00:04, 54.9MiB/s]
30%|██▉ | 105M/351M [00:02<00:06, 38.6MiB/s]
33%|███▎ | 116M/351M [00:03<00:04, 50.6MiB/s]
35%|███▌ | 123M/351M [00:03<00:05, 38.8MiB/s]
38%|███▊ | 134M/351M [00:03<00:04, 50.3MiB/s]
40%|████ | 141M/351M [00:03<00:05, 38.2MiB/s]
43%|████▎ | 152M/351M [00:03<00:03, 50.0MiB/s]
46%|████▌ | 160M/351M [00:04<00:04, 47.5MiB/s]
49%|████▊ | 170M/351M [00:04<00:04, 42.4MiB/s]
50%|█████ | 176M/351M [00:04<00:04, 42.1MiB/s]
52%|█████▏ | 183M/351M [00:04<00:03, 47.3MiB/s]
54%|█████▍ | 189M/351M [00:04<00:03, 43.5MiB/s]
55%|█████▌ | 194M/351M [00:04<00:03, 41.4MiB/s]
57%|█████▋ | 199M/351M [00:05<00:04, 32.1MiB/s]
58%|█████▊ | 203M/351M [00:05<00:04, 35.3MiB/s]
59%|█████▉ | 208M/351M [00:05<00:05, 24.8MiB/s]
60%|██████ | 211M/351M [00:05<00:05, 24.1MiB/s]
61%|██████ | 214M/351M [00:05<00:05, 23.8MiB/s]
64%|██████▎ | 223M/351M [00:06<00:03, 32.7MiB/s]
66%|██████▋ | 233M/351M [00:06<00:02, 44.5MiB/s]
69%|██████▊ | 240M/351M [00:06<00:02, 50.3MiB/s]
70%|███████ | 246M/351M [00:06<00:02, 41.1MiB/s]
73%|███████▎ | 257M/351M [00:06<00:01, 55.5MiB/s]
75%|███████▌ | 264M/351M [00:06<00:01, 51.4MiB/s]
78%|███████▊ | 274M/351M [00:06<00:01, 62.3MiB/s]
80%|████████ | 281M/351M [00:07<00:01, 50.4MiB/s]
83%|████████▎ | 290M/351M [00:07<00:01, 53.0MiB/s]
84%|████████▍ | 296M/351M [00:07<00:01, 48.3MiB/s]
88%|████████▊ | 308M/351M [00:07<00:00, 61.8MiB/s]
90%|████████▉ | 315M/351M [00:07<00:00, 42.9MiB/s]
93%|█████████▎| 326M/351M [00:07<00:00, 55.7MiB/s]
95%|█████████▍| 333M/351M [00:08<00:00, 31.4MiB/s]
98%|█████████▊| 344M/351M [00:08<00:00, 37.8MiB/s]
100%|█████████▉| 350M/351M [00:08<00:00, 39.5MiB/s]
100%|██████████| 351M/351M [00:08<00:00, 40.0MiB/s]
While using COCO format dataset, the input is the json annotation file of the dataset split.
In this example, usersplit_train_cocoformat.json is the annotation file of the train split.
usersplit_val_cocoformat.json is the annotation file of the validation split.
And usersplit_test_cocoformat.json is the annotation file of the test split.
We select the YOLOX-small model pretrained on COCO dataset. With this setting, it is fast to finetune or inference,
and easy to deploy. Note that you can use a larger model by setting the checkpoint_name to corresponding checkpoint name for better performance (but usually with slower speed).
And you may need to change the lr and per_gpu_batch_size for a different model.
An easier way is to use our predefined presets "medium_quality", "high_quality", or "best_quality".
For more about using presets, see Quick Start Coco.
checkpoint_name = "yolox_s"
num_gpus = 1 # only use one GPU
We create the MultiModalPredictor with selected checkpoint name and number of GPUs.
We need to specify the problem_type to "object_detection",
and also provide a sample_data_path for the predictor to infer the categories of the dataset.
Here we provide the train_path, and it also works using any other split of this dataset.
predictor = MultiModalPredictor(
hyperparameters={
"model.mmdet_image.checkpoint_name": checkpoint_name,
"env.num_gpus": num_gpus,
},
problem_type="object_detection",
sample_data_path=train_path,
)
We set the learning rate to be 1e-4.
Note that we use a two-stage learning rate option during finetuning by default,
and the model head will have 100x learning rate.
Using a two-stage learning rate with high learning rate only on head layers makes
the model converge faster during finetuning. It usually gives better performance as well,
especially on small datasets with hundreds or thousands of images.
We set batch size to be 16, and you can increase or decrease the batch size based on your available GPU memory.
We set max number of epochs to 30, number of validation check per interval to 1.0,
and validation check per n epochs to 3 for fast finetuning.
We also compute the time of the fit process here for better understanding the speed.
predictor.fit(
train_path,
tuning_data=val_path,
hyperparameters={
"optim.lr": 1e-4, # we use two stage and detection head has 100x lr
"env.per_gpu_batch_size": 16, # decrease it when model is large or GPU memory is small
"optim.max_epochs": 30, # max number of training epochs, note that we may early stop before this based on validation setting
"optim.val_check_interval": 1.0, # Do 1 validation each epoch
"optim.check_val_every_n_epoch": 3, # Do 1 validation each 3 epochs
"optim.patience": 3, # Early stop after 3 consective validations are not the best
},
)
loading annotations into memory...
Done (t=0.24s)
creating index...
index created!
loading annotations into memory...
Done (t=0.00s)
creating index...
index created!
Downloading yolox_s_8x8_300e_coco_20211121_095711-4592a793.pth from https://download.openmmlab.com/mmdetection/v2.0/yolox/yolox_s_8x8_300e_coco/yolox_s_8x8_300e_coco_20211121_095711-4592a793.pth...
Loads checkpoint by local backend from path: yolox_s_8x8_300e_coco_20211121_095711-4592a793.pth
The model and loaded state dict do not match exactly
size mismatch for bbox_head.multi_level_conv_cls.0.weight: copying a param with shape torch.Size([80, 128, 1, 1]) from checkpoint, the shape in current model is torch.Size([1, 128, 1, 1]).
size mismatch for bbox_head.multi_level_conv_cls.0.bias: copying a param with shape torch.Size([80]) from checkpoint, the shape in current model is torch.Size([1]).
size mismatch for bbox_head.multi_level_conv_cls.1.weight: copying a param with shape torch.Size([80, 128, 1, 1]) from checkpoint, the shape in current model is torch.Size([1, 128, 1, 1]).
size mismatch for bbox_head.multi_level_conv_cls.1.bias: copying a param with shape torch.Size([80]) from checkpoint, the shape in current model is torch.Size([1]).
size mismatch for bbox_head.multi_level_conv_cls.2.weight: copying a param with shape torch.Size([80, 128, 1, 1]) from checkpoint, the shape in current model is torch.Size([1, 128, 1, 1]).
size mismatch for bbox_head.multi_level_conv_cls.2.bias: copying a param with shape torch.Size([80]) from checkpoint, the shape in current model is torch.Size([1]).
=================== System Info ===================
AutoGluon Version: 1.2b20241213
Python Version: 3.11.9
Operating System: Linux
Platform Machine: x86_64
Platform Version: #1 SMP Tue Sep 24 10:00:37 UTC 2024
CPU Count: 8
Pytorch Version: 2.5.1+cu124
CUDA Version: 12.4
Memory Avail: 28.42 GB / 30.95 GB (91.8%)
===================================================
No path specified. Models will be saved in: "AutogluonModels/ag-20241213_073534"
Using default root folder: ./pothole/pothole/Annotations/... Specify `model.mmdet_image.coco_root=...` in hyperparameters if you think it is wrong.
Using default root folder: ./pothole/pothole/Annotations/... Specify `model.mmdet_image.coco_root=...` in hyperparameters if you think it is wrong.
AutoMM starts to create your model. ✨✨✨
To track the learning progress, you can open a terminal and launch Tensorboard:
```shell
# Assume you have installed tensorboard
tensorboard --logdir /home/ci/autogluon/docs/tutorials/multimodal/object_detection/advanced/AutogluonModels/ag-20241213_073534
```
Seed set to 0
0%| | 0.00/36.1M [00:00<?, ?iB/s]
0%| | 180k/36.1M [00:00<00:20, 1.72MiB/s]
1%| | 403k/36.1M [00:00<00:17, 2.01MiB/s]
2%|▏ | 698k/36.1M [00:00<00:14, 2.42MiB/s]
3%|▎ | 1.03M/36.1M [00:00<00:12, 2.72MiB/s]
4%|▍ | 1.39M/36.1M [00:00<00:11, 3.02MiB/s]
5%|▌ | 1.81M/36.1M [00:00<00:10, 3.42MiB/s]
6%|▋ | 2.27M/36.1M [00:00<00:09, 3.74MiB/s]
8%|▊ | 2.80M/36.1M [00:00<00:07, 4.20MiB/s]
9%|▉ | 3.39M/36.1M [00:00<00:06, 4.69MiB/s]
11%|█ | 4.04M/36.1M [00:01<00:06, 5.23MiB/s]
13%|█▎ | 4.78M/36.1M [00:01<00:05, 5.85MiB/s]
15%|█▌ | 5.55M/36.1M [00:01<00:04, 6.40MiB/s]
18%|█▊ | 6.35M/36.1M [00:01<00:04, 6.85MiB/s]
20%|█▉ | 7.15M/36.1M [00:01<00:04, 7.19MiB/s]
22%|██▏ | 7.99M/36.1M [00:01<00:03, 7.53MiB/s]
24%|██▍ | 8.82M/36.1M [00:01<00:03, 7.75MiB/s]
27%|██▋ | 9.66M/36.1M [00:01<00:03, 7.94MiB/s]
29%|██▉ | 10.5M/36.1M [00:01<00:03, 7.99MiB/s]
31%|███▏ | 11.3M/36.1M [00:01<00:03, 8.14MiB/s]
34%|███▍ | 12.2M/36.1M [00:02<00:02, 8.00MiB/s]
36%|███▌ | 13.1M/36.1M [00:02<00:02, 8.23MiB/s]
39%|███▊ | 14.0M/36.1M [00:02<00:02, 8.26MiB/s]
41%|████ | 14.8M/36.1M [00:02<00:02, 8.36MiB/s]
44%|████▎ | 15.7M/36.1M [00:02<00:02, 8.51MiB/s]
46%|████▌ | 16.6M/36.1M [00:02<00:02, 8.30MiB/s]
48%|████▊ | 17.4M/36.1M [00:02<00:02, 8.41MiB/s]
51%|█████ | 18.3M/36.1M [00:02<00:02, 8.57MiB/s]
53%|█████▎ | 19.2M/36.1M [00:02<00:02, 8.38MiB/s]
56%|█████▌ | 20.1M/36.1M [00:02<00:01, 8.44MiB/s]
58%|█████▊ | 20.9M/36.1M [00:03<00:01, 8.35MiB/s]
60%|██████ | 21.8M/36.1M [00:03<00:01, 8.48MiB/s]
63%|██████▎ | 22.7M/36.1M [00:03<00:01, 8.32MiB/s]
65%|██████▌ | 23.6M/36.1M [00:03<00:01, 8.44MiB/s]
68%|██████▊ | 24.5M/36.1M [00:03<00:01, 8.35MiB/s]
70%|███████ | 25.3M/36.1M [00:03<00:01, 8.49MiB/s]
73%|███████▎ | 26.2M/36.1M [00:03<00:01, 8.36MiB/s]
75%|███████▌ | 27.1M/36.1M [00:03<00:01, 8.50MiB/s]
78%|███████▊ | 28.0M/36.1M [00:03<00:00, 8.33MiB/s]
80%|████████ | 28.9M/36.1M [00:04<00:00, 8.47MiB/s]
82%|████████▏ | 29.8M/36.1M [00:04<00:00, 8.38MiB/s]
85%|████████▍ | 30.7M/36.1M [00:04<00:00, 8.55MiB/s]
87%|████████▋ | 31.5M/36.1M [00:04<00:00, 8.33MiB/s]
90%|████████▉ | 32.4M/36.1M [00:04<00:00, 8.50MiB/s]
92%|█████████▏| 33.3M/36.1M [00:04<00:00, 8.41MiB/s]
95%|█████████▍| 34.2M/36.1M [00:04<00:00, 8.52MiB/s]
97%|█████████▋| 35.1M/36.1M [00:04<00:00, 8.33MiB/s]
100%|█████████▉| 35.9M/36.1M [00:04<00:00, 8.42MiB/s]
/home/ci/opt/venv/lib/python3.11/site-packages/mmengine/runner/checkpoint.py:347: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
checkpoint = torch.load(filename, map_location=map_location)
GPU Count: 1
GPU Count to be Used: 1
Using 16bit Automatic Mixed Precision (AMP)
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
HPU available: False, using: 0 HPUs
`Trainer(val_check_interval=1.0)` was configured so validation will run at the end of the training epoch..
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
| Name | Type | Params | Mode
-------------------------------------------------------------------------------
0 | model | MMDetAutoModelForObjectDetection | 8.9 M | train
1 | validation_metric | MeanAveragePrecision | 0 | train
-------------------------------------------------------------------------------
8.9 M Trainable params
0 Non-trainable params
8.9 M Total params
35.751 Total estimated model params size (MB)
376 Modules in train mode
0 Modules in eval mode
/home/ci/opt/venv/lib/python3.11/site-packages/mmdet/models/backbones/csp_darknet.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
with torch.cuda.amp.autocast(enabled=False):
/home/ci/opt/venv/lib/python3.11/site-packages/torch/functional.py:534: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:3595.)
return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined]
/home/ci/opt/venv/lib/python3.11/site-packages/torchmetrics/utilities/prints.py:43: UserWarning: Encountered more than 100 detections in a single image. This means that certain detections with the lowest scores will be ignored, that may have an undesirable impact on performance. Please consider adjusting the `max_detection_threshold` to suit your use case. To disable this warning, set attribute class `warn_on_many_detections=False`, after initializing the metric.
warnings.warn(*args, **kwargs) # noqa: B028
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
Cell In[8], line 1
----> 1 predictor.fit(
2 train_path,
3 tuning_data=val_path,
4 hyperparameters={
5 "optim.lr": 1e-4, # we use two stage and detection head has 100x lr
6 "env.per_gpu_batch_size": 16, # decrease it when model is large or GPU memory is small
7 "optim.max_epochs": 30, # max number of training epochs, note that we may early stop before this based on validation setting
8 "optim.val_check_interval": 1.0, # Do 1 validation each epoch
9 "optim.check_val_every_n_epoch": 3, # Do 1 validation each 3 epochs
10 "optim.patience": 3, # Early stop after 3 consective validations are not the best
11 },
12 )
File ~/autogluon/multimodal/src/autogluon/multimodal/predictor.py:529, in MultiModalPredictor.fit(self, train_data, presets, tuning_data, max_num_tuning_data, id_mappings, time_limit, save_path, hyperparameters, column_types, holdout_frac, teacher_predictor, seed, standalone, hyperparameter_tune_kwargs, clean_ckpts, predictions, labels, predictors)
526 assert isinstance(predictors, list)
527 learners = [ele if isinstance(ele, str) else ele._learner for ele in predictors]
--> 529 self._learner.fit(
530 train_data=train_data,
531 presets=presets,
532 tuning_data=tuning_data,
533 max_num_tuning_data=max_num_tuning_data,
534 time_limit=time_limit,
535 save_path=save_path,
536 hyperparameters=hyperparameters,
537 column_types=column_types,
538 holdout_frac=holdout_frac,
539 teacher_learner=teacher_learner,
540 seed=seed,
541 standalone=standalone,
542 hyperparameter_tune_kwargs=hyperparameter_tune_kwargs,
543 clean_ckpts=clean_ckpts,
544 id_mappings=id_mappings,
545 predictions=predictions,
546 labels=labels,
547 learners=learners,
548 )
550 return self
File ~/autogluon/multimodal/src/autogluon/multimodal/learners/object_detection.py:243, in ObjectDetectionLearner.fit(self, train_data, presets, tuning_data, max_num_tuning_data, time_limit, save_path, hyperparameters, column_types, holdout_frac, seed, standalone, hyperparameter_tune_kwargs, clean_ckpts, **kwargs)
236 self.fit_sanity_check()
237 self.prepare_fit_args(
238 time_limit=time_limit,
239 seed=seed,
240 standalone=standalone,
241 clean_ckpts=clean_ckpts,
242 )
--> 243 fit_returns = self.execute_fit()
244 self.on_fit_end(
245 training_start=training_start,
246 strategy=fit_returns.get("strategy", None),
(...)
249 clean_ckpts=clean_ckpts,
250 )
252 return self
File ~/autogluon/multimodal/src/autogluon/multimodal/learners/base.py:577, in BaseLearner.execute_fit(self)
575 return dict()
576 else:
--> 577 attributes = self.fit_per_run(**self._fit_args)
578 self.update_attributes(**attributes) # only update attributes for non-HPO mode
579 return attributes
File ~/autogluon/multimodal/src/autogluon/multimodal/learners/object_detection.py:438, in ObjectDetectionLearner.fit_per_run(self, max_time, save_path, ckpt_path, resume, enable_progress_bar, seed, hyperparameters, advanced_hyperparameters, config, df_preprocessor, data_processors, model, standalone, clean_ckpts)
419 config = self.post_update_config_per_run(
420 config=config,
421 num_gpus=num_gpus,
422 precision=precision,
423 strategy=strategy,
424 )
425 trainer = self.init_trainer_per_run(
426 num_gpus=num_gpus,
427 config=config,
(...)
435 enable_progress_bar=enable_progress_bar,
436 )
--> 438 self.run_trainer(
439 trainer=trainer,
440 litmodule=litmodule,
441 datamodule=datamodule,
442 ckpt_path=ckpt_path,
443 resume=resume,
444 )
445 self.on_fit_per_run_end(
446 save_path=save_path,
447 standalone=standalone,
(...)
452 model=model,
453 )
455 return dict(
456 config=config,
457 df_preprocessor=df_preprocessor,
(...)
461 strategy=strategy,
462 )
File ~/autogluon/multimodal/src/autogluon/multimodal/learners/base.py:1211, in BaseLearner.run_trainer(self, trainer, litmodule, datamodule, ckpt_path, resume, pred_writer, is_train)
1209 warnings.filterwarnings("ignore", filter)
1210 if is_train:
-> 1211 trainer.fit(
1212 litmodule,
1213 datamodule=datamodule,
1214 ckpt_path=ckpt_path if resume else None, # this is to resume training that was broken accidentally
1215 )
1216 else:
1217 blacklist_msgs = []
File ~/opt/venv/lib/python3.11/site-packages/lightning/pytorch/trainer/trainer.py:538, in Trainer.fit(self, model, train_dataloaders, val_dataloaders, datamodule, ckpt_path)
536 self.state.status = TrainerStatus.RUNNING
537 self.training = True
--> 538 call._call_and_handle_interrupt(
539 self, self._fit_impl, model, train_dataloaders, val_dataloaders, datamodule, ckpt_path
540 )
File ~/opt/venv/lib/python3.11/site-packages/lightning/pytorch/trainer/call.py:47, in _call_and_handle_interrupt(trainer, trainer_fn, *args, **kwargs)
45 if trainer.strategy.launcher is not None:
46 return trainer.strategy.launcher.launch(trainer_fn, *args, trainer=trainer, **kwargs)
---> 47 return trainer_fn(*args, **kwargs)
49 except _TunerExitException:
50 _call_teardown_hook(trainer)
File ~/opt/venv/lib/python3.11/site-packages/lightning/pytorch/trainer/trainer.py:574, in Trainer._fit_impl(self, model, train_dataloaders, val_dataloaders, datamodule, ckpt_path)
567 assert self.state.fn is not None
568 ckpt_path = self._checkpoint_connector._select_ckpt_path(
569 self.state.fn,
570 ckpt_path,
571 model_provided=True,
572 model_connected=self.lightning_module is not None,
573 )
--> 574 self._run(model, ckpt_path=ckpt_path)
576 assert self.state.stopped
577 self.training = False
File ~/opt/venv/lib/python3.11/site-packages/lightning/pytorch/trainer/trainer.py:981, in Trainer._run(self, model, ckpt_path)
976 self._signal_connector.register_signal_handlers()
978 # ----------------------------
979 # RUN THE TRAINER
980 # ----------------------------
--> 981 results = self._run_stage()
983 # ----------------------------
984 # POST-Training CLEAN UP
985 # ----------------------------
986 log.debug(f"{self.__class__.__name__}: trainer tearing down")
File ~/opt/venv/lib/python3.11/site-packages/lightning/pytorch/trainer/trainer.py:1025, in Trainer._run_stage(self)
1023 self._run_sanity_check()
1024 with torch.autograd.set_detect_anomaly(self._detect_anomaly):
-> 1025 self.fit_loop.run()
1026 return None
1027 raise RuntimeError(f"Unexpected state {self.state}")
File ~/opt/venv/lib/python3.11/site-packages/lightning/pytorch/loops/fit_loop.py:205, in _FitLoop.run(self)
203 try:
204 self.on_advance_start()
--> 205 self.advance()
206 self.on_advance_end()
207 self._restarting = False
File ~/opt/venv/lib/python3.11/site-packages/lightning/pytorch/loops/fit_loop.py:363, in _FitLoop.advance(self)
361 with self.trainer.profiler.profile("run_training_epoch"):
362 assert self._data_fetcher is not None
--> 363 self.epoch_loop.run(self._data_fetcher)
File ~/opt/venv/lib/python3.11/site-packages/lightning/pytorch/loops/training_epoch_loop.py:140, in _TrainingEpochLoop.run(self, data_fetcher)
138 while not self.done:
139 try:
--> 140 self.advance(data_fetcher)
141 self.on_advance_end(data_fetcher)
142 self._restarting = False
File ~/opt/venv/lib/python3.11/site-packages/lightning/pytorch/loops/training_epoch_loop.py:212, in _TrainingEpochLoop.advance(self, data_fetcher)
210 else:
211 dataloader_iter = None
--> 212 batch, _, __ = next(data_fetcher)
213 # TODO: we should instead use the batch_idx returned by the fetcher, however, that will require saving the
214 # fetcher state so that the batch_idx is correct after restarting
215 batch_idx = self.batch_idx + 1
File ~/opt/venv/lib/python3.11/site-packages/lightning/pytorch/loops/fetchers.py:133, in _PrefetchDataFetcher.__next__(self)
130 self.done = not self.batches
131 elif not self.done:
132 # this will run only when no pre-fetching was done.
--> 133 batch = super().__next__()
134 else:
135 # the iterator is empty
136 raise StopIteration
File ~/opt/venv/lib/python3.11/site-packages/lightning/pytorch/loops/fetchers.py:60, in _DataFetcher.__next__(self)
58 self._start_profiler()
59 try:
---> 60 batch = next(self.iterator)
61 except StopIteration:
62 self.done = True
File ~/opt/venv/lib/python3.11/site-packages/lightning/pytorch/utilities/combined_loader.py:341, in CombinedLoader.__next__(self)
339 def __next__(self) -> _ITERATOR_RETURN:
340 assert self._iterator is not None
--> 341 out = next(self._iterator)
342 if isinstance(self._iterator, _Sequential):
343 return out
File ~/opt/venv/lib/python3.11/site-packages/lightning/pytorch/utilities/combined_loader.py:78, in _MaxSizeCycle.__next__(self)
76 for i in range(n):
77 try:
---> 78 out[i] = next(self.iterators[i])
79 except StopIteration:
80 self._consumed[i] = True
File ~/opt/venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:701, in _BaseDataLoaderIter.__next__(self)
698 if self._sampler_iter is None:
699 # TODO(https://github.com/pytorch/pytorch/issues/76750)
700 self._reset() # type: ignore[call-arg]
--> 701 data = self._next_data()
702 self._num_yielded += 1
703 if (
704 self._dataset_kind == _DatasetKind.Iterable
705 and self._IterableDataset_len_called is not None
706 and self._num_yielded > self._IterableDataset_len_called
707 ):
File ~/opt/venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:1465, in _MultiProcessingDataLoaderIter._next_data(self)
1463 else:
1464 del self._task_info[idx]
-> 1465 return self._process_data(data)
File ~/opt/venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:1491, in _MultiProcessingDataLoaderIter._process_data(self, data)
1489 self._try_put_index()
1490 if isinstance(data, ExceptionWrapper):
-> 1491 data.reraise()
1492 return data
File ~/opt/venv/lib/python3.11/site-packages/torch/_utils.py:715, in ExceptionWrapper.reraise(self)
711 except TypeError:
712 # If the exception takes multiple arguments, don't try to
713 # instantiate since we don't know how to
714 raise RuntimeError(msg) from None
--> 715 raise exception
TypeError: Caught TypeError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/opt/venv/lib/python3.11/site-packages/torch/utils/data/_utils/worker.py", line 351, in _worker_loop
data = fetcher.fetch(index) # type: ignore[possibly-undefined]
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/opt/venv/lib/python3.11/site-packages/torch/utils/data/_utils/fetch.py", line 52, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/opt/venv/lib/python3.11/site-packages/torch/utils/data/_utils/fetch.py", line 52, in <listcomp>
data = [self.dataset[idx] for idx in possibly_batched_index]
~~~~~~~~~~~~^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 148, in _load_item
raise e
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
To evaluate the model we just trained, run:
predictor.evaluate(test_path)
Note that it’s always recommended to use our predefined presets to save customization time with following code script:
predictor = MultiModalPredictor(
problem_type="object_detection",
sample_data_path=train_path,
presets="medium_quality",
)
predictor.fit(train_path, tuning_data=val_path)
predictor.evaluate(test_path)
For more about using presets, see Quick Start Coco.
And the evaluation results are shown in command line output. The first value is mAP in COCO standard, and the second one is mAP in VOC standard (or mAP50). For more details about these metrics, see COCO’s evaluation guideline.
We can get the prediction on test set:
pred = predictor.predict(test_path)
Let’s also visualize the prediction result:
!pip install opencv-python
from autogluon.multimodal.utils import visualize_detection
conf_threshold = 0.25 # Specify a confidence threshold to filter out unwanted boxes
visualization_result_dir = "./" # Use the pwd as result dir to save the visualized image
visualized = visualize_detection(
pred=pred[12:13],
detection_classes=predictor.classes,
conf_threshold=conf_threshold,
visualization_result_dir=visualization_result_dir,
)
from PIL import Image
from IPython.display import display
img = Image.fromarray(visualized[0][:, :, ::-1], 'RGB')
display(img)
Under this fast finetune setting, we reached a good mAP number on a new dataset with a few hundred seconds!
For how to finetune with higher performance,
see AutoMM Detection - High Performance Finetune on COCO Format Dataset, where we finetuned a VFNet model with
5 hours and reached mAP = 0.450, mAP50 = 0.718 on this dataset.
Other Examples¶
You may go to AutoMM Examples to explore other examples about AutoMM.
Customization¶
To learn how to customize AutoMM, please refer to Customize AutoMM.
Citation¶
@article{DBLP:journals/corr/abs-2107-08430,
author = {Zheng Ge and
Songtao Liu and
Feng Wang and
Zeming Li and
Jian Sun},
title = {{YOLOX:} Exceeding {YOLO} Series in 2021},
journal = {CoRR},
volume = {abs/2107.08430},
year = {2021},
url = {https://arxiv.org/abs/2107.08430},
eprinttype = {arXiv},
eprint = {2107.08430},
timestamp = {Tue, 05 Apr 2022 14:09:44 +0200},
biburl = {https://dblp.org/rec/journals/corr/abs-2107-08430.bib},
bibsource = {dblp computer science bibliography, https://dblp.org},
}