Inference using IceVision
Install IceVision
The following downloads and runs a short shell script. The script installs IceVision, IceData, the MMDetection library, and Yolo v5 as well as the fastai and pytorch lightning engines.
Install from pypi...
# Torch - Torchvision - IceVision - IceData - MMDetection - YOLOv5 - EfficientDet Installation
!wget https://raw.githubusercontent.com/airctic/icevision/master/icevision_install.sh
# Choose your installation target: cuda11 or cuda10 or cpu
!bash icevision_install.sh cuda11
... or from icevision master
# # Torch - Torchvision - IceVision - IceData - MMDetection - YOLOv5 - EfficientDet Installation
# !wget https://raw.githubusercontent.com/airctic/icevision/master/icevision_install.sh
# # Choose your installation target: cuda11 or cuda10 or cpu
# !bash icevision_install.sh cuda11 master
# Restart kernel after installation
import IPython
IPython.Application.instance().kernel.do_shutdown(True)
Imports
All of the IceVision components can be easily imported with a single line.
from icevision.all import *
The history saving thread hit an unexpected error (DatabaseError('database disk image is malformed')).History will not be written to the database.
[1m[1mINFO [0m[1m[0m - [1mThe mmdet config folder already exists. No need to downloaded it. Path : /home/farid/.icevision/mmdetection_configs/mmdetection_configs-2.10.0/configs[0m | [36micevision.models.mmdet.download_configs[0m:[36mdownload_mmdet_configs[0m:[36m17[0m
List of images for inference
Please store your images in a folder, and populate the path_to_folder variable with the corresponding folder name.
# Pick your images folder
path_to_image_folder = "./images"
img_files = get_image_files(path_to_image_folder)
# img_files
img = PIL.Image.open(img_files[4])
img
Loading a checkpoint and creating the corresponding model
The checkpoint file can either a local file or an URL
# checkpoint_path = 'checkpoints/fridge-retinanet-save-checkpoint-full.pth'
checkpoint_path = 'http://download.openmmlab.com/mmdetection/v2.0/retinanet/retinanet_r50_fpn_1x_coco/retinanet_r50_fpn_1x_coco_20200130-c2398f9e.pth'
from icevision.models import *
# The model is automatically recreated in the evaluation mode. To unset that mode, you only need to pass `eval_mode=Fales`)
checkpoint_and_model = model_from_checkpoint(checkpoint_path,
model_name='mmdet.retinanet',
backbone_name='resnet50_fpn_1x',
img_size=640,
is_coco=True)
# Just logging the info
model_type = checkpoint_and_model["model_type"]
backbone = checkpoint_and_model["backbone"]
class_map = checkpoint_and_model["class_map"]
img_size = checkpoint_and_model["img_size"]
model_type, backbone, class_map, img_size
Use load_from_http loader
/home/farid/anaconda3/envs/icevision/lib/python3.8/site-packages/mmcv/cnn/utils/weight_init.py:99: UserWarning: init_cfg without layer key, if you do not define override key either, this init_cfg will do nothing
warnings.warn(
(<module 'icevision.models.mmdet.models.retinanet' from '/home/farid/repos/airctic/icevision/icevision/models/mmdet/models/retinanet/__init__.py'>,
<icevision.models.mmdet.models.retinanet.backbones.resnet_fpn.MMDetRetinanetBackboneConfig at 0x7fa2024c34f0>,
<ClassMap: {'background': 0, 'person': 1, 'bicycle': 2, 'car': 3, 'motorcycle': 4, 'airplane': 5, 'bus': 6, 'train': 7, 'truck': 8, 'boat': 9, 'traffic light': 10, 'fire hydrant': 11, 'stop sign': 12, 'parking meter': 13, 'bench': 14, 'bird': 15, 'cat': 16, 'dog': 17, 'horse': 18, 'sheep': 19, 'cow': 20, 'elephant': 21, 'bear': 22, 'zebra': 23, 'giraffe': 24, 'backpack': 25, 'umbrella': 26, 'handbag': 27, 'tie': 28, 'suitcase': 29, 'frisbee': 30, 'skis': 31, 'snowboard': 32, 'sports ball': 33, 'kite': 34, 'baseball bat': 35, 'baseball glove': 36, 'skateboard': 37, 'surfboard': 38, 'tennis racket': 39, 'bottle': 40, 'wine glass': 41, 'cup': 42, 'fork': 43, 'knife': 44, 'spoon': 45, 'bowl': 46, 'banana': 47, 'apple': 48, 'sandwich': 49, 'orange': 50, 'broccoli': 51, 'carrot': 52, 'hot dog': 53, 'pizza': 54, 'donut': 55, 'cake': 56, 'chair': 57, 'couch': 58, 'potted plant': 59, 'bed': 60, 'dining table': 61, 'toilet': 62, 'tv': 63, 'laptop': 64, 'mouse': 65, 'remote': 66, 'keyboard': 67, 'cell phone': 68, 'microwave': 69, 'oven': 70, 'toaster': 71, 'sink': 72, 'refrigerator': 73, 'book': 74, 'clock': 75, 'vase': 76, 'scissors': 77, 'teddy bear': 78, 'hair drier': 79, 'toothbrush': 80}>,
640)
Get Model Object
model_from_checkpoint(checkpoint_path) returns a dictionary: checkpoint_and_model. The model object is stored in checkpoint_and_model["model"].
# Get model object
# The model is automatically set in the evaluation mode
model = checkpoint_and_model["model"]
# Check device
device=next(model.parameters()).device
device
# Transforms
img_size = checkpoint_and_model["img_size"]
valid_tfms = tfms.A.Adapter([*tfms.A.resize_and_pad(img_size), tfms.A.Normalize()])
device(type='cpu')
Single Image Inference
The end2end_detect() not only compute predictions for a single image but also automatically adjust predicted boxes to the original image size
img = PIL.Image.open(img_files[4])
pred_dict = model_type.end2end_detect(img, valid_tfms, model, class_map=class_map, detection_threshold=0.5)
pred_dict['img']
Batch Inference
The following option shows to do generate inference for a set of images. The latter is processed in batches.
# Create a dataset
imgs_array = [PIL.Image.open(Path(fname)) for fname in img_files]
infer_ds = Dataset.from_images(imgs_array, valid_tfms, class_map=class_map)
# Batch Inference
infer_dl = model_type.infer_dl(infer_ds, batch_size=4, shuffle=False)
preds = model_type.predict_from_dl(model, infer_dl, keep_images=True)
show_preds(preds=preds[0:5])
0%| | 0/2 [00:00<?, ?it/s]
How to save a model and its metadata in IceVision
When saving a model weights, we could also store the model metadata that are retrieved by the model_from_checkpoint(checkpoint_path) method
# How to save a model and its metadata
checkpoint_path = 'coco-retinanet-checkpoint-full.pth'
save_icevision_checkpoint(model,
model_name='mmdet.retinanet',
backbone_name='resnet50_fpn_1x',
classes = class_map.get_classes(),
img_size=img_size,
filename=checkpoint_path,
meta={'icevision_version': '0.9.1'})
Loading models already containing metadata
If you have saved your model weights with the model metadata, you only need to call model_from_checkpoint(checkpoint_path): No other arguments (model_name, backbone_name, classes, img_size) are needed. All the information is already embedded in the checkpoint file.
checkpoint_path = 'https://github.com/airctic/model_zoo/releases/download/m6/fridge-retinanet-checkpoint-full.pth'
checkpoint_and_model = model_from_checkpoint(checkpoint_path)
# Just logging the info
model_type = checkpoint_and_model["model_type"]
backbone = checkpoint_and_model["backbone"]
class_map = checkpoint_and_model["class_map"]
img_size = checkpoint_and_model["img_size"]
model_type, backbone, class_map, img_size
# Inference
# Model
model = checkpoint_and_model["model"]
# Transforms
img_size = checkpoint_and_model["img_size"]
valid_tfms = tfms.A.Adapter([*tfms.A.resize_and_pad(img_size), tfms.A.Normalize()])
# Pick your images folder
path_to_image_folder = "../samples/fridge/odFridgeObjects/images"
img_files = get_image_files(path_to_image_folder)
# Create a dataset with appropriate images
imgs_array = [PIL.Image.open(Path(fname)) for fname in img_files]
infer_ds = Dataset.from_images(imgs_array, valid_tfms, class_map=class_map)
# Batch Inference
infer_dl = model_type.infer_dl(infer_ds, batch_size=4, shuffle=False)
preds = model_type.predict_from_dl(model, infer_dl, keep_images=True)
show_preds(preds=preds[0:2])
Use load_from_http loader
(<module 'icevision.models.mmdet.models.retinanet' from '/home/farid/repos/airctic/icevision/icevision/models/mmdet/models/retinanet/__init__.py'>,
<icevision.models.mmdet.models.retinanet.backbones.resnet_fpn.MMDetRetinanetBackboneConfig at 0x7fa2024c34f0>,
<ClassMap: {'background': 0, 'carton': 1, 'milk_bottle': 2, 'can': 3, 'water_bottle': 4}>,
384)
Happy Learning!
If you need any assistance, feel free to join our forum.