RCNNForObjectDetection
PretrainedModelRCNNForObjectDetection(config: RCNNConfig)R-CNN object detector (Girshick et al., CVPR 2014).
The first deep-learning detector to demonstrate that ImageNet-pretrained
CNN features transfer to localisation. At inference the pipeline runs an
externally-supplied region-proposal set (typically from selective search)
through an AlexNet-style convolutional trunk per proposal, then feeds
the flattened pool5 features into a two-layer FC trunk with sibling
classification (cls_head) and bounding-box regression (bbox_head)
outputs. This module covers the CNN + heads — proposals must be supplied
by the caller; postprocess() applies score thresholding and per-class
NMS to recover final detections.
Parameters
configRCNNConfigrcnn factory for the
paper-cited AlexNet configuration (227x227 RoIs, 80 COCO classes);
override individual fields via **kwargs on the factory.Attributes
configRCNNConfigconv_features_ConvFeaturesroi_size=227.fc_head_FCHeadfc6, fc7) followed by sibling
lucid.nn.Linear heads:
cls_head produces class logits (background +
num_classes foreground classes) and bbox_head produces
class-specific bounding-box deltas.Notes
See Girshick et al., "Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation", CVPR 2014 (arXiv:1311.2524). For each proposal with centre and size , the bounding-box regressor predicts a parameterised offset so that the refined box centre and size are
The original paper trains per-class linear SVMs on top of frozen CNN features; this implementation follows the more common modern softmax-end-to-end form (matching the Caffe reference code). Cost is dominated by the redundant per-proposal CNN forward — the main motivation for the Fast R-CNN successor's shared feature map.
Examples
Run inference on a single image with three external proposals:
>>> import lucid
>>> from lucid.models.vision.rcnn import rcnn
>>> model = rcnn(num_classes=80)
>>> x = lucid.randn(1, 3, 600, 600)
>>> proposals = [lucid.tensor(
... [[10.0, 10.0, 200.0, 200.0],
... [50.0, 60.0, 300.0, 280.0],
... [100.0, 80.0, 450.0, 400.0]])]
>>> out = model(x, proposals)
>>> out.logits.shape, out.pred_boxes.shape
((3, 81), (3, 4))
Apply post-processing (score threshold + per-class NMS) on the raw
output to obtain final per-image detections:
>>> detections = model.postprocess(out, proposals)
>>> set(detections[0].keys()) == {"boxes", "scores", "labels"}
TrueUsed by 2
Constructors
1Instance methods
2forward(x: Tensor, proposals: list[Tensor] | None = None)Run R-CNN on a batch of images.
Parameters
Returns
ObjectDetectionOutputObjectDetectionOutput with:
logits : (Σ N_i, num_classes + 1) raw class logits.
pred_boxes : (Σ N_i, 4) decoded xyxy boxes (top class).
postprocess(output: ObjectDetectionOutput, proposals: list[Tensor] | None = None)Apply score threshold + per-class NMS to raw R-CNN output.
Parameters
outputObjectDetectionOutputObjectDetectionOutput from forward().forward(). When
omitted, falls back to output.proposals.Returns
list[dict[str, Tensor]]List of per-image result dicts, each with:
"boxes" : (K, 4) xyxy kept detections
"scores" : (K,) class confidence scores
"labels" : (K,) integer class indices (1-based, 0=bg)