Scene Understanding

What crosses the wire

SOLO, the detector, is Southwest Research Institute's component — a hybrid of YOLO, Faster R-CNN and Multi-box SSD, built in Python on Caffe, exposed as a REST microservice, running at around 10 frames per second. It emits bounding boxes and an objectness confidence, and nothing else. No class labels. The network never tells the reasoner what it thinks it has seen — only that something is there, and how sure it is.

That JSON crosses an HTTP boundary into VIStology's Java Semantic Reasoner, where a Tracker turns per-frame boxes into tracks with stable identity, a Semantic Annotator lifts those tracks and their sensor metadata into OWL/RDF individuals and typed property assertions — the same handoff, in a modern syntax, as the Prolog facts of 2002 — and a Semantic Engine runs BaseVISor over them with OWL 2 RL inference and user-defined BVR rules.

Symbolic reasoning is where pixels become meters

The reasoner is not a second opinion on the same evidence. It is holding evidence the network structurally cannot represent: a range map, the sensor's field of view, and a semantic segmentation of the scene into sky, water, cloud, road, building and vegetation. Box width plus range plus field of view gives an object's real-world size in meters — a quantity no pixel-space detector computes, because it has no concept of a meter.

From that, the rules follow, and they are blunt: a declaration whose estimated width exceeds 15 meters has its confidence dropped to zero. Declarations falling in water, cloud or sky are penalized; those on roads, buildings and man-made surfaces are boosted; those in vegetation are discounted. A tank-sized blob in a patch of sky, or a forty-meter-wide "vehicle," is not a plausible target — and now something in the system knows it.

The reasoner overrules the network. It never retrains it.

Be exact about the direction of the loop, because this is where neurosymbolic marketing usually starts lying. FLOCK closes the loop at the level of belief, not at the level of weights. The Semantic Engine removes high-confidence false alarms from the world model and raises the confidence of genuine detections. It sends no gradient back, updates no weight, and adapts no network. There is no arrow returning to the detector, and we have not drawn one.

Be equally exact about the rules: in the work reported, they were hand-authored and hand-tuned. Rule parameters were varied manually from one experiment to the next; learning them is identified by the authors as the next step, not delivered as a result. The paper's title says online learning; the paper's experiments do not implement it, and neither does this page.

What the work does establish, it establishes on its own terms. The evaluation is offline, on recorded infrared video from a government-furnished dataset with ground truth, on scenarios selected to exercise occlusion. It reports no mAP, no precision, no recall — its evidence is the shape of the detection-versus-false-alarm curves, and the authors' own conclusion: "The experiments decisively confirm the claim that semantic reasoning is a valuable tool for improving the quality of target detection."

Now substitute your LLM

Nothing in that architecture depends on the extractor being SOLO. Swap in a vision-language model, a modern detector, or an LLM reading text instead of pixels, and the rest of the diagram is unchanged: the annotator lifts its output into the ontology, the reasoner checks it against constraints the model cannot represent, and the answer that comes out is traceable to the axiom or the rule that produced it. The extractor gets better every year. The obligation to be right does not get easier — and that obligation is met on the far side of the socket, or it is not met at all.