Social Media Situational Awareness
People post what they see; a commander needs what it means. Across three funded programs, VIStology turned raw public text — a crawled foreign blogosphere, then half a million tweets — into ranked, mapped answers, with a sound reasoner under the tweet stream: every pin traceable to the phrases that produced it, and a published formal answer to the question under it all — how do you reason soundly over reports you cannot fully trust?
The Challenge: The Report Stream No One Can Read
In a crisis, a large fraction of the earliest and most timely information is public: people posting what they see, in unstructured natural language, at a volume no staff can read. The commander of a disaster-relief operation “must therefore have access to what people are saying on social media” — and, as VIStology’s STIDS 2013 paper puts it, “neither the commander, nor his or her staff, has time to read all of those messages and identify what is relevant in order to assess the situation.”
The same problem confronts an analyst facing an unfamiliar foreign blogosphere: which of thousands of sources matter, and which posts? The machinery of standard web search was built for a web of densely interlinked standalone documents — and a social-political blogosphere is not that. Posts are short, are largely commentary on the news stories they link to (so what a post is about is frequently not in its own text), and attract few inbound links for authority metrics to feed on.
One number frames why the text itself — not its metadata — is the hard requirement: of the 509,795 tweets sent in the hours after the Boston Marathon bombing, only about 1.6% carried geocoded origins. The locations that mattered were the ones people were writing about, and they existed only in the words.
2010 — The Statistical Front End, at Corpus Scale
Under an AFOSR contract, VIStology's IBlogs project built an automated monitor for an unfamiliar foreign blogosphere and published the metrics behind it in IEEE Internet Computing. The extractor of that era was statistical information retrieval and link analysis: a crawl that produced 220,320 unique URLs across 4,693 sites; tf·idf over the content plus a second, term-frequency-weighted vector of proper names drawn from a post and from every news story it cites; in-degree and eigenvector centrality for authority; and four metrics of information value — relevance, specificity, timeliness, credibility.
There is no machine learning in that work and we will not claim any. What there is, is measurement discipline of the kind we still hold ourselves to: the authors measured their own date extractors at 91.6% and 87% accuracy, then volunteered the compounded figure — roughly 79% — for the metric that depends on them. An extractor is a fallible instrument. The right response is to characterize it honestly and put something sound underneath it.
2011 — Trust Becomes Part of the Model
Between the statistics and the map sits a question neither answers: what should a reasoner do with a fact it cannot fully trust? NATO STANAG 2022 and U.S. Army doctrine require every item of intelligence to carry a reliability and credibility rating — and then stop. In the words of the CogSIMA 2011 paper VIStology wrote with two U.S. Army co-authors: “Once a report with its associated STANAG 2022 evaluations enters the information stream, there is no Army-prescribed (let alone overarching normative) theory for reasoning with this knowledge.”
Under U.S. Army contract W15P7T-09-C-S031 — with co-authors from the Army Research Laboratory’s Human Research and Engineering Directorate and the Army Intelligence Center of Excellence — VIStology’s answer was to put trust into the ontology itself, and to put each trust quantity on the right thing: reliability on the source, credibility on the proposition (because the same utterance expresses different propositions in different situations), and confidence on the utterance. A trust annotation is not metadata bolted onto a fact; it is itself a fact in the knowledge base, addressable by the reasoner.
The initial ratings come from human analysts, per doctrine. Rules executed by BaseVISor then revise them: corroboration by provably distinct, independent sources upgrades a proposition’s credibility; conflicting reports downgrade it; and the paper’s most novel rule caps a source’s assertions at doubtful when its payoff depends only on what it reports. The paper also shows, with worked numbers, why the quantitative alternatives break: Bayesian aggregation needs 2n joint prior probabilities, and “On the battlefield, priors are typically lacking” — while weighted-majority scoring lets three not-usually-reliable sources outvote one completely reliable one.
Be clear about what this work is: a formalization with worked examples, published at IEEE CogSIMA 2011 — not a fielded system, and not an evaluation. It contains no machine learning and no text extraction of any kind. What it contributes is the piece the rest of this page depends on: a sound, inspectable way to reason about the trustworthiness of what a fallible front end reports, with the same machinery the reasoner uses to reason about the world.
2013 — A Lossy Extractor Under a Sound Reasoner, Measured
Under AFRL SBIR contract FA8650-13-C-6381, VIStology's HADRian system assembled a common operational picture for disaster-relief operations out of social media. The test case was the hours after the Boston Marathon bombing: 509,795 tweets, and a commander's question — where are people reporting that additional or unexploded devices have been found?
Read that question precisely, because the system answered exactly it and nothing more. Every one of those Boston reports turned out to be false. HADRian never detected a device; it found, ranked, and mapped the places people were reporting — which is what officials actually had to triage, and could not have read for themselves at that volume.
The extractor was of its era: generic pattern matching with no gazetteer and no scenario-specific patterns, a regular expression carried in the ontology by the OWL class it recognizes, and commercial geocoding APIs to ground surface phrases into real places. It was lossy, and the paper says so: the topic filter reached 94.5% precision but only 50% recall (F1 65%); location phrases were identified at 95% precision. Tweets were then lifted into OWL by a declarative template, and BaseVISor — OWL 2 RL, forward-chaining — reasoned twice: once over repository metadata to decide which data source could even answer the query, and again over the lifted content to pull the locations out.
Here is the result that matters, and it is the neurosymbolic bet in one line. A half-lossy extractor under a sound reasoner still produced an operationally correct answer: 38 ranked places, benchmarked against the locations named in the live blogs of CNN, the New York Times and the Boston Globe over the same window — and the crowd-derived locations surfaced, on average, 11 minutes ahead of the newsrooms. Sites prominent on Twitter never appeared on the news blogs at all. Redundancy at scale absorbs an imperfect extractor. Soundness, provenance and auditability are the reasoner's job — and every pin on that map could be clicked back to the raw phrases that produced it.
Proof Points
HADRian — 11 Minutes Ahead of the Newsroom
AFRL SBIR contract FA8650-13-C-6381. Across 509,795 tweets from the hours after the Boston Marathon bombing, HADRian surfaced and ranked the locations people were reporting as sites of additional devices — every such report proved false, and the system never claimed otherwise. Its 38 ranked places, benchmarked against the CNN, New York Times and Boston Globe live blogs, appeared on average 11 minutes earlier. Topic filtering ran at 94.5% precision and 50% recall: a lossy extractor, an operationally correct answer.
IBlogs — Statistical Front End, Corpus Scale
An AFOSR-funded, three-year effort to monitor an unfamiliar foreign blogosphere automatically. A crawl of 220,320 unique URLs across 4,693 sites fed four metrics of information value — relevance, specificity, timeliness, credibility — built on tf·idf, term-frequency-weighted proper-name vectors, and link analysis, and the credibility metric validated against an independent panel of subject-matter experts. Published in IEEE Internet Computing.
Reasoning Over Facts You Cannot Trust
US Army contract W15P7T-09-C-S031, with co-authors from Army Research Laboratory HRED and the Army Intelligence Center of Excellence. NATO STANAG 2022 requires every item of intelligence to carry a reliability and credibility rating — and prescribes no way to reason with them. VIStology put them in the ontology: reliability on the source, credibility on the proposition, confidence on the utterance. Initial ratings come from human analysts, per doctrine; BaseVISor rules then revise them — independent corroboration upgrades credibility, a source whose payoff depends only on what it reports gets its assertions capped at doubtful.
Selected Publications
Metrics for Monitoring a Social-Political Blogosphere: A Malaysian Case Study
IEEE Internet Computing 14(2)
View PublicationAugmenting the Analyst via Situation-Dependent Reasoning with Trust-Annotated Facts
CogSIMA 2011
View PublicationRelated
Read the Stream No One Can Read
If your operation needs to know what thousands of public voices are reporting — and needs every conclusion to trace back to the words that produced it — we can help you put a sound reasoner over the stream.
Get in Touch