Building a Cybersecurity News Aggregator: Story Clustering, Seven Importance Signals and Strict Filtering Thresholds

The concept of a dedicated information security news aggregator originated several years ago. The objective was to receive only the most important developments without information noise.

Initial implementations used the rut5_base_sum_gazeta summarization model together with TextRank for importance ranking, yet results were inconsistent. Later commercial services that curate feeds still failed to balance breadth and relevance, often covering either everything or only narrow topics.

After weeks of prompt tuning proved ineffective, a hybrid architecture was adopted. Importance is now calculated by a formula consisting of seven explicit, logged features whose weights can be inspected and adjusted. A language model performs two supporting tasks: filtering out non-relevant items and generating readable text.

Core data flow and filtering funnel

Approximately one thousand materials arrive daily from more than 200 sources. Only around 0.5 percent reach publication. The first stage applies the seven-feature formula; the second stage uses the model to discard irrelevant content.

Seven importance features

• Confirmation – number of independent outlets covering the same event
• Severity – CVSS score above 9 receives maximum weight; active exploitation in the wild grants full weight regardless of CVSS
• Primary source – presence of a CERT, KEV catalog entry, or original vendor research
• Reader proximity – Russian company or Russian-context stories receive higher weight
• Analysis depth – detailed technical breakdowns and novel attack classes score higher
• Authority – best source tier present in the cluster (primary, research, security media)
• Speed – time between first and last publication; one post per hour or faster earns full points

Penalties are subtracted for vacancies, webinars, awards, and vendor self-promotion unless an independent outlet later confirms the story.

Story clustering methods

Materials describing the same incident are merged into a single story using three decreasingly strict techniques: shared CVE, GHSA, Microsoft KB or BDU identifiers; overlapping trigrams within the same language; and shared named entities such as Microsoft, SharePoint or Fortinet that link Russian and English reports.