OSINT / Alerting · Independent project
OSINT Radar: Near Real-Time Open-Source Intelligence
Dmitri Derevjanko built OSINT Radar, a near real-time dashboard that pulls 27+ open sources into one triage view with de-duplication, scoring, and AI briefs.
This is one of 10+ independent public projects by Dmitri Derevjanko. 100+ additional workflows were delivered professionally, mostly under confidentiality, and are not documented here.
Open live demo Back to portfolio
Problem
Open-source monitoring usually means many tabs, duplicate stories, and no shared event schema. Analysts spend time reconciling feeds instead of ranking what matters.
Solution
A radar surface that ingests RSS, APIs, and HTML sources, normalizes events, clusters duplicates, scores relevance, and writes short evidence-linked briefs for clusters rather than wrapping a chatbot around headlines.
What Dmitri Derevjanko built
Dmitri designed the ingestion pipeline, canonical event schema, de-duplication and clustering, worker queues, enrichment (language, entities, geo hints), alert scoring, AI summaries, the analyst UI, and the production-style observability around source health.
Architecture
- 27+ heterogeneous feeds enter one ingestion pipeline with retries, backoff, and defensive parsing.
- Events normalize to a shared schema (identity, time, URL, tags, language, entities, geo, severity) while raw payloads stay available for audit.
- Fingerprinting and fuzzy matching cluster duplicates across sources without merging distinct stories.
- Workers apply backpressure and priority so high-signal items surface first.
- Deterministic heuristics plus NLP signals rank the inbox; AI writes concise, source-linked cluster briefs.
- The dashboard is filters, saved views, timelines, and drill-down to original URLs, not a chat window.
Technologies
- Python
- FastAPI
- PostgreSQL
- Redis
- AsyncIO
- NLP
- OpenAI API
- Docker
Result
Operators can scan a ranked, de-duplicated stream from 27+ sources with AI briefs and source drill-down instead of a pile of unread feeds.
Limitations
- Sources are public open-web feeds; this is not classified intelligence and not a substitute for a dedicated SOC.
- AI briefs can miss nuance; the original URL remains the source of truth.
- Feed quality and anti-bot behaviour on scraped sites can delay or drop individual sources.
Live demo and source
Live demo: https://osint.dmitriderevjanko.com/
GitHub: source is not public. The live demo is the public artifact for this project.
Portfolio case study: https://dmitriderevjanko.com/projects/osint-radar/