Stream Forge Publishing and Identity - mdebellis/SemanticKG-Design GitHub Wiki

Stream Forge — Publishing and Identity

Overview

This story introduces the Stream Forge example by focusing on the ingestion of content metadata from multiple source systems and the challenge of establishing stable identity across heterogeneous identifiers.

It represents the earliest stage in the lifecycle of a semantic knowledge graph, where the primary concerns are understanding the domain, defining what counts as an entity, and determining how externally supplied identifiers relate to those entities.

Narrative

Stream Forge aggregates metadata about published video content from several external platforms. Each platform exposes its own identifiers, naming conventions, and metadata structures. Some platforms provide rich metadata, while others provide only minimal information.

A single video may appear in multiple feeds, sometimes with different titles, timestamps, or creator identifiers. In some cases, the same identifier refers to different real-world videos depending on the source system. In other cases, different identifiers refer to what appears to be the same video.

At this stage, Stream Forge does not attempt to compute analytics or derive insights. Its immediate goal is more fundamental: to ingest published content and establish a coherent representation of videos, creators, and platforms that can support later enrichment and analysis.

Core Questions

This story drives a set of foundational questions:

  • What is a video as an entity, distinct from any particular source record?
  • How do source-specific identifiers relate to persistent identifiers in the knowledge graph?
  • When should two records be treated as referring to the same video?
  • What information should be considered authoritative, and by whom?
  • How should uncertainty or ambiguity in identity resolution be represented?

These questions must be addressed before higher-level modeling, analytics, or AI integration can be meaningfully applied.

Requirements Implied by the Story

From this narrative, several requirements emerge:

  • The system must distinguish entities from identifiers
  • Multiple external identifiers may refer to the same