STIX 2.0 Proposal11 : Abstract Sightings into an independent construct rather than embedded within Indicator (#306) - STIXProject/specifications GitHub Wiki

Issue Summary

Sightings are a very important and likely to be highly used part of the CTI domain.

They give confirmation of asserted relationships between particular observable patterns and particular TTPs thus increasing confidence in the effectiveness of particular Indicators.
They reveal information as to the frequency and tempo of use for particular TTP yielding useful context for prioritization of investigation and other courses of action.
They reveal useful contextual information to understand victim targeting for particular adversary activity yielding useful context for getting ahead of the adversary and knowing where you should actively hunt.
They spur collaborative interchange and speed evolutionary understanding of threat context

Core Problem

In the current model the structure for conveying sightings is embedded within the Indicator structure. This enables sightings to be specified for that particular Indicator but brings with it some significant limitations:

Adding a Sighting to any Indicator involves actually modifying/versioning the Indicator itself
Specifying a Sighting on a 3rd party Indicator requires modifying the Indicator into ones own ID space even though the content of the actual Indicator has not changed and is still "owned" by the originating 3rd party
Versioning of Indicators and Sightings is more complex than necessary
It is not really possible to make any sort of analytical assertion on the Indicator or the Sighting separately

Observations vs Sightings: Common Confusions with the Current Model

There exists some confusion on the intent of the current model for handling various contexts in regards to observations and sightings.

If you are simply reporting new observations without any context, the current intent of the model is to support this via specification of a simple Observation.
If you are reporting new observations and wish to convey a simple context that they may be of interest or seem suspicious/funky without any further detail, the current intent of the model is to convey the observations as simple Observations and then include a Report linking these Observations and applying a simple "suspicious" context to them.
If you have decided that some new observations are of significant interest to dig further into and investigate in an attempt to discovering and capturing further context for them (and possibly share what you have learned), the current intent of the model is to convey this using an Incident with appropriate associations to the Observations and to any other context discovered (TTPs, Victims, ThreatActors, other Observations or Indicators, etc.).
If you wish to convey that a new observation appears to match an observable pattern defined in an Indicator (either shared or internally developed), the current intent of the model is to convey this as a Sighting of that Indicator with the option of including the actual Observation if desired.

"Sighting" of objects other than Indicators

There exists a lack of consensus among the community on the core meaning of a "sighting".

While the current intent and semantics of the model are that Sightings are instances of a given Indicator being "seen", some in the community feel that you should also be allowed to use Sightings to assert that other things (TTPs, ThreatActors, etc.) have been "seen".

Those arguing for the looser definition feel that it is useful to allow someone to quickly assert that they have seen a given TTP or Threat Actor without requiring any of the context behind such an assertion.

Those arguing for the tighter definition feel that enabling the assertions of such shortcuts actually short-circuits the context and logic behind such an assertion making acceptance of the assertion inherently lower confidence and making appropriate pivoting and discovery on that context impractical or impossible. They would argue that in the cyber domain you never really "see" a Threat Actor but rather you see some Observation of characteristics that matches the pattern defined within an Indicator that asserts it means a particular TTP is in play which has at some point been associated with the Threat Actor. Each of the points along that path bear with them uncertainty/confidence the aggregate of which affect the confidence that the given Threat Actor was "seen". Similarly, each of those points along the path represent pivoting points for analysis and discovery of context that may not be obvious if you only assert that the Threat Actor was "seen".

Proposed

Break Sighting out from Indicator. Create new Sighting construct derived from AssertionType which is in turn derived from IDableConstructType and remove the Sighting structure from Indicator.

Actions

Remove the Sighting structure from IndicatorType
Create a new SightingType as a specialization of AssertionType (presumes approval of #291) with two native properties:
- To : IndicatorType [1] (referencing the identifier for the Indicator being sighted)
- Count : NonNegativeInteger [0..1]
Create a new QuickSightingType (supporting the very simple +1 use case) as a specialization of SightingType with the following constraints applied to the existing properties:
- Title is prohibited ([0])
- Description is prohibited ([0])
- Confidence is prohibited ([0])
- Count is prohibited ([0])
Create a new RelatedSightingType (presumes approval of #291) with its To property constrained to only SightingType constructs

Non-sighting observations will still be handled according to the current model intent described in the "Observations vs Sightings: Common Confusions with the Current Model" section above.

Sighting will be constrained to only Indicators and not to other IDable construct types.

Proposed Model

Model for IndicatorType showing Sightings removed

Model for new Sightings constructs

Examples

Example #1: simple +1 Quick Sighting

Example #2: simple +1 Quick Sighting with Source attribution

Example #3: simple Sighting

Example #4: simple sighting with simple associated Observation

JSON Serialization example snippets

Example #1:

{
	"id": "example:sight-b686e902-3ee1-4d7e-a54f-448ccd741651",
	"type": "quick-sighting",
	"timestamp": {"value" : "2015-12-21T20:49:08.000000+00:00"},
	"to": "example:ind-b8e37090-5d62-45a1-ac2e-a88601b08432"
}

Example #2:

{
	"id": "example:sight-b686e902-3ee1-4d7e-a54f-448ccd741651",
	"type": "quick-sighting",
	"timestamp": {"value" : "2015-12-21T20:49:08.000000+00:00"},
	"to": "example:ind-b8e37090-5d62-45a1-ac2e-a88601b08432"
}

{
	"id": "example:rel-9d0c539e-a874-42c7-a055-3e900b98724f",
	"type": "related-source",
	"timestamp": {"value" : "2015-12-21T19:59:12.000000+00:00"},
	"from": "example:sight-b686e902-3ee1-4d7e-a54f-448ccd741651",
	"to": "example:src-83dc6b53-ac3d-40e0-82ef-eab173c7ee1e",
	"relationship_nature": {
		"value": "Has Source"
	}
}

Example #3:

{
	"id": "example:sight-cbc7d591-87f1-4765-a15e-c39e7b9d12ba",
	"type": "sighting",
	"timestamp": {"value" : "2015-12-21T20:49:08.000000+00:00"},
	"to": "example:ind-b8e37090-5d62-45a1-ac2e-a88601b08432",
	"count": "2",
	"confidence": {
		"value": {
			"value": "High",
			"vocab": "high-medium-low-vocab-1.0"
		}
}

{
	"id": "example:rel-9d0c539e-a874-42c7-a055-3e900b98724f",
	"type": "related-source",
	"timestamp": {"value" : "2015-12-21T19:59:12.000000+00:00"},
	"from": "example:sight-cbc7d591-87f1-4765-a15e-c39e7b9d12ba",
	"to": "example:src-83dc6b53-ac3d-40e0-82ef-eab173c7ee1e",
	"relationship_nature": {
		"value": "Has Source"
	}
}

Example #4:

{
	"id": "example:sight-cbc7d591-87f1-4765-a15e-c39e7b9d12ba",
	"type": "sighting",
	"timestamp": {"value" : "2015-12-21T20:49:08.000000+00:00"},
	"to": "example:ind-b8e37090-5d62-45a1-ac2e-a88601b08432",
	"count": "2",
	"confidence": {
		"value": {
			"value": "High",
			"vocab": "high-medium-low-vocab-1.0"
		}
}

{
	"id": "example:obj-fa7de07b-ee02-4773-8cac-fa7161a7f998",
	"type": "file-object-observation",
	"timestamp": { "value" : "2015-12-21T22:45:12.000000+00:00" },
	"title": "Suspicious File",
	"hashes": [
		{
			"type": "md5",
			"hash_value": "3773a88f65a5e780c8dff9cdc3a056f3",
		}
	]
}

{
	"id": "example:rel-9d0c539e-a874-42c7-a055-3e900b98724f",
	"type": "related-observation",
	"timestamp": {"value" : "2015-12-21T19:59:12.000000+00:00"},
	"confidence": {
		"value": {
			"value": "High",
			"vocab": "high-medium-low-vocab-1.0"
		}
	},
	"from": "example:sight-cbc7d591-87f1-4765-a15e-c39e7b9d12ba",
	"to": "example:obj-fa7de07b-ee02-4773-8cac-fa7161a7f998",
	"relationship_nature": {
		"value": "Sighted As"
	}
}

JSON Schema Serialization snippets

Open Questions

Should Sightings be able to point to IDable constructs other than Indicators?
Is there consensus on the semantic and mechanism differences between observations and sightings?