Histogram in stats - cryptocode/notes GitHub Wiki

A few notes related to the histogram PR

In addition to a counter, any stat can have an associated histogram. Separate stats can be created for histogram purposes, such as vote_byhash_histogram, or a histogram can be attached to an existing stat, like confirm_req.

API

Histograms should be defined in the nano::stat constructor by calling define_histogram.

Examples:

// Uniform histogram, total range 12, and 12 bins (each bin has width 1)
define_histogram (type::vote, detail::confirm_ack, dir::in, {1,13}, 12);

// Specific bins with intervals [1,4] [5,19] [20,99]
define_histogram (type::vote, detail::something, dir::out, {1,5,20,100});

// Logarithmic bins matching half-open intervals [1..10) [10..100) [100 1000)
define_histogram(type::vote, detail::log, dir::out, {1,10,100,1000});

These can then be updated using update_histogram:

// Add 1 to the bin representing a 4-item vbh
stats.update_histogram(type::vote, detail::confirm_ack, dir::in, 4, 1)

// Add 5 to the second bin where 17 falls
stats.update_histogram(type::vote, detail::something, dir::in, 17, 5)

// Add 3 to the last bin as the histogram clamps. You can also add a final bin 
// with maximum end value to effectively prevent this.
stats.update_histogram(type::vote, detail::log, dir::out, 1001, 3)

RPC stats output

The RPC "stats" output for a histogram contains all bins, with interval, value and time of last update. Even entries with a zero value is emitted so that tools can choose to display the intervals anyway.

...
{
    "time": "00:51:03",
    "type": "vote",
    "detail": "confirm_req",
    "dir": "in",
    "value": "1",
    "histogram": [
        {
            "start_inclusive": "1",
            "end_exclusive": "2",
            "value": "24",
            "time": "00:51:03"
        },
        {
            "start_inclusive": "2",
            "end_exclusive": "3",
            "value": "2",
            "time": "00:51:03"
        },
        {
            "start_inclusive": "3",
            "end_exclusive": "4",
            "value": "20",
            "time": "00:51:03"
        },
        {
            "start_inclusive": "4",
            "end_exclusive": "5",
            "value": "1",
            "time": "00:51:03"
        },
        {
            "start_inclusive": "5",
            "end_exclusive": "6",
            "value": "0",
            "time": "00:51:03"
        },
        {
            "start_inclusive": "6",
            "end_exclusive": "7",
            "value": "0",
            "time": "00:51:03"
        },
        {
            "start_inclusive": "7",
            "end_exclusive": "8",
            "value": "0",
            "time": "00:51:03"
        },
        {
            "start_inclusive": "8",
            "end_exclusive": "9",
            "value": "7",
            "time": "00:51:03"
        },
        {
            "start_inclusive": "9",
            "end_exclusive": "10",
            "value": "0",
            "time": "00:51:03"
        },
        {
            "start_inclusive": "10",
            "end_exclusive": "11",
            "value": "0",
            "time": "00:51:03"
        },
        {
            "start_inclusive": "11",
            "end_exclusive": "12",
            "value": "0",
            "time": "00:51:03"
        },
        {
            "start_inclusive": "12",
            "end_exclusive": "13",
            "value": "49813",
            "time": "00:51:03"
        }
    ]
},
...

Future enhancements

Support automatic log distribution of bins, and labels for intervals.