In observability/metrics, histograms are primarily used to record distributions in a space efficient manner that can be aggregated and summarized (usually with quantile measurements) at a later point.
But how can you pull out something like the 99.9 percentile from a few buckets? The answer is you can't. What you can do is find the bucket it would land in and attempt a (linear) interpolation between the bucket bounds and generate an estimate.
So it becomes apparent that having more buckets gives you higher resolutions. Now for a look at different implentations.
Prometheus essentially stores each bucket as its own time series with the bucket bounds recorded as labels. This is quite innefficient and as a result, by default it only gives a few buckets, which while it covers the common range for things like http request latency, is subject to large error bars if the boundaries aren't close to the actual values (which is what is interesting... the outlier events).
1DefBuckets = []float64{.005, .01, .025, .05, .1, .25, .5, 1, 2.5, 5, 10}
HDR-Histograms
based on this analysis
divides the space between a given (min, max) pair into exponential buckets (2^x),
then subdivides each bucket linearly.
This apprently scales to ~1800 buckets in a ~1500 byte packet?
In other places this is called "log linear bounds" (logarithmic big buckets, linear sub buckets).
DDSketch
based on this analysis
also uses exponential buckets, but this time choosing a base that guarantees relative error.
For a 1% error, (1+0.1) / (1-0.1) = 1.02, and so buckets are chosen with 1.02^x.
A fixed number of buckets are chosen,
and the starting buckets (lowest indices) are merged if it ever grows out of space.
Their blog post
mentions somewhere on the order of a few hundred buckets (eg 802 for ns~day scale),
with their implementation setting a default limit at 2048.
In a comment on an opentelemetry histogram proposal
Google said they use a (base = minimum_base^2^compression)^x for buckets.
After
much
discussion
opentelemetry proposed
and finally defined
their buckets with
base = 2 ^ (2 ^ -scale).
There are apparently no hard limits on bucket count, though the go implementation chooses a default of 160 buckets, adjusting scaling as necessary.
Implementations already exist in the form of: NrSketch DynaHist