Examples
Rolling mean
Here we show a computation of the rolling mean over fixed windows. We could use a FixedWindowAssociativeOp
to keep track of a rolling sum, then divide by the window length, only including values where the window has filled.
using AssociativeWindowAggregation
using Plots
x = range(1, 10; length=100)
y = sin.(x) + 0.5 * rand(length(x))
plot(x, y; label="raw", title="Rolling means")
for window in [5, 10, 20]
# Use this to keep track of a windowed sum.
state = FixedWindowAssociativeOp{Float64,+}(window)
z = []
for value in y
update_state!(state, value)
if window_full(state)
push!(z, window_value(state) / window)
else
push!(z, NaN)
end
end
plot!(x, z; label="mean $window", lw=2)
end
However, if we were to be using very large windows, we should be nervous with the implementation above. This is because, by taking the ratio of two large numbers, we may suffer a loss of precision.
Here is a better implementation, where we create a new object Mean
, whcih internally stores a mean and a count. We then define a merge
function, which is binary and associative.
struct Mean
n::Int64
mean::Float64
end
Mean(x::Real) = Mean(1, x)
"""
merge(x::Mean, y::Mean) -> Mean
Combine two `Mean` objects into a new `Mean`.
"""
function merge(x::Mean, y::Mean)
n = x.n + y.n
return Mean(n, (x.n / n) * x.mean + (y.n / n) * y.mean)
end
plot(x, y; label="raw", title="Rolling means")
for window in [5, 10, 20]
# Unlike in the previous example, we now combine our custom `Mean` object.
state = FixedWindowAssociativeOp{Mean,merge}(window)
z = []
for value in y
# Wrap individual values in a `Mean` to allow us to add them to the window.
update_state!(state, Mean(value))
if window_full(state)
# `window_value` returns a `Mean` object now, so we must extract the mean.
push!(z, window_value(state).mean)
else
push!(z, NaN)
end
end
plot!(x, z; label="mean $window", lw=2)
end
OnlineStats.jl
Many of the estimators from OnlineStats.jl can be merged associatively. Therefore, we can compute online windowed versions of these statistics easily with this framework.
Here is a simple example computing an online windowed standard deviation.
using AssociativeWindowAggregation
using OnlineStatsBase
using Plots
using Statistics
# Use a window of 100 values.
state = FixedWindowAssociativeOp{Variance,merge,merge!}(100)
function _wrap(v)
x = Variance()
fit!(x, v)
return x
end
values = rand(1000) .- 0.5
output = [std(window_value(update_state!(state, _wrap(v)))) for v in values]
plot(values; alpha=0.4, label="Input")
plot!(output; label="std (window=100)")