Fundamentals
Data
Time-series data is stored internally in a Block
. More information on what we mean by a time-series is explained in Time-series.
TimeDag.Block
— TypeBlock{T}()
Block(times::AbstractVector{DateTime}, values::AbstractVector{T})
Block(unchecked, times, values)
Represent some data in timeseries.
Conceptually this is a list of (time, value)
pairs, or "knots". Times must be strictly increasing — i.e. no repeated timestamps are allowed.
The constructor Block(times, values)
will verify that the input data satisfies this constraint, however Block(unchecked, times, values)
will skip the checks. This is primarily intended for internal use, where the caller assumes responsibility for the validity of times
& values
.
TimeDag
considers instances of Block
to be completely immutable. Thus, when working with functions that accept blocks (e.g. TimeDag.run_node!
), you must not modify times
or values
members.
Computational graph
The computational graph is formed of TimeDag.Node
objects. A node is an abstract representation of a time-series, i.e. a sequence of (time, value)
pairs. A node knows the type of its values, which can be queries with TimeDag.value_type
.
Note that nodes should never be constructed directly by the user. Typically one will call a function like block_node
or lag
, which will construct a node.
TimeDag
includes functions to construct many useful nodes, but often you will need to create a custom node. See Creating nodes
for instructions on how to do this.
All nodes should eventually be constructed with TimeDag.obtain_node
. This uses the global Identity map to ensure that we do not duplicate nodes.
TimeDag.Node
— TypeNode(parents, op)
A node in the computational graph that combines zero or more parents
with op
to produce a timeseries.
Note that a Node
is only declared mutable so that we can attach finalizers to instances. This is required for the WeakIdentityMap
to work. Nodes should NEVER actually be mutated!
Due to subgraph elimination, nodes that are equivalent should always be identical objects. We therefore leave hash
& ==
defined in terms of the objectid
.
TimeDag.value_type
— Functionvalue_type(node::Node{T}) -> T
The type of each value emitted for this node.
Every node contains parents, and a TimeDag.NodeOp
.
TimeDag.NodeOp
— Typeabstract type NodeOp{T} end
Represent a time-series operation whose output will be a time-series with value type T
.
TimeDag.obtain_node
— Functionobtain_node(parents::NTuple{N,Node}, op::NodeOp) -> Node
Get a node for the given op
and parents
. If an equivalent node already exists in the global identity map, use that one, otherwise create a new node, add to the identity map, and return it.
Constant propagation
If all parents
are constant nodes, and op
has a well-defined operation on constant inputs, we will immediately perform the computation and return a constant node wrapping the computed value.
Given a node, a rough-and-ready way to visualise the graph on the command line is with AbstractTrees.print_tree
. This will not directly indicated repeated nodes, but for small graphs the output can be useful.
Evaluation
In order to get a concrete time-series (as a Block
) for a node, it must be evaluated with evaluate
. Evaluation additionally requires a time range, and involves pulling data corresponding to this interval through the graph of ancestors of the given node(s).
When evaluating a graph in a production system, it may be desirable to have more control over evaluation. If this sounds like you, please read the Advanced evaluation section!
TimeDag.evaluate
— Functionevaluate(nodes::AbstractVector{Node}, t0, t1[; batch_interval]) -> Vector{Block}
evaluate(node::Node, t0, t1[; batch_interval]) -> Block
Evaluate the specified node(s) over the specified time range [t0, t1)
, and return the corresponding Block
(s).
If nodes
have common dependencies, work will not be repeated when performing this evaluation.