This paper introduces two other fundamental tasks to the set that baseline methods cannot solve. We also formalise several loss functions that encourage desirable properties on the trajectory paths.
It was recently shown that neural ordinary differential equation models
cannot solve fundamental and seemingly straightforward tasks even with
high-capacity vector field representations. This paper introduces two other
fundamental tasks to the set that baseline methods cannot solve, and proposes
mixtures of stochastic vector fields as a model class that is capable of
solving these essential problems. Dynamic vector field selection is of critical
importance for our model, and our approach is to propagate component
uncertainty over the integration interval with a technique based on forward
filtering. We also formalise several loss functions that encourage desirable
properties on the trajectory paths, and of particular interest are those that
directly encourage fewer expected function evaluations. Experimentally, we
demonstrate that our model class is capable of capturing the natural dynamics
of human behaviour; a notoriously volatile application area. Baseline
approaches cannot adequately model this problem.