Stochastic Coherence Over Attention Trajectory For Continuous Learning In Video Streams

1 Introduction

In the context of Artificial Intelligence, the idea of designing agents that exist in an environment and perceive and act [Russell and Norvig, 2009] is a longstanding goal that introduces a huge number of challenges. While Machine Learning solutions applied to Computer Vision might help in pursuing such a goal, most of their outstanding results are obtained in well-defined vision tasks, leveraging huge collections of supervised data or exploiting pretrained backbones [Ranftl et al., 2021]. Recent Self-Supervised Learning schemes aim at robust representations without human intervention, still exploiting very large image collections [Jing and Tian, 2020].

Supplementary material: https://arxiv.org/abs/2204.12193.

A lot of issues arise when trying to exploit neural models from other vision tasks in order to design a visual agent that learns while watching a video stream, especially when the agent is expected to parsimoniously interact with humans to get information on what it sees. Pretrained models might not always help in capturing properties of entities that belong to the particular environment in which the agent lives [Kornblith et al., 2019], and they are subject to inductive biases. Moreover, the agent must be able to learn synchronously with the continuous video stream, and the target classes are not known in advance. This setting not only implies redundancy in visual information, but it also introduces constraints in the data order, that cannot be shuffled as commonly done to implement stochastic gradient descent. To this regard, the scientific community is progressively paying more attention to continual learning [Parisi et al., 2019].

An often neglected element of crucial importance is the focus of attention, which guides the agent in wild visual scenes and attributes precise locations to the human-machine interaction. For example, consider an agent that asks for or receives a specific supervision in a crowded scene, or whenever there is a linguistic interface to exchange information with the human. Without contextualizing the dialogue to what is being precisely observed, the interaction is hardly meaningful. In particular, we are referring to the simulation of humanlike visual attention trajectories [Zanca et al., 2020], which is different from popular neural attention models [Chaudhari et al., 2019], that are learnt to cope with a certain task, still relying on large datasets processed offline. Supervisions can be in the form of a class/instance label about what is being observed, without a precise indication on the boundaries of what is supervised (as depicted in Fig. 1), differently from several Computer Vision tasks [Long et al., 2015].

In this paper, (i) we propose a novel approach to online learning from a video stream, that is rooted on the idea of using a scanpath-based focus of attention mechanism [Zanca et al., 2020] to explore the video and to drive the learning dynamics in conjunction with motion information. The attention coordinates offer a precise location for interaction purposes, and its trajectory has been recently proved to efficiently select the most salient information of the video stream when learning with deep architectures [Tiezzi et al., 2020].

We propose to learn representations that are coherent over the temporal attention trajectory during slow movements of the simulated gaze, that are likely to cover visual patterns with the same semantics. Attention is paired with information coming frommotion, that intrinsically suggests the spatial bounds of the attended area. This leads to a spatio-temporal unsupervised criterion that enforces coherence in the representations learned while observing what is moving (Fig. 1). In order to avoid trivial solutions, we augment the criterion with a contrastive term that favours the development of different representations in what is inside the moving area and what is right outside of it. Thanks to a graph-based formalization of this approach, we define a stochastic procedure that introduces variability in the information provided to the learning algorithm and leads to faster processing, also mitigating the effects of noisy motion information. (ii) We consider the case in which the human intervention is rare, and each supervision is about the coordinates of a single pixel with its class/instance label, thus not a signal that can strongly drive the features development. Target classes are not known in advance, and our model includes an open-set approach to avoid making predictions about unknown elements. In order to cope with continual learning, we propose a template-based schema with a dynamic update procedure that is synchronous with the processed stream and efficiently handled by modern hardware. (iii) Dealing with this setting introduces the further challenge of how to evaluate the artificial agent. We exploit the growing activity in realistic 3D Virtual Environments [Meloni et al., 2020], designing from scratch (and sharing) three ad-hoc streams with different visual difficulty levels. (iv) We compare the learned representations with those from state-of-the art pretrained models that exploited a massive supervision, showing that the latter are not as powerful as one might expect in the considered setting.

2 RelatedWork

Online, continual, open set learning. An online learning model progressively learns from a stream of data, continuously adapting to every new processed input instance [Hoi et al., 2018]. In the specific case of continual learning (also known as life-long, continuous or incremental learning [Parisi et al., 2019]), the goal of the agent is not fixed a priori but changes over time. In this paper, the goal of the agent is to learn to predict the class labels in every pixel of a video stream. Such goal is not fully defined in advance, since the agent becomes aware of classes in function of what the human supervisor tells him, being closer to task-free cases [Aljundi et al., 2019]. Open-set classifiers [Scheirer et al., 2012] can distinguish between examples belonging to different training classes and they can detect whether data do not belong to any of them, that is the case of what we propose. Our work is class-incremental [Geng et al., 2020], due to the progressive inclusion of new classes after human intervention. 1

Focus of attention. Several attempts to model human-like focus of attention mechanisms were presented [Borji and Itti, 2012], that not only differ in the way they are implemented, but also in the nature of the predicted attention (i.e., a temporal trajectory rather than saliency) [Borji, 2019]. Recently, an unsupervised dynamical model was proposed [Zanca et al., 2020] that can be applied both to static images and videos, also studied in the context of online learning in deep networks [Tiezzi et al., 2020]-without any loss of generality, it is the model we consider here.

Learning invariant features. Typical convolutional neural architectures require high sample complexity to learn representations invariant to factors that are not informative for the task at hand. Some solutions disentangle what and where features, each of them separately encoding informative and uninformative factors of variation [Burt et al., 2021]. This paper follows the perspective in which the attention trajectory, paired with motion information, offers a compact way to implicitly constrain the agent to learn invariances. This idea is linked to recent studies about learning invariance to motion in unsupervised learning over time [Betti et al., 2020].

Semantic segmentation. Semantic segmentation aims at associating a class label to each pixel of a given image. Deep architectures for this task usually rely on supervised learning from offline data, including fully convolutional networks [Long et al., 2015], models based on transposed convolutions, dilated convolutions, upsampling and/or unpooling (U/V-net architectures [Ronneberger et al., 2015]), transformers [Ranftl et al., 2021]. We will compare with these models.

Virtual environments. The significantly improved quality of the rendered scenes and the intrinsic versatility of 3D Virtual Environments have quickly increased their popularity in the Machine Learning community (e.g., [Gan et al., 2020]). SAILenv [Meloni et al., 2020] is a recently proposed environment based on Unity3D, specifically aimed at easy customization and interface to Machine Learning libraries. SAILenv yields pixel-level motion information from Unity3D, and provides utilities to ease the generation of adhoc scenes for continual learning scenarios [Meloni et al., 2021], making it well suited for what we propose.

3 Model

The basic concepts of the proposed unsupervised feature extractor are summarized in the example of Fig. 2. Our method is based on the assumption that a human-like attention trajectory [Zanca et al., 2020] generally spans the important location of the stream, and the gaze moves more slowly within areas with uniform semantic properties, whose bounds can be further guessed by motion information. Hence, we enforce both temporal and spatial coherence constraints to force pixel embeddings to be consistent in time and space, with respect to locations virtually connected by the attention trajectory and by motion, respectively. We exploit a contrastive loss to avoid trivial solutions, and a stochastic approach to lighten the computational burden. Given the pixel embeddings, a template-based classifier learns how to classify each of them in an open-set class-incremental setting (Fig. 3).

1 However, it differs from the protocol of the open-world scenario, that goes beyond what we describe in this paper [Geng et al., 2020]. One/few shot supervised models [Min et al., 2021] also learn new classes from few examples, exploiting prior knowledge.

Let us indicate with X the set Rw×h×c , and consider a continuous and potentially life-long video stream V ∈ (X)N that, at the discrete time index t ∈ N ↦→ Vt ∈ X, yields the video frame Vt at the resolution of w × h pixels with c channels. We also consider a neural network model that implements the function f(·, ω) : X → F , with F = Rw×h×d , where ω indicates the weights and biases of the net. The network is designed to process a frame Vt, and f(Vt, ω) is an encoded representation of the frame where, for each pixel, we have a vector with d components (Fig. 3). The network evolves over time and ωt are the weights and biases at time t. We denote with fx the output of f restricted to the pixel at 2D coordinates x ∈ Z ⋄ := {1, . . . ,w} × {1, . . . , h}. In online learning from a video stream V , the network weights ωt+1 are obtained updating the previous ωt with a law that depends on the gradient of a suitable loss function L with respect to ω.

Human-like attention. The first pillar sustaining our model is a function t ↦→ a(t), yielding an explicit estimate of the 2D coordinates where human would focus the attention at each time t. Among the various attention-prediction models yielding temporal trajectories [Borji and Itti, 2012], the recent unsupervised model [Zanca et al., 2020] achieved stateof-the art results in simulating human-like attention. The authors showed that visual cues of the frame can act as gravitational masses. The equation of the potential of the gravitational field, φ(x, t) := -(2π)-1 ∫Z log ∥x - z∥µ(z, t) dz, is at the basis of the attention model, where Z is the continuous set of frame coordinates and µ(x, t) is the total mass at (x, t). In particular, the magnitude of the brightness and an optical-flow-based measure of motion activity can be modeled as masses, and their impact is controlled by two positive scalars αb and αm, respectively, and an inhibitory signal is also included. The focus of attention is modeled as a pointlike particle subject to the above potential, and its trajectory a(t) is determined integrating the following equation with initial conditions a(0) = a0 and ˙a(0) = a1 ,

where the dissipation is controlled by ρ > 0 and ∇φ is the spatial gradient of the potential. The gaze performs fixations in locations of interest, with relatively low speed movements. Smooth pursuit consists of slow tracking movements performed to follow a considered stimulus. Differently, saccades are fast movements to relocate the attention toward a new target. In the most naive case, the first two categories of patterns can be distinguished by the latter exploiting a thresholding procedure on velocity, based on ν > 0.

Temporal coherence. During fixations, the attention spans a certain part of an object, with limited displacements of the gaze. Similarly, during smooth-pursuit the attended moving area has uniform semantic properties. Differently, during saccades, the attention switches the local context, shifting toward something that might or might not belong to the same object. We implement the notion of temporal coherence defining the loss function LT that (i.) restricts learning to the attention trajectory, filtering out the information in the visual scene [Tiezzi et al., 2020] and (ii.) avoids abrupt changes in the feature representation during fixations and smooth pursuit,

where ∥ · ∥ is the Euclidean norm, and δt is equal to 0 in case of saccades, otherwise it is 1. 2 When plugged into the online optimization, ω ← ωt and ˆω ← ωt-1. This loss is forced to zero for t = 0, and it will be paired with other penalties described in the following, thus avoiding trivial solutions.

Spatial coherence. Temporal coherence is not enough to capture the spatial extension of the encoded data, since it is only limited to the attention trajectory. Motion naturally provides such spatial information, and we follow this intuition designing agents that learn by observing moving attended regions. 3 As a matter of fact, motion plays a twofold role, being crucial in defining the attention masses and as a mean to extend the notion of coherence to a frame region, i.e., spatial coherence. Formally, for each fixed t ∈ N, we indicate with St ⊆ Z ⋄ the set of frame coordinates that belong to the region of connected moving pixels that includes at. 4 We introduce what we refer to as spatial attention graph at t, Gt, with a node for each pixel of the frame (∀x ∈ Z ⋄ ) and with two types of edges, referred to as positive and negative edges. Positive edges link pairs of nodes whose coordinates belong to St, while negative edges link nodes of St to nodes outside the moving region. The positive edges of the attention graph allow us to introduce a spatial coherence loss LS,

2 We also consider the case of feature vectors with unitary Euclidean norm-see supplementary material.

3 Filtering out camera motion, e.g., in agents aware of how the camera moves (devices with sensors) or using software techniques.

that encourages the agent to develop similar representations inside the attended moving area St, while ω ← ωt. The notion of learning driven by the fulfilment of spatio-temporal coherence over the attention trajectory (LS and LT of Eq. 2 and Eq. 3) is the second pillar on which our model is built.

Contrastive loss. In order to prevent the development of trivial constant solutions, which fulfill the spatio-temporal coherence, we add a contrastive loss LC that works the opposite way LS does. In particular, LC exploits negative edges of G to foster different representations between what is inside the moving area and what is outside of it,

where Ot = Z ⋄ \ St is composed of frame coordinates not in St and ε > 0 avoids divisions by zero. We notice that this contrastive loss is different from InfoNCE [Oord et al., 2018].

Stochastic coherence. These spatial and contrastive losses are plagued by two major issues. First, the number of pairs in Eq. 3 and Eq. 4 is large, being it (1/2)|St|(|St| - 1) and |St|(wh - |St|), respectively, making the computation of the loss terms pretty cumbersome. Secondly, whenever an exact copy of a moving object appears in a not-moving part of the scene, there will be pixels of the first instance that are fostered to develop different representations with respect to pixels of the second one, what we refer to as collision. However, this clashes with the idea of developing a common representation of the object pixels. Hence, we replace Gt with the subgraph ˜ Gt, that is the stochastic spatial attention graph at time t, composed by nodes that are the outcome of a stochastic subsampling of those belonging to Gt. In particular, the node set of ˜ Gt is the union of ˜ St ⊆ St and ˜Ot ⊆ Ot, where at is guaranteed to belong to the former. Edges of ˜ Gt are the positive and negative edges of Gt connecting the subsampled nodes. The key property of the stochastic graph is that the number of positive and negative edges is a chosen e > 0. 5 The set ˜ St is populated by uniformly sampling nodes in St, ensuring that at is always present, while ˜Ot is populated by sampling from a Gaussian distribution centered in at and with variance σ, discarding samples /∈ Ot. 6 Large σ's lead to sampling data also far away from the focus of attention, while small σ's will generate samples close to the boundary of the moving region. In the example of Fig. 2 we emphasized how the attention bridges multiple instances of ˜ Gt over time, yielding a stochastic attention graph. Such a graph reduces the probability of collisions, both due to the random sampling and to the control on the sampled area by means of σ, and it introduces variability in the loss functions also when computed on consecutive frames. Moreover, the impact of imperfect segmentation of the moving region is reduced, since only some pixels are actually exploited, re-sampled at every frame. Since the number of pairs is bounded, the stochastic graph makes the formulation suitable for real-time processing.

4 Moving pixels are selected as detailed in the suppl. material.

5 Once we select e, if s := | ˜ St| and o := | ˜Ot|, we have e = s(s1)/2 positive edges and e = so negative ones. We approximated the solution with s = ⌊(1 + √1 + 8e )/2⌋ and o = ⌈e/s⌉.

Cumulative loss. We define L as the cumulative loss at a certain time instant t, where the contributes of Eq. 2, Eq. 3, Eq. 4 are weighted by the positive scalars λT , λS and λC,

The losses ˜LS and ˜LC are the stochastic counterparts of LS and LC, respectively, in which the sets ˜ St and ˜Ot are used in place of St and Ot. We define ∇L to be the gradient of the loss with respect to its first argument, that drives the online learning process, ω0 = ϖ, ωt+1 = ωt - α∇L(ωt, ωt-1, t), t ∈ N, with α > 0 (learning rate) and ϖ some random initialization of the parameters of the network. We remark that other recent learning schemes for temporal domains could be exploited as well [Tiezzi et al., 2020].

Pixel-wise classification. Ahuman supervisor occasionally provides a supervision at coordinates at about a certain class yt. 7 Let us define an open-set classifier c(·, ξ, ζ) : F ↦→ Q that predicts the class-membership scores (belonging to a generic set indicated with Q), over a certain number of classes that can be attached to each pixel-level representation. The main parameters of the classifier are collected in ζ, and when all the membership scores are below threshold ξ the classifier assumes to be in front on an unknown visual element and it does not provide a decision (open-set)-see Fig. 3. Whenever a supervision on a never-seen-before class is received, the classifier becomes capable of computing the membership score of such class for all the following time steps (classincremental). We consider the case in which supervisions are extremely rare, not offering a suitable basis for gradientbased learning of c or further refinements of f. The previously described unsupervised learning process is what is crucial to learn compact representations that can be easily classified, since it favours pixels of the same object to be represented in similar ways over time and space.

The most straightforward way to implement the open-set c is with a distance-based model, storing the feature vectors associated to the supervised pixels as templates. 8 This allows the model to not make predictions when the minimum distance from all the templates is greater than ξ. We indicate with (kt, yt) a supervised pair where kt = fat (Vt, ωt) is the template at coordinates at of frame Vt. The intrinsic dependence of kt on the time index could make templates become outdated during the agent life, for example due to the evolution of the system, so that for some t′ > t we might have ∥kt - fat (Vt, ωt′)∥ ≫ 0, leading to potentially wrong predictions. In order to solve this issue, we propose to dynamically update the templates. Modern hardware and software libraries for Machine Learning are designed to efficiently exploit batched computations In online learning, this feature is commonly not exploited, since only one data sample becomes available for each t. We indicate with Bt a special type of mini-batch of frames, Bt = {Vt} ∪ {Vr, r ∈ Ht}, composed of the current Vt and the frames associated to supervised time instants whose indices are in Ht. Due to the tiny number of supervisions, storing supervised frames does not introduce any critical issues, and a maximum size b > 1 for each mini-batch can also be defined, populating Bt with up to b-1 previously supervised frames, chosen with or without priority. This way, batched computations can be efficiently exploited to keep templates up-to-date.

6 We selected σ to be ∝ √|St| by means of an integer spread factor β ≥ 1. We repeat the Gaussian sampling until we collect the target number of points in ˜Ot, up to a max number of iterations.

7 Notice that only one pixel gets a supervision a at time t, thus it is different from few-shot semantic segmentation [Min et al., 2021].

4 Experiments

In order to create the right setting to evaluate what we propose, we need continuous streams able to provide pixel labels at spatio-temporal coordinates that are not defined in advance. Virtual environments. We consider photo-realistic 3DVirtual Environments within the SAILenv platform [Meloni et al., 2020], that include pixel-wise semantic labeling and motion vectors of potentially endless streams, creating (from scratch) three 3D scenes to emphasize different and challenging aspects on which we measure the skills of the agents. 9 . The agent observes the scene from a fixed location, and some objects of interest move, one at the time, along pre-designed smooth trajectories while rotating and getting closer to/farther from the camera. We denote with the term lap a complete route traveled by each object to its starting location.

We designed three different scenes. (i) EMPTYSPACE: four photo-realistic textured object from the SAILenv library (chair, laptop, pillow, ewer) move over a uniform background. The goal is to distinguish them in a non-ambiguous setting. (ii) SOLID: a gray-scale environment with three white solids (cube, cylinder, sphere) is considered. Due to the lack of color-related features, the agent must necessarily develop the capability of encoding information from larger contexts around each pixel. (iii) LIVINGROOM: the objects from EMPTYSPACE are placed in a photo-realistic living room composed by other non-target objects (i.e., an heterogeneous background with a couch, tables, staircase, door, floor), and multiple static instances of the objects of interest. Samples of are shown in Fig. 4 (top) and, in what follows, n is the number of target objects in each stream, while m = n + 1 is the total number of categories (including the "unknown" class).

8 We tested the squared Euclidean distance and cosine similarity. 9 Code, data and selected hyper-parameters can be downloaded at https://github.com/sailab-code/cl stochastic coherence

Setup. We created three pre-rendered 2D visual streams by observing the moving scenes (256 × 256 pixels, with ≈ 51 k, 12 k and 20 k frames, respectively, corresponding to 31 completed laps for each object), both in grayscale (BW) and color (RGB)-SOLID is BW only. The agent learns by watching the first 25 laps per object and, only in the subsequent laps, receives a total of 3 supervisions (kt, yt) per object, spaced out by at least 100 frames. Learning stops when all the objects complete 30 laps and, finally, performances are measured in the last lap, considering the F1 score (averaged over the m categories), either along the attention trajectory or in the whole frame area. We evaluated the proposed approach considering two different families of deep convolutional architectures yielding d output features, referred to as HOURGLASS ( UNet-like [Ronneberger et al., 2015]) and FCN-ND (6-layer Fully-Convolutional without any downsamplings [Sherrah, 2016]), respectively (see the supplementary material for all the details). We compared the obtained features against those produced by massively pretrained state-of-the-art models in Semantic Segmentation. We considered the Dense Prediction Transformer (DPT) [Ranftl et al., 2021] and DEEPLABV3 [Chen et al., 2017] with ResNet101 backbone, exploiting both the features produced by the penultimate layer in the classification heads (-C suffix) and the ones obtained by the backbones (i.e., upsampling the representations if needed, - B suffix). In this way, we investigate both lower-level features based on backbones pretrained on millions of images (ImageNet), and task-specialized higher-level features for semantic segmentation (COCO [Lin et al., 2014] and ADE20k [Zhou et al., 2019] datasets-the latter explicitly includes the categories of the considered textured objects). As BASELINE model we considered the case in which the pixel representations are left untouched (i.e., pixel color/brightness).

Parameters. Parameters of the attention model were either fixed (αm = 1), or adapted according to a preliminary run of the model (αb, ρ). For each video stream, we searched for the model hyper-parameters that maximize the F1 along the attention trajectory, measured during the 30-th laps (hyperparams grids and the best selected are reported in the suppl. material). Fig. 4 (bottom) shows the effects of the parameters e and β in modeling the stochastic coherence graph.

Results. Table 1 (top) shows the F1 along the attention trajectory measured during the latest object lap, averaged over 3 runs with different initialization. The proposed learning mechanism is competitive and able to overcome models pretrained on large supervised data. This is mostly evident in the case of HOURGLASS, while FCN-ND is less accurate, mostly due to the lack of implicit spatial aggregation. Table 1 (bottom) is about the F1 computed considering predictions on all the pixels of all the frames. This measure is not directly affected by the attention model, and while our model performs worse, it is still on par with some of the competitors, with the exception of LIVINGROOM (RGB). We investigated our results, showing in Fig. 5 some sample predictions comparing HOURGLASS with transformers (DPT-B). Indeed, stateof-the-art models have troubles in recognizing closer objects and also in discriminating from the background, due to the fact that their pixel representations are typically strongly affected by a large context. When presenting objects in unusual orientations, likely different from what observed during the fully supervised training, they tend to perform badly. Differently, our model adapts to the video stream, learning more coherent representations when the object transforms. Overall, the attention model allows to focus on what is more important, and, although just a tiny number of supervisions are provided, our model can online-learn to make predictions that competes with the massively offline trained competitors.

In-depth studies and ablations. In order to evaluate the sensitivity of the proposed approach to the key elements of the considered setup, we selected the HOURGLASS model of Table 1. Fig. 6 reports results of experiments in which we changed the number of supervisions per object, the number of edges e (per type) in the stochastic graph, we disabled the temporal coherence, and we changed the length of the streams (discarding SOLID in which differences were less appreciable). Even with a single supervision, the model is able to distinguish the target objects in EMPTYSPACE, while in the more cluttered LIVINGROOM it benefits from multiple supervisions, as expected. Our proposal is better than using a non-stochastic criterion (LIVINGROOM), and works well even with a limited value for e. Moreover, the agent benefits from relatively longer streams (going from 15 to 30 laps), that allow it to develop more coherent representations (we tuned the model in the 30-lap case, that is the reason for the slight performance drop in 55-lap), and temporal coherence has an important role in the overall results. For completeness, we highlighted in Fig. 6 (last) the computational benefits, in terms of time (one lap per object), brought by the stochastic subsampling, showing that they are more evident for large d.

5 Conclusions

We presented a novel approach to the design of agents that continuously learn from a visual environment dealing with an open-set class-incremental setting, leveraging a focus of attention mechanism and spatio-temporal coherence. We devised an innovative way of benchmarking this class of algorithms using 3D Virtual Environments, in which our proposal leads to results that, on average, are comparable to those obtained using state-of-the art strongly-supervised models.

Acknowledgements

This work was partly supported by the PRIN 2017 project RexLearn, funded by the Italian Ministry of Education, University and Research (grant no. 2017TWNMH2).