Harnessing Vision Models for Time Series Analysis: A Survey

1 Introduction

Vision models have long been applied to time series analysis. Early successes with 1D CNNs [Bai et al., 2018], such as WaveNet [Van Den Oord et al., 2016], led to their widespread use across time series tasks [Koprinska et al., 2018; Zhang et al., 2020]. Recently, advances in language-based sequence modeling have shifted attention to Transformers [Wen et al., 2023] and Large Language Models (LLMs) [Zhang et al., 2024]. Meanwhile, the demands for universal modeling has driven the emergence of time series foundation models such as TimesFM [Das et al., 2024], Chronos [Ansari et al., 2024], and Time-MoE [Shi et al., 2024].

As Large Vision Models (LVMs), such as ViT [Dosovitskiy et al., 2021], BEiT [Bao et al., 2022] and MAE [He et al., 2022], become achieving a similar success as LLMs (but in vision domain), a great deal of emergent efforts has been invested to explore the potential of LVMs in time series modeling [Chen et al., 2024]. This is inspired by the plenty of ways for visualizing time series as images such as line plots of univariate time series (UTS) and heatmaps of multivariate time series (MTS). Such images provide a more straightforward view of time series than the counterpart textual representations to humans and, presumably, AI bots.

Taking a closer inspection reveals more advantages favoring LVMs over LLMs: (1) There is an inherent relationship between images and time series - each row/column in an image (per channel) is a sequence of continuous pixel values. By pre-training on massive images, LVMs may have learned important sequential patterns such as trends, periods, and spikes [Chen et al., 2024]. In contrast, LLMs are pre-trained on discrete tokens, thus are less aligned with continuous time series. In fact, LLMs' effectiveness on time series modeling is in question [Tan et al., 2024]; (2) Instead of using channelindependence assumption [Nie et al., 2023] to individually model each variate in an MTS, some imaging methods (§3.7) can naturally represent MTS, enabling explicit correlation encoding; (3) When prompting LLMs, existing methods often struggle with properly verbalizing a long sequence (or a matrix) of floating numbers in a UTS (or MTS), which may be limited by the context length or induce high API costs. In contrast, existing works find that using LVMs on imaged time series is more prompt-friendly and less API-costly [Daswani et al., 2024]; (4) Some imaging methods can encode long time series in a compact manner [Naiman et al., 2024], thus have a great potential in modeling long-term dependency.

Also, the concurrent developments of LLMs and LVMs for time series pave the way for a confluence, i.e., leveraging Large Multimodal Models (LMMs), such as LLaVA [Liu et al., 2023], Gemini [Team, 2023] and Claude-3 [Anthropic, 2024], to consolidate the two complementary modalities, which may revolutionize the way (e.g., visually, linguistically, etc.) that users interact with time series.

Despite the significance, a thorough review of relevant works is absent in the existing literature to the best of our knowledge. The survey [Zhang et al., 2024] discusses a few vision models, but its focus is LLMs for time series. In light of this, in this survey, we comprehensively investigate the traditional and the state-of-the-art (SOTA) methods. Fig. 1 identifies the general process of applying vision models for time series analysis, which also serves as the structure of this survey. Our taxonomy has a dual view: (1) in Time Series to Image Transformation (§3), we review 5 primary imaging methods including Line Plot, Heatmap, Spectrogram, Gramian Angular Field (GAF), Recurrence Plot (RP), and some other methods; (2) in Imaged Time Series Modeling (§4), we discuss conventional vision models, LVMs and the initial efforts in LMMs. To highlight the taxonomy, we defer the discussion on the desiderata of pre- and post-processing to the end of this survey (§5). For comparison, we provide Table 1 to summarize the existing methods. Finally, we discuss future directions in this promising field (§6). A Github repository 1 is also maintained to provide up-to-date resources including our code of the imaging methods in §3. We hope this survey could be an orthogonal complement to the existing surveys on Transformer [Wen et al., 2023], LLMs [Zhang et al., 2024; Jiang et al., 2024] and foundation models [Liang et al., 2024] for time series, and provide a complete view on the process of using vision models for time series analysis, so as to be an insightful guidebook to the developers in this area.

2 Preliminaries and Taxonomy

In this paper, a UTS is represented by x = [x1, ..., xT ] ∈ R1×T where T is the length of the UTS, xt (1 ≤ t ≤ T ) is the value at time step t. Suppose there are d variates (or features), let xi ∈ R1×T (1 ≤ i ≤ d) be a UTS of the i-th variate, an MTS can be represented by X = [x⊤1 , ..., x⊤d ]⊤ ∈ Rd×T .

As illustrated in Fig. 1, this survey focuses on methods that transform time series to images, namely imaged time series, and then apply vision models on the imaged time series for tackling time series tasks, such as classification, forecasting and anomaly detection. It is noteworthy that methods on videos or sequential images (a.k.a. image time series [Tarasiou et al., 2023]) do not belong to this category because they don't transform time series to images. Similarly, methods for spaciotemporal traffic data are out of our scope if the methods focus on streams of images (e.g., traffic flows in a stream of grid maps [Zhang et al., 2017]), but methods on imaging time-space matrices [Ma et al., 2017] that resemble MTS are included. For vision models on audios, this survey only discusses some representative works in §3.3 due to space limit. The focus of the survey will remain on general time series.

1 https://github.com/D2I-Group/awesome-vision-time-series

2.1 Taxonomy

We propose a taxonomy from the two views of Time Series to Image Transformation (§3) and Imaged Time Series Modeling (§4) as illustrated in Fig. 1. For the former, we discuss 5 primary methods for imaging UTS or MTS, and remark on their pros and cons. For the latter, we classify the existing methods by conventional vision models, LVMs and LMMs. We discuss their strategies on pre-training, fine-tuning, prompting, and the deigns of task-specific heads. We also discuss the challenges and solutions in pre-/post-processing in §5. Table 1 presents a summary. In the following two sections, we will delve into the existing methods from the two views.

3 Time Series To Image Transformation

This section summarizes the methods for imaging time series (§3.1-§3.6) and their extensions to encode MTS (§3.7).

3.1 Line Plot

Line Plot is a straightforward way for visualizing UTS for human analysis (e.g., stocks, power consumption, etc.). As illustrated by Fig. 2(a), the simplest approach is to draw a 2D image with x-axis representing time steps and y-axis representing time-wise values, with a line connecting all values of the series over time. This image can be either three-channel (i.e., RGB) or single-channel as the colors may not be informative [Cohen et al., 2020; Sood et al., 2021; Zhang et al., 2023]. ForCNN [Semenoglou et al., 2023] even uses a single 8-bit integer to represent each pixel for blackwhite images. So far, there is no consensus on whether other graphical components, such as legend, grids and tick labels, could provide extra benefits in any task. For example, ViTST [Li et al., 2023b] finds these components are superfluous in a classification task, while TAMA [Zhuang et al., 2024] finds grid-like auxiliary lines help enhance anomaly detection.

In addition to the regular Line Plot, MV-DTSA [Yang et al., 2023] and ViTime [Yang et al., 2024] divide an image into h × L grids, and define a function to map each time step of a UTS to a grid, producing a grid-like Line Plot. Also, we include methods that use Scatter Plot [Daswani et al., 2024; Prithyani et al., 2024] in this category because a Scatter Plot resembles a Line Plot but doesn't connect data points with a line. By comparing them, [Prithyani et al., 2024] finds a Line Plot could induce better time series classification.

For MTS, we defer the discussion on Line Plot to §3.7.

3.2 Heatmap

As shown in Fig. 2(b), a Heatmap visualizes the magnitudes of the values in matrix using color. It has been used to represent the matrix of MTS, i.e., X ∈ Rd×T , as a onechannel d × T image [Li et al., 2022; Yazdanbakhsh and Dick, 2019]. TimEHR [Karami et al., 2024] extends this to irregular MTS, where the intervals between time steps are uneven, by grouping the uneven time steps into H uniform bins, creating a d ×H Heatmap image. In [Zeng et al., 2021], a different method is used for visualizing a 9-variate financial MTS. It reshapes the 9 variates at each time step to a 3 × 3 Heatmap, and uses a sequence of the images to forecast future frames, achieving time series forecasting. In contrast, VisionTS [Chen et al., 2024] uses Heatmap to visualize UTS. Similar to TimesNet [Wu et al., 2023] and TimeMixer++ [Wang et al., 2025], it segments a length-T UTS into ⌊T/P⌋ length-P subsequences, where P denotes a period, stacks them into a P × ⌊T/P⌋ matrix, and replicates the channel 3 times to form a grayscale image for input to an LVM. For MTS, VisionTS adopts channel-independence strategy [Nie et al., 2023] and individually models each variate.

Remark. Heatmap can be used to visualize matrices of various forms. It is also used for matrices generated by the subsequent methods (e.g., Spectrogram, GAF, RP) in this section. In this paper, the name Heatmap refers specifically to images that use color to visualize the (normalized) values in UTS x or MTS X without performing other transformations.

3.3 Spectrogram

A spectrogram is a visual representation of the spectrum of frequencies of a signal as it varies with time, which are extensively used for analyzing audio signals [Gong et al., 2021]. Since audio signals are a type of UTS, spectrogram can be considered as a method for imaging a UTS. As shown in Fig. 2(c), a common format is a 2D heatmap image with x-axis representing time steps and y-axis representing frequency, a.k.a. a time-frequency space. Each pixel in the image represents the (logarithmic) amplitude of a specific frequency at a specific time point. Typical methods for producing a spectrogram include Short-Time Fourier Transform (STFT) [Griffin and Lim, 1984], Wavelet Transform [Daubechies, 1990], and Filterbank [Vetterli and Herley, 1992].

STFT. Discrete Fourier transform (DFT) can be used to describe the intensity f(w) of each constituent frequency w of a UTS signal x ∈ R1×T . However, f(w) has no time dependency. It cannot provide dynamic information such as when a specific frequency appears in the UTS. STFT addresses this deficiency by sliding a window function g(t) over the time steps in x, and computing the DFT within each window by

where w is frequency, τ is the position of the window, f(w, τ ) describes the intensity of frequency w at time step τ .

By selecting a proper window function g(·) (e.g., Gaussian/Hamming/Bartlett window), a 2D spectrogram (e.g., Fig.

2(c)) can be drawn via a heatmap on the squared values |f(w, τ)|2 , with w as the y-axis, and τ as the x-axis. For example, [Dixit et al., 2024] uses STFT based spectrogram as an input to LMMs for time series classification.

Wavelet Transform. Continuous Wavelet Transform (CWT) uses the inner product to measure the similarity between a signal function x(t) and an analyzing function. The analyzing function is a wavelet ψ(t), where the typical choices include Morse wavelet, Morlet wavelet, etc. CWT compares x(t) to the shifted and scaled (i.e., stretched or shrunk) versions of the wavelet, and output a CWT coefficient by

where ∗ denotes complex conjugate, τ is the time step to shift, and s represents the scale. In practice, a discretized version of CWT in Eq. (2) is implemented for UTS [x1, ..., xT ].

It is noteworthy that the scale s controls the frequency encoded in a wavelet - a larger s leads to a stretched wavelet with a lower frequency, and vice versa. As such, by varying s and τ, a 2D spectrogram (e.g., Fig. 2(d)) can be drawn on |c(s, τ)|, where s is the y-axis and τ is the x-axis. Compared to STFT, which uses a fixed window size, Wavelet Transform allows variable wavelet sizes - a larger size for more precise low frequency information. Thus, the methods in [Du et al., 2020; Namura et al., 2024; Zeng et al., 2023] choose CWT (with Morlet wavelet) to generate the spectrogram.

Filterbank. This method resembles STFT and is often used in processing audio signals. Given an audio signal, it firstly goes through a pre-emphasis filter to boost high frequencies, which helps improve the clarity of the signal. Then, STFT is applied on the signal. Finally, multiple "triangle-shaped" filters spaced on a Mel-scale are applied to the STFT power spectrum |f(w, τ )|2 to extract frequency bands. The outcome filterbank features ˆ f (w, τ ) can be used to yield a spectrogram with w as the y-axis, and τ as the x-axis.

Filterbank was adopted in AST [Gong et al., 2021] with a 25ms Hamming window that shifts every 10ms for classifying audio signals using Vision Transformer (ViT). It then becomes widely used in the follow-up works such as SSAST [Gong et al., 2022], MAE-AST [Baade et al., 2022], and AST-SED [Li et al., 2023a], as summarized in Table 1.

3.4 Gramian Angular Field (GAF)

GAF was introduced for classifying UTS using CNNs by [Wang and Oates, 2015a]. It was then extended to an imputation task in [Wang and Oates, 2015b]. Similarly, [Barra et al., 2020] applied GAF for financial time series forecasting.

Given a UTS x ∈ R1×T , the first step is to rescale each xt to a value ˜xt within [0, 1] (or [-1, 1]). This range enables mapping ˜xt to polar coordinates by ϕt = arccos(˜xi), with a radius r = t/N encoding the time stamp, where N is a constant factor to regularize the span of the polar coordinates. Two types of GAF, Gramian Sum Angular Field (GASF) and Gramian Difference Angular Field (GADF) are defined as

which exploits the pairwise temporal correlations in the UTS. Thus, the outcome is a T × T matrix Gwith Gt,t′ specified by either type in Eq. (3). A GAF image is a heatmap on G with both axes representing time, as illustrated by Fig. 2(e).

3.5 Recurrence Plot (RP)

RP [Eckmann et al., 1987] encodes a UTS into an image that captures its periodic patterns by using its reconstructed phase space. The phase space of x ∈ R1×T can be reconstructed by time delay embedding - a set of new vectors v1, ..., vl with

where τ is the time delay, m is the dimension of the phase space, both are hyperparameters. Hence, l = T - (m- 1)τ . With vectors v1, ..., vl, an RP image measures their pairwise distances, results in an l × l image whose element is

where Θ(·) is the Heaviside step function, ε is a threshold, and ∥·∥ is a norm function such as ℓ2 norm. Eq. (5) generates a binary matrix with RPi,j = 1 if vi and vj are sufficiently similar, producing a black-white image (e.g., Fig. 2(f)).

An advantage of RP is its flexibility in image size by tuning m and τ. Thus it has been used for time series classification [Silva et al., 2013; Hatami et al., 2018], forecasting [Li et al., 2020], anomaly detection [Lin et al., 2024] and explanation [Kim et al., 2024]. The method in [Hatami et al., 2018], and similarly in HCR-AdaAD [Lin et al., 2024], omit the thresholding in Eq. (5) and uses ∥vi -vj∥ to produce continuously valued images to avoid information loss.

3.6 Other Methods

Additionally, [Wang and Oates, 2015a] introduces Markov Transition Field (MTF). MTF is a matrix M ∈ RQ×Q encoding transition probabilities among Q time segments, to visualize UTS. ImagenTime [Naiman et al., 2024] stacks the delay embeddings v1, ..., vl in Eq. (4) to an l×mτ matrix for visualizing UTS. MSCRED [Zhang et al., 2019] applies heatmaps to d × d correlation matrices of d-variate MTS for anomaly detection. Earlier, [Kumar et al., 2005] proposes Bitmaps to image discretized UTS and defines distances among them. Some methods use a mixture of imaging methods for richer representations, e.g., [Wang and Oates, 2015b] stacks GASF, GADF, and MTF into a 3-channel image. FIRTS [Costa et al., 2024] combines GASF, MTF, and RP. These multi-view representations have shown greater robustness than single-view images in these works for classification tasks.

3.7 How toModel MTS

In the above methods, Heatmap (§3.2) can be used to visualize the variate-time matrices, X, of MTS (e.g., Fig. 1(b)), where correlated variates should be spatially close to each other. Line Plot (§3.1) can be used to visualize MTS by plotting all variates in the same image [Wimmer and Rekabsaz, 2023; Daswani et al., 2024] or combining all univariate images to compose a bigger image [Li et al., 2023b], but these methods only work for a small number of variates. Spectrogram (§3.3), GAF (§3.4), and RP (§3.5) were designed specifically for UTS. For these methods and Line Plot, which are not straightforward in imaging MTS, the general approaches include using channel independence assumption to model each variate individually [Nie et al., 2023], or stacking the images of d variates to form a d-channel image [Naiman et al., 2024; Kim et al., 2024]. However, the latter does not fit some vision models pre-trained on RGB images which requires 3-channel inputs (more discussions are deferred to §5).

Remark. As a summary, Table 2 recaps the salient advantages and limitations of the five primary imaging methods that are introduced in this section.

4 Imaged Time Series Modeling

With image representations, time series analysis can be readily performed with vision models. This section discusses such solutions from traditional models to the SOTA models.

4.1 Conventional VisionModels

Following traditional image classification, [Silva et al., 2013] applies a K-NN classifier on the RPs of time series, [Cohen et al., 2020] applies an ensemble of fundamental classifiers such as SVM and AdaBoost on the Line Plots for time series classification. As an image encoder, CNNs have been widely used for learning image representations. Different from using 1D CNNs on sequences [Bai et al., 2018], 2D and 3D CNNs can be applied on imaged time series as shown in Fig. 3(a). For example, regular CNNs have been used on Spectrograms [Du et al., 2020] and Heatmaps [Wang et al., 2025], tiled CNNs have been used on GAF images [Wang and Oates, 2015a; Wang and Oates, 2015b], dilated CNNs have been used on Heatmaps [Yazdanbakhsh and Dick, 2019]. More frequently, ResNet [He et al., 2016], Inception-v1 [Szegedy et al., 2015], and VGG-Net [Simonyan and Zisserman, 2015] have been used on Line Plots [Semenoglou et al., 2023], Heatmap images [Zeng et al., 2021; Wu et al., 2023], RP images [Li et al., 2020; Kim et al., 2024], GAF images [Barra et al., 2020; Kaewrakmuk et al., 2024], and even a mixture of GAF, MTF and RP images [Costa et al., 2024]. In particular, for time series generation tasks, GAN frameworks of CNNs [Li et al., 2022; Karami et al., 2024] and a diffusion model with U-Nets [Naiman et al., 2024] have also been explored.

Due to their small to medium sizes, these models are often trained from scratch using task-specific training data. Meanwhile, fine-tuning pre-trained vision models have already been found promising in cross-modal knowledge transfer for time series anomaly detection [Namura et al., 2024] and forecasting [Li et al., 2020].

4.2 Large VisionModels (LVMs)

Vision Transformer (ViT) [Dosovitskiy et al., 2021] has inspired the development of modern LVMs such as Swin [Liu et al., 2021], BEiT [Bao et al., 2022], and MAE [He et al., 2022]. As Fig. 3(b) shows, ViT splits an image into patches of fixed size, then embeds each patch and augments it with a positional embedding. The vectors of patches are processed by a Transformer as if they were token embeddings. Compared to CNNs, ViTs are less data-efficient, but have higher capacity. Thus, pre-trained ViTs have been explored for modeling imaged time series. For example, AST [Gong et al., 2021] fine-tunes DeiT [Touvron et al., 2021] on the filterbank spetrogram of audios for classification tasks and finds ImageNet-pretrained DeiT is remarkably effective in knowledge transfer. The fine-tuning paradigm has also been adopted in [Zeng et al., 2023; Li et al., 2023b] but with different pre-trained models such as Swin by [Li et al., 2023b]. VisionTS [Chen et al., 2024] attributes LVMs' superiority over LLMs in knowledge transfer to the small gap between the pre-trained images and imaged time series. It finds that with one-epoch fine-tuning, MAE becomes the SOTA time series forecasters on some benchmark datasets.

Similar to time series foundation models such as TimesFM [Das et al., 2024], there are some initial efforts in pre-training ViT architectures with imaged time series. Following AST, SSAST [Gong et al., 2022] introduced a masked spectrogram patch prediction framework for pre-training ViT on a large dataset - AudioSet-2M. Then it becomes a backbone of some follow-up works such as AST-SED [Li et al., 2023a] for sound event detection. For UTS, ViTime [Yang et al., 2024] generates a large set of Line Plots of synthetic UTS for pre-training ViT, which was found superior over TimesFM in zero-shot forecasting tasks on benchmark datasets.

4.3 Large Multimodal Models (LMMs)

As LMMs get growing attentions, some notable LMMs, such as LLaVA [Liu et al., 2023], Gemini [Team, 2023], GPT-4o [Achiam et al., 2023] and Claude-3 [Anthropic, 2024], have been explored to consolidate the power of LLMs and LVMs for time series analysis. Since LMMs support multimodal input via prompts, methods in this thread typically prompt LMMs with textual and imaged representations of time series, and instructions on what tasks to perform (e.g., Fig. 3(c)).

InsightMiner [Zhang et al., 2023] is a pioneer work that uses the LLaVA architecture to generate texts describing the trend of each input UTS. It extracts the trend of a UTS by Seasonal-Trend decomposition, encodes the Line Plot of the trend, and concatenates the embedding of the Line Plot with the embeddings of a textual instruction, which includes a sequence of numbers representing the UTS, e.g., "[1.1, 1.7, ..., 0.3]". The concatenated embeddings are taken by a language model for generating trend descriptions. Similarly, [Prithyani et al., 2024] adopts the LLaVA architecture, but for MTS classification. An MTS is encoded by the visual embeddings of the stacked Line Plots of all variates. The matrix of the MTS is also verbalized in a prompt as the textual modality. By integrating token embeddings, both methods fine-tune some layers of the LMMs with some synthetic data.

Moreover, zero-shot and in-context learning performance of several commercial LMMs have been evaluated for audio classification [Dixit et al., 2024], anomaly detection [Zhuang et al., 2024], and some synthetic tasks [Daswani et al., 2024], where the image and textual representations of a query time series are integrated into a prompt. For in-context learning, these methods inject the images of a few example time series and their labels (e.g., classes) into an instruction to prompt LMMs for assisting the prediction of the query time series.

4.4 Task-Specific Heads

As Fig. 3(d) illustrates, for classification tasks, most of the methods in Table 1 adopt a fully connected (FC) layer or multilayer perceptron (MLP) to transform an embedding into a probability distribution over all classes. For forecasting tasks, there are two approaches: (1) using a de ×W MLP/FC layer to directly predict (from the de-dimensional embedding) the time series values in a future time window of size W [Li et al., 2020; Semenoglou et al., 2023]; (2) predicting the pixel values that represent the future part of the time series and then recovering the time series from the predicted image [Yang et al., 2023; Chen et al., 2024; Yang et al., 2024] (§5 discusses the recovery methods). Imputation and generation tasks resemble forecasting as they also predict time series values. Thus approach (2) has been used for imputation [Wang and Oates, 2015b] and generation [Naiman et al., 2024; Karami et al., 2024]. When using LMMs for classification, text generation, and anomaly detection, most of the methods prompt LMMs to produce the desired outputs in textual answers, circumventing task-specific heads [Zhang et al., 2023; Dixit et al., 2024; Zhuang et al., 2024].

5 Pre-Processing and Post-Processing

To be successful in using vision models, some subtle design desiderata include time series normalization, image alignment and time series recovery.

Time Series Normalization. Vision models are usually trained on standardized images. To be aligned, the images introduced in §3 should be normalized with a controlled mean and standard deviation, as did by [Gong et al., 2021] on spectrograms. In particular, as Heatmap is built on raw time series values, the commonly used Instance Normalization (IN) [Kim et al., 2022] can be applied on the time series as suggested by VisionTS [Chen et al., 2024] since IN share similar merits as Standardization. Using Line Plot requires a proper range of y-axis. In addition to rescaling time series [Zhuang et al., 2024], ViTST [Li et al., 2023b] introduced several methods to remove extreme values from the plot. GAF requires min-max normalization on its input, as it transforms time series values withtin [0, 1] to polar coordinates (i.e., arccos). In contrast, input to RP is usually normalization-free as an ℓ2 norm is involved in Eq. (5) before thresholding.

Image Alignment. When using pre-trained models, it is imperative to fit the image size to the input requirement of the models. This is especially true for Transformer based models as they use a fixed number of positional embeddings to encode the spatial information of image patches. For 3-channel RGB images such as Line Plot, it is straightforward to meet a pre-defined size by adjusting the resolution when producing the image. For images built upon matrices such as Heatmap, Spectrogram, GAF, RP, the number of channels and matrix size need adjustment. For the channels, one method is to duplicate a matrix to 3 channels [Chen et al., 2024], another way is to average the weights of the 3-channel patch embedding layer into a 1-channel layer [Gong et al., 2021]. For the image size, bilinear interpolation is a common method to resize input images [Chen et al., 2024]. Alternatively, AST [Gong et al., 2021] resizes the positional embeddings instead of the images to fit the model to a desired input size. However, the interpolation in these methods may either alter the time series or the spatial information in positional embeddings.

Time Series Recovery. As stated in §4.4, tasks such as forecasting, imputation and generation requires predicting time series values. For models that predict pixel values of images, post-processing involves recovering time series from the predicted images. Recovery from Line Plots is tricky, it requires locating pixels that represent time series and mapping them back to the original values. This can be done by manipulating a grid-like Line Plot as introduced in [Yang et al., 2023; Yang et al., 2024], which has a recovery function. In contrast, recovery from Heatmap is straightforward as it directly stores the predicted time series values [Zeng et al., 2021; Chen et al., 2024]. Spectrogram is underexplored in these tasks and it remains open on how to recover time series from it. The existing work [Zeng et al., 2023] uses Spectrogram for forecasting only with an MLP head that directly predicts time series. GAF supports accurate recovery by an inverse mapping from polar coordinates to normalized time series [Wang and Oates, 2015b]. However, RP lost time series information during thresholding (Eq. 5), thus may not fit recoverydemanded tasks without using an ad-hoc prediction head.

6 Challenges and Future Directions

Fundamental Understanding. With various imaging methods available, most existing works select them based on intuition. There remains a gap in both theoretical and empirical understanding of questions such as which imaging methods fit which tasks and whether LVMs truly learn patterns that make them better suited to time series than LLMs. Some existing works evaluate multiple imaging methods, but in limited tasks. For example, ImagenTime [Naiman et al., 2024] compares the representation abilities of GAF, STFT, and delay embedding (§3.6) in a time series generation task. However, a thorough understanding to guide the development of LVMs and LMMs across imaging methods is absent. This survey provides an initial comparative discussion in §3. Further empirical and theoretical investigations are essential to the synergy between LVMs/LMMs and time series analysis.

Modeling the Correlation of Variates in MTS. §3.7 reviewed existing methods for imaging MTS, each with limitations. For example, Heatmaps (e.g., Fig. 2(b)) encode spatial relationships, so the row order of variates affects how correlations are modeled, implying correlated variates should be spatially close to each other. Similarly, Line Plots do not explicitly capture inter-variate correlations. Stacking one channel per variate into a d-channel input hinders the use of pre-trained LVMs, which expect 3-channel RGB inputs. Therefore, there is a need for more effective imaging or modeling techniques (e.g., incorporating graph neural networks (GNNs) on variates) to better capture correlations in MTS.

Advanced Imaging for Time Series. In addition to the basic methods introduced in §3, advanced image representations hold promise. For example, InsightMiner [Zhang et al., 2023] adopts seasonal-trend decomposition, which is often used to extract components that can serve as inductive biases for time series models. Extending this idea to decompose images such as Spectrogram, GAF, and RP into finer-grained components may enhance vision models' effectiveness. Moreover, combining multiple imaging methods from different views, such as frequency (Spectrogram), temporal structure (GAF), and recurrence patterns (RP), can provide richer representations. FIRTS [Costa et al., 2024] stacks images across channels for a classification task, but is limited to images of the same size. Modeling a mixture of arbitrary images by methods such as multi-view learning may enable more flexibility.

Multimodal Time Series Models and Agents. As can be seen from Table 1, the existing research on multimodal analysis (with vision modality) is much less than unimodal analysis, with a limited scope of time series tasks. Given the existing LLMs for time series such as Time-LLM [Jin et al., 2024] and S 2 IP [Pan et al., 2024], it is appealing to introduce vision modality to further boost the performance in wide tasks such as forecasting, classification and anomaly detection. Furthermore, the visual representation of time series provides the foundation for exploring multimodal AI agents [Xie et al., 2024] for more intricate and nuanced tasks that requires reasoning and interactions with environments, such as root cause analysis in AI for IT Operations (AIOps).

Vision-based Time Series Foundation Models. A foundation model (FM) is a deep learning model trained on vast datasets that is applicable to a wide range of tasks. Recent time series FMs, such as TimesFM [Das et al., 2024], MOMENT [Goswami et al., 2024], Chronos [Ansari et al., 2024] and Time-MoE [Shi et al., 2024], are mostly built upon LLM architectures and trained on raw time series. Given the potential of image representation, it is promising to explore vision models as a new architecture to revolutionize time series FMs. This research direction not only leverages the advantages of LVMs as introduced in §1 (e.g., the prior knowledge extracted from the vast pre-training images), but also enables future development of vision-language FMs for time series.

7 Conclusion

In this paper, we present the first survey on leveraging vision models for time series analysis, whose general process structures the survey. We propose a new taxonomy consisting of imaging and modeling methods for time series. We discuss the pre- and post-processing steps as well. Each category encompasses representative methods and relevant remarks. The survey also highlights the challenges and future directions for further advancing time series analysis with vision models.

Acknowledgments

This work was partially supported by NSF (2134079), CAREER Award IIS-2338878, IIS-2331908, and research gifts from NEC Laboratories America and Morgan Stanley.