Towards Accurate Video Text Spotting with Text-wise Semantic Reasoning

1 Introduction

In recent years, we have witnessed rapid growth in new methods [Wang et al., 2017; Cheng et al., 2019; Cheng et al., 2020; Wu et al., 2021; Wu et al., 2022] and datasets [Karatzas et al., 2013; Karatzas et al., 2015; Wu et al., 2021; Cheng et al., 2019] for video text spotting (VTS). It has been drawing increasing research interest due to its crucial role in computer vision tasks, such as video retrieval [Spolaor et al., 2020] and video subtitle recognition [Xu et al., 2018], and downstream industrial applications, such as license plate recognition [Shashirangana et al., 2020] and machine translation [Stahlberg, 2020].

There have been some works [Wang et al., 2017; Cheng et al., 2019; Cheng et al., 2020; Wu et al., 2021; Wu et al., 2022]

∗Corresponding author

aiming to tackle the VTS problem, most of which tend to design a multi-task pipeline containing text detection, tracking and recognition as three major components. For example, a Transformer-based VTS model is proposed to boost performance in [Wu et al., 2021]. Some methods [Cheng et al., 2019; Cheng et al., 2020] utilize a scoring mechanism to save recognition time. In addition, a light-weighted VTS model is proposed in [Wu et al., 2022] to achieve real-time inference. However, these methods mainly focus on improving the backbones of models and may ignore the underlying semantic relationships among texts within a frame. In this case, most existing methods leverage only two types of information, i.e., visual information and temporal information.

In addition to the aforementioned two types of helpful information, inter-text semantic relationships among texts within a frame may also be beneficial for improving the performance of VTS. For example, the first frame in Fig. 1 is captured from a food store, where the texts therein are related to the semantic concept of food. One can consider a situation that 'Steak' and 'Panini' are correctly predicted, while 'Hamburger' is wrongly predicted as 'lamberger' due to occlusion. As the texts within this particular frame share similar semantics, humans can naturally figure out that 'lamberger' is wrongly spelled based on the observation that both 'Steak' and 'Panini' belong to the semantic concept of food, and can further infer the correct text 'Hamburger' belonging to the same semantic concept. There exist a non-ignorable number of such cases in scene text videos, which hatches our core motivation: We can introduce inter-text semantic reasoning into VTS to achieve more accurate results.

Based on this motivation, we propose an accurate video text spotter, VLSpotter, to read texts visually, linguistically, and semantically. Following the fashion of existing VTS

methods [Wang et al., 2017; Cheng et al., 2019; Cheng et al., 2020; Wu et al., 2021; Wu et al., 2022], VLSpotter adopts the paradigm of performing text detection, tracking and recognition simultaneously. In addition to leveraging visual and temporal information carried in text videos, the unique design of the proposed VLSpotter further explores the potential of inter-text semantic relationships. Specifically, for 'visually', we propose a text-focused super-resolution (TFSR) module to effectively alleviate motion blur and produce high-quality frames for subsequent detection, tracking and recognition; moreover, we propose a mask-adaptive super-resolution loss to supervise TFSR, making it focus more on text areas. For 'linguistically', a language model is employed to mitigate wrongly predicted texts through capturing intra-text context. For 'semantically', we propose a text-wise semantic reasoning (TWSR) module to model the semantic relationships among texts with similar semantics. To train the proposed TWSR, we categorize 21K commonly used texts into 20 semantic concepts, and perform semantic reasoning among texts that belong to the same concept. In this way, better recognition results can be obtained with fused visual, linguistic and semantic features.

Extensive experiments of text spotting, tracking and detection are conducted on three VTS benchmarks, ICDAR2013 Video [Karatzas et al., 2013], ICDAR2015 Video [Karatzas et al., 2015], and BOVText [Wu et al., 2021], to evaluate the effectiveness of the proposed VLSpotter. The experimental results demonstrate that VLSpotter outperforms the existing methods by a clear margin under the criterion of end-to-end text spotting, and achieves comparable performance under the criterion of text detection and tracking. The code of VLSpotter is available at GitHub 1 .

The contributions of this paper are three-fold:

• Based on the observation that texts within a video frame usually exhibit similar semantics, we propose a textwise semantic reasoning module to model inter-text semantic relationships.
• We propose a plug-and-play text-focused superresolution module to enhance video quality and design a mask-adaptive super-resolution loss to force this module to focus more on text areas.
• The proposed VLSpotter outperforms the state-of-theart methods by a clear margin on ICDAR2015 Video and BOVText in end-to-end text spotting.

2 RelatedWorks

In this section, we first introduce text recognition. Then, we introduce the recent development of end-to-end video text spotting. Finally, we briefly introduce the text image superresolution task.

2.1 Text Recognition

Some existing text recognition works have introduced semantic reasoning to text recognition. [Yu et al., 2020] proposes a Transformer-based global semantic reasoning module to capture global semantic context via parallel transmissions; its semantic context is actually the embedded features of attended visual features aligned to characters. Additionally, [Bhunia et al., 2021] develops a multi-stage and multi-scale attention decoder to perform reasoning procedure. Different from [Yu et al., 2020], this method conducts a joint visual and semantic reasoning. Inspired by [Bhunia et al., 2021], [He et al., 2022] proposes a reasoning module implemented based on graph convolution to infer the text results with both visual and textual (language) features in an iterative manner. Technically, the aforementioned works conduct semantic reasoning on top of intra-text semantics, thus still ignoring intertext semantic relationships. Speaking of the diversity of languages in text recognition, a large portion of Chinese texts appear in the VTS benchmarks such as BOVText [Wu et al., 2021] and LSVTD [Cheng et al., 2019]. Some recently proposed Chinese text recognition methods [Yu et al., 2022; Zu et al., 2022; Yu et al., 2021a; Su et al., 2023; Chen et al., 2021b] are capable of tackling them.

1 https://github.com/FudanVI/FudanOCR/VLSpotter

2.2 End-to-End Video Text Spotting

Spotting text from videos has already been studied before the era of deep learning. Recently, the rapid growth of deep learning techniques and datasets for text detection and recognition have hatched several new approaches to end-to-end video text spotting. In [Wang et al., 2017], temporal information is used to enable multi-frame tracking, thus improving spotting performance. Subsequently, [Cheng et al., 2019] and [Cheng et al., 2020] design a scoring network to select high-quality instances from the tracking stream, thus allowing their methods to recognize only once. The framework for tracking, recognition, and scoring is also made end-to-end in these works. Recently, [Wu et al., 2021] employs a Transformer-based detection and tracking system to spot arbitrarily oriented texts in videos, and brings the end-to-end spotting performance to a higher level. [Wu et al., 2022] develops an extremely efficient solution while not harming the performance by conducting GPU-parallel detection post-processing. [Wu et al., 2022] introduces contrastive learning to VTS for managing longrange dependencies across multiple frames. Different from these methods, the proposed VLSpotter attempts to leverage the inter-text semantic relationships within video frames.

2.3 Text Image Super-Resolution

Image super-resolution aims to enhance the quality of lowresolution images, and it is also effective in blurred situations. However, generic image super-resolution methods like [Chen et al., 2022b] have limited effect on text regions. To fill this gap, [Mou et al., 2020] employs super-resolution blocks to benefit text recognizer. [Chen et al., 2021a] makes use of text position and content priors; and [Ma et al., 2023] further leverage text probability priors in a concurrent manner. [Chen et al., 2022a] reconstructs high-quality text images by focusing on text strokes. However, these methods only work on text-line images instead of an entire scene image with multiple texts, which is highly demanded in VTS tasks. In this paper, the proposed text-focused super-resolution module aims to handle text-focused super-resolution of the entire image.

3 Methodology

We introduce the details of the proposed VLSpotter in the following. First, we present the proposed Text-Focused SuperResolution (TFSR) module, which is developed to alleviate the motion blur in text videos and enhance video quality. We next introduce the procedures for text detection, recognition and tracking in VLSpotter. Finally, we introduce the proposed Text-Wise Semantic Reasoning (TWSR) module for correcting text predictions with inter-text semantic relationships within a frame. The overall architecture of VLSpotter is shown in Fig. 2.

3.1 Text-Focused Super-Resolution

To avoid the side effect of motion blur for text detection and recognition, the previous methods [Cheng et al., 2019; Cheng et al., 2020] design a scoring mechanism to select a frame with the best quality in the temporal tracking stream to perform one-time text recognition. However, when all frames in a text video are blurred and achieve low scores, the quality of text areas in the selected frame may still pose challenges for subsequent text detection and recognition. Therefore, we introduce a plug-and-play Text-Focused Super-Resolution (TFSR) module to conquer motion blur in text videos, thus improving the quality of input for subsequent modules.

Considering that super-resolving frames of text video only plays as a preprocessing role for the subsequent modules, we adopt a light-weighted super-resolution framework [Chen et al., 2022b] as the main body of TFSR to reduce the time overhead. Specifically, TFSR consists of a ResNet-based backbone G, a super-resolution block R, and a pixel-shuffle module S. The entire process of TFSR can be formulated as:

where R(i) represents that the image features extracted by G(·) are iteratively processed by the super-resolution block R for multiple iterations. Eventually, the input frames VtL with the size of H ×W are super-resolved into the corresponding super-resolution frame VtS with the size of 2H × 2W.

The TFSR module aims to obtain the frames with better quality for subsequent text detection and recognition. However, existing datasets for super-resolution mainly focus on scene images with few texts. Although TextZoom [Wang et al., 2020] is proposed to solve the text image super-resolution task, the samples are all cropped text areas rather than scene images with multiple texts. Therefore, it is necessary to build a super-resolution dataset for text images to train the proposed TFSR module. As it lacks a scene-level dataset for TSFR training, we synthesize a dataset by replacing the text areas in video frames with the high-resolution (HR) and low-resolution (LR) images in TextZoom to construct a HR-LR pair of text images. In addition, we also observe that the image-quality labels of frames are available in ICDAR2013 [Karatzas et al., 2013], ICDAR2015 [Karatzas et al., 2015], and LSVTD [Cheng et al., 2019]. Thus, two adjacent frames with different quality scores in these datasets can be regarded as a HR-LR text image pair. Through the aforementioned two strategies, we can construct a sufficient super-resolution dataset, containing both synthetic and realworld samples, for text images to train the proposed TFSR.

To make TFSR focus on text areas, we further propose a mask-adaptive super-resolution (MASR) loss Lmasr to supervise this module. Specifically, we first perform text segmentation on an HR text image to obtain the corresponding text mask M, where a pre-trained text segmentation model [Xu et al., 2021] is employed. Then, the text mask M is utilized to distinguish text and non-text areas in the text image and further adaptively adjust the loss weight of them. The MASR loss can be computed by:

where VH and VS represent the HR and SR text images, respectively, ⊙ denotes the pixel-wise product, and ϵ is a hyperparameter. Based on the proposed MASR loss, TFSR module could pay more attention to text areas as the super-resolution quality of the whole text image improves.

3.2 Text Detection, Recognition and Tracking

The output of the TFSR module, i.e. high-quality video frames, are then fed into the pipeline of subsequent text detection, tracking and recognition.

Detection. Taking the super-resolved frame VtS as input, the detection head [Wang et al., 2021] Det(·) is utilized to predict the location of each text area:

(3)

where Bt = {bt1 , bt2 , ..., btN } denotes all the bounding boxes predicted by detector Det(·) in the t-th frame; btn and N denote the n-th bounding box and the total number of detected text boxes, respectively.

Recognition. According to Bt , we can obtain N RoIs cropped from VS. Through transformation, these RoIs can be resized into fix-sized text image patches, denoted as Pt = {pt1 , pt2 , ..., ptN }, which are further sent to the text recognition model. For text recognition, we employ the vision model proposed in [Fang et al., 2021], which consists of a visual feature extractor VM(·) and a prediction head. VM(·) is used to extract the visual features of text image patches FtC :

(4)

Then, the extracted features FtC go through the prediction head to obtain the coarse text prediction Yt = {yt1 , yt2 , ..., ytN }:

(5)

Subsequently, the coarse text prediction Yt will be sent to a pre-trained language model and the proposed TWSR module in a parallel manner. The language model conducts intratext spell correction and generates intra-text semantic features FtL; and the TWSR module will be detailed in Section 3.3.

Tracking. As shown in Fig. 2, three types of features are used for text tracking: the visual features FtV , the contextual features FtC from the vision model VM(·), and the embedding of the coordinates of texts FtB . The visual features FtV are extracted by a CNN-based backbone for text tracking. For the purpose of introducing more text-related features to improve the performance of tracking, we fuse the extracted contextual features FtC fromVM(·). In addition, the position information of a text is also crucial for text tracking since the positional offset of the text in adjacent frames is tiny. Therefore, we feed the predicted coordinates of texts Bt in the text

detector to an embedding layer to obtain the corresponding features FtB . Finally, we concatenate the three types of features and employ a 1×1 convolution layer to fuse them, which can be represented as follows:

where FtM denotes the fused features for tracking. In the inference stage, we employ the KM [Kuhn, 1955] algorithm to match the texts between two adjacent frames, and thus creating a tracking trajectory for each text:

where T t denotes the set of all matched texts from the t-1th frame and the detected new texts in the current frame.

To further increase the effectiveness of FtM , we train the learnable fusion parameters in Eq. 6 by maximizing the similarity of the fused tracking features FM between temporaladjacent texts (positive pairs) that belong to the same tracking trajectory, while minimizing the similarity of the fused tracking features FM between text pairs that do not exist in the same trajectory (negative pairs). The contrastive loss is capable of unifying both the positive and the negative objectives above: Let T t = {τ1, ..., τL} where τl denotes the l-th text, the tracking loss of the trajectory can be computed by:

where τk /∈ T t denotes a text in the other trajectories and σ is a temperature parameter. In the beginning of the given trajectory (t = 1) , the positive pair is set to be τl and itself.

3.3 Text-Wise Semantic Reasoning

The prediction results outputted by the text recognizer may be wrongly spelled due to occlusion or other reasons. However, based on our observation that texts within a video frame usually exhibit similar semantics, which suggests that the wrongly predicted texts may be corrected via semantic reasoning. To this end, we propose a text-wise semantic reasoning (TWSR) module to model inter-text semantic relationships among the texts within a frame.

Let ytn denote the n-th text prediction in Yt , and yt j denote an arbitrary text in Yt other than ytn , i.e., 1 ≤ j ̸= n ≤ N. As shown in Fig. 3, the TWSR module takes a pair of texts (ytn , yt j ) as input, where the support text yt j may have semantic relationships with the coarsely predicted text ytn . Subsequently, TWSR processes the input text pair through an embedding layer, a local semantic reasoning layer, and a global semantic reasoning layer. After going through these layers, the inter-text semantic features FtS between ytn and yt j can be extracted. Finally, TWSR comes up with a single text output stn . We expect the semantic reasoning result stn to be the correction of ytn , if ytn is wrongly predicted, or to remain ytn , if ytn and yt j are irrelevant in semantics.

To achieve this goal, we construct a text-to-concept (T2C) taxonomy, which can be used for training TWSR to successfully model the inter-text semantic relationships. The data of T2C are collected from WordNet, Wikipedia, and Amazon Reviews 2 , including 20 concepts (food, feeling, animal, location, art, movie, traffic, weather, act, event, brand, sport, celebrity, book, clothing, health, science, politic, travel and date), which comprises more than 21K words. Each word in T2C belongs to a specific concept (e.g. 'apple-food' and 'dog-animal').

Based on the collected T2C taxonomy, we propose a tailored pre-training strategy to provide flexible supervision for TWSR. Let cq denote the q-th word in the concept set C cpt , which contains all words with the same semantic concept 'cpt'. The insight of our strategy is to set a flexible ground truth ˆx that alters conditionally as follows:

where E(·) denotes the character-wise edit-distance and ϕ is a threshold. In the training stage, x is randomly selected from the T2C taxonomy during training and the semantic support text of x, namely cq, should be randomly selected from C cpt . Random noises will be manually added to the input text x to mimic the wrongly spelled situations, where the editdistance between the noised text and the original text is constrained to be smaller than the threshold ϕ. In this case, if the shortest edit-distance minq(E(x, cq)) is smaller than ϕ, cargminq (E(x,cq)) is taken as the ground truth. Otherwise, if minq(E(x, cq)) is too large, indicating that x does not belong to the concept 'cpt', we use x itself as the ground truth to encourage the TWSR to turn off the reasoning mode and maintain the original coarse predicted texts from the vision model. During the pre-training stage, the cross-entropy loss Lsec is adopted to supervise this module.

In inference, the target text ytn and each support text yt j are input into the TWSR module to obtain the inter-text semantic features FtS . Following the fashion of ABInet [Fang et al., 2021], a gated fusion operation is adopted to fuse three types of features: visual contextual features FtC , intra-text semantic features FtL, and inter-text semantic features FtS . Finally, the fused visual, linguistic and semantic features FtVLS can be produced and further used for final prediction, which is supervised by the cross-entropy loss Lrec.

3.4 Overall Objective

Except for the aforementioned pre-training losses Lmasr for TFSR and Lsec for TWSR, the rest modules of VLSpotter are jointly trained as follows:

where λ1 and λ2 are hyper-parameters for trade-off.

2 https://s3.amazonaws.com/amazon-reviews-pds/readme.html

4 Experiments

In this section, we first introduce the adopted VTS datasets and the implementation details of the proposed VLSpotter. Subsequently, we compare VLSpotter with the state-of-theart methods on each VTS benchmark for end-to-end video text spotting, video text tracking and detection. Finally, ablation studies are conducted to evaluate the effectiveness of the proposed modules in VLSpotter.

4.1 Datasets

We conduct experiments on three commonly-used datasets: ICDAR2013 Video [Karatzas et al., 2013], ICDAR2015 Video [Karatzas et al., 2015], and BOVText [Wu et al., 2021].

• ICDAR2013 Video was published in the ICDAR Robust Reading Competition. It contains 28 wild videos captured by a moving camera, where 13 videos are used for training and 15 videos for testing. The total number of frames is 15,277, containing 93,934 texts. This dataset only contains English texts.
• ICDAR2015 Video is the upgrade revision of the ICDAR2013 Video dataset, and the number of videos expands to 49, where 25 videos for training and 24 videos for testing. The total number of frames is 27,824, containing 143,588 texts. Both ICDAR2013 and ICDAR2015 Video datasets are quite challenging due to frequent motion blurs.
• BOVText is the largest video text spotting dataset that includes 2,021 text videos in a variety of scenarios. This dataset covers various scenarios, e.g., life vlog, driving, movie, etc. It contains 1,757,598 frames with 7,292,261 texts, where 1,328,575 frames from 1,541 videos are used for training and 429,023 frames from 480 videos are used for testing.

4.2 Implementation Details

The proposed VLSpotter is implemented with PyTorch. All experiments are conducted on a single RTX 3090 GPU with

24GBmemory. For VLSpotter, the TFSRmodule, the TWSR module, and the vision model are all pre-trained in advance for better results. The language model used in VLSpotter is off-the-shelf and frozen during training. We use the Adadelta optimizer with an initial learning rate 0.1, which further shrinks every 200 epochs. Based on the empirical experiments, we set the hyper-parameters ε, σ, ϕ, λ1, λ2 to 1, 0.5, 3, 1, 1, respectively.

4.3 Performance Comparison

Following [Wu et al., 2021], we adopt IDF1 , MOTA, MOTP, M-Matched (M-M), and M-Lost (M-L) as the evaluation metrics to evaluate the performance of VLSpotter and the compared methods in video text spotting and tracking. Precision, Recall, and F-Measure are used to evaluate the performance in video text detection. In addition, FPS is also reported to evaluate the efficiency of the compared methods.

End-to-End Video Text Spotting. In VTS, improving the performance of end-to-end video text spotting is the most crucial and ultimate goal. Following the fashion of existing works, we compare VLSpotter with six existing methods (i.e., USTB-TV/USTB-TV(2) [Karatzas et al., 2015], StardVision-1 [Karatzas et al., 2015], FREE [Cheng et al., 2020], TransVTSpotter [Wu et al., 2021] and CoText [Wu et al., 2022]) on the ICDAR2015 Video dataset. The detailed experimental results are reported in Tab. 1. The proposed VLSpotter achieves the best spotting performance in most metrics. Compared to the most recent state-of-the-art method CoText, the results of VLSpotter in IDF1 , MOTA, MOTP, M-Matched, and M-Lost are boosted by 0.8%, 1.1%, 1.9%, 1.4%, and 0.3%, respectively. On the BOVText dataset, except for the result inMOTA, the proposed VLSpotter still outperforms the state-of-the-art method CoText by 0.5%, 2.6%, 2.1%, and 2.2% in IDF1 , MOTP, M-Matched, and M-Lost, respectively.

To validate the effectiveness of inter-text semantic reasoning, we demonstrate four examples that the TWSR module successfully corrects wrongly predicted texts in Fig. 4. The presented examples show different scenarios related to food, traffic, location, and brand, respectively. Through the visualization, we observe that the texts in a frame indeed exist semantic relationships among one another, which is fully explored in our method to improve the performance of text spotting. For example, 'UREO' is corrected to 'OREO' with the help of 'FONTANEDA' (shown in Fig. 4(d)).

Video Text Tracking. As shown in Tab. 2, we evaluate the tracking performance of VLSpotter on ICDAR2013 Video, ICDAR2015 Video, and BOVText, and adopt multiple state-of-the-art text tracking methods, i.e., USTB-TV/USTBTV(2) [Karatzas et al., 2015], StradVision-1 [Karatzas et al., 2015], AJOU [Koo and Kim, 2013], FREE [Cheng et al., 2020], TransVTSpotter [Wu et al., 2021], SVRep [Li et al., 2021], CoText [Wu et al., 2022], YORO [Cheng et al., 2019], EAST [Zhou et al., 2017], PSENet [Wang et al., 2019], and DB [Liao et al., 2020] for comparison. According to the experimental results, VLSpotter achieves comparable tracking performance with the state-of-the-art method CoText; while on challenging BOVText dataset, VLSpotter even outperforms CoText by 0.1% in IDF1 . Although CoText models the long-range temporal dependency with the tailored contrastive loss, VLSpotter can still achieve comparable performance in video text tracking, which may benefit from the high-quality frames produced by the TFSR module.

Video Text Detection. In video text detection, we evaluate the performance of VLSpotter on ICDAR2013 Video. Multiple state-of-the-art text detectors, i.e., [Epshtein et al., 2010], [Zhao et al., 2010], [Yin et al., 2013], [Khare et al., 2017], [Wang et al., 2018], [Shivakumara et al., 2017], [Wu et al., 2015], [Yu et al., 2021b], ASGD [Feng et al., 2021],

_

(c) Entranc

(d) UREO -- OREO

Figure 4: Visualization of the effectiveness of inter-text semantic reasoning in frames that contain different semantic concepts: (a) Food, (b) Location, (c) Traffic, and (d) Brand. Red bounding boxes denote wrongly predicted texts, which are finally corrected through semantic reasoning by the support texts in green bounding boxes.

FREE [Cheng et al., 2020], SVRep [Li et al., 2021] and CoText [Wu et al., 2022] are used for comparison, and the experimental results are shown in Tab. 3. According to the experimental results, the proposed VLSpotter is tied with the state-of-the-art method CoText [Wu et al., 2022]. Specifically, VLSpotter is superior in Recall, but it slightly falls behind CoText in Precision and F-Measure.

4.4 Ablation Study

The proposed VLSpotter contains a pluggable superresolution module TFSR and a semantic reasoning module TWSR. Therefore, it is important to investigate the effectiveness of both modules. As shown in Tab. 4, we conduct ablation experiments to measure the contributions of TFSR and TWSR. Unsurprisingly, if both of them are unequipped, the performance of VLSpotter will decrease by nearly 2% in all metrics. On one hand, when TWSR is equipped to the baseline model, the performance on recognition-related tasks can be improved clearly. On the other hand, the performance can be boosted in a comprehensive manner by equipping TFSR because it generally increases the visual quality of the input frame and suppresses motion blurs.

4.5 Temporal Semantic Relationships

Intuitively, it is likely that there also exist inter-text semantic relationships between texts from temporal-adjacent frames, considering that scenes of adjacent frames are usually the same ones. Tab. 5 shows interesting results of establishing inter-text semantic reasoning among temporal-adjacent frames. In this case, by sacrificing inference speed, the performance of text spotting can be further improved, i.e., IDF1 , MOTA and MOTP boosted by 0.2%, 0.4% and 0.2%, respectively. A possible reason for the performance improvement is that, when texts of temporal-adjacent frames are taken into account to perform semantic reasoning, some occluded texts can be correctly predicted since they may not be occluded in the adjacent frames.

5 Conclusion

This paper proposes an accurate video text spotter, VLSpotter, which leverages the semantic relationships among texts within a video frame to correct those wrongly spelled texts from the vision model of the text recognizer. To model the inter-text semantic relationships, we propose a text-wise semantic reasoning module to perform semantic reasoning for mitigating wrongly spelled text predictions. Moreover, we improve VTS performance by proposing a pluggable textfocused super-resolution module along with a mask-adaptive super-resolution loss that significantly alleviates motion blurs and enhances video quality. The experimental results demonstrate that VLSpotter outperforms the state-of-the-art methods by a clear margin in end-to-end video text spotting.

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China (No.62176060), STCSM projects (No.20511100400, No.22511105000), Shanghai Municipal Science and Technology Major Project (No.2021SHZDZX0103), Shanghai Research and Innovation Functional Program (No.17DZ2260900), and the Program for Professor of Special Appointment (Eastern Scholar) at Shanghai Institutions of Higher Learning.