SketchEdit: Editing Freehand Sketches at the Stroke-Level

1 Introduction

People may draw sketches to express their abstract concepts for the real world, and humans possess an extraordinary ability to create imaginative sketches. The objective of sketch synthesis is to mimic the human drawing process through machines, and the task is challenging due to the sketch's abstractness, sparsity, and lack of details. Recently, efforts have been made to learn efficient sketch representations and generate realistic sketches, such as Sketch-RNN [Ha and Eck, 2017], SketchHealer [Su et al., 2020], SketchLattice [Qi et al., 2021] and SP-gra2seq [Zang et al., 2023a].

*Corresponding Author

However, whilst existing methods [Zang et al., 2021; Zang et al., 2023b; Wang et al., 2023] exhibit effective control on generating sketches with certain global properties, they are unable to perform finer control on strokes. For example, researchers have focused on synthesizing sketches of particular categories, such as generating a "cat", but have difficulty in manipulating the shape of certain parts (e.g., the body) of the "cat". Furthermore, during the sketch creation process, users may incorporate fresh strokes or choose strokes for multiple revisions based on inspiration. This paper attempts to present a framework, which generates imaginative editing outcomes and may assist in the heuristic education of children, to mimic human sketch editing at the stroke-level as in Figure 1.

To achieve stroke-level editing, it is a key obstacle to pinpoint the strokes that require editing. For the conventional method [Ha and Eck, 2017] using a sequence of points to represent sketches, although the segments determined by the pen states can be directly used as strokes, the lengths of the obtained strokes are not the same, which is not convenient for editing the strokes and updating the sketch sequence. Rasterizing a sketch into an image is a common operation in sketch studies [Chen et al., 2017; Yu et al., 2015; Yu et al., 2016]. However, these image-based methods lost details of the drawing order and the way sketches are drawn, making it more difficult to get the stroke information. Recently, the work [Qu et al., 2023] provided an effective way to break down the sketch sequence into strokes for downstream tasks, where the stroke segments are padded to be of the same length. Inspired by this idea, we develop a stroke encoder to encode each stroke separately, without exchanging information with another stroke. This approach provides the flexibility to select strokes and edit them in the latent space of the encoder while minimizing the impact on the content of the rest part of the sketch.

Another challenge for stroke-level editing is how to appropriately place the strokes after the editing is done. As given in the second box of Figure 1, if we replace the cat's body with the sheep's body, the cat's head moves from the right to the left side of the image. If the cat's head is still in its original position, the generated sketch will be unrealistic. Here, we develop a diffusion model [Ho et al., 2020] for accurate stroke placement. The diffusion model generates the stroke locations progressively through the denoising process, based on the features of all strokes to be synthesized. The diffusion model extends beyond the generation of single-category sketches, enabling the creation of more diverse results, e.g., a pig with wing-like ears. Furthermore, we fuse the stroke embeddings with the generated stroke locations and devise a sequence decoder to synthesize the final manipulated sketch. The stroke encoder and the sequence decoder are jointly pretrained under the autoencoder paradigm, with an extra image decoder to learn the local structure of sketches.

In summary, we propose a novel sketch editing method called SketchEdit and our contributions are as follows: (i) We develop the traditional task of sketch synthesis into a more controllable sketch editing task at the stroke-level for the first time. The proposed SketchEdit achieves this purpose well and enables the generation of creative sketches. (ii) We present a fresh perspective on the placement of sketch strokes without labeling, where strokes are synthesized akin to assembling building blocks. Given a set of base strokes, we first generate meaningful placements for them, and then combine the strokes into a meaningful sketch. (iii) Experiments show that our method performs significantly better than the state-of-the-art sketch generation models for the task of sketch reconstruction. This guarantees that the edited sketch effectively retains the visual properties of the original sketch for sketch editing at the stroke-level.

2 RelatedWork

Sketch generation. Sketching, as a practical communication tool and medium for emotional expression, is impressive and expressive [Xu et al., 2022; Ribeiro et al., 2020; Alaniz et al., 2022]. Its related generative tasks have attracted the interest of researchers [Zhou et al., 2018; Das et al., 2021; Pourreza et al., 2023]. An essential work to this is Sketch-RNN [Ha and Eck, 2017], which is facilitating research into deep learning for the imitation of human drawing. Although Sketch-RNN is capable of accurately capturing the connection between drawing points, it falls short in perceiving the local structural information of images. Therefore, the subsequent methods [Chen et al., 2017; Song et al., 2018] convert the sequence of sketches into rasterized images and introduce Convolutional Neural Networks (CNNs) as a replacement or supplement to the RNN encoder. To improve the representational capabilities of the models, graph neural networks (GNNs) are introduced on top of the image representation [Su et al., 2020; Qi et al., 2022; Qi et al., 2021; Zang et al., 2023a]. These methods construct graphs by temporal proximity, spatial proximity, or synonymous proximity. Another methods to improve performance are to use a Gaussian Mixture Model (GMM) to model the latent space [Zang et al., 2021; Zang et al., 2023b] or design a Lmser-based network to learn stable sketch representations [Li et al., 2024]. These models have struggled to decouple specific strokes, so our SketchEdit takes strokes as input rather than images or drawing points. There is also a similar class to our methodology, which views sketches as being comprised of multiple parts and requires labeling the components of the sketch, such as the head of a bird [Ge et al., 2020]. However, due to the manual labeling being costly, our method is geared towards unlabeled and more basic strokes. Recently, some studies utilized a parametric representation of sketches [Vinker et al., 2022; Xing et al., 2023] for easy generation. These approaches lack an important feature of sketching, which is the ability to maintain the order in which human strokes are drawn.

Diffusionmodels. Diffusion models [Sohl-Dickstein et al., 2015] have led to a boom in research, particularly in the field of image synthesis [Ho et al., 2020; Dhariwal and Nichol, 2021]. Text-to-image (T2I) generation is a widely recognized application of diffusion models, which enables the rapid generation of artwork by providing prompts as a cue to large models [Ramesh et al., 2021; Rombach et al., 2022]. However, certain information remains difficult to convey solely through text, leading to the emergence of visual cues as conditions for diffusion models. Sketches are an effective tool for responding to structural information and are therefore regarded as control conditions by PITI [Voynov et al., 2023], ControlNet [Zhang and Agrawala, 2023], T2I-Adapter [Mou et al., 2023], and other methods. Recently some diffusion models [Wang et al., 2023; Das et al., 2023] about sketches have been proposed, which focus on modeling the points of the sketch rather than the stroke locations. Our approach differs from these pure diffusion models used for sketch generation in that SketchEdit is able to utilize the highly semantic latent space of the AE paradigm for flexible stroke editing.

3 Methodology

SketchEdit is constructed based on diffusion model to edit sketches at the stroke-level. The key step is to generate the locations of the strokes. This is achieved by the reverse denoising process of the diffusion model conditioned on stroke embeddings, as shown in Figure 2(a). The SketchEdit decouples sketch into several strokes without position information, allowing the user to conveniently select strokes for editing. Strokes and generated locations are eventually fed into a sequence decoder to synthesis the edited sketch. The pipeline of editing sketches are illustrated in Figure 2(b).

3.1 Sketch Representation

A sketch is represented by a sequence of Lp points, i.e., τ = (p1, p2, ..., pLp ). Each point pi is a vector containing five elements. The first two are the coordinates of the absolute position, while the last three use the one-hot vector format to represent the three pen states of lift, touch, and the end of sketch. To proceed in the stroke-level, the sketch sequence is broken down into a series of strokes, i.e., (s1, s2, ..., sLs ), where Ls denotes the number of strokes. We use (x, y) = [(x1, y1), (x2, y2), . . . , (xLs , yLs )] to record the locations of the strokes, which are the coordinates of the first point of the stroke. In this paper, we also define the normalized stroke sequence ˜ si by subtracting the location (xi, yi) of the stroke from the coordinates of all the points in the stroke.

3.2 DiffusionModel for Generating Locations

Forward process. Given a set of stroke locations (x, y)0 ∼ q((x, y)0), we apply the Markov diffusion process in DDPMs [Ho et al., 2020] here. The noise sampled from Gaussian distribution is gradually added to x and y:

where βt represents the noise schedule at time t.

Reverse process. The reverse process aims to recreate the true locations from a Gaussian noise input (x, y)T . Similar with the DDPMs [Ho et al., 2020], A U-Net [Ronneberger et al., 2015] like network is utilized to predict the noise ϵθ((x, y)t, t). However, stroke locations have no explicit semantic information, so it is necessary to introduce strokes as a condition. Thus, the network for predicting noise is modified to ϵθ((x, y)t, t, ˜ s). To decrease computational complexity and leverage high-level semantic information, as illustrated in Figure 2, we utilize the stroke embeddings ˜ z as the condition rather than the strokes ˜ s. The reverse denoising process can be formalized as:

where αt = 1 - βt and ¯αt = ∏ti=1 αi. In practice, we use the DDIM-based [Song et al., 2020] generation process for accelerated sampling.

3.3 Editing Freehand Sketches at the Stroke-level

In this subsection, we provide the process of editing sketch at the stroke-level. First, users pick the stroke ˜ si they want to edit from the sketch τ . The edited stroke ˆ si can either be drawn by the users or selected from the stroke gallery to replace ˜ si. Taking the angle shown in Figure 2(b) as an example, we have obtained the strokes ˆ s( ˜ s1, ˜ s2, ˜ s3, ˆ s4, ˜ s5) after editing. Then, the stroke encoder calculates the stroke embeddings ˆ z( ˜ z1, ˜ z2, ˜ z3, ˆ z4, ˜ z5). As the encoding process does not involve the exchange of stroke information, stroke substitution in the latent space, such as replacing ˜ z4 with ˆ z4, is also possible.

Next, we apply the reverse process of diffusion model to denoise random noise (ˆx, ˆ y)T conditional on ˆ z, resulting in generated stroke locations (ˆx, ˆ y)0. Finally, the stroke embeddings ˆ z and the stroke locations (ˆx, ˆ y)0 are fed into the token mixture block and sequence decoder to synthesis the target sketch ˆ τ .

3.4 Constructing the Stroke Encoder, the Sequence Decoder, and the Image Decoder

After converting the sketch sequence to the normalized stroke representation, the resulting tensor ˜ s ∈ RLs×Ln×5 is obtained, where Ln is the number of points in a stroke. A position-sensitive block must act as the backbone of the stroke encoder to extract features form ˜ s because significant changes in the shape of the stroke occur when any two points in the sequence are interchanged. Token-based MLPs [Tolstikhin et al., 2021] fulfil this requirement, and thus we consider gMLP [Liu et al., 2021] as the basic component. Since we do not wish for any exchange of information to occur during the encoding stage between the strokes, we can intuitively treat the first dimension of ˜ s as the batch size.

Several layers are used to extract the stroke embeddings ˜ z. Firstly, each point in a stroke is treated as a token, which then interacts through the network with other points. Next, these tokens are summed for aggregation to get ˜ zenc ∈ RLs×dmodel1 , where dmodel1 denotes the dimension of the tokens. The stroke embeddings ˜ z ∈ RLs×dmodel2 are calculated as followings:

where flinear(·) and dmodel2 represents a linear projection and the dimension of stroke embeddings, respectively. The reparameterization trick [Kingma and Welling, 2013] employed in Equation (3) serves to effectively constrain the latent space, resulting in improved continuity.

Then, we map the stroke locations (x, y) ∈ RLs×2 to the location embeddings zloc ∈ RLs×dmodel2 . The summation of ˜ z and zloc is fed into a token mixture block to mixture the information of different strokes. The resulting zmix ∈ RLs×dmodel2 is subsequently sent to both the sequence decoder and the image decoder. The decoders utilize spatial projection to increase the number of tokens before reconstructing either the sequence ˜ τ (˜ p1, ˜ p2, ..., ˜ pLp ) or the image ˜ I. The backbone of token mixing block and sequence decoder is gMLP, while the image decoder is built based on CNNs. Thanks to the powerful global capture capability of gMLP, we can decode all sequence points simultaneously, rather than using the autoregressive approach [Ha and Eck, 2017; Chen et al., 2017; Su et al., 2020]. This still result in good reconstruction outcomes.

3.5 Two-stage Training

Pre-training the stroke encoder, the sequence decoder, and the image decoder. After completing end-to-end training, the stroke encoder and the sequence decoder can effectively reconstruct sketches. There are three training objectives. The first is for the output of the sequence decoder, where our goal is to minimize the negative log-likelihood function of the generated probability distribution:

The training goal in Sketch-RNN [Ha and Eck, 2017] also pursues this aim, with the difference being the absolute or relative coordinates modeling. Second, for calculating the image reconstruction loss Limg, we utilize the traditional mean square error (MSE). Finally, to improve the representational power of the model [Zang et al., 2021; Zang et al., 2023b], GMM modeling is carried out in the encoder's latent space. We initialize K Gaussian components and the appropriate number is determined automatically with the aid of RPCL [Xu et al., 1993]. The corresponding loss function is formalized as follows:

where ˜ zi is the stroke embedding correspond to the stroke si and the KL term is calculated as in [Jiang et al., 2016]. The parameters of the GMM are learned by an EM-like algorithm, details of which can be found in [Zang et al., 2021]. In summary, the overall objective is:

where λ is a hyperparameter and we set it to 0.0001 in practice.

Training the diffusion model. In this stage, the previously trained parameters of the stroke encoder and the sequence decoder are fixed, and the following are the training objectives of the diffusion model:

4 Experiment

4.1 Preparation

Dataset. Two datasets are selected from the largest sketch dataset QuickDraw [Ha and Eck, 2017] for experiments. DS1 is a 17-category dataset [Su et al., 2020; Qi et al., 2022]. The specific categories are: airplane, angel, alarm clock, apple, butterfly, belt, bus, cake, cat, clock, eye, fish, pig, sheep, spider, umbrella, the Great Wall of China. These categories are common in life and the instances in the categories are globally similar in appearance. DS2 [Zang et al., 2021] is a multi-style and comparatively small dataset for synthesized sketches, comprising five categories: bee, bus, flower, giraffe, and pig. Each category contains 70000 sketches for training and 2500 sketches for testing.

Implementation details. The AdamW optimizer [Loshchilov and Hutter, 2017] is applied to train the proposed model with parameters β1 = 0.9, β2 = 0.999, ϵ = 10-8 and weight decay = 0.01. We use the CosineAnnealingLR scheduler [Smith and Topin, 2019] with the peak learning rates are 0.002 and 0.0005 for the pre-trained model and the diffusion model, respectively. We set drop path rate to 0.1. All the sketch is padded to the same length, i.e. Lp = 180. Each sketch is break down into Ls = 25 strokes and each stroke contains 96 points. For the pre-trained network, we train it with 15 epochs and the batch size is 200. There are 8 gMLP blocks in the stroke encoder with dmodel1 = 96 and dffn1 = 384. The token mixture block and the sequence decoder includes 2 and 12 gMLP blocks, respectively. We set dmodel2 = 128 and dffn2 = 512 for these blocks. We train the U-Net of the diffusion model with 40 epochs with the batch size is 768. The encoder and the decoder both consist of 12 gMLP blocks. The dmodel and dffn in these blocks are 96 and 384, respectively. We consider the linear noise schedule for the model with βt ∈ (0.0001, 0.02). We take 60 steps for DDIM sampling in default and truncate the stroke locations at (-1, 1) for better performance. More implementation details are in the Supplementary material.

Baselines. We consider 3 types of models as the baselines. Sketch-RNN [Ha and Eck, 2017] employs a VAE [Kingma and Welling, 2013] framework to learn sketch representations from sequences. Sketch-pix2seq [Chen et al., 2017] and RPCL-pix2seq [Zang et al., 2021] take sketch images as input to learn local structural information form sketches. Based on the rasterized sketch images, SketchHealer [Su et al., 2020], SketchLattice [Qi et al., 2021], and SP-gra2seq [Zang et al., 2023a] introduce the graphs for better representations.

Metrics. To evaluate the performance of the SketchEdit, we select Rec [Zang et al., 2021], FID [Heusel et al., 2017], LPIPS [Zhang et al., 2018], and CLIP Score [Radford et al., 2021; Hessel et al., 2021] as the metrics. To classify whether the recreated sketches belong to the original category, two sketch-a-nets [Yu et al., 2015] are trained on DS1 and DS2, respectively. Rec is the success rate of recognition named by [Zang et al., 2021]. The CLIP Score in this paper specifically measures the similarity between the generated sketch and the original sketch.

4.2 Editing Sketches at the Stroke-level

Stroke-level sketch editing involves modifying distinct strokes while minimizing the impact on the overall structure. In this subsection, we provide qualitative analysis and some applications related to the stroke-level sketch editing, including stroke replacement, interpolation between strokes, and stroke addition.

Replacing strokes. Sketches typically consist of various basic shapes and strokes from other sketches can be conveniently reused to edit the intended sketch, as illustrated in Figure 3 and Figure 4. The recycled shapes may comprise constituents from the identical class with clearly defined meanings, for example, an airplane fuselage, an umbrella handle, and so on. Apart from that, SketchEdit enables a sensible synthesis of strokes from different categories of sketches. Some examples are provided in Figure 3, for instance, the alarm clock's bells have been replaced by apple stems, and the SketchEdit has found a "logical" place for the apple stem. Since the stroke encoder encodes only the structure of the strokes and extracts features with high-level semantics while avoiding the influence of sketch position information, the resulting stroke embeddings are of high-quality. This assists the diffusion model in learning stroke position relationships in a more effective manner and enables the sequence decoder to efficiently generate after incorporating position information. Figure 3 also gives the synthesis results of the other methods and our technique of using the original locations. The methods of comparison encode the entirety of the sketch, and misplaced strokes can significantly impact the synthesis, making it difficult to generate recognizable sketches. Editing sketches to produce creative results using existing basic shapes can be challenging if the adverse impact of the initial stroke position on the outcome is not reduced. Our objective in normalizing strokes and implementing a diffusion model is specifically to address this issue. Although our sub-method cannot produce high-quality sketches utilizing the initial stroke placements, it expertly preserves the visual features of each individual stroke. Effective stroke reconstruction is a crucial requirement for stroke-level editing, ensuring minimal alteration to the overall sketch structure. More reconstruction-related content will be discussed in the next subsection.

Interpolating between strokes. Figure 4 provides some examples of interpolation between strokes. For edited strokes, the transition from source to target is smooth as the stroke encoder acquires a well-organized and impact latent space. During the process of interpolation, some stroke positions experience a perturbation, e.g., the cats move up and down slightly. This effect is partly attributed to a degree of randomness, which is caused by the initial Gaussian noise of the inverse process in the diffusion model [Ho et al., 2020], and the sampling production of the sequences [Ha and Eck, 2017]. This randomness does not typically affect the overall structure of the edited sketch, and only has negative consequences in certain instances, as exemplified by the bees illustrated in Figure 4.

Adding new strokes. In our method, theway to adding new strokes to a sketch is similar with stroke replacement. Initially, the first n padded strokes are replaced with the target strokes in the input field or in the latent space. The new stroke set is then used to generate the stroke locations and the edited sketch. Figure 5 shows the results of adding strokes. The diffusion model generates reasonable locations for the added strokes, an interesting example of which is the clock in the sixth column. In the referenced sketch, the stroke '1' and the stroke '2' are used as a whole to represent the clock scale '12'. However, in the original sketch, there is not sufficient space to include scale '12', and therefore the two strokes are more appropriately represented as scale '1' and scale '2', respectively. Reusing the strokes from the original sketch, our method can also respond effectively. For example, the nostrils of the pig in the penultimate column are reused, and the diffusion model predicts them to the location of the eyes.

4.3 Sketch Reconstruction

Sketch reconstruction (also called controllable sketch synthesis [Zang et al., 2021; Zang et al., 2023a]) requires the model to recreate the sketch ˜ τ from the input τ . High-quality sketch reconstruction is essential to maintaining a consistent visual appearance between the edited sketch and the original sketch. In this subsection, we compare the SketchEdit with other sketch synthesis methods. It should be noticed that, when we use the original locations of the strokes rather than the generated locations for SketchEdit, the comparison becomes fair, because the inputs to the baselines are full sketches (including stroke locations). In this subsection, SketchEdit(w ol) denotes that SketchEdit recreates sketches with original locations instead of generated locations.

Qualitative analysis. Figure 6 presents the qualitative comparisons. Compared to other approaches, SketchEdit(w ol) is capable of reconstructing sketches with high-quality, without introducing additional noisy strokes, while preserving the structural patterns of the sketches. To prevent generated sketches from changing category, the model must first learn an accurate representation of the category-level. A failure case is that Sketch-pix2seq [Chen et al., 2017] reconstructs the last column of the Great Wall into a belt. Capturing structural information at the instance-level is a challenging undertaking. While nearly all the competitors reproduced "cakes" as "cakes", the generated results displayed significant structural changes. Furthermore, the existence of multiple styles within the same sketch category poses a challenge to sketch reconstruction. The proposed SketchEdit(w ol) shows significant preservation of detail about sketch instances, which is the basis for our sketch editing task. When we reconstructed the sketch using the completed SketchEdit, the results are slightly degraded, mainly reflecting the subtle movement of some of the strokes. Nevertheless, SketchEdit is still superior to baselines in terms of visualization.

Quantitative analysis. Table 1 reports the sketch reconstruction performance of the proposed method and its competitors. SketchEdit(w ol) significantly outperforms other methods across all metrics. The proposed method captures global dependencies in sketch sequences more efficiently, while the proposed sequence decoder addresses the challenge of stacked layers in RNN and the deeper network improves reconstruction results. However, due to the data-driven nature of the gMLP block, it lacks adequate inductive bias, resulting in a less prominent advantage of SketchEdit(w ol) on the smaller DS2 compared to DS 1. Similar to the results of the qualitative analysis, the metrics for SketchEdit, which employs generated locations for synthesizing sketches, decreased in comparison to SketchEdit(w ol). On DS1, SketchEdit outperforms the other methods significantly, whereas on DS2, it is able to achieve comparable results. For Sketch-RNN [Ha and Eck, 2017], the FID metrics and other metrics present a distinct phenomenon because there exists a considerable domain gap between the sequences and the images, resulting in a disparity between the distributions learned by the image-based approach and the sequence sketches.

4.4 Ablation Study

In this subsection, we discuss the sampling process, model components, and why the diffusion model was chosen. We conduct the ablation study on DS1.

Sampling process. The DDIM [Song et al., 2020] sampling method features a hyperparameter η. If η = 0, the inverse process is deterministic sampling. Conversely, if η = 1, the inverse process involves the original DDPM [Ho et al., 2020] generation process. Both scenarios are examined, alongside an evaluation of the impact of truncating stroke locations in the sampling process. Table 2 reports the reconstruction performance for different sampling settings. When the positions are not truncated at each sampling step, η = 1 gives better results than deterministic sampling for the same step. This issue may arise if the initial stroke generation occurs outside the canvas area. DDIM sampling is significantly impacted by this phenomenon and faces challenges in producing satisfactory outcomes, therefore, limiting stroke position can be considered a viable solution. We observe that a sampling of 10 steps gives better results than a sampling of 5 steps, with additional steps having possible negative consequences. Due to the sensitivity and lack of semantic information in the sketch stroke locations, the predicted noise by U-Net may not be entirely accurate. Further steps may lead to an accumulation of errors which could negatively impact performance.

Why diffusionmodel? To demonstrate the effectiveness of the diffusion model, we train an MLP to directly predict the stroke locations as a competitor. Its architecture is identical to the noise predictor's U-Net and we fixed the noise input and time step to '0'. Compared with MLP, the autoregressive generation process of the diffusion model is better suited to capturing the structural relationship between sketch strokes, resulting in a more effective approximation of the original stroke position distribution. As shown in Table 2, the greatest discrepancy between the MLP and diffusion models is in the FID metric. This implies that MLP finds it challenging to accurately anticipate appropriate locations via stroke embeddings to produce sketches resembling human creations.

Model components. The role of components for pretrained models is evaluated. No image decoder is included and no token mixture block is shared between the two decoders denoted by SketchEdit(wo i) and SketchEdit(wo s), respectively. Table 3 reports the results of the models containing different components. SketchEdit(full) and SketchEdit(wo s) with image encoders have performance advantages over SketchEdit(wo i). This is because the use of image reconstruction allows the network to learn shape information and spatial relationships. Similarly, SketchEdit(wo s) would make learning image-related information difficult for the token mixture block at the sequence decoder. As shown in Figure 7, some strokes overlap in the results produced by SketchEdit(wo s) and SketchEdit(wo i) which reduces the quality of the recreated sketch. In addition, SketchEdit(full) has marginally fewer parameters compared to SketchEdit(wo s) as it only employs a single token mixture block.

5 Conclusion

In this paper, we develop the traditional sketch synthesis task to the more controllable sketch editing task at the strokelevel and propose the SketchEdit to realize it. We have focused on decoupling independent strokes from sketches to enable editing operations at the stroke-level. The core of our methodology is to employ the diffusion model to acquire reasonable positions and recreate meaningful sketches based on the strokes. Experimental results demonstrate that SketchEdit can edit sketches without altering categories and facilitate the production of innovative sketches across various categories. Meanwhile, SketchEdit which efficiently preserves the spatial structure of sketches and supports the parallel reconstruction of sketch sequences, surpasses the state-of-the-art methods significantly in preserving visual features of sketches.

Acknowledgements

This work was supported by the Shanghai Municipal Science and Technology Major Project, China (Grant No. 2021SHZDZX0102), and by the National Natural Science Foundation of China (62172273). Shikui Tu and Lei Xu are co-corresponding authors.