Ten Words Only Still Help: Improving Black-Box AI-Generated Text Detection via Proxy-Guided Efficient Re-Sampling

1 Introduction

Recent breakthroughs in large language models (LLMs) have significantly improved the quality of AI-generated text (AIGT) and further boosted applications in diverse scenarios. People can easily instruct LLM-supported services like ChatGPT [OpenAI, 2022] to generate texts that are almost imperceptible to humans [Jakesch et al., 2023; Uchendu et al., 2023]. Though LLMs bring much convenience, new societal threats caused by their abuses also emerged: Political manipulators produce AI-generated fake news to risk democracy [Lucas et al., 2023]; students cheat by submitting AIgenerated works without paying expected efforts [Bohacek, 2023]; and content farms accelerate information pollution with AI-generated low-quality articles [Brewster et al., 2023]. To build the first barrier against such threats, developing techniques for detecting AI-generated text is of urgent need.

AI-generated text detection is generally defined as a binary or multiclass classification task. The former is to distinguish human-written and AI-generated text, while the latter subsequently recognizes which LLM generates the given text (usually for forensics needs). According to whether the detector can access the source LLM's internal states [Yang et al., 2023b], existing methods can be categorized as whitebox and black-box methods. White-box methods [Mitchell et al., 2023; Li et al., 2023a] distinguish LLMs using delicate features reflected by internal states like output token probabilities and usually achieve high detection performance, but its applicability is limited due to the widely existing unavailability of internal states in commercial LLM services. In contrast, black-box methods [Guo et al., 2023] require output texts only for feature acquirement and in principle could be applied to any LLM. However, they generally underperform white-box methods and are more likely to suffer generalization issues on texts from a new domain [Bhattacharjee et al., 2023]. Such a dilemma poses a key challenge for effectively detecting AI-generated text in reality.

To address this issue, a possible solution is to estimate features that proved effective in white-box detection for black-box scenarios (see Figure 1). Inspired by the recent study on inferring the decoding strategy of an LLM with the multiple re-sampling [Ippolito et al., 2023], we suppose that the statistics on re-sampling results on black-box LLMs could serve as a good estimation of word output probabilities, which reflects the nuance of internal states among different LLMs and subsequently help improve black-box detection. In this paper, we conduct the first empirical exploration along this line. Preliminarily, we implement a naive but costly solution that prompts the LLM with a continuation instruction multiple times on each position of the given text (i.e., full-text re-sampling). Results show that even using estimated word probabilities, the detector still outperforms the typical blackbox RoBERTa-based detector by 14.3%, validating the feasibility of re-sampling-based black-box detection.

To reduce the sampling cost and improve the practicality, we further design POGER, a proxy-guided efficient resampling method for black-box AIGT detection. The core idea of POGER is to select a subset of words possibly indicative of the LLMs' unique word use characteristics from the given text. As LLMs are usually trained on large-scale human language corpora, the word generated with high probability is often similar across different LLMs under the same context, reflecting human language preferences. Instead, words with lower probabilities are more likely to expose LLMs' uniqueness. Therefore, we employ a proxy white-box LLM to nominate words of relatively low probabilities across the text sequence and then preserve the words with low probability estimation error. By performing re-sampling only for the positions of these words, POGER largely reduces the required sampling times while still maintaining the advantages over existing methods in the challenging 8-class black-box setting. Our contributions are as follows:

• We propose to use estimated word generation probabilities to empower black-box AI-generated text detection and empirically show its feasibility.
• We design POGER, a proxy-guided efficient resampling method that largely reduces sampling cost and maintains detection performance by recognizing words that reflect LLMs' uniqueness.
• Extensive experiments on texts from humans and seven popular LLMs show the superiority of POGER over existing methods for binary, multiclass, and out-ofdistribution detection scenarios. 1

2 Background

2.1 Task Formulation

Given a text including n words x = (x1, x2, · · · , xn), AIGT detection aims to obtain a classifier f : x → y, where y is the source of x. The task can be further categorized into:

1. Binary AIGT Detection: Distinguish whether a text is generated by AI, i.e., y ∈ {human, AI}.
2. Multiclass AIGT Detection: Distinguish where a text is from human or a specific AI model, i.e., y ∈ {human, θ1, θ2, · · · , θM}, where θi is an AI model that can generate text. Similar concepts include origin tracing [Li et al., 2023a] and authorship attribution [Uchendu et al., 2020].

1 Code, Dataset, and Extended version: https://github.com/ ICTMCG/POGER

2.2 RelatedWorks

The detection of AIGT can be active or passive. Active methods add pre-designed watermarks to LLM-generated text for later identification [Kirchenbauer et al., 2023; Yoo et al., 2023; Liu et al., 2023; Gu et al., 2023; Wang et al., 2023a]. They show promising results but require extra efforts from stakeholders like LLM providers, not applicable to non-watermarked LLMs. We focus on passive detection, which is more flexible as it operates without modifying LLM workflow. They could be categorized as:

White-box detection methods exploit information from probabilities, which reflects the essentials of language modeling. Earlier works use overall probability [Solaiman et al., 2019], perplexity [Beresneva, 2016; Tian, 2023], or entropy [Lavergne et al., 2008] of the given text on LMs as features. Subsequent works focus on finer-grained token-level probabilities [Gehrmann et al., 2019; Verma et al., 2023]. For example, Sniffer [Li et al., 2023a] and SeqXGPT [Wang et al., 2023b] utilize the probability lists of token sequences on each candidate LLM for multiclass detection. To obtain richer probability information, recent works also consider perturbation of given text on candidate LLMs through mask-filling [Mitchell et al., 2023] or re-generating [Yang et al., 2023a]. White-box detection methods generally perform better and more robustly than black-box ones [Wang et al., 2023b], but the required access to LLMs' internal states largely limits their application to black-box LLMs. Some variants use other models as proxies but performances drop significantly [Mitchell et al., 2023].

Black-box detection methods typically mine effective features from the given text based on semantic representation [Guo et al., 2023; Chen et al., 2023; Zhan et al., 2023] or stylistic expert knowledge [Fr¨ohling and Zubiaga, 2021; Aich et al., 2022]. Recent works [Yang et al., 2023a; Yu et al., 2023] compute the distance between the given text and re-generated texts to reflect the familiarity the candidate LLM has with the given text. Black-box methods have better applicability, but their performance and generalizability generally fall behind white-box ones, especially for multiclass tasks [Li et al., 2023b]. Our POGER combines both their advantages of applicability under the black-box setting and effectiveness brought by (estimated) white-box features.

3 Preliminary Study on Re-Sampling-Based Black-Box AIGT Detection

A recent study reveals that multiple re-sampling could be used to infer the internal decoding strategy of LLMs [Ippolito et al., 2023] under black-box access. Inspired by this, we propose a naive black-box solution that estimates word generation probabilities (proved effective for white-box detection) using multiple re-sampling on the given text to preliminarily validate the feasibility.

3.1 ANaive Solution

We implement the straightforward full-text re-sampling as the naive solution, which samples multiple times at each position of the given text to compute word probabilities on the blackbox LLM. For a given text x, to obtain the word xi's probability ˆ p(xi|x<i), we instruct the black-box LLM for N times using the following prompt:

Please continue writing the following text, starting from the next word: {x<i}

For each prompting, we obtain the generated word at position i by restricting the maximum output length. The estimated probability of xi given {x<i} is computed as the frequency of xi in the output word set {oj}Nj=1 :

where I(·) is the indicator function. By repeating the above process for each word in x, we obtain an estimated probability list of x on this black-box LLM, denoted as ˆp = {ˆ p(x1), ˆ p(x2|x1), · · · , ˆ p(xn|x<n)}. With ˆp as an alternative input, we can now use white-box methods to validate the feasibility of re-sampling-based detection.

3.2 Experimental Settings

Dataset. Our experiments are based on a dataset consisting of 10,608 text items from humans and seven popular opensourced or API-based LLMs in two scenarios, covering realworld threats such as low-quality content production, news faking, and student cheating. Table 1 details the statistics.

We first obtain human-written samples from Quora and ELI5 dataset [Fan et al., 2019] for the QA domain and IELTS essay and BBC news dataset for the writing domain [Greene and Cunningham, 2006], respectively (500 each source). Subsequently, we prompt the seven LLMs with questions or writing instructions from the original datasets, including GPT-2 XL [Radford et al., 2019], GPT-J [Wang and Komatsuzaki, 2021], LLaMA-2 13B [Touvron et al., 2023], Alpaca 7B [Taori et al., 2023], Vicuna 13B [Zheng et al., 2023], GPT-3.5 Turbo [OpenAI, 2022], and GPT-4 Turbo [OpenAI, 2023] and collects their responses. We group human and AI texts with the same prompts and remove the groups that contain answers expressing rejection or exceeding 350 words. All samples are split into the train/validation/test sets with a 7:2:1 ratio at the group level.

Metrics. We compute F1 for each class and Macro F1 (MacF1) to evaluate overall performance.

3.3 Results &Analysis

We implement our naive solution combined with two powerful white-box detectors, Sniffer [Li et al., 2023a] and SeqXGPT [Wang et al., 2023b]. To facilitate comparison between white-box and black-box settings, we only consider the five open-sourced LLMs to easily obtain the true probabilities and set a 6-class task in this part.

Does Re-Sampling Work? As presented in Figure 2(a), detection performance increases as the sampling times increase for both Sniffer and SeqXGPT. At sampling times of 100, the macro F1 exceeds that of black-box baseline RoBERTa by 14.3%. This result demonstrates the feasibility of estimating word probabilities for black-box AIGT detection, though the requirement of sampling times makes this solution costly.

How does Estimation Error Impact Detection Performance? Inevitably, estimation errors exist with re-sampling of limited times. Though having promising results in Figure 2(a), the detectors still underperform those using true probabilities. To analyze the impact of estimated errors, we adjust the sampling temperature, which changes the probability differences between words and indirectly influences the error. In the resulting Figure 2(b), errors lead to low F1 scores on both left and right sides. On the left, a lower temperature indicates the situation that the target word might not be sampled (estimation probability is 0), resulting in a performance drop. On the right, a higher temperature makes the probabilities of all tokens closer. Even slight sampling randomness causes the ranking of probabilities to change and finally distort the unique characteristics of the LLM. This reveals the importance of error control in probability estimation.

Through the preliminary study, we validated the feasibility of estimated probabilities obtained by re-sampling for blackbox AIGT detection. We also identify two issues of the naive solution: 1) the sampling cost on the full text is extremely high; 2) the influence of estimation error is not well controlled. Our improved method to be introduced POGER will tackle these issues.

4 POGER: Proxy-Guided Efficient Re-Sampling for Black-Box AIGT Detection

To tackle the cost and error control issues exposed in Section 3 and design a more practical black-box detector, we propose POGER. Figure 3 presents the overall architecture of POGER. It operates with three steps: First, POGER forms a small word subset from the given text by selecting words of low probabilities and low estimation errors. Subsequently, multiple re-sampling is applied to words in the subset only to obtain a pseudo probabilistic feature. Finally, the feature is enhanced by compensating contextual information and then fed into a classifier for final detection. Details are as follows.

4.1 Error-AwareWord Selection

Proxy-Based Candidate Nomination. To lower sampling times, we aim to select a small subset of k words from x that reflect the LLM's unique word use characteristics. Here, we use an easy-to-use LM (e.g., GPT-2) as the proxy for candidate nomination. The intuition is as follows: As LLMs are usually trained on large-scale human language corpora, they would learn well on common word use and even different LLMs may output similar texts with high probabilities; in contrast, other words with a lower probability in a text are more likely to expose the unique word use shaped by nuances of LLM training process. Specifically, we use a proxy LM θ to infer on the given text x and obtain token probabilities on it. We transform the list into word-level by computing joint probabilities of corresponding tokens for multi-token words, denoted as pθ = (pθ1 , pθ2 , · · · , pθn ). A lower pθi would make the xi more likely to be selected.

Error-aware bottom-k Word Selection. To mitigate the negative impacts of estimated errors on feature effectiveness, we adopt an error-aware bottom-k word selector. For a word xi with true probability pi, if the estimated probability of xi obtained by re-sampling N times is ˆ pi, the standard error (SE) of ˆ pi is given by:

For low-probability words, we constrain a lower bound on their true probability to ensure that the error in the estimated probability does not exceed ∆ times itself:

& '!|'"! '! ∈ S}

By controlling the relative error, we remove the items in pθ that do not meet the error requirements and obtain pθ′ . Words which are with lowest k probabilities are selected from pθ′ using MINK(·) function and finally form the representative word set S:

4.2 Probability Estimation

We again use the sampling and probability calculation process described in Section 3.1, but only for the selected k words in S on the given M candidate black-box LLMs (denoted as {θi}Mi=1 ) by N times. For efficiency needs, we constrain the maximum context length as b. We get the pseudo log probabilistic feature matrix L = [li]ki=1 ∈ Rk×M , where the M-dimentional feature vector for the i-th word is li = [ ˆ pθj (xIDX[i]|xIDX[i]-b:IDX[i]-1 )]Mj=1 .

4.3 Context-Compensated Classification

In the final step, the source of the target text x is classified based on L. Following [Wang et al., 2023b], we first transform the L into another L′ ∈ Rk×d using convolutional neural network and Transformer to enrich the representation. Furthermore, since we discontinuously select representative words, the information about the local context around the word and their relative positions in the original text is lost. As context compensation, we introduce the contextual semantic representation of the k words to bootstrap the probabilistic representation L′ .

Specifically, we input the given text x into RoBERTa [Liu et al., 2019] and obtain the last-layer word representation as E = {e1, e2, · · · , en}. The representations for the k representative words are then mapped to d dimension through a multilayer perceptron (MLP), forming the contextual feature matrix C ∈ Rk×d :

Then a bidirectional cross-attention is adopted to build interaction between C and L′ , outputting the enhance feature F ∈ Rk×2d :

where ⊕ is a concatenation operation. Since the representation of each position in C implies contextual information, this interaction allows relative positional information to be fused into the final enhanced feature.

Finally, F is fed into another MLP for final classification. The network is optimized using cross-entropy loss.

5 Evaluation

5.1 Experimental Settings

Settings. We continue using the datasets and LLMs introduced in Section 3. To simulate real-world situations, we have two settings: 1) Partial White-Box Setting provides true probabilities for the five open-sourced LLMs; and 2) Black-box Setting treats all models as black-box LLMs. For the former setting, we provide a variant POGER-Mixture, which uses true probabilities from white-box LLMs and estimated ones from black-box LLMs. The estimated probability list is expanded to the same dimension as the true lists by padding with 0. We evaluate for both binary and multiclass detection tasks.

Baselines. 1) GPTZero [Tian, 2023]: Distinguish between human and generated text using perplexity and burstiness of text; 2) RoBERTa [Guo et al., 2023]: A widely used and powerful pre-training-based detector; 3) T5-Sentinel [Chen et al., 2023] Another pretraining-based method for reframing the classification task as a next-token prediction task. 4) DNA-GPT [Yang et al., 2023a]: Determine the source of text based on multiple re-generation, can works in two forms under black-box and white-box settings. 5) DetectGPT [Mitchell et al., 2023]: Determine whether a text is generated by comparing the probability of the original text with a large number of perturbed texts. 6) Sniffer [Li et al., 2023a]: Determine the source of the text using the contrastive features of the probability lists on each candidate model. 7) SeqXGPT [Wang et al., 2023b]: Also based on probability lists of the text, but reframes the classification task as a sequence labeling task.

Implementation Details. For POGER and POGERMixture, we use GPT-2 Large as the proxy for representative word selection, with maximum error tolerance ∆ = 1.2, representative word set size k = 10, re-sampling times N = 100, and sampling temperature t = 1.0. For the five open-source models, we perform re-sampling locally, and for GPT-3.5 Turbo and GPT-4 Turbo, we call OpenAI API for resampling and set max tokens = 2. We input b = 20 words before the target word as context. For white-box detection methods under the black-box setting, we employ GPT-Neo 2.7B and LLaMA 7B as proxy probability providers to ensure their proper functionality (some methods require at least two proxies). For zero-shot detection methods, we do a grid search on classification thresholds and report their optimal performance in the search interval.

5.2 Main Results

Multiclass AIGT Detection

Table 2 shows the performance comparison of POGER and its variants with other baselines. Based on the result, we have the following observations:

• POGER and POGER-Mixture outperform all baselines in macro F1 in both the black-box and partial white-box settings. In particular, POGER and POGER-Mixture

outperform all black-box LLMs (shaded in the table) in single-class F1. This demonstrates that POGER has superior performance, especially for black-box detection.

• Compared with the partial white-box setting, all baseline models experience significant performance degradation in the black-box setting. Among the baseline models, semantic-based RoBERTa and T5-Sentinel perform the best, but they still fall behind POGER by more than 15.9% in macro F1, which indicates that POGER could achieve a good balance between applicability and performance.
• We evaluate the effectiveness of the Context Compensation module in POGER and POGER-Mixture in both settings. We can see that, on the one hand, w/o Context Compensation brings a decrease of over 5 macro F1 scores in POGER performance, highlighting the significance of this module. On the other hand, even the w/o Context Compensation variant of POGER still outperforms all baselines, demonstrating the effectiveness of the resampling strategy we proposed.

Binary AIGT Detection

Table 3 shows the performance of each method in binary detection. Among the baselines, RoBERTa, SeqXGPT, and T5Sentinel perform well (macro F1 of over 90) while others gain unsatisfying performance. This might be influenced by the method nature of focusing more on distinguishing different LLMs. Still, POGER gains the best performance in both settings, showing its wide applicability in different AIGT detection tasks.

5.3 Out-of-Distribution Results

Due to variations in semantic and stylistic features across different domains in AIGT, existing training-based black-box detection methods often exhibit poor performance in OutOf-Distribution (OOD) scenarios. We conducted multi-class AIGT detection experiments between two domains within our dataset, where training samples were sourced from one domain and testing was performed on the other domain. The results are presented in Table 4.

It can be seen that POGER still outperforms all baselines, both in terms of generalizing from QA to writing and vice versa. Meanwhile, compared with their respective InDistribution performance, the relative performance degradation of POGER in the OOD scenario is significantly smaller than the two black-box baselines RoBERTa and T5-Sentinel, and is comparable to the two white-box baselines Sniffer and SeqXGPT. This indicates that, despite being a blackbox AIGT detector, POGER benefits from the excellent OOD generalization capabilities inherited from white-box detection methods through the pseudo probabilistic feature.

6 Analysis

6.1 Representativeness of SelectedWords

We conduct empirical analysis to validate if our word selection methods will select representative words.

Are Low-Probability Words More Helpful in Detection? We implement a simple white-box AIGT detector with an attention layer that takes true probabilities on candidate LLMs as inputs. Figure 4 shows the attention distribution on words in different probability ranking intervals. We observe that the 10% lowest-probability words gain over 30% of the attention weights. As word probability increases, their attention weights decline, indicating low-probability words play more important roles in AIGT detection.

Are Low-Probability Word Sets Similar Between the Proxy and Candidate LLMs? We use each LLM as a proxy model and the other LLMs as target models and show the proportion of words with the lowest 5% probability on the proxy model whose probability on the target model was in the lowest 20% in Figure 5. Even in the worst case, 72% of low-probability words on the proxy model are also hit in the set of the target model. This suggests that proxy model could be used as a good indicator for word selection.

6.2 Hyperparameter Senstivity

We conduct an analysis of four hyper-parameters on a subset containing text from humans and the five open-sourced LLMs for brevity:

Impact of MaximumError Tolerance. Figure 6(a) shows the performance impact of the maximum error tolerance ∆ in the representative word selector. As ∆ increases, we find that the performance curve shows a trend of rising first and then falling. This is due to the fact that when ∆ takes a small value, as the selector has a strict constraint on the error of empirical probability, the low-probability words that best reflect the characteristics of the text source are filtered out, rendering the words in set S insufficiently representative. Representative; when ∆ takes a large value, the error of empirical probability also increases, resulting in the weakening of the effectiveness of probabilistic features. When ∆ is between 1.2 and 2.2, our selector is able to strike a good balance between word representativeness and estimation error.

Impact of Representative Word Set Size & Re-Sampling Times. Intuitively, if the representative word set size k and the re-sampling time N are increased at any cost, the performance of the detector will also increase. However, in practice, we expect POGER to achieve the best possible detection performance while meeting certain cost and efficiency requirements. Therefore, we examine the effect of k and N on POGER performance at the same level of total sampling number on the black-box LLM. Figure 6(b) shows the performance of POGER under different (k, N) pairs when the total sampling number k · N = 1000. It is observed that POGER performs best when k is between 10 and 50 (i.e., N is between 20 and 100). When k is too small, the detector obtains too little information about probability lists; and when N is too small, the error-aware selector filters out a large number of words with excessive errors, resulting in the selected words not being representative enough. Both situations result in POGER failing to make accurate classifications.

We find that compared to the version without content compensation, the full POGER has less variation in performance in response to changes in either N, k, or ∆. We believe this is due to the fact that the contextual semantic information is able to compensate for a portion of the performance in cases where the probabilistic features are not effective enough.

Proxy Model for Representative Words Selection. We conduct small-scale experiments using GPT-2 Large, GPTNeo 2.7B, and LLaMA 7B as proxy models in representative word selection, and the Macro F1s are 85.77, 85.42, and 86.14, respectively, indicating that POGER is not sensitive to the selection of proxy models. It also proves once again that not using the probability value of the proxy model but determining the approximate range of the probability is a proper way of using proxy models for AIGT detection.

6.3 Cost Comparison

With a well-designed re-sampling strategy, we achieve highperformance black-box AIGT detection at a relatively low cost. Although POGER requires re-sampling for each representative word, we only infer for the next position to obtain the generated word instead of repeatedly generating the whole text sequence, thus requiring limited LLM inference cost. If the maximum number of generated tokens is set to 1, the inference length of LLM is even independent of the re-sampling times, since no additional inference beyond prompt is required. Table 5 shows a comparison of the number of inference tokens on a target LLM for POGER and other regeneration-based AIGT detection methods, indicating that POGER's inference cost is similar to DNA-GPT and much less than DetectGPT and Full Sampling.

7 Conclusion and Discussion

In this paper, we proposed to estimate features that proved effective in the white-box setting to help improve blackbox AIGT detection. We first developed a naive solution that leverages multiple re-sampling to estimate word generation probabilities for black-box detection. To further reduce the sampling cost, we designed POGER, which leverages a proxy model to select a subset of representative words with an awareness of sampling errors. Experiments on texts from humans and seven LLMs demonstrated the superiority of POGER for binary, multiclass, and OOD scenarios. Further cost analysis indicates that POGER keeps lower resampling costs than its counterparts. In the future, we plan to further improve the efficiency by introducing result storage.

Ethical Statement

Considering that white-box and black-box LLMs like LLaMA-2 [Touvron et al., 2023] and GPT-4 [OpenAI, 2023] have been widely used in daily lives, and the AI-generated texts have posed real-world threats and are believed to bring more serious societal harms, our research attempts to propose a new method for AI-generated text detection to help defense against the unknown threats in the future. Considering the performance is not perfect for now, the text that is flagged as AI-generated text by POGER and its counterparts should go through extra checking before making an official accusation regarding rule violations to a certain person in practice.

Acknowledgments

This work is supported by the Strategic Priority Research Program of the Chinese Academy of Sciences (XDB068 0202) and the Postdoctoral Fellowship Program of CPSF (GZC20232738). The corresponding author is Qiang Sheng.