Repairing General-Purpose ASR Output to Improve Accuracy of Spoken Sentences in Specific Domains Using Artificial Development Approach

1 Introduction

Many working examples of automatic speech recognition (ASR) systems that convert human spoken speech into text can be seen in day to day use; popular examples are Google Now, IBM Watson and Siri. For freely spoken natural language sentences, the typical recognition accuracy achievable even for such state-of-the-art speech recognition systems have been observed to be about 60% to 90% in realworld environments [Fusayasu et al., 2015; Morbini et al., 2013]. However, such ASR engines may at times perform even worse when used in specific domains where specialized domain terms appear that are not available in general corpus [Twiefel et al., 2014]. Other factors such as variations in speaker's accent, background noise, poor ability to express on the part of the user also effect the accuracy of the ASR's output text. As a consequence if one takes the ASR output asis and attempts natural language processing such as automated question-answering, of such erroneously and partially recognized text, then it becomes rather problematic. It is important, therefore, to repair the ASR output to improve the accuracy of the output-text vis-à-vis the specific domain.

In this demonstration, we present a mechanism to repair the ASR output of popular speech engines like Google Now and IBM Watson. Our mechanism is motivated by evolutionary development (Evo-Devo) inspired artificial development processes (Art-Dev) [Tufte, 2009; Harding and Banzhaf, 2008], and considers an erroneous sentence as a biological cell (zygote) and grows it through evolution and development of the partial gene present in the input sentence with respect to the genes in the domain. Once the genotypes are identified, we grow the genotypes into phenotypes to fill the missing gaps and replace erroneous words with appropriate domain concepts in the sentence. We present the results of such repair mechanism and demonstrate its usefulness.

2 Related Work

[Peng et al., 2013] have proposed a mechanism to improve ASR accuracy by re-ranking the N-best speech recognition hypothesis. Their method uses ASR output text as a set of queries for a web-search system and examine the search results to re-ranking the ASR hypothesis. [Fusayasu et al.,2015] use 'normalized relevance distance' as a measure for semantic similarity between words and use that to replace possibly erroneous words in the recognition. [Tur et al., 2013] use word confusion networks (WCNs) for more robust semantic parsing in a conditional random fields (CRF) framework to repair the errors and show significant semantic parsing performance improvements using WCNs. All the above methods, however, do not repair at the sentence level. In our work, we consider the ASR output as a biological cell that is repaired with respect to the environment through repair and development of its "genes" [Anantaram et al., 2015].

3 Repairing ASR output by Art-Dev method

In our approach the ASR output is treated as a biological cell that encodes 'species-specific' genes of the domain representing concepts in the sentences spoken by a speaker. However, in the process of ASR, wherein the spoken-audio is converted to text output, the genes are 'injured' during the recognition process. We repair all 'injured' genes through an artificial development approach, with evolution and development of the partial gene present in the ASR output with respect to the species-specific genes present in the domain (which encodes the concepts of the domain). In our context, the 'fittest' domain gene replaces the partial gene in the sentence. This is the first-level of repair. The set of all genes remaining in the sentence forms the genotypes of the sentence. Once the genotypes are identified, we grow them into phenotypes to remove the grammatical and linguistic errors in the sentence. The paritally repaired ASR sentence is parsed and the POS tags are evaluated to find any linguistic inconsistencies, which are then repaired. For example, a sentence may have a WP-tag (Wh-pronoun), but a WDT-tag may be missing (Wh-determiner). Using such clues we find appropriate parts and repair the sentence. Linguistic and semantic repairs form the genotype to phenotype repair - the second level of repair - and improves the sentence accuracy.

As an example, let the sentence spoken by a Human speaker (H1) on retail sales domain be "In two thousand fourteen which industry had the peak sales". In our experiment Google Now recognised this sentence as (G1): "in two thousand fourteen which industry had the pixels". Through our partial gene matching method, we find that there is no relationship in the domain ontology between industry and pixels, whereas there is a relationship between industry and peak sales. Further the phonetic match between pixels and peak sales gives us the fitness match to replace the inaccurately recognized word pixels with the domain word peak sales. Thus we repair (G1) to (E1): "in two thousand fourteen which industry had the peak sales" which is more accurate with respect to the domain.

4 Experiments

We show results of our experiments on a financial domain.

5 Conclusions

Thus our evo-devo inspired artificial development process helps repair ASR output of spoken sentences in a domain.