Skip to main content

Design of a Phonetically Balanced Code-Mixed Hindi-English Read Speech Corpus for Automatic Speech Recognition





by -Ayushi Pandey, B M L Srivastava , Rohit Kumar, B T Nellore , K S Teja, S V Gangashetty





"Hungry kya?"
"What your bahana is?"

 few advertisement slogans
                   
        Pepsi: "Yeh Dil Maange More"
        Coke: "Life ho to aisi"
Have you come across the above conversations and native half baked pure language :) .

 New pattern emerged known as Hinglish. The mix of Hindi and English is the language of the street and the college campus, and its sound sets many parents' teeth on edge. It's a bridge between two cultures that has become an island of its own, a distinct hybrid culture for people who aspire to make it rich abroad without sacrificing the sassiness of the mother tongue. And it may soon claim more native speakers worldwide than English. full article on Hinglish

 Bilingual and multilingual speech communities recognize code-switching and code-mixing as predominant phenomena in conversational speech. While code-switching is regarded as an inter-sentential alternation between two languages, code-mixing is a word-level embedding of one language in the matrix of another. The phenomenon holds particular relevance in speech communities where the mother tongue and the medium of instruction are different languages.
According to the census of 2001, 12.1% of the speakers in India are speakers of English as their second or third language. With widespread usage and growth of this phenomenon of code-mixing mandates a shift in paradigm from monolingual automatic speech recognition (ASR).

 Types of code-switching that occur in data from various bilingual communities. 
  •  Insertion, where words or elements from one language are inserted into the frame of another. For example: “ मïजाते वक़्त उन्हì drop कर दँगी । ू ” meaning :  "I will drop them when I go."
  • Alternation is described by the act of alternating larger chunks of the sentence, for example a clausal level switch. For example: “ मुझे अच्छा लगेगा if you could come” meaning : "I would like it if you could come."
  • Congruent lexicalisation is described by how a common language structure emerges by overlapping the words/morphemes of the two languages in question. For example, in the word कम्प्यूटरƑ = कम्प्यूटर + ओं, meaning computers = computer + s where a Hindi inflection is being accepted on an English word, computer.


Paper, presented a Phonetically Balanced Code Mixed (PBCM) speech corpus, sampled from a standardized code-mixed text corpus, the Large Code Mixed (LCM) corpus. An optimal text selection procedure has been used to extract 6,126 utterances from the LCM. The PBCM corpus is currently in the process of being recorded and post-processed for speech recognition purposes at IIITHyderabad.
 The primary objectives of the work include:

 • To introduce selected sections of Hindi newspapers as a reliable site for code mixed HindiEnglish.

 • To develop an optimal text selection procedure towards a Phonetically Balanced read speech corpus in Code Mixed (PBCM) Hindi-English.

 • To record the utterances collected in the PBCM, through the contribution of Hindi-English bilingual speakers.

 • To construct a baseline speech recognition system for code-mixed speech, extrapolating on monolingual Hindi and English training resources.

Design of data corpus

As a first step, a large body of data was scraped from three sections, namely Gadgets and
Technology, Lifestyle and Sports from the newspapers DainikBhaskar  and Sanjeevani . 
The following example represents the word level English insertion in the matrix of a Hindi sentence.

Example:
अनहल्थी फ ै ूड्स को अ￸धकतर अवॉइड करना चािहए ।
Gloss:
[unhealthy-ENG] [foods-ENG] [case marker-HIN] [avoid-ENG] [mostly-HIN] [do-HIN] [should-HIN]
Translation:
One should mostly avoid unhealthy foods. 

Here, the English insertion has been transcribed in a matrix sentence of Devanagari. The newspaper corpus contains both English words transcribed in Devanagari, as in the example above, but also a sizeable amount of English words in their Roman transcriptions.


Equation (1) describes the Pearson’s correlation r, where n is the number of pairs to be scored, x is the value contained in the first variable (in our case, the phonetic distribution of the LCM corpus), and y is the value contained in the second variable (phonetic distribution of the PBCM corpus).






The paper presents a phonetically balanced read speech corpus for code-mixed Hindi-English automatic speech recognition. The PBCM corpus has been sampled from a Large newspaper Corpus (LCM), which contains rich lexical insertions from English in a matrix of Hindi sentences. The inclusion of rare triphones in the sampled corpus has resulted in a high phonetic coverage (correlation: 0.996), even with a small number of sentences. 

To the best of knowledge, the PBCM can be safely proposed as one of the first phonetically balanced corpus of code-mixed speech in an Indian language pair. Recordings through the contribution of 100 Hindi-English bilinguals is aimed for the corpus, of which 78 speakers have been recorded. Once post-processed, the PBCM corpus will be made available for research and related purposes.


Comments

Post a Comment

Popular posts from this blog

DDSP: DIFFERENTIABLE DIGITAL SIGNAL PROCESSING

 -By Google Research {jesseengel,hanoih,gcj,adarob}@google.com Paper Link Code Demo Abstract Most generative models of audio directly generate samples in one of two domains: time or frequency. While sufficient to express any signal, these representations are inefficient, as they do not utilize existing knowledge of how sound is generated and perceived. A third approach (vocoders/synthesizers) successfully incorporates strong domain knowledge of signal processing and perception, but has been less actively researched due to limited expressivity and difficulty integrating with modern auto-differentiation-based machine learning methods. In this paper, we introduce the Differentiable Digital Signal Processing (DDSP) library, which enables direct integration of classic signal processing elements with deep learning methods. Focusing on audio synthesis, we achieve high-fidelity generation without the need for large autoregressive models or adversarial losses, demonstrating that DDSP enable...
Post a Comment
Read more

Capsule Neural Networks

-By Sara Sabour, Nicholas Frosst,  Geoffrey E. Hinton  Google Brain Toronto Paper Link Abstract   A capsule is a group of neurons whose activity vector represents the instantiation parameters of a specific type of entity such as an object or an object part. We use the length of the activity vector to represent the probability that the entity exists and its orientation to represent the instantiation parameters. Active capsules at one level make predictions, via transformation matrices, for the instantiation parameters of higher-level capsules. When multiple predictions agree, a higher level capsule becomes active. We show that a discrimininatively trained, multi-layer capsule system achieves state-of-the-art performance on MNIST and is considerably better than a convolutional net at recognizing highly overlapping digits. To achieve these results we use an iterative routing-by-agreement mechanism: A lower-level capsule prefers to send its output to higher...
1 comment
Read more

Detecting Anomaly in Big Data System Logs Using Convolutional Neural Network

By - Siyang Lu, Xiang Wei, Yandong Li, Liqiang Wang Department of Computer Science, University of Central Florida, Orlando, FL, USA School of Software Engineering, Beijing Jiaotong University, China Paper Link Previous week we did a scan of the survey of Anomaly detection   , that covers breath of the topic (Types of Anomalies, Types of Models and their applications). In this summary paper, we look in to the core Heavy lifting part of Detecting Anomaly in Big Data System Logs Using Convolutional Neural Network. Abstract Nowadays, big data systems are being widely adopted by many domains for offering effective data solutions, such as manufacturing, healthcare, education, and media. Big data systems produce tons of unstructured logs that contain buried valuable information. However, it is a daunting task to manually unearth the information and detect system anomalies. A few automatic methods have been developed, where the cutting-edge machine lear...
Post a Comment
Read more