Professor Pawan Goyal was pursuing his bachelor's degree when he found about the intricacies of Sanskrit. He then became a part of projects related to Sanskrit which involved analysis of text and speech to text conversion. His pursuit to unravel the mysteries of the language took him on the path of machine learning and NLP until he ended up doing a PhD after which he began working on a platform for the Sanskrit language. Intrigued?
Firstly, what is NLP?
NLP stands for Natural Language Processing. It is a subfield of linguistics, computer science, and artificial intelligence. It basically helps computers communicate with humans in their own language and scales other language-related tasks. It analyzes different aspects of human language like syntax, semantics, pragmatics, and morphology. Then this linguistic knowledge is converted into machine learning algorithms which are used to perform expected tasks.
But there were a few issues with Sanskrit-
Sanskrit is unique. Let me tell you how. Other languages like English or Hindi follow a specific typology.
For example, in English we say ‘I ate an Apple.’ Its format would be: subject + verb + object.
In Hindi, ‘Me seb khata hu’ (I eat an apple), it's: subject + object + verb.
See? Every language follows a specific typology. But in Sanskrit, your way of writing depends on the meter rather than the typology. When you pronounce it, you have a specific meter in your mind. So this makes Sanskrit translation a tad bit difficult. One way to do this is, you can convert a poetry in prose format and then translate, so it will save the typology.
What Prof. Goyal says on this, All of these problems that generally when you go to a Sanskrit expert and he or she will do it for you, but, we are trying to do it through NLP and computers, so that the experience of learning some skills or reading Sanskrit text becomes much more exciting.
- Technical aspects of Translation:
Context is really important while translating. Because, the same word in different contexts can mean different things. For example ‘cool’, cool can be ‘thanda’ or cool can be ‘smart’. So the accuracy of the translation depends on how much of the context you can encode. In NLP, there is a famous hypothesis called ‘Distributional Hypothesis’ which says ‘the words occurring in the same context tend to have the same meaning’. It happens in real life also, like even if you don't know the exact meaning of the word, you can guess it by knowing the context.
The important requirement of Al/ML is data. You must look at a lot of data to achieve accuracy. By looking at the data you get two things, First, a particular word can be used in how many different contexts. Second, which different words can be used in a particular context. By knowing this information, you can use some machine learning idea saying that these two words are occurring in the same context, so they should have the same representation. Essentially, you can not know what the word exactly means. This is a symbol grounding problem in AI, that you can not claim what the word exactly means but you can use some sort of differential saying that these two words are similar so they should have the same representation and these two words are different, so they should have different representation.
When you get an input sentence, first you get the meaning of each word, then you use context to enhance the meaning. So finally you know how to represent the sentence. Now you need to convert this representation into another language. So you try to find the words which would mean the same and then you give the final output of the translation to the user. So this is the technology behind the translation using ML.
- Plan for extension of this technology to other languages:
The work of Prof Goyal and his team on Sanskrit has been acknowledged in top conferences, in CS , in publication. Sanskrit has unique problems, unique formulas. So there is an attempt to use this to help other Indian languages. Their team is working on machine translation between various Indian languages. The rich linguistic traditions in Sanskrit are being used to enhance the translation quality. Also not only the final translation is sufficient every time. We need logic. Translation must be explainable. So there is an attempt to trace the path using the grammar. Like when we are translating from one language to another can we get the logic behind the final translation, what is the meaning of every word, what is the grammar rule and hence the translation.
What Prof Goyal says on this, We are trying to use the rich linguistic tradition in Sanskrit to help other Indian languages. Also we are trying to make it more explainable. Like how do we know that if I'm translating from Kannada to Hindi, can I tell why this actually came? Can we actually trace the path making use of the grammar?
