HomeMioTech ResearchArticle Detail

Fano Labs: Multilingual ASR and NLP

In this article, we speak to Albert Lam, Chief Scientist and Acting CTO at Fano Labs on their expertise in Multilingual Speech Recognition and NLP technologies and how it’s being employed in the finance sector.

Albert Lam2019-07-12

Please tell me about Fano Labs and how has it evolved into the company it is today.

Fano Labs is a high-tech spin-off of the University of Hong Kong. Founded by Dr. Miles Wen, an HKU Electrical and Electronic Engineering (EEE) PhD graduate and Professor Victor Li On Kwok, Chair Professor in Information Engineering, HKU, we specialize in Automatic Speech Recognition (ASR), Natural Language Processing (NLP), and big data technologies to help enterprises with the customer services, compliance, and other lines of businesses. With an in-house research team led by our Chief Scientist, Dr. Albert Lam, PhD and Adjunct Assistant Professor in EEE, HKU, we have the capability to conduct research on the most frontier speech and NLP technologies and to transfer the knowledge into products and solutions for the industries.

In the early days, we have been very lucky to be supported by many great organizations and mentors. Headquartered and incubated in HKSTP, we have received funds from HKU and Hong Kong government which enabled us to invest into the research and development of our core technologies and products. In November 2017, Fano Labs raised Pre-A round and became the first HK-based high-tech startup ever invested by Horizons Ventures, who is a leading investor in some of the world’s most innovative companies and disruptive technologies, including Facebook, DeepMind, Skype, and Siri.

In 2018, we established branches in Chengdu, Shenzhen, and Guangzhou in the Mainland. Our Artificial Intelligence Customer Service System was awarded Gold and Grand Awards of Hong Kong ICT Awards 2018, and Winner of Asia Pacific ICT Alliance (APICTA) Awards 2018. Moreover, our solutions are well recognized by the market and widely adopted by clients from telecom, finance, government, and other private and public sectors.

The NLP market is said to be worth USD16.07 billion by 2021, why such explosive growth?

Smart city, smart business, and smart future have been very popular technological topics over the years. Artificial Intelligence, especially in terms of speech recognition and NLP, is playing an inevitable role. For instance, people are using personal assistant in our smartphones to send messages, play music, and even place an online shopping order. To understand the voice commands or enquiries, the software needs to transcribe the voice input into text, and then understand the text using NLP technology. The same thing happens in applications like Virtual Bank, Internet of Things, and Call Centers, which explains why the NLP market is considered very huge in the future.

Fano Labs specializes in speech recognition and NLP technologies on dialects, and focuses on enterprise call center applications. In China, there are two million agents working in call centers, with running costs over USD10 billion every year. As the largest market in the world and one of the leaders on the development of AI technologies, China will definitely be a key player in the market and will witness significant growth in the coming years.

Fano Labs specialises specifically in the Chinese language and Chinese dialects, why the focus in this specifically?

As a startup that was founded and has grown in Hong Kong, we find that many companies in Hong Kong are facing the same pain points of customer services as in the other parts of the world. They want to make their customer services smarter by employing AI assistants. They are facing the challenge of high turnover rate and high labor cost in call centers. Some of them are losing their customers because the agents do not understand their needs well enough. However, there is no suitable solution developed for Cantonese in the market. Some have tried the solutions provided by particular vendors from the Mainland or America, but it turns out that their solutions do not work well for the Hong Kong-style Cantonese. So, we were thinking: We can try solving this problem using our AI technology. From there we started the research and development of our AI Customer Service System.

Starting with the development of Cantonese ASR and NLP technologies, we also build engines to handle various languages including English, Mandarin, Sichuanese, and other minor languages. There are nearly 100 million people that speak Cantonese and even more speaking other dialects and minor languages in the world. We believe it is important and valuable to adopt AI technologies in these markets to make their business and life smarter.

What are the differences between NLP classification for Chinese (Mandarin) and other dialects?

The biggest difference comes from the data. As you may know, data is of great importance in the development of Artificial Intelligence, especially Machine Learning technology. There are enormous speech and text data generated, transferred and archived on the Internet in China, which could be used as inexhaustible fuel to train the automatic speech recognition (ASR) and NLP engines. However, when it comes to dialects, it’s a totally different story.

Since the data resources we can get for model training has been very limited, we made much effort to maximize the performance of our system by optimizing our algorithm. Moreover, to get better NLP results, we keep collecting domain-specific and local language datasets to enable our model to understand specific knowledges, which makes our solutions capable of being used for many different industries and applications.

What sets Fano Labs Speech Recognition technology apart?

Different from the ASR engine we usually use on our smartphones, which can only understand daily conversation with standard languages, our engines, in most cases, are designed for enterprises call center where many factors need to be taken into consideration, including the accent, domain knowledge, noise, equipment, and so on. Taking the ASR engine for a bank call center as an example, we may expect it to understand some domain-specific words such as “future price” or “time deposit”, which are likely to be mistaken when using a generic ASR model. Besides, the engine should be able to handle the voice signal transferred via the telephone system to avoid poor recognition accuracy, since the sampling rate of which is much lower than the smartphone.

With an in-house research team composed of professors and PhDs from prestigious universities, we are able to build customized speech recognition models for our customers which can meet their unique requirements. Moreover, unlike most of the vendors in the market who provide cloud-based solutions only, we are able to deploy on-premise speech recognition model for our customers which ensures all their data is secure and well protected.

As a company that was born in Hong Kong, how do you compete with steep competition in the Mainland?

What is unique about Fano Labs is that we have a strong R&D team and we have a great understanding of the local market. With specialties in dialects and minor languages, we are able to provide tailor-made solutions for our clients in Hong Kong and the Mainland. Most big companies focus on the big market, but usually neglect the importance of the minority. A fun fact we found out through a call center in Sichuan, China, is that more than 70% of the callers speak Sichuanese, the dialect commonly used in southwestern China and spoken by more than 300 million people.

To be honest, it is not easy for a start-up like Fano Labs to compete with the big companies in the Mainland as well as in the United States. However, we think there are still many opportunities for us in the market, because NLP is an emerging technology and the burst of the industry is yet to come. Other than competition, cooperation is more welcomed by us. With different focuses on technologies and target markets, Fano Labs can also work together with other ASR and NLP companies to provide a total solution for clients.

With new data protection regulations introduced in China, how does Fano Labs manage this?

Data security is always one of the biggest concerns of our clients and their customers, especially in finance industries. It is also Fano’s mission to ensure the security of clients’ data and the users’ information. Our systems are built under a robust and secure architecture that can prevent the threats and attacks from the outside. Having the system deployed on the private cloud or on-promise, all the data processed by our system is transferred and stored in a secure network environment. Through multiple methodologies taken in the software, hardware and management procedure, we manage to ensure that the data processing complies with the regulations of the clients and the local government.

Could you give me some use cases of how your ASR & NLP solutions are supporting your clients?

The first project we implemented in Hong Kong is the AI Customer Service System for CLP, in which we built a voice-enabled chatbot to answer the frequently asked questions raised by their customer. The chatbot is able to recognize and understand the voice inquiries in Cantonese and even mixed language of Cantonese and English, which is typically used in Hong Kong. Using NLP technology, the chatbot can understand the real meaning behind different expressions and provide appropriate responses to the users.

Our Speech Analytics system helps banks and insurance companies with customer service quality assurance and compliance check by recognizing and analyzing their call recordings and providing business insights to the managers. Fano Labs has successfully delivered AI solutions for clients from various industries, including government departments, telecom service providers, and finance companies, helping them significantly reduce their labor costs, and improve their customer service quality.


What's the most challenging moment in your entrepreneurial journey that has forced you to rethink the business?

One of the biggest challenges comes from human resources. It is not easy to hire speech and NLP specialists in Hong Kong. Although there are some PhD graduates on speech and NLP trained in some local universities, many of them prefer to work overseas or in the Mainland, maybe due to the reason that the whole job package in Hong Kong is not competitive enough. With more encouraging government policies, the situation is turning better but there is still a long way to go. We hope there will be a better research atmosphere on various AI technologies in universities and local institutions, resulting in a better research ecosystem in Hong Kong.


How do you think your technologies can be applied to and benefit the finance industries?

There are actually many applications in the finance industry where our speech and NLP technologies could help. A text-based chatbot or voicebot could understand the customers’ inquiries and communicate with them with text, voice or any other ways you want. The Speech Analytics system can accurately detect non-compliant behaviors in sales calls, optimizing the process of KYC and AML, so as to reduce fines or lawsuits caused by compliance issues and improve the quality of customer services. Our Voice Biometrics technology can shorten the lengthy authentication process through voice verification and provide better user experience.


What do you foresee happening in the future in terms of NLP, especially for Chinese NLP? What is in store for Fano Labs?

In these years, NLP technologies have been developing rapidly in China and overseas. The technologies are widely used for customer services, smart home, smartphone assistant, and so on, which will definitely become more and more popular in the future as the technologies evolve. As one of the most complicated languages in the world, Chinese NLP research is a challenging and also promising topic. As our business grows, we will encounter the use cases of different industries and different languages which we are not familiar with. However, with our research capability and increasing experience in the field, Fano Labs will be well prepared for the changes and challenges in the future.


Dr. Albert Lam is the Chief Scientist and Acting Chief Technology Officer, Fano Labs