Innovation & Technology
• 6 minute read
How Emotions Affects Our Investing Behaviours
With the use of AI technology, CUHK research reveals how Hong Kong stock market is greatly vulnerable to market sentiment
By Jaymee Ng, China Business Knowledge @ CUHK
Human beings are emotional creatures. As much as we try not to let emotions cloud our judgements, we are still led by them from time to time. The power of sentiment affects not only our decisions in daily routines but also our financial investments. A research study by The Chinese University of Hong Kong (CUHK) Business School has looked into the power of sentiment and how it influences the Hong Kong stock markets.
In the study entitled “Sentiment Analysis in financial texts”, Prof. Samuel Chan Wai-kwong, Associate Professor in the Department of Decision Sciences and Managerial Economics at The Chinese University of Hong Kong (CUHK) Business School introduces a sentiment analysis engine (SAE) that conducts linguistic analyses based on grammars. The study utilizes a total of 46,842 financial news articles and commentaries of 133 trading days during the period from 9 September 2014 to 24 March 2015.
According to Prof. Chan, their SAE has significant implications in its applications in market prediction.
“Our sentiment analysis of finance data takes advantage of linguistic analysis based on grammar, which extends the assessment process not only at the token level, but also at the phrase level within each sentence,” says Prof. Chan.
He further explains that the inclusiveness is possible because such linguistic analysis identifies various phrases in relation to the recursive structure of parse trees.
“Along with demonstrating the functionality of our mathematical economic models, the patterns of sentiment uncovered by our engine are able to reveal the conceptual drifts in financial markets,” he says.
Sentiment Analysis and AI
Sentiment analysis, also known as opinion mining, is a type of data mining that analyses the general public’s sentiments or opinions through natural language processing (NLP), a sub-field of Artificial Intelligence (AI) that enables computers to acquire a human-level understanding of language.
Prof. Chan’s sentiment analysis engine is based on the same principle, and further divided into five major components: feature extraction, sentence chunker, phrase recognizer, parse tree resolution and sentiment assessment module.
For example, every sentence fed into the system is extracted into words of the sentence, part-of-speech sequences, etc. The sentence chunker then locates the boundaries and the spans of the phrases in the sentence. The phrase recognizer predicts the phrasal structure before the parsing (also known as syntax analysis) and sentiment assessment processes. Apart from the usual “thumb up” and “thumb down” sentiments, the system is able to identify eight additional emotions including joy, trust, fear, surprise, sadness, disgust, anger and anticipation.
“Sentences and their syntactic structures are not important in and of themselves. They are important because they are shaped by an author’s experiences and point to his or her expectations or predictions,” says Prof. Chan.
“It would be highly misleading to determine the polarity of a sentence by simply counting the number of positive or negative words. The polarity of a sentence resides not only in the words and taxonomy used, but also in how the words relate to others,” he adds.
“AI is certainly promising. The trend is so overwhelming that there is no way we can stop or revert it. What really matters is who is using this technology.” – Prof. Samuel Chan
Role of Sentiment in Stock Market Investments
Sentiment plays a huge role in stock market investments. Any investors would be familiar with the term “market sentiment”, which is the overall feeling or tone of a specific stock or financial market. When the market sentiment is bearish, people usually expect stock prices to fall. On the contrary, the stock prices are expected to rise when the market sentiment is bullish.
There are countless financial reports and commentaries on stocks or the market being made on a daily basis. It is simply impossible for human beings to analyse every piece of information in order to derive the most accurate market sentiment. And that is where AI comes into play – With the advancement in AI technology and big data, researchers are now able to turn the impossible into possible.
“Financial text streams are more than just collections of isolated documents, in that they exhibit the traits of collective intentions and actions. These text streams contain prominent hints for investors, as they convey many expressions of sentiment drawn from the public. They can predict the correct sentiment in financial texts and also hint on the changes in the stock market index, at least in the sense of Granger-cause,” Prof. Chan explains.
According to Prof. Chan, the accuracy in predicting market movement by analysing technical indicators only resulted to around 55 per cent.
“However, after adding the analysis of sentiment in financial news and commentaries from eight sources including AA Stocks, we are able to achieve up to 60 per cent accuracy in predicting the market movement in day trade.”
Prof. Chan believes one possible reason is that the Hong Kong stock market is particularly vulnerable to market sentiment.
“The Hong Kong stock market is very emotional, unlike the financial markets in the West. There are more individual investors in Hong Kong whereas the western markets are dominated by institutional investors,” says Prof. Chan, adding that the engine needs to be fed with big data so that it can better identify and predict market trends and investor behaviours.
Making AI Work for Humans
AI adoption in the business world is certainly expected to keep thriving in the near future. Deloitte’s State of AI in the Enterprise 2018 report revealed that 63 per cent of businesses in the US adopted machine learning as a key technology in 2018 and 82 per cent of early adopters of AI technology reported positive ROI.
Coming from a computer science background, Prof. Chan sees the limitless potential of the sentiment analysis engine beyond the financial markets, and for good causes that benefit our community.
A few years ago, he started building a text analytics open platform known as HanMosaic to analyse Chinese texts in real time. Based on this platform, he further developed a language learning site called HM-Learn for ethnic minorities to learn Chinese in an interactive way.
Similar to the sentiment analysis engine, HM-Learn splits a Chinese sentence into different words and phrases and provides explanations to each phrase in English. The site also supports image input for non-Chinese users who cannot type or copy the correct Chinese text into the system. With images, users can simply take a picture of any object for the system to identify and provide the Chinese text for it.
Since AI is an inevitable development in human life, the key questions then are who has control of this powerful technology and how it should be applied.
“AI is certainly promising. The trend is so overwhelming that there is no way we can stop or revert it. What really matters is who is using this technology,” he says.
“If the person who has control of the technology cares about privacy, then there shouldn’t be a huge problem. If not, AI will become a dangerous tool of mass surveillance. There are always two sides of a coin.”