Cause-Effect Relationship Extraction from Text of Financial Events
Research Assistant, Advisor: Prof. Weirong Liu (Central South University)
June 2021-January 2022
In this research, I Designed an NER model based on BiGRU and self-attention mechanism to extract cause-effect relationships from the text of financial events, achieving 78% accuracy.
The model employs two inputs. Starting with the left input, I utilize the Bert-base-Chinese model from Huggingface to encode the sentence into a vector, with each word represented by a number. This vector, denoted as input1, undergoes processing through Bert, resulting in 768 features per word. Subsequently, a Bidirectional GRU is applied, reducing each word to 128 features and enhancing word-to-word relationships. A linear layer is then employed to transform the feature count to 14.
Shifting to the right input, input2, I utilize jieba to extract Part-of-Speech information, converting it into 29-feature vectors. The Attention mechanism is applied to capture Part-of-Speech relationships.
Next, the fusion layer combines both inputs via its concat layer, resulting in 43 features (29 from input2 and 14 from input1). Subsequently, a linear layer reduces these to 14 features, representing the potential results. For loss calculation, I employ the torchcrf API, which utilizes log likelihood. The final result is obtained through the API’s decode function, based on the Viterbi algorithm.
Materials