Comparing LSTM and FOFE-based Architectures for Named Entity Recognition
(2018) Seventh Swedish Language Technology Conference- Abstract
- LSTM architectures (Hochreiter and Schmidhuber, 1997) have become standard to recognize named entities (NER) in text (Lample et al., 2016; Chiu and Nichols, 2016). Nonetheless, Zhang et al. (2015) recently proposed an approach based on fixed-size ordinally forgetting encoding (FOFE) to translate variable-length contexts into fixed-length features. This encoding method can be used with feed-forward neural networks and, despite its simplicity, reach accuracy rates matching those of LTSMs in NER tasks (Xu et al., 2017). However, the figures reported in the NER articles are difficult to compare precisely as the experiments often use external resources such as gazetteers and corpora. In this paper, we describe an experimental setup, where we... (More)
- LSTM architectures (Hochreiter and Schmidhuber, 1997) have become standard to recognize named entities (NER) in text (Lample et al., 2016; Chiu and Nichols, 2016). Nonetheless, Zhang et al. (2015) recently proposed an approach based on fixed-size ordinally forgetting encoding (FOFE) to translate variable-length contexts into fixed-length features. This encoding method can be used with feed-forward neural networks and, despite its simplicity, reach accuracy rates matching those of LTSMs in NER tasks (Xu et al., 2017). However, the figures reported in the NER articles are difficult to compare precisely as the experiments often use external resources such as gazetteers and corpora. In this paper, we describe an experimental setup, where we reimplemented the two core algorithms, to level the differences in initial conditions. This allowed us to measure more precisely the accuracy of both architectures and to report what we believe are unbiased results on English and Swedish datasets. (Less)
Please use this url to cite or link to this publication:
https://lup.lub.lu.se/record/99deb7a8-01f7-481a-9bd0-98862307c203
- author
- Klang, Marcus LU and Nugues, Pierre LU
- organization
- publishing date
- 2018-11-07
- type
- Contribution to conference
- publication status
- published
- subject
- keywords
- LSTM, FOFE, Named Entity Recognition
- conference name
- Seventh Swedish Language Technology Conference
- conference location
- Stockholm, Sweden
- conference dates
- 2018-11-07 - 2018-11-09
- language
- English
- LU publication?
- yes
- id
- 99deb7a8-01f7-481a-9bd0-98862307c203
- date added to LUP
- 2019-04-30 14:02:12
- date last changed
- 2021-05-06 17:36:43
@misc{99deb7a8-01f7-481a-9bd0-98862307c203, abstract = {{LSTM architectures (Hochreiter and Schmidhuber, 1997) have become standard to recognize named entities (NER) in text (Lample et al., 2016; Chiu and Nichols, 2016). Nonetheless, Zhang et al. (2015) recently proposed an approach based on fixed-size ordinally forgetting encoding (FOFE) to translate variable-length contexts into fixed-length features. This encoding method can be used with feed-forward neural networks and, despite its simplicity, reach accuracy rates matching those of LTSMs in NER tasks (Xu et al., 2017). However, the figures reported in the NER articles are difficult to compare precisely as the experiments often use external resources such as gazetteers and corpora. In this paper, we describe an experimental setup, where we reimplemented the two core algorithms, to level the differences in initial conditions. This allowed us to measure more precisely the accuracy of both architectures and to report what we believe are unbiased results on English and Swedish datasets.}}, author = {{Klang, Marcus and Nugues, Pierre}}, keywords = {{LSTM; FOFE; Named Entity Recognition}}, language = {{eng}}, month = {{11}}, title = {{Comparing LSTM and FOFE-based Architectures for Named Entity Recognition}}, url = {{https://lup.lub.lu.se/search/files/63588503/SLTC2018_final.pdf}}, year = {{2018}}, }