Skip to main content

Lund University Publications

LUND UNIVERSITY LIBRARIES

Comparing LSTM and FOFE-based Architectures for Named Entity Recognition

Klang, Marcus LU orcid and Nugues, Pierre LU orcid (2018) Seventh Swedish Language Technology Conference
Abstract
LSTM architectures (Hochreiter and Schmidhuber, 1997) have become standard to recognize named entities (NER) in text (Lample et al., 2016; Chiu and Nichols, 2016). Nonetheless, Zhang et al. (2015) recently proposed an approach based on fixed-size ordinally forgetting encoding (FOFE) to translate variable-length contexts into fixed-length features. This encoding method can be used with feed-forward neural networks and, despite its simplicity, reach accuracy rates matching those of LTSMs in NER tasks (Xu et al., 2017). However, the figures reported in the NER articles are difficult to compare precisely as the experiments often use external resources such as gazetteers and corpora. In this paper, we describe an experimental setup, where we... (More)
LSTM architectures (Hochreiter and Schmidhuber, 1997) have become standard to recognize named entities (NER) in text (Lample et al., 2016; Chiu and Nichols, 2016). Nonetheless, Zhang et al. (2015) recently proposed an approach based on fixed-size ordinally forgetting encoding (FOFE) to translate variable-length contexts into fixed-length features. This encoding method can be used with feed-forward neural networks and, despite its simplicity, reach accuracy rates matching those of LTSMs in NER tasks (Xu et al., 2017). However, the figures reported in the NER articles are difficult to compare precisely as the experiments often use external resources such as gazetteers and corpora. In this paper, we describe an experimental setup, where we reimplemented the two core algorithms, to level the differences in initial conditions. This allowed us to measure more precisely the accuracy of both architectures and to report what we believe are unbiased results on English and Swedish datasets. (Less)
Please use this url to cite or link to this publication:
author
and
organization
publishing date
type
Contribution to conference
publication status
published
subject
keywords
LSTM, FOFE, Named Entity Recognition
conference name
Seventh Swedish Language Technology Conference
conference location
Stockholm, Sweden
conference dates
2018-11-07 - 2018-11-09
language
English
LU publication?
yes
id
99deb7a8-01f7-481a-9bd0-98862307c203
date added to LUP
2019-04-30 14:02:12
date last changed
2021-05-06 17:36:43
@misc{99deb7a8-01f7-481a-9bd0-98862307c203,
  abstract     = {{LSTM architectures (Hochreiter and Schmidhuber, 1997) have become standard to recognize named entities (NER) in text (Lample et al., 2016; Chiu and Nichols, 2016). Nonetheless, Zhang et al. (2015) recently proposed an approach based on fixed-size ordinally forgetting encoding (FOFE) to translate variable-length contexts into fixed-length features. This encoding method can be used with feed-forward neural networks and, despite its simplicity, reach accuracy rates matching those of LTSMs in NER tasks (Xu et al., 2017). However, the figures reported in the NER articles are difficult to compare precisely as the experiments often use external resources such as gazetteers and corpora. In this paper, we describe an experimental setup, where we reimplemented the two core algorithms, to level the differences in initial conditions. This allowed us to measure more precisely the accuracy of both architectures and to report what we believe are unbiased results on English and Swedish datasets.}},
  author       = {{Klang, Marcus and Nugues, Pierre}},
  keywords     = {{LSTM; FOFE; Named Entity Recognition}},
  language     = {{eng}},
  month        = {{11}},
  title        = {{Comparing LSTM and FOFE-based Architectures for Named Entity Recognition}},
  url          = {{https://lup.lub.lu.se/search/files/63588503/SLTC2018_final.pdf}},
  year         = {{2018}},
}