Natural Language Processing in Artificial Neural Networks: Sentence analysis in medical papers

Abstract

Convolutional Neural Networks (CNNs) and pre-trained word embeddings have
revolutionized the field of Natural Language Processing (NLP) during the last years.
In this project, CNNs are used on top of the Word2Vec word representation for a
sentence classification task on medical research articles. Both individual networks for
each category as well as a combined classification network are optimized and achieve
average AUC scores of 0.89 and 0.82 respectively. A comparison with the results
of a collaborating group using Support Vector Machines (SVMs) shows that simple
CNNs can now compete with SVMs in this formerly SVM dominated area. In an
extension, Recurrent Neural Networks (RNNs) are also trained on the same task and
shown to be unfavorable compared to CNNs because of their lack of stability.

Popular Abstract

When Alan Turing stated his world famous ”Turing Test” in 1950 he made clear
that for a machine to be considered intelligent it must, among other skills, have con-
versational capacities comparable to a human being. Since then, scientists in the
field of Natural Language Processing (NLP) have undertaken uncountable attempts
to come closer to this objective. While we are still far from reaching human-like
language skills of machines, the field of Artificial Neural Networks (ANNs) has con-
tributed very promising improvements in the past years.
ANNs mimic the functioning of the human brain by passing signals from node to
node (the artificial neurons) via weighted connections (the artificial synapses). The
connection weights are flexible and can be tweaked and twisted just like synapses in
the brain can grow stronger or weaker. The effect is that weights and synapses alike
can adapt to a problem and finally solve it, a process which is known to everyone
as ”learning”. Apart from a strong increase of computational power, the substantial
progress of the field in the last years is mainly related to new architectures of how
the ANNs are built and connected. For instance, the so called Convolutional Neural
Network (CNN) was inspired by the human visual cortex and is set up in a way that
it can detect shapes and objects in a picture independently of the objects location
and scale. Even though this may sound rather trivial to a human observer it is a
great challenge for a computer. While it was initially used for image recognition
tasks, scientists have now come to appreciate its value in NLP as well. Just like
objects in pictures, words and phrases can also appear in different locations in a
sentence: ”I will eat today” and ”Today I will eat” have the same meaning (except
for the emphasis) but look very different to a computer. The ability to recognize
locations within the sentence also creates a concept of context and thus memory,
essential parts of understanding language.
Before language can be handled by machines, its most basic building blocks, the
words, need to be transformed into something the machine can process, namely num-
bers. In 2013 researchers made a groundbreaking discovery when they fed the entire
content of Wikipedia into a powerful ANN and trained it to create its own language
which they called Word2Vec. In this language, every human word is associated with
a vector, an array of numbers, in a 300 dimensional vector space. While the 300
dimensions of these word vectors do not correspond to any concepts that are known
to humans, the computer seemed to have encoded both the semantic and syntactic
meaning of the words. For example, subtracting vector(”Man”) from vector(”King”)
and adding vector(”Woman”) to it did indeed result in vector(”Queen”).
In this project, CNNs are used together with the machine language Word2Vec in
order to solve a problem in medical science. Due to the high publication rate of
medical papers, the findings of these papers do not reach their application with the
common doctors and hospitals any more. An Artificial Intelligence is thus needed
to extract and process the contents of the articles and present them to the doctors
when consulted about a specific topic. The first step for this purpose is done in this
project, the training of an ANN which can classify sentences according to their type
of content such as ”Methods”, ”Results” or ”Aim”. The future step will then be to extract
the content which had been located previously by our network.

• author: Lurz, Konstantin-Klemens
• supervisor: Mattias Ohlsson
• organization: Department of Astronomy and Theoretical Physics - Undergoing reorganization; Computational Biology and Biological Physics - Undergoing reorganization
• course: FYTM04 20181
• year: 2018
• type: H2 - Master's Degree (Two Years)
• subject: Physics and Astronomy
• keywords: Machine Learning, Natural Language Processing, Medical Papers, Convolutional Neural Networks, Word2Vec
• language: English
• id: 8948103
• date added to LUP: 2018-06-14 11:02:35
• date last changed: 2018-06-14 11:02:35