diff --git a/lessons/5-NLP/19-NER/README.md b/lessons/5-NLP/19-NER/README.md index 8bc38a54a2153befdff17a047855e23c3e3b5061..294af8c12d463676b16dc0cb7f9d06a206272692 100644 --- a/lessons/5-NLP/19-NER/README.md +++ b/lessons/5-NLP/19-NER/README.md @@ -1,20 +1,20 @@ # Named Entity Recognition -Up to now, we have mostly been concentrating on one NLP task - classification. However, there are also other NLP tasks that can be accomplished with neural networks. One of those tasks is **[Named Entity Recognition](https://en.wikipedia.org/wiki/Named-entity_recognition)** (NER), which deals with recognizing specific entities within text, such as places, person names, date-time intervals, chemical formulae and so on. +Up to now, we have mostly been concentrating on one NLP task - classification. However, there are also other NLP tasks that can be accomplished with neural networks. One of those tasks is **[Named Entity Recognition](https://wikipedia.org/wiki/Named-entity_recognition)** (NER), which deals with recognizing specific entities within text, such as places, person names, date-time intervals, chemical formulae and so on. ## [Pre-lecture quiz](https://black-ground-0cc93280f.1.azurestaticapps.net/quiz/119) ## Example of Using NER -Suppose you want to develop a natural language chat bot, similar to Amazon Alexa or Google Assistant. The way intelligent chat bots work is to *understand* what the user want by doing text classification on the input sentence. The result of this classification is so-called **intent**, which determines what a chat bot should do. +Suppose you want to develop a natural language chat bot, similar to Amazon Alexa or Google Assistant. The way intelligent chat bots work is to *understand* what the user wants by doing text classification on the input sentence. The result of this classification is so-called **intent**, which determines what a chat bot should do. <img alt="Bot NER" src="images/bot-ner.png" width="50%"/> -> *Image by author* +> Image by the author -However, a user may provide some parameters as part of the phrase. For example, when asking for a weather, she may specify a location or date. A bot should be able to understand those entities, and fill in the parameter slots accordingly before performing the action. This is exactly when NER comes in. +However, a user may provide some parameters as part of the phrase. For example, when asking for the weather, she may specify a location or date. A bot should be able to understand those entities, and fill in the parameter slots accordingly before performing the action. This is exactly where NER comes in. -Another example would be [analyzing scientific medical papers](https://soshnikov.com/science/analyzing-medical-papers-with-azure-and-text-analytics-for-health/), one of the main things we need to look for are specific medical terms, such as diseases and medical substances. While a small number of diseases can probably be extracted using substring search, more complex entities, such as chemical compounds and medication names, need more complex approach. +> ✅ Another example would be [analyzing scientific medical papers](https://soshnikov.com/science/analyzing-medical-papers-with-azure-and-text-analytics-for-health/). One of the main things we need to look for are specific medical terms, such as diseases and medical substances. While a small number of diseases can probably be extracted using substring search, more complex entities, such as chemical compounds and medication names, need a more complex approach. ## NER as Token Classification @@ -29,7 +29,7 @@ Entities here are: * Lithium carbonate is a chemical substance (`CHEM`) * Toxicity is also a disease (`DIS`) -Notice that one entity can span several tokens. And, as in this case, we need to distinguish between two consecutive entities. Thus it is common to use two classes for each entity - one specifying the first token of the entity (often `B-` prefix is used, for beginning), and another - the continuation of an entity (`I-`, for inner token). We also use `O` as a class to represent all other tokens. Such token tagging is called [BIO tagging](https://en.wikipedia.org/wiki/Inside%E2%80%93outside%E2%80%93beginning_(tagging)) (or IOB). When tagged, our title will look like this: +Notice that one entity can span several tokens. And, as in this case, we need to distinguish between two consecutive entities. Thus it is common to use two classes for each entity - one specifying the first token of the entity (often the `B-` prefix is used, for **b**eginning), and another - the continuation of an entity (`I-`, for **i**nner token). We also use `O` as a class to represent all **o**ther tokens. Such token tagging is called [BIO tagging](https://en.wikipedia.org/wiki/Inside%E2%80%93outside%E2%80%93beginning_(tagging)) (or IOB). When tagged, our title will look like this: Token | Tag ------|----- @@ -46,23 +46,37 @@ newborn | O infant | O . | O -Since we need to build one-to-one correspondence between tokens and this classes, we can train rightmost **many-to-many** neural network model from this picture: +Since we need to build a one-to-one correspondence between tokens and classes, we can train a rightmost **many-to-many** neural network model from this picture:  -> *Image from blog post [Unreasonable Effectiveness of Recurrent Neural Networks](http://karpathy.github.io/2015/05/21/rnn-effectiveness/) by [Andrej Karpaty](http://karpathy.github.io/). NER token classification models correspond to right-most network architecture on this picture.* +> *Image from [this blog post](http://karpathy.github.io/2015/05/21/rnn-effectiveness/) by [Andrej Karpathy](http://karpathy.github.io/). NER token classification models correspond to the right-most network architecture on this picture.* ## Training NER models -Since NER model is essentially a token classification model, we can use RNNs that we are already familiar with for this task. In this case, each block of recurrent network will return the token ID. Example notebook shows how to train LSTM for token classification. +Since a NER model is essentially a token classification model, we can use RNNs that we are already familiar with for this task. In this case, each block of recurrent network will return the token ID. The following example notebook shows how to train LSTM for token classification. ## ✍️ Example Notebooks: NER -Continue your learning in the following notebooks: +Continue your learning in the following notebook: * [NER with TensorFlow](NER-TF.ipynb) +## Conclusion + +A NER model is a **token classification model**, which means that it can be used to perform token classification. This is a very common task in NLP, helping to recognize specific entities within text including places, names, dates, and more. + +## 🚀 Challenge + +Complete the assignment linked below to train a named entity recognition model for medical terms, then try it on a different dataset. + +## [Post-lecture quiz](https://black-ground-0cc93280f.1.azurestaticapps.net/quiz/219) + +## Review & Self Study + +Read through the blog [The Unreasonable Effectiveness of Recurrent Neural Networks](http://karpathy.github.io/2015/05/21/rnn-effectiveness/) and follow along with the Further Reading section in that article to deepen your knowledge. + ## [Assignment](lab/README.md) -In the assignment for this lesson, you will have to train medical entity recognition model. You can start with training LSTM model as described in this lesson, and proceed with using BERT transformer model. Read [the instructions](lab/README.md) to get all the details. +In the assignment for this lesson, you will have to train a medical entity recognition model. You can start with training an LSTM model as described in this lesson, and proceed with using the BERT transformer model. Read [the instructions](lab/README.md) to get all the details.