diff --git a/README.md b/README.md index e123bbce08a304674414c1d1acd398b19c570eb5..9166eb9542f199bc40b04f5192089ceebd1295ea 100644 --- a/README.md +++ b/README.md @@ -78,7 +78,7 @@ For a gentle introduction to *AI in the Cloud* topics you may consider taking th <tr><td>14</td><td>Semantic word embeddings. Word2Vec and GloVe</td><td><a href="lessons/5-NLP/14-Embeddings/README.md">Text</td><td><a href="lessons/5-NLP/14-Embeddings/EmbeddingsPyTorch.ipynb">PyTorch</a></td><td><a href="lessons/5-NLP/14-Embeddings/EmbeddingsTF.ipynb">TensorFlow</a></td><td></td></tr> <tr><td>15</td><td>Language Modeling. Training your own embeddings</td><td><a href="lessons/5-NLP/15-LanguageModeling">Text</a></td><td>PyTorch</td><td>TensorFlow</td><td></td></tr> <tr><td>16</td><td>Recurrent Neural Networks</td><td><a href="lessons/5-NLP/16-RNN/README.md">Text</a></td><td><a href="lessons/5-NLP/16-RNN/RNNPyTorch.ipynb">PyTorch</a></td><td><a href="lessons/5-NLP/16-RNN/RNNTF.ipynb">TensorFlow</a></td><td></td></tr> -<tr><td>17</td><td>Generative Recurrent Networks</td><td><a href="lessons/5-NLP/17-GenerativeNetworks/README.md">Text</a></td><td><a href="lessons/5-NLP/17-GenerativeNetworks/GenerativePyTorch.md">PyTorch</a></td><td><a href="lessons/5-NLP/17-GenerativeNetworks/GenerativeTF.md">TensorFlow</a></td><td></td></tr> +<tr><td>17</td><td>Generative Recurrent Networks</td><td><a href="lessons/5-NLP/17-GenerativeNetworks/README.md">Text</a></td><td><a href="lessons/5-NLP/17-GenerativeNetworks/GenerativePyTorch.md">PyTorch</a></td><td><a href="lessons/5-NLP/17-GenerativeNetworks/GenerativeTF.md">TensorFlow</a></td><td><a href="lessons/5-NLP/17-GenerativeNetworks/lab/README.md">Lab</a></td></tr> <tr><td>18</td><td>Transformers. BERT.</td><td><a href="lessons/5-NLP/18-Transformers/README.md">Text</a></td><td><a href="lessons/5-NLP/18-Transformers/TransformersPyTorch.md">PyTorch</a></td><td><a href="lessons/5-NLP/18-Transformers/TransformersTF.md">TensorFlow</a></td><td></td></tr> <tr><td>19</td><td>Named Entity Recognition</td><td>Text</td><td>PyTorch</td><td>TensorFlow</td><td></td></tr> <tr><td>20</td><td>Large Language Models, Prompt Programming and Few-Shot Tasks</td><td>Text</td><td>PyTorch</td><td>TensorFlow</td><td></td></tr> diff --git a/lessons/5-NLP/17-GenerativeNetworks/README.md b/lessons/5-NLP/17-GenerativeNetworks/README.md index 5b3002f50ddd09cdca6c80185211a98612fa2336..8a91433235fe3f376916290e0861629bde4c8641 100644 --- a/lessons/5-NLP/17-GenerativeNetworks/README.md +++ b/lessons/5-NLP/17-GenerativeNetworks/README.md @@ -50,4 +50,16 @@ Have a look at how this soft text generation is implemented in the notebooks. ## [Post-lecture quiz](https://black-ground-0cc93280f.1.azurestaticapps.net/quiz/217) -> 鉁� Todo: conclusion, Assignment, challenge, reference. +## [Assignment](lab/README.md) + +We have seen how to generate text character-by-character. In the lab, you will explore word-level text generation. + +## References + +* Different approaches to text generation with Markov Chain, LSTM and GPT-2: [blog post](https://towardsdatascience.com/text-generation-gpt-2-lstm-markov-chain-9ea371820e1e) +* Learn Unit on Text Generation with [PyTorch](https://docs.microsoft.com/learn/modules/intro-natural-language-processing-pytorch/6-generative-networks/?WT.mc_id=academic-15963-dmitryso)/[TensorFlow](https://docs.microsoft.com/learn/modules/intro-natural-language-processing-tensorflow/5-generative-networks/?WT.mc_id=academic-15963-dmitryso) +* Text generation sample in [Keras documentation](https://keras.io/examples/generative/lstm_character_level_text_generation/) + +## Takeaway + +While text generation may be useful in its own right, the major benefits come from the ability to generate text using RNNs from some initial feature vector. For example, text generation is used as part of machine translation (sequence-to-sequence, in this case state vector from *encoder* is used to generate or *decode* translated message), generating textual description of an image (in which case feature vector would come from CNN extractor), etc. diff --git a/lessons/5-NLP/17-GenerativeNetworks/lab/README.md b/lessons/5-NLP/17-GenerativeNetworks/lab/README.md new file mode 100644 index 0000000000000000000000000000000000000000..6db7214a96dce71db5766f4d5b9e19ca6464dfeb --- /dev/null +++ b/lessons/5-NLP/17-GenerativeNetworks/lab/README.md @@ -0,0 +1,12 @@ +# Word-level Text Generation using RNNs + +Lab Assignment from [AI for Beginners Curriculum](https://github.com/microsoft/ai-for-beginners). + +## Task + +In this lab, you need to take any book, and use it as a dataset to train word-level text generator. + +## The Dataset + +You are welcome to use any book. You can find a lot of free texts at [Project Gutenberg](https://www.gutenberg.org/), for example, here is a direct link to [Alice's Adventures in Wonderland](https://www.gutenberg.org/files/11/11-0.txt)) by Lewis Carroll. +