{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Multi-Layered Perceptrons\n",
"## Building our own Neural Framework\n",
"\n",
"> This notebook is a part of [AI for Beginners Curricula](http://github.com/microsoft/ai-for-beginners). Visit the repository for complete set of learning materials.\n",
"\n",
"In this notebook, we will gradually build our own neural framework capable of solving multi-class classification tasks as well as regression with multi-layered preceptrons.\n",
"\n",
"First, let's import some required libraries."
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"outputs": [],
"source": [
"%matplotlib nbagg\n",
"import matplotlib.pyplot as plt \n",
"from matplotlib import gridspec\n",
"from sklearn.datasets import make_classification\n",
"import numpy as np\n",
"# pick the seed for reproducibility - change it to explore the effects of random variations\n",
"np.random.seed(0)\n",
"import random"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Sample Dataset\n",
"\n",
"As before, we will start with a simple sample dataset with two parameters.\n"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {
"scrolled": false,
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [],
"source": [
"n = 100\n",
"X, Y = make_classification(n_samples = n, n_features=2,\n",
" n_redundant=0, n_informative=2, flip_y=0.2)\n",
"X = X.astype(np.float32)\n",
"Y = Y.astype(np.int32)\n",
"\n",
"# Split into train and test dataset\n",
"train_x, test_x = np.split(X, [n*8//10])\n",
"train_labels, test_labels = np.split(Y, [n*8//10])"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {
"scrolled": false,
"slideshow": {
"slide_type": "skip"
}
},
"outputs": [],
"source": [
"def plot_dataset(suptitle, features, labels):\n",
" # prepare the plot\n",
" fig, ax = plt.subplots(1, 1)\n",
" #pylab.subplots_adjust(bottom=0.2, wspace=0.4)\n",
" fig.suptitle(suptitle, fontsize = 16)\n",
" ax.set_xlabel('$x_i[0]$ -- (feature 1)')\n",
" ax.set_ylabel('$x_i[1]$ -- (feature 2)')\n",
"\n",
" colors = ['r' if l else 'b' for l in labels]\n",
" ax.scatter(features[:, 0], features[:, 1], marker='o', c=colors, s=100, alpha = 0.5)\n",
" fig.show()"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {
"scrolled": false,
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [
{
"data": {
"application/javascript": "/* Put everything inside the global mpl namespace */\n/* global mpl */\nwindow.mpl = {};\n\nmpl.get_websocket_type = function () {\n if (typeof WebSocket !== 'undefined') {\n return WebSocket;\n } else if (typeof MozWebSocket !== 'undefined') {\n return MozWebSocket;\n } else {\n alert(\n 'Your browser does not have WebSocket support. ' +\n 'Please try Chrome, Safari or Firefox ≥ 6. ' +\n 'Firefox 4 and 5 are also supported but you ' +\n 'have to enable WebSockets in about:config.'\n );\n }\n};\n\nmpl.figure = function (figure_id, websocket, ondownload, parent_element) {\n this.id = figure_id;\n\n this.ws = websocket;\n\n this.supports_binary = this.ws.binaryType !== undefined;\n\n if (!this.supports_binary) {\n var warnings = document.getElementById('mpl-warnings');\n if (warnings) {\n warnings.style.display = 'block';\n warnings.textContent =\n 'This browser does not support binary websocket messages. ' +\n 'Performance may be slow.';\n }\n }\n\n this.imageObj = new Image();\n\n this.context = undefined;\n this.message = undefined;\n this.canvas = undefined;\n this.rubberband_canvas = undefined;\n this.rubberband_context = undefined;\n this.format_dropdown = undefined;\n\n this.image_mode = 'full';\n\n this.root = document.createElement('div');\n this.root.setAttribute('style', 'display: inline-block');\n this._root_extra_style(this.root);\n\n parent_element.appendChild(this.root);\n\n this._init_header(this);\n this._init_canvas(this);\n this._init_toolbar(this);\n\n var fig = this;\n\n this.waiting = false;\n\n this.ws.onopen = function () {\n fig.send_message('supports_binary', { value: fig.supports_binary });\n fig.send_message('send_image_mode', {});\n if (fig.ratio !== 1) {\n fig.send_message('set_dpi_ratio', { dpi_ratio: fig.ratio });\n }\n fig.send_message('refresh', {});\n };\n\n this.imageObj.onload = function () {\n if (fig.image_mode === 'full') {\n // Full images could contain transparency (where diff images\n // almost always do), so we need to clear the canvas so that\n // there is no ghosting.\n fig.context.clearRect(0, 0, fig.canvas.width, fig.canvas.height);\n }\n fig.context.drawImage(fig.imageObj, 0, 0);\n };\n\n this.imageObj.onunload = function () {\n fig.ws.close();\n };\n\n this.ws.onmessage = this._make_on_message_function(this);\n\n this.ondownload = ondownload;\n};\n\nmpl.figure.prototype._init_header = function () {\n var titlebar = document.createElement('div');\n titlebar.classList =\n 'ui-dialog-titlebar ui-widget-header ui-corner-all ui-helper-clearfix';\n var titletext = document.createElement('div');\n titletext.classList = 'ui-dialog-title';\n titletext.setAttribute(\n 'style',\n 'width: 100%; text-align: center; padding: 3px;'\n );\n titlebar.appendChild(titletext);\n this.root.appendChild(titlebar);\n this.header = titletext;\n};\n\nmpl.figure.prototype._canvas_extra_style = function (_canvas_div) {};\n\nmpl.figure.prototype._root_extra_style = function (_canvas_div) {};\n\nmpl.figure.prototype._init_canvas = function () {\n var fig = this;\n\n var canvas_div = (this.canvas_div = document.createElement('div'));\n canvas_div.setAttribute(\n 'style',\n 'border: 1px solid #ddd;' +\n 'box-sizing: content-box;' +\n 'clear: both;' +\n 'min-height: 1px;' +\n 'min-width: 1px;' +\n 'outline: 0;' +\n 'overflow: hidden;' +\n 'position: relative;' +\n 'resize: both;'\n );\n\n function on_keyboard_event_closure(name) {\n return function (event) {\n return fig.key_event(event, name);\n };\n }\n\n canvas_div.addEventListener(\n 'keydown',\n on_keyboard_event_closure('key_press')\n );\n canvas_div.addEventListener(\n 'keyup',\n on_keyboard_event_closure('key_release')\n );\n\n this._canvas_extra_style(canvas_div);\n this.root.appendChild(canvas_div);\n\n var canvas = (this.canvas = document.createElement('canvas'));\n canvas.classList.add('mpl-canvas');\n canvas.setAttribute('style', 'box-sizing: content-box;');\n\n this.context = canvas.getContext('2d');\n\n var backingStore =\n this.context.backingStorePixelRatio ||\n this.context.webkitBackingStorePixelRatio ||\n this.context.mozBackingStorePixelRatio ||\n this.context.msBackingStorePixelRatio ||\n this.context.oBackingStorePixelRatio ||\n this.context.backingStorePixelRatio ||\n 1;\n\n this.ratio = (window.devicePixelRatio || 1) / backingStore;\n\n var rubberband_canvas = (this.rubberband_canvas = document.createElement(\n 'canvas'\n ));\n rubberband_canvas.setAttribute(\n 'style',\n 'box-sizing: content-box; position: absolute; left: 0; top: 0; z-index: 1;'\n );\n\n // Apply a ponyfill if ResizeObserver is not implemented by browser.\n if (this.ResizeObserver === undefined) {\n if (window.ResizeObserver !== undefined) {\n this.ResizeObserver = window.ResizeObserver;\n } else {\n var obs = _JSXTOOLS_RESIZE_OBSERVER({});\n this.ResizeObserver = obs.ResizeObserver;\n }\n }\n\n this.resizeObserverInstance = new this.ResizeObserver(function (entries) {\n var nentries = entries.length;\n for (var i = 0; i < nentries; i++) {\n var entry = entries[i];\n var width, height;\n if (entry.contentBoxSize) {\n if (entry.contentBoxSize instanceof Array) {\n // Chrome 84 implements new version of spec.\n width = entry.contentBoxSize[0].inlineSize;\n height = entry.contentBoxSize[0].blockSize;\n } else {\n // Firefox implements old version of spec.\n width = entry.contentBoxSize.inlineSize;\n height = entry.contentBoxSize.blockSize;\n }\n } else {\n // Chrome <84 implements even older version of spec.\n width = entry.contentRect.width;\n height = entry.contentRect.height;\n }\n\n // Keep the size of the canvas and rubber band canvas in sync with\n // the canvas container.\n if (entry.devicePixelContentBoxSize) {\n // Chrome 84 implements new version of spec.\n canvas.setAttribute(\n 'width',\n entry.devicePixelContentBoxSize[0].inlineSize\n );\n canvas.setAttribute(\n 'height',\n entry.devicePixelContentBoxSize[0].blockSize\n );\n } else {\n canvas.setAttribute('width', width * fig.ratio);\n canvas.setAttribute('height', height * fig.ratio);\n }\n canvas.setAttribute(\n 'style',\n 'width: ' + width + 'px; height: ' + height + 'px;'\n );\n\n rubberband_canvas.setAttribute('width', width);\n rubberband_canvas.setAttribute('height', height);\n\n // And update the size in Python. We ignore the initial 0/0 size\n // that occurs as the element is placed into the DOM, which should\n // otherwise not happen due to the minimum size styling.\n if (fig.ws.readyState == 1 && width != 0 && height != 0) {\n fig.request_resize(width, height);\n }\n }\n });\n this.resizeObserverInstance.observe(canvas_div);\n\n function on_mouse_event_closure(name) {\n return function (event) {\n return fig.mouse_event(event, name);\n };\n }\n\n rubberband_canvas.addEventListener(\n 'mousedown',\n on_mouse_event_closure('button_press')\n );\n rubberband_canvas.addEventListener(\n 'mouseup',\n on_mouse_event_closure('button_release')\n );\n rubberband_canvas.addEventListener(\n 'dblclick',\n on_mouse_event_closure('dblclick')\n );\n // Throttle sequential mouse events to 1 every 20ms.\n rubberband_canvas.addEventListener(\n 'mousemove',\n on_mouse_event_closure('motion_notify')\n );\n\n rubberband_canvas.addEventListener(\n 'mouseenter',\n on_mouse_event_closure('figure_enter')\n );\n rubberband_canvas.addEventListener(\n 'mouseleave',\n on_mouse_event_closure('figure_leave')\n );\n\n canvas_div.addEventListener('wheel', function (event) {\n if (event.deltaY < 0) {\n event.step = 1;\n } else {\n event.step = -1;\n }\n on_mouse_event_closure('scroll')(event);\n });\n\n canvas_div.appendChild(canvas);\n canvas_div.appendChild(rubberband_canvas);\n\n this.rubberband_context = rubberband_canvas.getContext('2d');\n this.rubberband_context.strokeStyle = '#000000';\n\n this._resize_canvas = function (width, height, forward) {\n if (forward) {\n canvas_div.style.width = width + 'px';\n canvas_div.style.height = height + 'px';\n }\n };\n\n // Disable right mouse context menu.\n this.rubberband_canvas.addEventListener('contextmenu', function (_e) {\n event.preventDefault();\n return false;\n });\n\n function set_focus() {\n canvas.focus();\n canvas_div.focus();\n }\n\n window.setTimeout(set_focus, 100);\n};\n\nmpl.figure.prototype._init_toolbar = function () {\n var fig = this;\n\n var toolbar = document.createElement('div');\n toolbar.classList = 'mpl-toolbar';\n this.root.appendChild(toolbar);\n\n function on_click_closure(name) {\n return function (_event) {\n return fig.toolbar_button_onclick(name);\n };\n }\n\n function on_mouseover_closure(tooltip) {\n return function (event) {\n if (!event.currentTarget.disabled) {\n return fig.toolbar_button_onmouseover(tooltip);\n }\n };\n }\n\n fig.buttons = {};\n var buttonGroup = document.createElement('div');\n buttonGroup.classList = 'mpl-button-group';\n for (var toolbar_ind in mpl.toolbar_items) {\n var name = mpl.toolbar_items[toolbar_ind][0];\n var tooltip = mpl.toolbar_items[toolbar_ind][1];\n var image = mpl.toolbar_items[toolbar_ind][2];\n var method_name = mpl.toolbar_items[toolbar_ind][3];\n\n if (!name) {\n /* Instead of a spacer, we start a new button group. */\n if (buttonGroup.hasChildNodes()) {\n toolbar.appendChild(buttonGroup);\n }\n buttonGroup = document.createElement('div');\n buttonGroup.classList = 'mpl-button-group';\n continue;\n }\n\n var button = (fig.buttons[name] = document.createElement('button'));\n button.classList = 'mpl-widget';\n button.setAttribute('role', 'button');\n button.setAttribute('aria-disabled', 'false');\n button.addEventListener('click', on_click_closure(method_name));\n button.addEventListener('mouseover', on_mouseover_closure(tooltip));\n\n var icon_img = document.createElement('img');\n icon_img.src = '_images/' + image + '.png';\n icon_img.srcset = '_images/' + image + '_large.png 2x';\n icon_img.alt = tooltip;\n button.appendChild(icon_img);\n\n buttonGroup.appendChild(button);\n }\n\n if (buttonGroup.hasChildNodes()) {\n toolbar.appendChild(buttonGroup);\n }\n\n var fmt_picker = document.createElement('select');\n fmt_picker.classList = 'mpl-widget';\n toolbar.appendChild(fmt_picker);\n this.format_dropdown = fmt_picker;\n\n for (var ind in mpl.extensions) {\n var fmt = mpl.extensions[ind];\n var option = document.createElement('option');\n option.selected = fmt === mpl.default_extension;\n option.innerHTML = fmt;\n fmt_picker.appendChild(option);\n }\n\n var status_bar = document.createElement('span');\n status_bar.classList = 'mpl-message';\n toolbar.appendChild(status_bar);\n this.message = status_bar;\n};\n\nmpl.figure.prototype.request_resize = function (x_pixels, y_pixels) {\n // Request matplotlib to resize the figure. Matplotlib will then trigger a resize in the client,\n // which will in turn request a refresh of the image.\n this.send_message('resize', { width: x_pixels, height: y_pixels });\n};\n\nmpl.figure.prototype.send_message = function (type, properties) {\n properties['type'] = type;\n properties['figure_id'] = this.id;\n this.ws.send(JSON.stringify(properties));\n};\n\nmpl.figure.prototype.send_draw_message = function () {\n if (!this.waiting) {\n this.waiting = true;\n this.ws.send(JSON.stringify({ type: 'draw', figure_id: this.id }));\n }\n};\n\nmpl.figure.prototype.handle_save = function (fig, _msg) {\n var format_dropdown = fig.format_dropdown;\n var format = format_dropdown.options[format_dropdown.selectedIndex].value;\n fig.ondownload(fig, format);\n};\n\nmpl.figure.prototype.handle_resize = function (fig, msg) {\n var size = msg['size'];\n if (size[0] !== fig.canvas.width || size[1] !== fig.canvas.height) {\n fig._resize_canvas(size[0], size[1], msg['forward']);\n fig.send_message('refresh', {});\n }\n};\n\nmpl.figure.prototype.handle_rubberband = function (fig, msg) {\n var x0 = msg['x0'] / fig.ratio;\n var y0 = (fig.canvas.height - msg['y0']) / fig.ratio;\n var x1 = msg['x1'] / fig.ratio;\n var y1 = (fig.canvas.height - msg['y1']) / fig.ratio;\n x0 = Math.floor(x0) + 0.5;\n y0 = Math.floor(y0) + 0.5;\n x1 = Math.floor(x1) + 0.5;\n y1 = Math.floor(y1) + 0.5;\n var min_x = Math.min(x0, x1);\n var min_y = Math.min(y0, y1);\n var width = Math.abs(x1 - x0);\n var height = Math.abs(y1 - y0);\n\n fig.rubberband_context.clearRect(\n 0,\n 0,\n fig.canvas.width / fig.ratio,\n fig.canvas.height / fig.ratio\n );\n\n fig.rubberband_context.strokeRect(min_x, min_y, width, height);\n};\n\nmpl.figure.prototype.handle_figure_label = function (fig, msg) {\n // Updates the figure title.\n fig.header.textContent = msg['label'];\n};\n\nmpl.figure.prototype.handle_cursor = function (fig, msg) {\n var cursor = msg['cursor'];\n switch (cursor) {\n case 0:\n cursor = 'pointer';\n break;\n case 1:\n cursor = 'default';\n break;\n case 2:\n cursor = 'crosshair';\n break;\n case 3:\n cursor = 'move';\n break;\n }\n fig.rubberband_canvas.style.cursor = cursor;\n};\n\nmpl.figure.prototype.handle_message = function (fig, msg) {\n fig.message.textContent = msg['message'];\n};\n\nmpl.figure.prototype.handle_draw = function (fig, _msg) {\n // Request the server to send over a new figure.\n fig.send_draw_message();\n};\n\nmpl.figure.prototype.handle_image_mode = function (fig, msg) {\n fig.image_mode = msg['mode'];\n};\n\nmpl.figure.prototype.handle_history_buttons = function (fig, msg) {\n for (var key in msg) {\n if (!(key in fig.buttons)) {\n continue;\n }\n fig.buttons[key].disabled = !msg[key];\n fig.buttons[key].setAttribute('aria-disabled', !msg[key]);\n }\n};\n\nmpl.figure.prototype.handle_navigate_mode = function (fig, msg) {\n if (msg['mode'] === 'PAN') {\n fig.buttons['Pan'].classList.add('active');\n fig.buttons['Zoom'].classList.remove('active');\n } else if (msg['mode'] === 'ZOOM') {\n fig.buttons['Pan'].classList.remove('active');\n fig.buttons['Zoom'].classList.add('active');\n } else {\n fig.buttons['Pan'].classList.remove('active');\n fig.buttons['Zoom'].classList.remove('active');\n }\n};\n\nmpl.figure.prototype.updated_canvas_event = function () {\n // Called whenever the canvas gets updated.\n this.send_message('ack', {});\n};\n\n// A function to construct a web socket function for onmessage handling.\n// Called in the figure constructor.\nmpl.figure.prototype._make_on_message_function = function (fig) {\n return function socket_on_message(evt) {\n if (evt.data instanceof Blob) {\n var img = evt.data;\n if (img.type !== 'image/png') {\n /* FIXME: We get \"Resource interpreted as Image but\n * transferred with MIME type text/plain:\" errors on\n * Chrome. But how to set the MIME type? It doesn't seem\n * to be part of the websocket stream */\n img.type = 'image/png';\n }\n\n /* Free the memory for the previous frames */\n if (fig.imageObj.src) {\n (window.URL || window.webkitURL).revokeObjectURL(\n fig.imageObj.src\n );\n }\n\n fig.imageObj.src = (window.URL || window.webkitURL).createObjectURL(\n img\n );\n fig.updated_canvas_event();\n fig.waiting = false;\n return;\n } else if (\n typeof evt.data === 'string' &&\n evt.data.slice(0, 21) === 'data:image/png;base64'\n ) {\n fig.imageObj.src = evt.data;\n fig.updated_canvas_event();\n fig.waiting = false;\n return;\n }\n\n var msg = JSON.parse(evt.data);\n var msg_type = msg['type'];\n\n // Call the \"handle_{type}\" callback, which takes\n // the figure and JSON message as its only arguments.\n try {\n var callback = fig['handle_' + msg_type];\n } catch (e) {\n console.log(\n \"No handler for the '\" + msg_type + \"' message type: \",\n msg\n );\n return;\n }\n\n if (callback) {\n try {\n // console.log(\"Handling '\" + msg_type + \"' message: \", msg);\n callback(fig, msg);\n } catch (e) {\n console.log(\n \"Exception inside the 'handler_\" + msg_type + \"' callback:\",\n e,\n e.stack,\n msg\n );\n }\n }\n };\n};\n\n// from http://stackoverflow.com/questions/1114465/getting-mouse-location-in-canvas\nmpl.findpos = function (e) {\n //this section is from http://www.quirksmode.org/js/events_properties.html\n var targ;\n if (!e) {\n e = window.event;\n }\n if (e.target) {\n targ = e.target;\n } else if (e.srcElement) {\n targ = e.srcElement;\n }\n if (targ.nodeType === 3) {\n // defeat Safari bug\n targ = targ.parentNode;\n }\n\n // pageX,Y are the mouse positions relative to the document\n var boundingRect = targ.getBoundingClientRect();\n var x = e.pageX - (boundingRect.left + document.body.scrollLeft);\n var y = e.pageY - (boundingRect.top + document.body.scrollTop);\n\n return { x: x, y: y };\n};\n\n/*\n * return a copy of an object with only non-object keys\n * we need this to avoid circular references\n * http://stackoverflow.com/a/24161582/3208463\n */\nfunction simpleKeys(original) {\n return Object.keys(original).reduce(function (obj, key) {\n if (typeof original[key] !== 'object') {\n obj[key] = original[key];\n }\n return obj;\n }, {});\n}\n\nmpl.figure.prototype.mouse_event = function (event, name) {\n var canvas_pos = mpl.findpos(event);\n\n if (name === 'button_press') {\n this.canvas.focus();\n this.canvas_div.focus();\n }\n\n var x = canvas_pos.x * this.ratio;\n var y = canvas_pos.y * this.ratio;\n\n this.send_message(name, {\n x: x,\n y: y,\n button: event.button,\n step: event.step,\n guiEvent: simpleKeys(event),\n });\n\n /* This prevents the web browser from automatically changing to\n * the text insertion cursor when the button is pressed. We want\n * to control all of the cursor setting manually through the\n * 'cursor' event from matplotlib */\n event.preventDefault();\n return false;\n};\n\nmpl.figure.prototype._key_event_extra = function (_event, _name) {\n // Handle any extra behaviour associated with a key event\n};\n\nmpl.figure.prototype.key_event = function (event, name) {\n // Prevent repeat events\n if (name === 'key_press') {\n if (event.key === this._key) {\n return;\n } else {\n this._key = event.key;\n }\n }\n if (name === 'key_release') {\n this._key = null;\n }\n\n var value = '';\n if (event.ctrlKey && event.key !== 'Control') {\n value += 'ctrl+';\n }\n else if (event.altKey && event.key !== 'Alt') {\n value += 'alt+';\n }\n else if (event.shiftKey && event.key !== 'Shift') {\n value += 'shift+';\n }\n\n value += 'k' + event.key;\n\n this._key_event_extra(event, name);\n\n this.send_message(name, { key: value, guiEvent: simpleKeys(event) });\n return false;\n};\n\nmpl.figure.prototype.toolbar_button_onclick = function (name) {\n if (name === 'download') {\n this.handle_save(this, null);\n } else {\n this.send_message('toolbar_button', { name: name });\n }\n};\n\nmpl.figure.prototype.toolbar_button_onmouseover = function (tooltip) {\n this.message.textContent = tooltip;\n};\n\n///////////////// REMAINING CONTENT GENERATED BY embed_js.py /////////////////\n// prettier-ignore\nvar _JSXTOOLS_RESIZE_OBSERVER=function(A){var t,i=new WeakMap,n=new WeakMap,a=new WeakMap,r=new WeakMap,o=new Set;function s(e){if(!(this instanceof s))throw new TypeError(\"Constructor requires 'new' operator\");i.set(this,e)}function h(){throw new TypeError(\"Function is not a constructor\")}function c(e,t,i,n){e=0 in arguments?Number(arguments[0]):0,t=1 in arguments?Number(arguments[1]):0,i=2 in arguments?Number(arguments[2]):0,n=3 in arguments?Number(arguments[3]):0,this.right=(this.x=this.left=e)+(this.width=i),this.bottom=(this.y=this.top=t)+(this.height=n),Object.freeze(this)}function d(){t=requestAnimationFrame(d);var s=new WeakMap,p=new Set;o.forEach((function(t){r.get(t).forEach((function(i){var r=t instanceof window.SVGElement,o=a.get(t),d=r?0:parseFloat(o.paddingTop),f=r?0:parseFloat(o.paddingRight),l=r?0:parseFloat(o.paddingBottom),u=r?0:parseFloat(o.paddingLeft),g=r?0:parseFloat(o.borderTopWidth),m=r?0:parseFloat(o.borderRightWidth),w=r?0:parseFloat(o.borderBottomWidth),b=u+f,F=d+l,v=(r?0:parseFloat(o.borderLeftWidth))+m,W=g+w,y=r?0:t.offsetHeight-W-t.clientHeight,E=r?0:t.offsetWidth-v-t.clientWidth,R=b+v,z=F+W,M=r?t.width:parseFloat(o.width)-R-E,O=r?t.height:parseFloat(o.height)-z-y;if(n.has(t)){var k=n.get(t);if(k[0]===M&&k[1]===O)return}n.set(t,[M,O]);var S=Object.create(h.prototype);S.target=t,S.contentRect=new c(u,d,M,O),s.has(i)||(s.set(i,[]),p.add(i)),s.get(i).push(S)}))})),p.forEach((function(e){i.get(e).call(e,s.get(e),e)}))}return s.prototype.observe=function(i){if(i instanceof window.Element){r.has(i)||(r.set(i,new Set),o.add(i),a.set(i,window.getComputedStyle(i)));var n=r.get(i);n.has(this)||n.add(this),cancelAnimationFrame(t),t=requestAnimationFrame(d)}},s.prototype.unobserve=function(i){if(i instanceof window.Element&&r.has(i)){var n=r.get(i);n.has(this)&&(n.delete(this),n.size||(r.delete(i),o.delete(i))),n.size||r.delete(i),o.size||cancelAnimationFrame(t)}},A.DOMRectReadOnly=c,A.ResizeObserver=s,A.ResizeObserverEntry=h,A}; // eslint-disable-line\nmpl.toolbar_items = [[\"Home\", \"Reset original view\", \"fa fa-home icon-home\", \"home\"], [\"Back\", \"Back to previous view\", \"fa fa-arrow-left icon-arrow-left\", \"back\"], [\"Forward\", \"Forward to next view\", \"fa fa-arrow-right icon-arrow-right\", \"forward\"], [\"\", \"\", \"\", \"\"], [\"Pan\", \"Left button pans, Right button zooms\\nx/y fixes axis, CTRL fixes aspect\", \"fa fa-arrows icon-move\", \"pan\"], [\"Zoom\", \"Zoom to rectangle\\nx/y fixes axis, CTRL fixes aspect\", \"fa fa-square-o icon-check-empty\", \"zoom\"], [\"\", \"\", \"\", \"\"], [\"Download\", \"Download plot\", \"fa fa-floppy-o icon-save\", \"download\"]];\n\nmpl.extensions = [\"eps\", \"jpeg\", \"pgf\", \"pdf\", \"png\", \"ps\", \"raw\", \"svg\", \"tif\"];\n\nmpl.default_extension = \"png\";/* global mpl */\n\nvar comm_websocket_adapter = function (comm) {\n // Create a \"websocket\"-like object which calls the given IPython comm\n // object with the appropriate methods. Currently this is a non binary\n // socket, so there is still some room for performance tuning.\n var ws = {};\n\n ws.binaryType = comm.kernel.ws.binaryType;\n ws.readyState = comm.kernel.ws.readyState;\n function updateReadyState(_event) {\n if (comm.kernel.ws) {\n ws.readyState = comm.kernel.ws.readyState;\n } else {\n ws.readyState = 3; // Closed state.\n }\n }\n comm.kernel.ws.addEventListener('open', updateReadyState);\n comm.kernel.ws.addEventListener('close', updateReadyState);\n comm.kernel.ws.addEventListener('error', updateReadyState);\n\n ws.close = function () {\n comm.close();\n };\n ws.send = function (m) {\n //console.log('sending', m);\n comm.send(m);\n };\n // Register the callback with on_msg.\n comm.on_msg(function (msg) {\n //console.log('receiving', msg['content']['data'], msg);\n var data = msg['content']['data'];\n if (data['blob'] !== undefined) {\n data = {\n data: new Blob(msg['buffers'], { type: data['blob'] }),\n };\n }\n // Pass the mpl event to the overridden (by mpl) onmessage function.\n ws.onmessage(data);\n });\n return ws;\n};\n\nmpl.mpl_figure_comm = function (comm, msg) {\n // This is the function which gets called when the mpl process\n // starts-up an IPython Comm through the \"matplotlib\" channel.\n\n var id = msg.content.data.id;\n // Get hold of the div created by the display call when the Comm\n // socket was opened in Python.\n var element = document.getElementById(id);\n var ws_proxy = comm_websocket_adapter(comm);\n\n function ondownload(figure, _format) {\n window.open(figure.canvas.toDataURL());\n }\n\n var fig = new mpl.figure(id, ws_proxy, ondownload, element);\n\n // Call onopen now - mpl needs it, as it is assuming we've passed it a real\n // web socket which is closed, not our websocket->open comm proxy.\n ws_proxy.onopen();\n\n fig.parent_element = element;\n fig.cell_info = mpl.find_output_cell(\"<div id='\" + id + \"'></div>\");\n if (!fig.cell_info) {\n console.error('Failed to find cell for figure', id, fig);\n return;\n }\n fig.cell_info[0].output_area.element.on(\n 'cleared',\n { fig: fig },\n fig._remove_fig_handler\n );\n};\n\nmpl.figure.prototype.handle_close = function (fig, msg) {\n var width = fig.canvas.width / fig.ratio;\n fig.cell_info[0].output_area.element.off(\n 'cleared',\n fig._remove_fig_handler\n );\n fig.resizeObserverInstance.unobserve(fig.canvas_div);\n\n // Update the output cell to use the data from the current canvas.\n fig.push_to_output();\n var dataURL = fig.canvas.toDataURL();\n // Re-enable the keyboard manager in IPython - without this line, in FF,\n // the notebook keyboard shortcuts fail.\n IPython.keyboard_manager.enable();\n fig.parent_element.innerHTML =\n '<img src=\"' + dataURL + '\" width=\"' + width + '\">';\n fig.close_ws(fig, msg);\n};\n\nmpl.figure.prototype.close_ws = function (fig, msg) {\n fig.send_message('closing', msg);\n // fig.ws.close()\n};\n\nmpl.figure.prototype.push_to_output = function (_remove_interactive) {\n // Turn the data on the canvas into data in the output cell.\n var width = this.canvas.width / this.ratio;\n var dataURL = this.canvas.toDataURL();\n this.cell_info[1]['text/html'] =\n '<img src=\"' + dataURL + '\" width=\"' + width + '\">';\n};\n\nmpl.figure.prototype.updated_canvas_event = function () {\n // Tell IPython that the notebook contents must change.\n IPython.notebook.set_dirty(true);\n this.send_message('ack', {});\n var fig = this;\n // Wait a second, then push the new image to the DOM so\n // that it is saved nicely (might be nice to debounce this).\n setTimeout(function () {\n fig.push_to_output();\n }, 1000);\n};\n\nmpl.figure.prototype._init_toolbar = function () {\n var fig = this;\n\n var toolbar = document.createElement('div');\n toolbar.classList = 'btn-toolbar';\n this.root.appendChild(toolbar);\n\n function on_click_closure(name) {\n return function (_event) {\n return fig.toolbar_button_onclick(name);\n };\n }\n\n function on_mouseover_closure(tooltip) {\n return function (event) {\n if (!event.currentTarget.disabled) {\n return fig.toolbar_button_onmouseover(tooltip);\n }\n };\n }\n\n fig.buttons = {};\n var buttonGroup = document.createElement('div');\n buttonGroup.classList = 'btn-group';\n var button;\n for (var toolbar_ind in mpl.toolbar_items) {\n var name = mpl.toolbar_items[toolbar_ind][0];\n var tooltip = mpl.toolbar_items[toolbar_ind][1];\n var image = mpl.toolbar_items[toolbar_ind][2];\n var method_name = mpl.toolbar_items[toolbar_ind][3];\n\n if (!name) {\n /* Instead of a spacer, we start a new button group. */\n if (buttonGroup.hasChildNodes()) {\n toolbar.appendChild(buttonGroup);\n }\n buttonGroup = document.createElement('div');\n buttonGroup.classList = 'btn-group';\n continue;\n }\n\n button = fig.buttons[name] = document.createElement('button');\n button.classList = 'btn btn-default';\n button.href = '#';\n button.title = name;\n button.innerHTML = '<i class=\"fa ' + image + ' fa-lg\"></i>';\n button.addEventListener('click', on_click_closure(method_name));\n button.addEventListener('mouseover', on_mouseover_closure(tooltip));\n buttonGroup.appendChild(button);\n }\n\n if (buttonGroup.hasChildNodes()) {\n toolbar.appendChild(buttonGroup);\n }\n\n // Add the status bar.\n var status_bar = document.createElement('span');\n status_bar.classList = 'mpl-message pull-right';\n toolbar.appendChild(status_bar);\n this.message = status_bar;\n\n // Add the close button to the window.\n var buttongrp = document.createElement('div');\n buttongrp.classList = 'btn-group inline pull-right';\n button = document.createElement('button');\n button.classList = 'btn btn-mini btn-primary';\n button.href = '#';\n button.title = 'Stop Interaction';\n button.innerHTML = '<i class=\"fa fa-power-off icon-remove icon-large\"></i>';\n button.addEventListener('click', function (_evt) {\n fig.handle_close(fig, {});\n });\n button.addEventListener(\n 'mouseover',\n on_mouseover_closure('Stop Interaction')\n );\n buttongrp.appendChild(button);\n var titlebar = this.root.querySelector('.ui-dialog-titlebar');\n titlebar.insertBefore(buttongrp, titlebar.firstChild);\n};\n\nmpl.figure.prototype._remove_fig_handler = function (event) {\n var fig = event.data.fig;\n if (event.target !== this) {\n // Ignore bubbled events from children.\n return;\n }\n fig.close_ws(fig, {});\n};\n\nmpl.figure.prototype._root_extra_style = function (el) {\n el.style.boxSizing = 'content-box'; // override notebook setting of border-box.\n};\n\nmpl.figure.prototype._canvas_extra_style = function (el) {\n // this is important to make the div 'focusable\n el.setAttribute('tabindex', 0);\n // reach out to IPython and tell the keyboard manager to turn it's self\n // off when our div gets focus\n\n // location in version 3\n if (IPython.notebook.keyboard_manager) {\n IPython.notebook.keyboard_manager.register_events(el);\n } else {\n // location in version 2\n IPython.keyboard_manager.register_events(el);\n }\n};\n\nmpl.figure.prototype._key_event_extra = function (event, _name) {\n var manager = IPython.notebook.keyboard_manager;\n if (!manager) {\n manager = IPython.keyboard_manager;\n }\n\n // Check for shift+enter\n if (event.shiftKey && event.which === 13) {\n this.canvas_div.blur();\n // select the cell after this one\n var index = IPython.notebook.find_cell_index(this.cell_info[0]);\n IPython.notebook.select(index + 1);\n }\n};\n\nmpl.figure.prototype.handle_save = function (fig, _msg) {\n fig.ondownload(fig, null);\n};\n\nmpl.find_output_cell = function (html_output) {\n // Return the cell and output element which can be found *uniquely* in the notebook.\n // Note - this is a bit hacky, but it is done because the \"notebook_saving.Notebook\"\n // IPython event is triggered only after the cells have been serialised, which for\n // our purposes (turning an active figure into a static one), is too late.\n var cells = IPython.notebook.get_cells();\n var ncells = cells.length;\n for (var i = 0; i < ncells; i++) {\n var cell = cells[i];\n if (cell.cell_type === 'code') {\n for (var j = 0; j < cell.output_area.outputs.length; j++) {\n var data = cell.output_area.outputs[j];\n if (data.data) {\n // IPython >= 3 moved mimebundle to data attribute of output\n data = data.data;\n }\n if (data['text/html'] === html_output) {\n return [cell, data, j];\n }\n }\n }\n }\n};\n\n// Register the function which deals with the matplotlib target/channel.\n// The kernel may be null if the page has been refreshed.\nif (IPython.notebook.kernel !== null) {\n IPython.notebook.kernel.comm_manager.register_target(\n 'matplotlib',\n mpl.mpl_figure_comm\n );\n}\n",
"text/plain": [
"<IPython.core.display.Javascript object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<img src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAoAAAAHgCAYAAAA10dzkAAAAAXNSR0IArs4c6QAAGJZJREFUeF7t1kEBAAAIAjHpX9ogNxswfLBzBAgQIECAAAECKYGl0gpLgAABAgQIECBwBqAnIECAAAECBAjEBAzAWOHiEiBAgAABAgQMQD9AgAABAgQIEIgJGICxwsUlQIAAAQIECBiAfoAAAQIECBAgEBMwAGOFi0uAAAECBAgQMAD9AAECBAgQIEAgJmAAxgoXlwABAgQIECBgAPoBAgQIECBAgEBMwACMFS4uAQIECBAgQMAA9AMECBAgQIAAgZiAARgrXFwCBAgQIECAgAHoBwgQIECAAAECMQEDMFa4uAQIECBAgAABA9APECBAgAABAgRiAgZgrHBxCRAgQIAAAQIGoB8gQIAAAQIECMQEDMBY4eISIECAAAECBAxAP0CAAAECBAgQiAkYgLHCxSVAgAABAgQIGIB+gAABAgQIECAQEzAAY4WLS4AAAQIECBAwAP0AAQIECBAgQCAmYADGCheXAAECBAgQIGAA+gECBAgQIECAQEzAAIwVLi4BAgQIECBAwAD0AwQIECBAgACBmIABGCtcXAIECBAgQICAAegHCBAgQIAAAQIxAQMwVri4BAgQIECAAAED0A8QIECAAAECBGICBmCscHEJECBAgAABAgagHyBAgAABAgQIxAQMwFjh4hIgQIAAAQIEDEA/QIAAAQIECBCICRiAscLFJUCAAAECBAgYgH6AAAECBAgQIBATMABjhYtLgAABAgQIEDAA/QABAgQIECBAICZgAMYKF5cAAQIECBAgYAD6AQIECBAgQIBATMAAjBUuLgECBAgQIEDAAPQDBAgQIECAAIGYgAEYK1xcAgQIECBAgIAB6AcIECBAgAABAjEBAzBWuLgECBAgQIAAAQPQDxAgQIAAAQIEYgIGYKxwcQkQIECAAAECBqAfIECAAAECBAjEBAzAWOHiEiBAgAABAgQMQD9AgAABAgQIEIgJGICxwsUlQIAAAQIECBiAfoAAAQIECBAgEBMwAGOFi0uAAAECBAgQMAD9AAECBAgQIEAgJmAAxgoXlwABAgQIECBgAPoBAgQIECBAgEBMwACMFS4uAQIECBAgQMAA9AMECBAgQIAAgZiAARgrXFwCBAgQIECAgAHoBwgQIECAAAECMQEDMFa4uAQIECBAgAABA9APECBAgAABAgRiAgZgrHBxCRAgQIAAAQIGoB8gQIAAAQIECMQEDMBY4eISIECAAAECBAxAP0CAAAECBAgQiAkYgLHCxSVAgAABAgQIGIB+gAABAgQIECAQEzAAY4WLS4AAAQIECBAwAP0AAQIECBAgQCAmYADGCheXAAECBAgQIGAA+gECBAgQIECAQEzAAIwVLi4BAgQIECBAwAD0AwQIECBAgACBmIABGCtcXAIECBAgQICAAegHCBAgQIAAAQIxAQMwVri4BAgQIECAAAED0A8QIECAAAECBGICBmCscHEJECBAgAABAgagHyBAgAABAgQIxAQMwFjh4hIgQIAAAQIEDEA/QIAAAQIECBCICRiAscLFJUCAAAECBAgYgH6AAAECBAgQIBATMABjhYtLgAABAgQIEDAA/QABAgQIECBAICZgAMYKF5cAAQIECBAgYAD6AQIECBAgQIBATMAAjBUuLgECBAgQIEDAAPQDBAgQIECAAIGYgAEYK1xcAgQIECBAgIAB6AcIECBAgAABAjEBAzBWuLgECBAgQIAAAQPQDxAgQIAAAQIEYgIGYKxwcQkQIECAAAECBqAfIECAAAECBAjEBAzAWOHiEiBAgAABAgQMQD9AgAABAgQIEIgJGICxwsUlQIAAAQIECBiAfoAAAQIECBAgEBMwAGOFi0uAAAECBAgQMAD9AAECBAgQIEAgJmAAxgoXlwABAgQIECBgAPoBAgQIECBAgEBMwACMFS4uAQIECBAgQMAA9AMECBAgQIAAgZiAARgrXFwCBAgQIECAgAHoBwgQIECAAAECMQEDMFa4uAQIECBAgAABA9APECBAgAABAgRiAgZgrHBxCRAgQIAAAQIGoB8gQIAAAQIECMQEDMBY4eISIECAAAECBAxAP0CAAAECBAgQiAkYgLHCxSVAgAABAgQIGIB+gAABAgQIECAQEzAAY4WLS4AAAQIECBAwAP0AAQIECBAgQCAmYADGCheXAAECBAgQIGAA+gECBAgQIECAQEzAAIwVLi4BAgQIECBAwAD0AwQIECBAgACBmIABGCtcXAIECBAgQICAAegHCBAgQIAAAQIxAQMwVri4BAgQIECAAAED0A8QIECAAAECBGICBmCscHEJECBAgAABAgagHyBAgAABAgQIxAQMwFjh4hIgQIAAAQIEDEA/QIAAAQIECBCICRiAscLFJUCAAAECBAgYgH6AAAECBAgQIBATMABjhYtLgAABAgQIEDAA/QABAgQIECBAICZgAMYKF5cAAQIECBAgYAD6AQIECBAgQIBATMAAjBUuLgECBAgQIEDAAPQDBAgQIECAAIGYgAEYK1xcAgQIECBAgIAB6AcIECBAgAABAjEBAzBWuLgECBAgQIAAAQPQDxAgQIAAAQIEYgIGYKxwcQkQIECAAAECBqAfIECAAAECBAjEBAzAWOHiEiBAgAABAgQMQD9AgAABAgQIEIgJGICxwsUlQIAAAQIECBiAfoAAAQIECBAgEBMwAGOFi0uAAAECBAgQMAD9AAECBAgQIEAgJmAAxgoXlwABAgQIECBgAPoBAgQIECBAgEBMwACMFS4uAQIECBAgQMAA9AMECBAgQIAAgZiAARgrXFwCBAgQIECAgAHoBwgQIECAAAECMQEDMFa4uAQIECBAgAABA9APECBAgAABAgRiAgZgrHBxCRAgQIAAAQIGoB8gQIAAAQIECMQEDMBY4eISIECAAAECBAxAP0CAAAECBAgQiAkYgLHCxSVAgAABAgQIGIB+gAABAgQIECAQEzAAY4WLS4AAAQIECBAwAP0AAQIECBAgQCAmYADGCheXAAECBAgQIGAA+gECBAgQIECAQEzAAIwVLi4BAgQIECBAwAD0AwQIECBAgACBmIABGCtcXAIECBAgQICAAegHCBAgQIAAAQIxAQMwVri4BAgQIECAAAED0A8QIECAAAECBGICBmCscHEJECBAgAABAgagHyBAgAABAgQIxAQMwFjh4hIgQIAAAQIEDEA/QIAAAQIECBCICRiAscLFJUCAAAECBAgYgH6AAAECBAgQIBATMABjhYtLgAABAgQIEDAA/QABAgQIECBAICZgAMYKF5cAAQIECBAgYAD6AQIECBAgQIBATMAAjBUuLgECBAgQIEDAAPQDBAgQIECAAIGYgAEYK1xcAgQIECBAgIAB6AcIECBAgAABAjEBAzBWuLgECBAgQIAAAQPQDxAgQIAAAQIEYgIGYKxwcQkQIECAAAECBqAfIECAAAECBAjEBAzAWOHiEiBAgAABAgQMQD9AgAABAgQIEIgJGICxwsUlQIAAAQIECBiAfoAAAQIECBAgEBMwAGOFi0uAAAECBAgQMAD9AAECBAgQIEAgJmAAxgoXlwABAgQIECBgAPoBAgQIECBAgEBMwACMFS4uAQIECBAgQMAA9AMECBAgQIAAgZiAARgrXFwCBAgQIECAgAHoBwgQIECAAAECMQEDMFa4uAQIECBAgAABA9APECBAgAABAgRiAgZgrHBxCRAgQIAAAQIGoB8gQIAAAQIECMQEDMBY4eISIECAAAECBAxAP0CAAAECBAgQiAkYgLHCxSVAgAABAgQIGIB+gAABAgQIECAQEzAAY4WLS4AAAQIECBAwAP0AAQIECBAgQCAmYADGCheXAAECBAgQIGAA+gECBAgQIECAQEzAAIwVLi4BAgQIECBAwAD0AwQIECBAgACBmIABGCtcXAIECBAgQICAAegHCBAgQIAAAQIxAQMwVri4BAgQIECAAAED0A8QIECAAAECBGICBmCscHEJECBAgAABAgagHyBAgAABAgQIxAQMwFjh4hIgQIAAAQIEDEA/QIAAAQIECBCICRiAscLFJUCAAAECBAgYgH6AAAECBAgQIBATMABjhYtLgAABAgQIEDAA/QABAgQIECBAICZgAMYKF5cAAQIECBAgYAD6AQIECBAgQIBATMAAjBUuLgECBAgQIEDAAPQDBAgQIECAAIGYgAEYK1xcAgQIECBAgIAB6AcIECBAgAABAjEBAzBWuLgECBAgQIAAAQPQDxAgQIAAAQIEYgIGYKxwcQkQIECAAAECBqAfIECAAAECBAjEBAzAWOHiEiBAgAABAgQMQD9AgAABAgQIEIgJGICxwsUlQIAAAQIECBiAfoAAAQIECBAgEBMwAGOFi0uAAAECBAgQMAD9AAECBAgQIEAgJmAAxgoXlwABAgQIECBgAPoBAgQIECBAgEBMwACMFS4uAQIECBAgQMAA9AMECBAgQIAAgZiAARgrXFwCBAgQIECAgAHoBwgQIECAAAECMQEDMFa4uAQIECBAgAABA9APECBAgAABAgRiAgZgrHBxCRAgQIAAAQIGoB8gQIAAAQIECMQEDMBY4eISIECAAAECBAxAP0CAAAECBAgQiAkYgLHCxSVAgAABAgQIGIB+gAABAgQIECAQEzAAY4WLS4AAAQIECBAwAP0AAQIECBAgQCAmYADGCheXAAECBAgQIGAA+gECBAgQIECAQEzAAIwVLi4BAgQIECBAwAD0AwQIECBAgACBmIABGCtcXAIECBAgQICAAegHCBAgQIAAAQIxAQMwVri4BAgQIECAAAED0A8QIECAAAECBGICBmCscHEJECBAgAABAgagHyBAgAABAgQIxAQMwFjh4hIgQIAAAQIEDEA/QIAAAQIECBCICRiAscLFJUCAAAECBAgYgH6AAAECBAgQIBATMABjhYtLgAABAgQIEDAA/QABAgQIECBAICZgAMYKF5cAAQIECBAgYAD6AQIECBAgQIBATMAAjBUuLgECBAgQIEDAAPQDBAgQIECAAIGYgAEYK1xcAgQIECBAgIAB6AcIECBAgAABAjEBAzBWuLgECBAgQIAAAQPQDxAgQIAAAQIEYgIGYKxwcQkQIECAAAECBqAfIECAAAECBAjEBAzAWOHiEiBAgAABAgQMQD9AgAABAgQIEIgJGICxwsUlQIAAAQIECBiAfoAAAQIECBAgEBMwAGOFi0uAAAECBAgQMAD9AAECBAgQIEAgJmAAxgoXlwABAgQIECBgAPoBAgQIECBAgEBMwACMFS4uAQIECBAgQMAA9AMECBAgQIAAgZiAARgrXFwCBAgQIECAgAHoBwgQIECAAAECMQEDMFa4uAQIECBAgAABA9APECBAgAABAgRiAgZgrHBxCRAgQIAAAQIGoB8gQIAAAQIECMQEDMBY4eISIECAAAECBAxAP0CAAAECBAgQiAkYgLHCxSVAgAABAgQIGIB+gAABAgQIECAQEzAAY4WLS4AAAQIECBAwAP0AAQIECBAgQCAmYADGCheXAAECBAgQIGAA+gECBAgQIECAQEzAAIwVLi4BAgQIECBAwAD0AwQIECBAgACBmIABGCtcXAIECBAgQICAAegHCBAgQIAAAQIxAQMwVri4BAgQIECAAAED0A8QIECAAAECBGICBmCscHEJECBAgAABAgagHyBAgAABAgQIxAQMwFjh4hIgQIAAAQIEDEA/QIAAAQIECBCICRiAscLFJUCAAAECBAgYgH6AAAECBAgQIBATMABjhYtLgAABAgQIEDAA/QABAgQIECBAICZgAMYKF5cAAQIECBAgYAD6AQIECBAgQIBATMAAjBUuLgECBAgQIEDAAPQDBAgQIECAAIGYgAEYK1xcAgQIECBAgIAB6AcIECBAgAABAjEBAzBWuLgECBAgQIAAAQPQDxAgQIAAAQIEYgIGYKxwcQkQIECAAAECBqAfIECAAAECBAjEBAzAWOHiEiBAgAABAgQMQD9AgAABAgQIEIgJGICxwsUlQIAAAQIECBiAfoAAAQIECBAgEBMwAGOFi0uAAAECBAgQMAD9AAECBAgQIEAgJmAAxgoXlwABAgQIECBgAPoBAgQIECBAgEBMwACMFS4uAQIECBAgQMAA9AMECBAgQIAAgZiAARgrXFwCBAgQIECAgAHoBwgQIECAAAECMQEDMFa4uAQIECBAgAABA9APECBAgAABAgRiAgZgrHBxCRAgQIAAAQIGoB8gQIAAAQIECMQEDMBY4eISIECAAAECBAxAP0CAAAECBAgQiAkYgLHCxSVAgAABAgQIGIB+gAABAgQIECAQEzAAY4WLS4AAAQIECBAwAP0AAQIECBAgQCAmYADGCheXAAECBAgQIGAA+gECBAgQIECAQEzAAIwVLi4BAgQIECBAwAD0AwQIECBAgACBmIABGCtcXAIECBAgQICAAegHCBAgQIAAAQIxAQMwVri4BAgQIECAAAED0A8QIECAAAECBGICBmCscHEJECBAgAABAgagHyBAgAABAgQIxAQMwFjh4hIgQIAAAQIEDEA/QIAAAQIECBCICRiAscLFJUCAAAECBAgYgH6AAAECBAgQIBATMABjhYtLgAABAgQIEDAA/QABAgQIECBAICZgAMYKF5cAAQIECBAgYAD6AQIECBAgQIBATMAAjBUuLgECBAgQIEDAAPQDBAgQIECAAIGYgAEYK1xcAgQIECBAgIAB6AcIECBAgAABAjEBAzBWuLgECBAgQIAAAQPQDxAgQIAAAQIEYgIGYKxwcQkQIECAAAECBqAfIECAAAECBAjEBAzAWOHiEiBAgAABAgQMQD9AgAABAgQIEIgJGICxwsUlQIAAAQIECBiAfoAAAQIECBAgEBMwAGOFi0uAAAECBAgQMAD9AAECBAgQIEAgJmAAxgoXlwABAgQIECBgAPoBAgQIECBAgEBMwACMFS4uAQIECBAgQMAA9AMECBAgQIAAgZiAARgrXFwCBAgQIECAgAHoBwgQIECAAAECMQEDMFa4uAQIECBAgAABA9APECBAgAABAgRiAgZgrHBxCRAgQIAAAQIGoB8gQIAAAQIECMQEDMBY4eISIECAAAECBAxAP0CAAAECBAgQiAkYgLHCxSVAgAABAgQIGIB+gAABAgQIECAQEzAAY4WLS4AAAQIECBAwAP0AAQIECBAgQCAmYADGCheXAAECBAgQIGAA+gECBAgQIECAQEzAAIwVLi4BAgQIECBAwAD0AwQIECBAgACBmIABGCtcXAIECBAgQICAAegHCBAgQIAAAQIxAQMwVri4BAgQIECAAAED0A8QIECAAAECBGICBmCscHEJECBAgAABAgagHyBAgAABAgQIxAQMwFjh4hIgQIAAAQIEDEA/QIAAAQIECBCICRiAscLFJUCAAAECBAgYgH6AAAECBAgQIBATMABjhYtLgAABAgQIEDAA/QABAgQIECBAICZgAMYKF5cAAQIECBAgYAD6AQIECBAgQIBATMAAjBUuLgECBAgQIEDAAPQDBAgQIECAAIGYgAEYK1xcAgQIECBAgIAB6AcIECBAgAABAjEBAzBWuLgECBAgQIAAAQPQDxAgQIAAAQIEYgIGYKxwcQkQIECAAAECBqAfIECAAAECBAjEBAzAWOHiEiBAgAABAgQMQD9AgAABAgQIEIgJGICxwsUlQIAAAQIECBiAfoAAAQIECBAgEBMwAGOFi0uAAAECBAgQMAD9AAECBAgQIEAgJmAAxgoXlwABAgQIECBgAPoBAgQIECBAgEBMwACMFS4uAQIECBAgQMAA9AMECBAgQIAAgZiAARgrXFwCBAgQIECAgAHoBwgQIECAAAECMQEDMFa4uAQIECBAgAABA9APECBAgAABAgRiAgZgrHBxCRAgQIAAAQIGoB8gQIAAAQIECMQEDMBY4eISIECAAAECBAxAP0CAAAECBAgQiAkYgLHCxSVAgAABAgQIGIB+gAABAgQIECAQEzAAY4WLS4AAAQIECBAwAP0AAQIECBAgQCAmYADGCheXAAECBAgQIGAA+gECBAgQIECAQEzAAIwVLi4BAgQIECBAwAD0AwQIECBAgACBmIABGCtcXAIECBAgQICAAegHCBAgQIAAAQIxAQMwVri4BAgQIECAAAED0A8QIECAAAECBGICBmCscHEJECBAgAABAgagHyBAgAABAgQIxAQMwFjh4hIgQIAAAQIEDEA/QIAAAQIECBCICRiAscLFJUCAAAECBAgYgH6AAAECBAgQIBATMABjhYtLgAABAgQIEDAA/QABAgQIECBAICZgAMYKF5cAAQIECBAgYAD6AQIECBAgQIBATMAAjBUuLgECBAgQIEDAAPQDBAgQIECAAIGYgAEYK1xcAgQIECBAgIAB6AcIECBAgAABAjEBAzBWuLgECBAgQIAAgQfuPgHhBVmL8wAAAABJRU5ErkJggg==\" width=\"640\">"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"plot_dataset('Scatterplot of the training data', train_x, train_labels)\n",
"plt.show()"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[[ 1.3382818 -0.98613256]\n",
" [ 0.5128146 0.43299454]\n",
" [-0.4473693 -0.2680512 ]\n",
" [-0.9865851 -0.28692 ]\n",
" [-1.0693829 0.41718036]]\n",
"[1 1 0 0 0]\n"
]
}
],
"source": [
"print(train_x[:5])\n",
"print(train_labels[:5])"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Machine Learning Problem\n",
"\n",
"Suppose we have input dataset $\\langle X,Y\\rangle$, where $X$ is a set of features, and $Y$ - corresponding labels. For regression problem, $y_i\\in\\mathbb{R}$, and for classification it is represented by a class number $y_i\\in\\{0,\\dots,n\\}$. \n",
"\n",
"Any machine learning model can be represented by function $f_\\theta(x)$, where $\\theta$ is a set of **parameters**. Our goal is to find such parameters $\\theta$ that our model fits the dataset in the best way. The criteria is defined by **loss function** $\\mathcal{L}$, and we need to find optimal value\n",
"\n",
"$$\n",
"\\theta = \\mathrm{argmin}_\\theta \\mathcal{L}(f_\\theta(X),Y)\n",
"$$"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"Loss function depends on the problem being solved.\n",
"\n",
"### Loss functions for regression\n",
"\n",
"For regression, we often use **abosolute error** $\\mathcal{L}_{abs}(\\theta) = \\sum_{i=1}^n |y_i - f_{\\theta}(x_i)|$, or **mean squared error**: $\\mathcal{L}_{sq}(\\theta) = \\sum_{i=1}^n (y_i - f_{\\theta}(x_i))^2$"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"outputs": [],
"source": [
"# helper function for plotting various loss functions\n",
"def plot_loss_functions(suptitle, functions, ylabels, xlabel):\n",
" fig, ax = plt.subplots(1,len(functions), figsize=(9, 3))\n",
" plt.subplots_adjust(bottom=0.2, wspace=0.4)\n",
" fig.suptitle(suptitle)\n",
" for i, fun in enumerate(functions):\n",
" ax[i].set_xlabel(xlabel)\n",
" if len(ylabels) > i:\n",
" ax[i].set_ylabel(ylabels[i])\n",
" ax[i].plot(x, fun)\n",
" plt.show()"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [
{
"data": {
"application/javascript": "/* Put everything inside the global mpl namespace */\n/* global mpl */\nwindow.mpl = {};\n\nmpl.get_websocket_type = function () {\n if (typeof WebSocket !== 'undefined') {\n return WebSocket;\n } else if (typeof MozWebSocket !== 'undefined') {\n return MozWebSocket;\n } else {\n alert(\n 'Your browser does not have WebSocket support. ' +\n 'Please try Chrome, Safari or Firefox ≥ 6. ' +\n 'Firefox 4 and 5 are also supported but you ' +\n 'have to enable WebSockets in about:config.'\n );\n }\n};\n\nmpl.figure = function (figure_id, websocket, ondownload, parent_element) {\n this.id = figure_id;\n\n this.ws = websocket;\n\n this.supports_binary = this.ws.binaryType !== undefined;\n\n if (!this.supports_binary) {\n var warnings = document.getElementById('mpl-warnings');\n if (warnings) {\n warnings.style.display = 'block';\n warnings.textContent =\n 'This browser does not support binary websocket messages. ' +\n 'Performance may be slow.';\n }\n }\n\n this.imageObj = new Image();\n\n this.context = undefined;\n this.message = undefined;\n this.canvas = undefined;\n this.rubberband_canvas = undefined;\n this.rubberband_context = undefined;\n this.format_dropdown = undefined;\n\n this.image_mode = 'full';\n\n this.root = document.createElement('div');\n this.root.setAttribute('style', 'display: inline-block');\n this._root_extra_style(this.root);\n\n parent_element.appendChild(this.root);\n\n this._init_header(this);\n this._init_canvas(this);\n this._init_toolbar(this);\n\n var fig = this;\n\n this.waiting = false;\n\n this.ws.onopen = function () {\n fig.send_message('supports_binary', { value: fig.supports_binary });\n fig.send_message('send_image_mode', {});\n if (fig.ratio !== 1) {\n fig.send_message('set_dpi_ratio', { dpi_ratio: fig.ratio });\n }\n fig.send_message('refresh', {});\n };\n\n this.imageObj.onload = function () {\n if (fig.image_mode === 'full') {\n // Full images could contain transparency (where diff images\n // almost always do), so we need to clear the canvas so that\n // there is no ghosting.\n fig.context.clearRect(0, 0, fig.canvas.width, fig.canvas.height);\n }\n fig.context.drawImage(fig.imageObj, 0, 0);\n };\n\n this.imageObj.onunload = function () {\n fig.ws.close();\n };\n\n this.ws.onmessage = this._make_on_message_function(this);\n\n this.ondownload = ondownload;\n};\n\nmpl.figure.prototype._init_header = function () {\n var titlebar = document.createElement('div');\n titlebar.classList =\n 'ui-dialog-titlebar ui-widget-header ui-corner-all ui-helper-clearfix';\n var titletext = document.createElement('div');\n titletext.classList = 'ui-dialog-title';\n titletext.setAttribute(\n 'style',\n 'width: 100%; text-align: center; padding: 3px;'\n );\n titlebar.appendChild(titletext);\n this.root.appendChild(titlebar);\n this.header = titletext;\n};\n\nmpl.figure.prototype._canvas_extra_style = function (_canvas_div) {};\n\nmpl.figure.prototype._root_extra_style = function (_canvas_div) {};\n\nmpl.figure.prototype._init_canvas = function () {\n var fig = this;\n\n var canvas_div = (this.canvas_div = document.createElement('div'));\n canvas_div.setAttribute(\n 'style',\n 'border: 1px solid #ddd;' +\n 'box-sizing: content-box;' +\n 'clear: both;' +\n 'min-height: 1px;' +\n 'min-width: 1px;' +\n 'outline: 0;' +\n 'overflow: hidden;' +\n 'position: relative;' +\n 'resize: both;'\n );\n\n function on_keyboard_event_closure(name) {\n return function (event) {\n return fig.key_event(event, name);\n };\n }\n\n canvas_div.addEventListener(\n 'keydown',\n on_keyboard_event_closure('key_press')\n );\n canvas_div.addEventListener(\n 'keyup',\n on_keyboard_event_closure('key_release')\n );\n\n this._canvas_extra_style(canvas_div);\n this.root.appendChild(canvas_div);\n\n var canvas = (this.canvas = document.createElement('canvas'));\n canvas.classList.add('mpl-canvas');\n canvas.setAttribute('style', 'box-sizing: content-box;');\n\n this.context = canvas.getContext('2d');\n\n var backingStore =\n this.context.backingStorePixelRatio ||\n this.context.webkitBackingStorePixelRatio ||\n this.context.mozBackingStorePixelRatio ||\n this.context.msBackingStorePixelRatio ||\n this.context.oBackingStorePixelRatio ||\n this.context.backingStorePixelRatio ||\n 1;\n\n this.ratio = (window.devicePixelRatio || 1) / backingStore;\n\n var rubberband_canvas = (this.rubberband_canvas = document.createElement(\n 'canvas'\n ));\n rubberband_canvas.setAttribute(\n 'style',\n 'box-sizing: content-box; position: absolute; left: 0; top: 0; z-index: 1;'\n );\n\n // Apply a ponyfill if ResizeObserver is not implemented by browser.\n if (this.ResizeObserver === undefined) {\n if (window.ResizeObserver !== undefined) {\n this.ResizeObserver = window.ResizeObserver;\n } else {\n var obs = _JSXTOOLS_RESIZE_OBSERVER({});\n this.ResizeObserver = obs.ResizeObserver;\n }\n }\n\n this.resizeObserverInstance = new this.ResizeObserver(function (entries) {\n var nentries = entries.length;\n for (var i = 0; i < nentries; i++) {\n var entry = entries[i];\n var width, height;\n if (entry.contentBoxSize) {\n if (entry.contentBoxSize instanceof Array) {\n // Chrome 84 implements new version of spec.\n width = entry.contentBoxSize[0].inlineSize;\n height = entry.contentBoxSize[0].blockSize;\n } else {\n // Firefox implements old version of spec.\n width = entry.contentBoxSize.inlineSize;\n height = entry.contentBoxSize.blockSize;\n }\n } else {\n // Chrome <84 implements even older version of spec.\n width = entry.contentRect.width;\n height = entry.contentRect.height;\n }\n\n // Keep the size of the canvas and rubber band canvas in sync with\n // the canvas container.\n if (entry.devicePixelContentBoxSize) {\n // Chrome 84 implements new version of spec.\n canvas.setAttribute(\n 'width',\n entry.devicePixelContentBoxSize[0].inlineSize\n );\n canvas.setAttribute(\n 'height',\n entry.devicePixelContentBoxSize[0].blockSize\n );\n } else {\n canvas.setAttribute('width', width * fig.ratio);\n canvas.setAttribute('height', height * fig.ratio);\n }\n canvas.setAttribute(\n 'style',\n 'width: ' + width + 'px; height: ' + height + 'px;'\n );\n\n rubberband_canvas.setAttribute('width', width);\n rubberband_canvas.setAttribute('height', height);\n\n // And update the size in Python. We ignore the initial 0/0 size\n // that occurs as the element is placed into the DOM, which should\n // otherwise not happen due to the minimum size styling.\n if (fig.ws.readyState == 1 && width != 0 && height != 0) {\n fig.request_resize(width, height);\n }\n }\n });\n this.resizeObserverInstance.observe(canvas_div);\n\n function on_mouse_event_closure(name) {\n return function (event) {\n return fig.mouse_event(event, name);\n };\n }\n\n rubberband_canvas.addEventListener(\n 'mousedown',\n on_mouse_event_closure('button_press')\n );\n rubberband_canvas.addEventListener(\n 'mouseup',\n on_mouse_event_closure('button_release')\n );\n rubberband_canvas.addEventListener(\n 'dblclick',\n on_mouse_event_closure('dblclick')\n );\n // Throttle sequential mouse events to 1 every 20ms.\n rubberband_canvas.addEventListener(\n 'mousemove',\n on_mouse_event_closure('motion_notify')\n );\n\n rubberband_canvas.addEventListener(\n 'mouseenter',\n on_mouse_event_closure('figure_enter')\n );\n rubberband_canvas.addEventListener(\n 'mouseleave',\n on_mouse_event_closure('figure_leave')\n );\n\n canvas_div.addEventListener('wheel', function (event) {\n if (event.deltaY < 0) {\n event.step = 1;\n } else {\n event.step = -1;\n }\n on_mouse_event_closure('scroll')(event);\n });\n\n canvas_div.appendChild(canvas);\n canvas_div.appendChild(rubberband_canvas);\n\n this.rubberband_context = rubberband_canvas.getContext('2d');\n this.rubberband_context.strokeStyle = '#000000';\n\n this._resize_canvas = function (width, height, forward) {\n if (forward) {\n canvas_div.style.width = width + 'px';\n canvas_div.style.height = height + 'px';\n }\n };\n\n // Disable right mouse context menu.\n this.rubberband_canvas.addEventListener('contextmenu', function (_e) {\n event.preventDefault();\n return false;\n });\n\n function set_focus() {\n canvas.focus();\n canvas_div.focus();\n }\n\n window.setTimeout(set_focus, 100);\n};\n\nmpl.figure.prototype._init_toolbar = function () {\n var fig = this;\n\n var toolbar = document.createElement('div');\n toolbar.classList = 'mpl-toolbar';\n this.root.appendChild(toolbar);\n\n function on_click_closure(name) {\n return function (_event) {\n return fig.toolbar_button_onclick(name);\n };\n }\n\n function on_mouseover_closure(tooltip) {\n return function (event) {\n if (!event.currentTarget.disabled) {\n return fig.toolbar_button_onmouseover(tooltip);\n }\n };\n }\n\n fig.buttons = {};\n var buttonGroup = document.createElement('div');\n buttonGroup.classList = 'mpl-button-group';\n for (var toolbar_ind in mpl.toolbar_items) {\n var name = mpl.toolbar_items[toolbar_ind][0];\n var tooltip = mpl.toolbar_items[toolbar_ind][1];\n var image = mpl.toolbar_items[toolbar_ind][2];\n var method_name = mpl.toolbar_items[toolbar_ind][3];\n\n if (!name) {\n /* Instead of a spacer, we start a new button group. */\n if (buttonGroup.hasChildNodes()) {\n toolbar.appendChild(buttonGroup);\n }\n buttonGroup = document.createElement('div');\n buttonGroup.classList = 'mpl-button-group';\n continue;\n }\n\n var button = (fig.buttons[name] = document.createElement('button'));\n button.classList = 'mpl-widget';\n button.setAttribute('role', 'button');\n button.setAttribute('aria-disabled', 'false');\n button.addEventListener('click', on_click_closure(method_name));\n button.addEventListener('mouseover', on_mouseover_closure(tooltip));\n\n var icon_img = document.createElement('img');\n icon_img.src = '_images/' + image + '.png';\n icon_img.srcset = '_images/' + image + '_large.png 2x';\n icon_img.alt = tooltip;\n button.appendChild(icon_img);\n\n buttonGroup.appendChild(button);\n }\n\n if (buttonGroup.hasChildNodes()) {\n toolbar.appendChild(buttonGroup);\n }\n\n var fmt_picker = document.createElement('select');\n fmt_picker.classList = 'mpl-widget';\n toolbar.appendChild(fmt_picker);\n this.format_dropdown = fmt_picker;\n\n for (var ind in mpl.extensions) {\n var fmt = mpl.extensions[ind];\n var option = document.createElement('option');\n option.selected = fmt === mpl.default_extension;\n option.innerHTML = fmt;\n fmt_picker.appendChild(option);\n }\n\n var status_bar = document.createElement('span');\n status_bar.classList = 'mpl-message';\n toolbar.appendChild(status_bar);\n this.message = status_bar;\n};\n\nmpl.figure.prototype.request_resize = function (x_pixels, y_pixels) {\n // Request matplotlib to resize the figure. Matplotlib will then trigger a resize in the client,\n // which will in turn request a refresh of the image.\n this.send_message('resize', { width: x_pixels, height: y_pixels });\n};\n\nmpl.figure.prototype.send_message = function (type, properties) {\n properties['type'] = type;\n properties['figure_id'] = this.id;\n this.ws.send(JSON.stringify(properties));\n};\n\nmpl.figure.prototype.send_draw_message = function () {\n if (!this.waiting) {\n this.waiting = true;\n this.ws.send(JSON.stringify({ type: 'draw', figure_id: this.id }));\n }\n};\n\nmpl.figure.prototype.handle_save = function (fig, _msg) {\n var format_dropdown = fig.format_dropdown;\n var format = format_dropdown.options[format_dropdown.selectedIndex].value;\n fig.ondownload(fig, format);\n};\n\nmpl.figure.prototype.handle_resize = function (fig, msg) {\n var size = msg['size'];\n if (size[0] !== fig.canvas.width || size[1] !== fig.canvas.height) {\n fig._resize_canvas(size[0], size[1], msg['forward']);\n fig.send_message('refresh', {});\n }\n};\n\nmpl.figure.prototype.handle_rubberband = function (fig, msg) {\n var x0 = msg['x0'] / fig.ratio;\n var y0 = (fig.canvas.height - msg['y0']) / fig.ratio;\n var x1 = msg['x1'] / fig.ratio;\n var y1 = (fig.canvas.height - msg['y1']) / fig.ratio;\n x0 = Math.floor(x0) + 0.5;\n y0 = Math.floor(y0) + 0.5;\n x1 = Math.floor(x1) + 0.5;\n y1 = Math.floor(y1) + 0.5;\n var min_x = Math.min(x0, x1);\n var min_y = Math.min(y0, y1);\n var width = Math.abs(x1 - x0);\n var height = Math.abs(y1 - y0);\n\n fig.rubberband_context.clearRect(\n 0,\n 0,\n fig.canvas.width / fig.ratio,\n fig.canvas.height / fig.ratio\n );\n\n fig.rubberband_context.strokeRect(min_x, min_y, width, height);\n};\n\nmpl.figure.prototype.handle_figure_label = function (fig, msg) {\n // Updates the figure title.\n fig.header.textContent = msg['label'];\n};\n\nmpl.figure.prototype.handle_cursor = function (fig, msg) {\n var cursor = msg['cursor'];\n switch (cursor) {\n case 0:\n cursor = 'pointer';\n break;\n case 1:\n cursor = 'default';\n break;\n case 2:\n cursor = 'crosshair';\n break;\n case 3:\n cursor = 'move';\n break;\n }\n fig.rubberband_canvas.style.cursor = cursor;\n};\n\nmpl.figure.prototype.handle_message = function (fig, msg) {\n fig.message.textContent = msg['message'];\n};\n\nmpl.figure.prototype.handle_draw = function (fig, _msg) {\n // Request the server to send over a new figure.\n fig.send_draw_message();\n};\n\nmpl.figure.prototype.handle_image_mode = function (fig, msg) {\n fig.image_mode = msg['mode'];\n};\n\nmpl.figure.prototype.handle_history_buttons = function (fig, msg) {\n for (var key in msg) {\n if (!(key in fig.buttons)) {\n continue;\n }\n fig.buttons[key].disabled = !msg[key];\n fig.buttons[key].setAttribute('aria-disabled', !msg[key]);\n }\n};\n\nmpl.figure.prototype.handle_navigate_mode = function (fig, msg) {\n if (msg['mode'] === 'PAN') {\n fig.buttons['Pan'].classList.add('active');\n fig.buttons['Zoom'].classList.remove('active');\n } else if (msg['mode'] === 'ZOOM') {\n fig.buttons['Pan'].classList.remove('active');\n fig.buttons['Zoom'].classList.add('active');\n } else {\n fig.buttons['Pan'].classList.remove('active');\n fig.buttons['Zoom'].classList.remove('active');\n }\n};\n\nmpl.figure.prototype.updated_canvas_event = function () {\n // Called whenever the canvas gets updated.\n this.send_message('ack', {});\n};\n\n// A function to construct a web socket function for onmessage handling.\n// Called in the figure constructor.\nmpl.figure.prototype._make_on_message_function = function (fig) {\n return function socket_on_message(evt) {\n if (evt.data instanceof Blob) {\n var img = evt.data;\n if (img.type !== 'image/png') {\n /* FIXME: We get \"Resource interpreted as Image but\n * transferred with MIME type text/plain:\" errors on\n * Chrome. But how to set the MIME type? It doesn't seem\n * to be part of the websocket stream */\n img.type = 'image/png';\n }\n\n /* Free the memory for the previous frames */\n if (fig.imageObj.src) {\n (window.URL || window.webkitURL).revokeObjectURL(\n fig.imageObj.src\n );\n }\n\n fig.imageObj.src = (window.URL || window.webkitURL).createObjectURL(\n img\n );\n fig.updated_canvas_event();\n fig.waiting = false;\n return;\n } else if (\n typeof evt.data === 'string' &&\n evt.data.slice(0, 21) === 'data:image/png;base64'\n ) {\n fig.imageObj.src = evt.data;\n fig.updated_canvas_event();\n fig.waiting = false;\n return;\n }\n\n var msg = JSON.parse(evt.data);\n var msg_type = msg['type'];\n\n // Call the \"handle_{type}\" callback, which takes\n // the figure and JSON message as its only arguments.\n try {\n var callback = fig['handle_' + msg_type];\n } catch (e) {\n console.log(\n \"No handler for the '\" + msg_type + \"' message type: \",\n msg\n );\n return;\n }\n\n if (callback) {\n try {\n // console.log(\"Handling '\" + msg_type + \"' message: \", msg);\n callback(fig, msg);\n } catch (e) {\n console.log(\n \"Exception inside the 'handler_\" + msg_type + \"' callback:\",\n e,\n e.stack,\n msg\n );\n }\n }\n };\n};\n\n// from http://stackoverflow.com/questions/1114465/getting-mouse-location-in-canvas\nmpl.findpos = function (e) {\n //this section is from http://www.quirksmode.org/js/events_properties.html\n var targ;\n if (!e) {\n e = window.event;\n }\n if (e.target) {\n targ = e.target;\n } else if (e.srcElement) {\n targ = e.srcElement;\n }\n if (targ.nodeType === 3) {\n // defeat Safari bug\n targ = targ.parentNode;\n }\n\n // pageX,Y are the mouse positions relative to the document\n var boundingRect = targ.getBoundingClientRect();\n var x = e.pageX - (boundingRect.left + document.body.scrollLeft);\n var y = e.pageY - (boundingRect.top + document.body.scrollTop);\n\n return { x: x, y: y };\n};\n\n/*\n * return a copy of an object with only non-object keys\n * we need this to avoid circular references\n * http://stackoverflow.com/a/24161582/3208463\n */\nfunction simpleKeys(original) {\n return Object.keys(original).reduce(function (obj, key) {\n if (typeof original[key] !== 'object') {\n obj[key] = original[key];\n }\n return obj;\n }, {});\n}\n\nmpl.figure.prototype.mouse_event = function (event, name) {\n var canvas_pos = mpl.findpos(event);\n\n if (name === 'button_press') {\n this.canvas.focus();\n this.canvas_div.focus();\n }\n\n var x = canvas_pos.x * this.ratio;\n var y = canvas_pos.y * this.ratio;\n\n this.send_message(name, {\n x: x,\n y: y,\n button: event.button,\n step: event.step,\n guiEvent: simpleKeys(event),\n });\n\n /* This prevents the web browser from automatically changing to\n * the text insertion cursor when the button is pressed. We want\n * to control all of the cursor setting manually through the\n * 'cursor' event from matplotlib */\n event.preventDefault();\n return false;\n};\n\nmpl.figure.prototype._key_event_extra = function (_event, _name) {\n // Handle any extra behaviour associated with a key event\n};\n\nmpl.figure.prototype.key_event = function (event, name) {\n // Prevent repeat events\n if (name === 'key_press') {\n if (event.key === this._key) {\n return;\n } else {\n this._key = event.key;\n }\n }\n if (name === 'key_release') {\n this._key = null;\n }\n\n var value = '';\n if (event.ctrlKey && event.key !== 'Control') {\n value += 'ctrl+';\n }\n else if (event.altKey && event.key !== 'Alt') {\n value += 'alt+';\n }\n else if (event.shiftKey && event.key !== 'Shift') {\n value += 'shift+';\n }\n\n value += 'k' + event.key;\n\n this._key_event_extra(event, name);\n\n this.send_message(name, { key: value, guiEvent: simpleKeys(event) });\n return false;\n};\n\nmpl.figure.prototype.toolbar_button_onclick = function (name) {\n if (name === 'download') {\n this.handle_save(this, null);\n } else {\n this.send_message('toolbar_button', { name: name });\n }\n};\n\nmpl.figure.prototype.toolbar_button_onmouseover = function (tooltip) {\n this.message.textContent = tooltip;\n};\n\n///////////////// REMAINING CONTENT GENERATED BY embed_js.py /////////////////\n// prettier-ignore\nvar _JSXTOOLS_RESIZE_OBSERVER=function(A){var t,i=new WeakMap,n=new WeakMap,a=new WeakMap,r=new WeakMap,o=new Set;function s(e){if(!(this instanceof s))throw new TypeError(\"Constructor requires 'new' operator\");i.set(this,e)}function h(){throw new TypeError(\"Function is not a constructor\")}function c(e,t,i,n){e=0 in arguments?Number(arguments[0]):0,t=1 in arguments?Number(arguments[1]):0,i=2 in arguments?Number(arguments[2]):0,n=3 in arguments?Number(arguments[3]):0,this.right=(this.x=this.left=e)+(this.width=i),this.bottom=(this.y=this.top=t)+(this.height=n),Object.freeze(this)}function d(){t=requestAnimationFrame(d);var s=new WeakMap,p=new Set;o.forEach((function(t){r.get(t).forEach((function(i){var r=t instanceof window.SVGElement,o=a.get(t),d=r?0:parseFloat(o.paddingTop),f=r?0:parseFloat(o.paddingRight),l=r?0:parseFloat(o.paddingBottom),u=r?0:parseFloat(o.paddingLeft),g=r?0:parseFloat(o.borderTopWidth),m=r?0:parseFloat(o.borderRightWidth),w=r?0:parseFloat(o.borderBottomWidth),b=u+f,F=d+l,v=(r?0:parseFloat(o.borderLeftWidth))+m,W=g+w,y=r?0:t.offsetHeight-W-t.clientHeight,E=r?0:t.offsetWidth-v-t.clientWidth,R=b+v,z=F+W,M=r?t.width:parseFloat(o.width)-R-E,O=r?t.height:parseFloat(o.height)-z-y;if(n.has(t)){var k=n.get(t);if(k[0]===M&&k[1]===O)return}n.set(t,[M,O]);var S=Object.create(h.prototype);S.target=t,S.contentRect=new c(u,d,M,O),s.has(i)||(s.set(i,[]),p.add(i)),s.get(i).push(S)}))})),p.forEach((function(e){i.get(e).call(e,s.get(e),e)}))}return s.prototype.observe=function(i){if(i instanceof window.Element){r.has(i)||(r.set(i,new Set),o.add(i),a.set(i,window.getComputedStyle(i)));var n=r.get(i);n.has(this)||n.add(this),cancelAnimationFrame(t),t=requestAnimationFrame(d)}},s.prototype.unobserve=function(i){if(i instanceof window.Element&&r.has(i)){var n=r.get(i);n.has(this)&&(n.delete(this),n.size||(r.delete(i),o.delete(i))),n.size||r.delete(i),o.size||cancelAnimationFrame(t)}},A.DOMRectReadOnly=c,A.ResizeObserver=s,A.ResizeObserverEntry=h,A}; // eslint-disable-line\nmpl.toolbar_items = [[\"Home\", \"Reset original view\", \"fa fa-home icon-home\", \"home\"], [\"Back\", \"Back to previous view\", \"fa fa-arrow-left icon-arrow-left\", \"back\"], [\"Forward\", \"Forward to next view\", \"fa fa-arrow-right icon-arrow-right\", \"forward\"], [\"\", \"\", \"\", \"\"], [\"Pan\", \"Left button pans, Right button zooms\\nx/y fixes axis, CTRL fixes aspect\", \"fa fa-arrows icon-move\", \"pan\"], [\"Zoom\", \"Zoom to rectangle\\nx/y fixes axis, CTRL fixes aspect\", \"fa fa-square-o icon-check-empty\", \"zoom\"], [\"\", \"\", \"\", \"\"], [\"Download\", \"Download plot\", \"fa fa-floppy-o icon-save\", \"download\"]];\n\nmpl.extensions = [\"eps\", \"jpeg\", \"pgf\", \"pdf\", \"png\", \"ps\", \"raw\", \"svg\", \"tif\"];\n\nmpl.default_extension = \"png\";/* global mpl */\n\nvar comm_websocket_adapter = function (comm) {\n // Create a \"websocket\"-like object which calls the given IPython comm\n // object with the appropriate methods. Currently this is a non binary\n // socket, so there is still some room for performance tuning.\n var ws = {};\n\n ws.binaryType = comm.kernel.ws.binaryType;\n ws.readyState = comm.kernel.ws.readyState;\n function updateReadyState(_event) {\n if (comm.kernel.ws) {\n ws.readyState = comm.kernel.ws.readyState;\n } else {\n ws.readyState = 3; // Closed state.\n }\n }\n comm.kernel.ws.addEventListener('open', updateReadyState);\n comm.kernel.ws.addEventListener('close', updateReadyState);\n comm.kernel.ws.addEventListener('error', updateReadyState);\n\n ws.close = function () {\n comm.close();\n };\n ws.send = function (m) {\n //console.log('sending', m);\n comm.send(m);\n };\n // Register the callback with on_msg.\n comm.on_msg(function (msg) {\n //console.log('receiving', msg['content']['data'], msg);\n var data = msg['content']['data'];\n if (data['blob'] !== undefined) {\n data = {\n data: new Blob(msg['buffers'], { type: data['blob'] }),\n };\n }\n // Pass the mpl event to the overridden (by mpl) onmessage function.\n ws.onmessage(data);\n });\n return ws;\n};\n\nmpl.mpl_figure_comm = function (comm, msg) {\n // This is the function which gets called when the mpl process\n // starts-up an IPython Comm through the \"matplotlib\" channel.\n\n var id = msg.content.data.id;\n // Get hold of the div created by the display call when the Comm\n // socket was opened in Python.\n var element = document.getElementById(id);\n var ws_proxy = comm_websocket_adapter(comm);\n\n function ondownload(figure, _format) {\n window.open(figure.canvas.toDataURL());\n }\n\n var fig = new mpl.figure(id, ws_proxy, ondownload, element);\n\n // Call onopen now - mpl needs it, as it is assuming we've passed it a real\n // web socket which is closed, not our websocket->open comm proxy.\n ws_proxy.onopen();\n\n fig.parent_element = element;\n fig.cell_info = mpl.find_output_cell(\"<div id='\" + id + \"'></div>\");\n if (!fig.cell_info) {\n console.error('Failed to find cell for figure', id, fig);\n return;\n }\n fig.cell_info[0].output_area.element.on(\n 'cleared',\n { fig: fig },\n fig._remove_fig_handler\n );\n};\n\nmpl.figure.prototype.handle_close = function (fig, msg) {\n var width = fig.canvas.width / fig.ratio;\n fig.cell_info[0].output_area.element.off(\n 'cleared',\n fig._remove_fig_handler\n );\n fig.resizeObserverInstance.unobserve(fig.canvas_div);\n\n // Update the output cell to use the data from the current canvas.\n fig.push_to_output();\n var dataURL = fig.canvas.toDataURL();\n // Re-enable the keyboard manager in IPython - without this line, in FF,\n // the notebook keyboard shortcuts fail.\n IPython.keyboard_manager.enable();\n fig.parent_element.innerHTML =\n '<img src=\"' + dataURL + '\" width=\"' + width + '\">';\n fig.close_ws(fig, msg);\n};\n\nmpl.figure.prototype.close_ws = function (fig, msg) {\n fig.send_message('closing', msg);\n // fig.ws.close()\n};\n\nmpl.figure.prototype.push_to_output = function (_remove_interactive) {\n // Turn the data on the canvas into data in the output cell.\n var width = this.canvas.width / this.ratio;\n var dataURL = this.canvas.toDataURL();\n this.cell_info[1]['text/html'] =\n '<img src=\"' + dataURL + '\" width=\"' + width + '\">';\n};\n\nmpl.figure.prototype.updated_canvas_event = function () {\n // Tell IPython that the notebook contents must change.\n IPython.notebook.set_dirty(true);\n this.send_message('ack', {});\n var fig = this;\n // Wait a second, then push the new image to the DOM so\n // that it is saved nicely (might be nice to debounce this).\n setTimeout(function () {\n fig.push_to_output();\n }, 1000);\n};\n\nmpl.figure.prototype._init_toolbar = function () {\n var fig = this;\n\n var toolbar = document.createElement('div');\n toolbar.classList = 'btn-toolbar';\n this.root.appendChild(toolbar);\n\n function on_click_closure(name) {\n return function (_event) {\n return fig.toolbar_button_onclick(name);\n };\n }\n\n function on_mouseover_closure(tooltip) {\n return function (event) {\n if (!event.currentTarget.disabled) {\n return fig.toolbar_button_onmouseover(tooltip);\n }\n };\n }\n\n fig.buttons = {};\n var buttonGroup = document.createElement('div');\n buttonGroup.classList = 'btn-group';\n var button;\n for (var toolbar_ind in mpl.toolbar_items) {\n var name = mpl.toolbar_items[toolbar_ind][0];\n var tooltip = mpl.toolbar_items[toolbar_ind][1];\n var image = mpl.toolbar_items[toolbar_ind][2];\n var method_name = mpl.toolbar_items[toolbar_ind][3];\n\n if (!name) {\n /* Instead of a spacer, we start a new button group. */\n if (buttonGroup.hasChildNodes()) {\n toolbar.appendChild(buttonGroup);\n }\n buttonGroup = document.createElement('div');\n buttonGroup.classList = 'btn-group';\n continue;\n }\n\n button = fig.buttons[name] = document.createElement('button');\n button.classList = 'btn btn-default';\n button.href = '#';\n button.title = name;\n button.innerHTML = '<i class=\"fa ' + image + ' fa-lg\"></i>';\n button.addEventListener('click', on_click_closure(method_name));\n button.addEventListener('mouseover', on_mouseover_closure(tooltip));\n buttonGroup.appendChild(button);\n }\n\n if (buttonGroup.hasChildNodes()) {\n toolbar.appendChild(buttonGroup);\n }\n\n // Add the status bar.\n var status_bar = document.createElement('span');\n status_bar.classList = 'mpl-message pull-right';\n toolbar.appendChild(status_bar);\n this.message = status_bar;\n\n // Add the close button to the window.\n var buttongrp = document.createElement('div');\n buttongrp.classList = 'btn-group inline pull-right';\n button = document.createElement('button');\n button.classList = 'btn btn-mini btn-primary';\n button.href = '#';\n button.title = 'Stop Interaction';\n button.innerHTML = '<i class=\"fa fa-power-off icon-remove icon-large\"></i>';\n button.addEventListener('click', function (_evt) {\n fig.handle_close(fig, {});\n });\n button.addEventListener(\n 'mouseover',\n on_mouseover_closure('Stop Interaction')\n );\n buttongrp.appendChild(button);\n var titlebar = this.root.querySelector('.ui-dialog-titlebar');\n titlebar.insertBefore(buttongrp, titlebar.firstChild);\n};\n\nmpl.figure.prototype._remove_fig_handler = function (event) {\n var fig = event.data.fig;\n if (event.target !== this) {\n // Ignore bubbled events from children.\n return;\n }\n fig.close_ws(fig, {});\n};\n\nmpl.figure.prototype._root_extra_style = function (el) {\n el.style.boxSizing = 'content-box'; // override notebook setting of border-box.\n};\n\nmpl.figure.prototype._canvas_extra_style = function (el) {\n // this is important to make the div 'focusable\n el.setAttribute('tabindex', 0);\n // reach out to IPython and tell the keyboard manager to turn it's self\n // off when our div gets focus\n\n // location in version 3\n if (IPython.notebook.keyboard_manager) {\n IPython.notebook.keyboard_manager.register_events(el);\n } else {\n // location in version 2\n IPython.keyboard_manager.register_events(el);\n }\n};\n\nmpl.figure.prototype._key_event_extra = function (event, _name) {\n var manager = IPython.notebook.keyboard_manager;\n if (!manager) {\n manager = IPython.keyboard_manager;\n }\n\n // Check for shift+enter\n if (event.shiftKey && event.which === 13) {\n this.canvas_div.blur();\n // select the cell after this one\n var index = IPython.notebook.find_cell_index(this.cell_info[0]);\n IPython.notebook.select(index + 1);\n }\n};\n\nmpl.figure.prototype.handle_save = function (fig, _msg) {\n fig.ondownload(fig, null);\n};\n\nmpl.find_output_cell = function (html_output) {\n // Return the cell and output element which can be found *uniquely* in the notebook.\n // Note - this is a bit hacky, but it is done because the \"notebook_saving.Notebook\"\n // IPython event is triggered only after the cells have been serialised, which for\n // our purposes (turning an active figure into a static one), is too late.\n var cells = IPython.notebook.get_cells();\n var ncells = cells.length;\n for (var i = 0; i < ncells; i++) {\n var cell = cells[i];\n if (cell.cell_type === 'code') {\n for (var j = 0; j < cell.output_area.outputs.length; j++) {\n var data = cell.output_area.outputs[j];\n if (data.data) {\n // IPython >= 3 moved mimebundle to data attribute of output\n data = data.data;\n }\n if (data['text/html'] === html_output) {\n return [cell, data, j];\n }\n }\n }\n }\n};\n\n// Register the function which deals with the matplotlib target/channel.\n// The kernel may be null if the page has been refreshed.\nif (IPython.notebook.kernel !== null) {\n IPython.notebook.kernel.comm_manager.register_target(\n 'matplotlib',\n mpl.mpl_figure_comm\n );\n}\n",
"text/plain": [
"<IPython.core.display.Javascript object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<img src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAA4QAAAEsCAYAAACbnn2RAAAAAXNSR0IArs4c6QAAFK1JREFUeF7t10EBAAAIAjHpX9ogNxsw/LBzBAgQIECAAAECBAgQIJAUWDK10AQIECBAgAABAgQIECBwBqEnIECAAAECBAgQIECAQFTAIIwWLzYBAgQIECBAgAABAgQMQj9AgAABAgQIECBAgACBqIBBGC1ebAIECBAgQIAAAQIECBiEfoAAAQIECBAgQIAAAQJRAYMwWrzYBAgQIECAAAECBAgQMAj9AAECBAgQIECAAAECBKICBmG0eLEJECBAgAABAgQIECBgEPoBAgQIECBAgAABAgQIRAUMwmjxYhMgQIAAAQIECBAgQMAg9AMECBAgQIAAAQIECBCIChiE0eLFJkCAAAECBAgQIECAgEHoBwgQIECAAAECBAgQIBAVMAijxYtNgAABAgQIECBAgAABg9APECBAgAABAgQIECBAICpgEEaLF5sAAQIECBAgQIAAAQIGoR8gQIAAAQIECBAgQIBAVMAgjBYvNgECBAgQIECAAAECBAxCP0CAAAECBAgQIECAAIGogEEYLV5sAgQIECBAgAABAgQIGIR+gAABAgQIECBAgAABAlEBgzBavNgECBAgQIAAAQIECBAwCP0AAQIECBAgQIAAAQIEogIGYbR4sQkQIECAAAECBAgQIGAQ+gECBAgQIECAAAECBAhEBQzCaPFiEyBAgAABAgQIECBAwCD0AwQIECBAgAABAgQIEIgKGITR4sUmQIAAAQIECBAgQICAQegHCBAgQIAAAQIECBAgEBUwCKPFi02AAAECBAgQIECAAAGD0A8QIECAAAECBAgQIEAgKmAQRosXmwABAgQIECBAgAABAgahHyBAgAABAgQIECBAgEBUwCCMFi82AQIECBAgQIAAAQIEDEI/QIAAAQIECBAgQIAAgaiAQRgtXmwCBAgQIECAAAECBAgYhH6AAAECBAgQIECAAAECUQGDMFq82AQIECBAgAABAgQIEDAI/QABAgQIECBAgAABAgSiAgZhtHixCRAgQIAAAQIECBAgYBD6AQIECBAgQIAAAQIECEQFDMJo8WITIECAAAECBAgQIEDAIPQDBAgQIECAAAECBAgQiAoYhNHixSZAgAABAgQIECBAgIBB6AcIECBAgAABAgQIECAQFTAIo8WLTYAAAQIECBAgQIAAAYPQDxAgQIAAAQIECBAgQCAqYBBGixebAAECBAgQIECAAAECBqEfIECAAAECBAgQIECAQFTAIIwWLzYBAgQIECBAgAABAgQMQj9AgAABAgQIECBAgACBqIBBGC1ebAIECBAgQIAAAQIECBiEfoAAAQIECBAgQIAAAQJRAYMwWrzYBAgQIECAAAECBAgQMAj9AAECBAgQIECAAAECBKICBmG0eLEJECBAgAABAgQIECBgEPoBAgQIECBAgAABAgQIRAUMwmjxYhMgQIAAAQIECBAgQMAg9AMECBAgQIAAAQIECBCIChiE0eLFJkCAAAECBAgQIECAgEHoBwgQIECAAAECBAgQIBAVMAijxYtNgAABAgQIECBAgAABg9APECBAgAABAgQIECBAICpgEEaLF5sAAQIECBAgQIAAAQIGoR8gQIAAAQIECBAgQIBAVMAgjBYvNgECBAgQIECAAAECBAxCP0CAAAECBAgQIECAAIGogEEYLV5sAgQIECBAgAABAgQIGIR+gAABAgQIECBAgAABAlEBgzBavNgECBAgQIAAAQIECBAwCP0AAQIECBAgQIAAAQIEogIGYbR4sQkQIECAAAECBAgQIGAQ+gECBAgQIECAAAECBAhEBQzCaPFiEyBAgAABAgQIECBAwCD0AwQIECBAgAABAgQIEIgKGITR4sUmQIAAAQIECBAgQICAQegHCBAgQIAAAQIECBAgEBUwCKPFi02AAAECBAgQIECAAAGD0A8QIECAAAECBAgQIEAgKmAQRosXmwABAgQIECBAgAABAgahHyBAgAABAgQIECBAgEBUwCCMFi82AQIECBAgQIAAAQIEDEI/QIAAAQIECBAgQIAAgaiAQRgtXmwCBAgQIECAAAECBAgYhH6AAAECBAgQIECAAAECUQGDMFq82AQIECBAgAABAgQIEDAI/QABAgQIECBAgAABAgSiAgZhtHixCRAgQIAAAQIECBAgYBD6AQIECBAgQIAAAQIECEQFDMJo8WITIECAAAECBAgQIEDAIPQDBAgQIECAAAECBAgQiAoYhNHixSZAgAABAgQIECBAgIBB6AcIECBAgAABAgQIECAQFTAIo8WLTYAAAQIECBAgQIAAAYPQDxAgQIAAAQIECBAgQCAqYBBGixebAAECBAgQIECAAAECBqEfIECAAAECBAgQIECAQFTAIIwWLzYBAgQIECBAgAABAgQMQj9AgAABAgQIECBAgACBqIBBGC1ebAIECBAgQIAAAQIECBiEfoAAAQIECBAgQIAAAQJRAYMwWrzYBAgQIECAAAECBAgQMAj9AAECBAgQIECAAAECBKICBmG0eLEJECBAgAABAgQIECBgEPoBAgQIECBAgAABAgQIRAUMwmjxYhMgQIAAAQIECBAgQMAg9AMECBAgQIAAAQIECBCIChiE0eLFJkCAAAECBAgQIECAgEHoBwgQIECAAAECBAgQIBAVMAijxYtNgAABAgQIECBAgAABg9APECBAgAABAgQIECBAICpgEEaLF5sAAQIECBAgQIAAAQIGoR8gQIAAAQIECBAgQIBAVMAgjBYvNgECBAgQIECAAAECBAxCP0CAAAECBAgQIECAAIGogEEYLV5sAgQIECBAgAABAgQIGIR+gAABAgQIECBAgAABAlEBgzBavNgECBAgQIAAAQIECBAwCP0AAQIECBAgQIAAAQIEogIGYbR4sQkQIECAAAECBAgQIGAQ+gECBAgQIECAAAECBAhEBQzCaPFiEyBAgAABAgQIECBAwCD0AwQIECBAgAABAgQIEIgKGITR4sUmQIAAAQIECBAgQICAQegHCBAgQIAAAQIECBAgEBUwCKPFi02AAAECBAgQIECAAAGD0A8QIECAAAECBAgQIEAgKmAQRosXmwABAgQIECBAgAABAgahHyBAgAABAgQIECBAgEBUwCCMFi82AQIECBAgQIAAAQIEDEI/QIAAAQIECBAgQIAAgaiAQRgtXmwCBAgQIECAAAECBAgYhH6AAAECBAgQIECAAAECUQGDMFq82AQIECBAgAABAgQIEDAI/QABAgQIECBAgAABAgSiAgZhtHixCRAgQIAAAQIECBAgYBD6AQIECBAgQIAAAQIECEQFDMJo8WITIECAAAECBAgQIEDAIPQDBAgQIECAAAECBAgQiAoYhNHixSZAgAABAgQIECBAgIBB6AcIECBAgAABAgQIECAQFTAIo8WLTYAAAQIECBAgQIAAAYPQDxAgQIAAAQIECBAgQCAqYBBGixebAAECBAgQIECAAAECBqEfIECAAAECBAgQIECAQFTAIIwWLzYBAgQIECBAgAABAgQMQj9AgAABAgQIECBAgACBqIBBGC1ebAIECBAgQIAAAQIECBiEfoAAAQIECBAgQIAAAQJRAYMwWrzYBAgQIECAAAECBAgQMAj9AAECBAgQIECAAAECBKICBmG0eLEJECBAgAABAgQIECBgEPoBAgQIECBAgAABAgQIRAUMwmjxYhMgQIAAAQIECBAgQMAg9AMECBAgQIAAAQIECBCIChiE0eLFJkCAAAECBAgQIECAgEHoBwgQIECAAAECBAgQIBAVMAijxYtNgAABAgQIECBAgAABg9APECBAgAABAgQIECBAICpgEEaLF5sAAQIECBAgQIAAAQIGoR8gQIAAAQIECBAgQIBAVMAgjBYvNgECBAgQIECAAAECBAxCP0CAAAECBAgQIECAAIGogEEYLV5sAgQIECBAgAABAgQIGIR+gAABAgQIECBAgAABAlEBgzBavNgECBAgQIAAAQIECBAwCP0AAQIECBAgQIAAAQIEogIGYbR4sQkQIECAAAECBAgQIGAQ+gECBAgQIECAAAECBAhEBQzCaPFiEyBAgAABAgQIECBAwCD0AwQIECBAgAABAgQIEIgKGITR4sUmQIAAAQIECBAgQICAQegHCBAgQIAAAQIECBAgEBUwCKPFi02AAAECBAgQIECAAAGD0A8QIECAAAECBAgQIEAgKmAQRosXmwABAgQIECBAgAABAgahHyBAgAABAgQIECBAgEBUwCCMFi82AQIECBAgQIAAAQIEDEI/QIAAAQIECBAgQIAAgaiAQRgtXmwCBAgQIECAAAECBAgYhH6AAAECBAgQIECAAAECUQGDMFq82AQIECBAgAABAgQIEDAI/QABAgQIECBAgAABAgSiAgZhtHixCRAgQIAAAQIECBAgYBD6AQIECBAgQIAAAQIECEQFDMJo8WITIECAAAECBAgQIEDAIPQDBAgQIECAAAECBAgQiAoYhNHixSZAgAABAgQIECBAgIBB6AcIECBAgAABAgQIECAQFTAIo8WLTYAAAQIECBAgQIAAAYPQDxAgQIAAAQIECBAgQCAqYBBGixebAAECBAgQIECAAAECBqEfIECAAAECBAgQIECAQFTAIIwWLzYBAgQIECBAgAABAgQMQj9AgAABAgQIECBAgACBqIBBGC1ebAIECBAgQIAAAQIECBiEfoAAAQIECBAgQIAAAQJRAYMwWrzYBAgQIECAAAECBAgQMAj9AAECBAgQIECAAAECBKICBmG0eLEJECBAgAABAgQIECBgEPoBAgQIECBAgAABAgQIRAUMwmjxYhMgQIAAAQIECBAgQMAg9AMECBAgQIAAAQIECBCIChiE0eLFJkCAAAECBAgQIECAgEHoBwgQIECAAAECBAgQIBAVMAijxYtNgAABAgQIECBAgAABg9APECBAgAABAgQIECBAICpgEEaLF5sAAQIECBAgQIAAAQIGoR8gQIAAAQIECBAgQIBAVMAgjBYvNgECBAgQIECAAAECBAxCP0CAAAECBAgQIECAAIGogEEYLV5sAgQIECBAgAABAgQIGIR+gAABAgQIECBAgAABAlEBgzBavNgECBAgQIAAAQIECBAwCP0AAQIECBAgQIAAAQIEogIGYbR4sQkQIECAAAECBAgQIGAQ+gECBAgQIECAAAECBAhEBQzCaPFiEyBAgAABAgQIECBAwCD0AwQIECBAgAABAgQIEIgKGITR4sUmQIAAAQIECBAgQICAQegHCBAgQIAAAQIECBAgEBUwCKPFi02AAAECBAgQIECAAAGD0A8QIECAAAECBAgQIEAgKmAQRosXmwABAgQIECBAgAABAgahHyBAgAABAgQIECBAgEBUwCCMFi82AQIECBAgQIAAAQIEDEI/QIAAAQIECBAgQIAAgaiAQRgtXmwCBAgQIECAAAECBAgYhH6AAAECBAgQIECAAAECUQGDMFq82AQIECBAgAABAgQIEDAI/QABAgQIECBAgAABAgSiAgZhtHixCRAgQIAAAQIECBAgYBD6AQIECBAgQIAAAQIECEQFDMJo8WITIECAAAECBAgQIEDAIPQDBAgQIECAAAECBAgQiAoYhNHixSZAgAABAgQIECBAgIBB6AcIECBAgAABAgQIECAQFTAIo8WLTYAAAQIECBAgQIAAAYPQDxAgQIAAAQIECBAgQCAqYBBGixebAAECBAgQIECAAAECBqEfIECAAAECBAgQIECAQFTAIIwWLzYBAgQIECBAgAABAgQMQj9AgAABAgQIECBAgACBqIBBGC1ebAIECBAgQIAAAQIECBiEfoAAAQIECBAgQIAAAQJRAYMwWrzYBAgQIECAAAECBAgQMAj9AAECBAgQIECAAAECBKICBmG0eLEJECBAgAABAgQIECBgEPoBAgQIECBAgAABAgQIRAUMwmjxYhMgQIAAAQIECBAgQMAg9AMECBAgQIAAAQIECBCIChiE0eLFJkCAAAECBAgQIECAgEHoBwgQIECAAAECBAgQIBAVMAijxYtNgAABAgQIECBAgAABg9APECBAgAABAgQIECBAICpgEEaLF5sAAQIECBAgQIAAAQIGoR8gQIAAAQIECBAgQIBAVMAgjBYvNgECBAgQIECAAAECBAxCP0CAAAECBAgQIECAAIGogEEYLV5sAgQIECBAgAABAgQIGIR+gAABAgQIECBAgAABAlEBgzBavNgECBAgQIAAAQIECBAwCP0AAQIECBAgQIAAAQIEogIGYbR4sQkQIECAAAECBAgQIGAQ+gECBAgQIECAAAECBAhEBQzCaPFiEyBAgAABAgQIECBAwCD0AwQIECBAgAABAgQIEIgKGITR4sUmQIAAAQIECBAgQICAQegHCBAgQIAAAQIECBAgEBUwCKPFi02AAAECBAgQIECAAAGD0A8QIECAAAECBAgQIEAgKmAQRosXmwABAgQIECBAgAABAgahHyBAgAABAgQIECBAgEBUwCCMFi82AQIECBAgQIAAAQIEDEI/QIAAAQIECBAgQIAAgaiAQRgtXmwCBAgQIECAAAECBAgYhH6AAAECBAgQIECAAAECUQGDMFq82AQIECBAgAABAgQIEDAI/QABAgQIECBAgAABAgSiAgZhtHixCRAgQIAAAQIECBAgYBD6AQIECBAgQIAAAQIECEQFDMJo8WITIECAAAECBAgQIEDAIPQDBAgQIECAAAECBAgQiAoYhNHixSZAgAABAgQIECBAgIBB6AcIECBAgAABAgQIECAQFTAIo8WLTYAAAQIECBAgQIAAAYPQDxAgQIAAAQIECBAgQCAqYBBGixebAAECBAgQIECAAAECBqEfIECAAAECBAgQIECAQFTAIIwWLzYBAgQIECBAgAABAgQMQj9AgAABAgQIECBAgACBqIBBGC1ebAIECBAgQIAAAQIECBiEfoAAAQIECBAgQIAAAQJRAYMwWrzYBAgQIECAAAECBAgQMAj9AAECBAgQIECAAAECBKICBmG0eLEJECBAgAABAgQIECBgEPoBAgQIECBAgAABAgQIRAUMwmjxYhMgQIAAAQIECBAgQOAB6HkBLXgQOl0AAAAASUVORK5CYII=\" width=\"900\">"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"x = np.linspace(-2, 2, 101)\n",
"plot_loss_functions(\n",
" suptitle = 'Common loss functions for regression',\n",
" functions = [np.abs(x), np.power(x, 2)],\n",
" ylabels = ['$\\mathcal{L}_{abs}}$ (absolute loss)',\n",
" '$\\mathcal{L}_{sq}$ (squared loss)'],\n",
" xlabel = '$y - f(x_i)$')"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"### Loss functions for classification\n",
"\n",
"Let's consider binary classification for a moment. In this case we have two classes, numbered 0 and 1. The output of the network $f_\\theta(x_i)\\in [0,1]$ essentially defines the probability of choosing the class 1.\n",
"\n",
"**0-1 loss**\n",
"\n",
"0-1 loss is the same as calculating accuracy of the model - we compute the number of correct classifications:\n",
"\n",
"$$\\mathcal{L}_{0-1} = \\sum_{i=1}^n l_i \\quad l_i = \\begin{cases}\n",
" 0 & (f(x_i)<0.5 \\land y_i=0) \\lor (f(x_i)<0.5 \\land y_i=1) \\\\\n",
" 1 & \\mathrm{ otherwise}\n",
" \\end{cases} \\\\\n",
"$$\n",
"\n",
"However, accuracy itself does not show how far are we from the right classification. It could be that we missed the correct class just by a little bit, and that is in a way \"better\" (in a sense that we need to correct weights much less) than missing significantly. Thus, more often logistic loss is used, which takes this into account.\n",
"\n",
"**Logistic Loss**\n",
"\n",
"$$\\mathcal{L}_{log} = \\sum_{i=1}^n -y\\log(f_{\\theta}(x_i)) - (1-y)\\log(1-f_\\theta(x_i))$$"
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {},
"outputs": [],
"source": [
"x = np.linspace(0,1,100)\n",
"def zero_one(d):\n",
" if d < 0.5:\n",
" return 0\n",
" return 1\n",
"zero_one_v = np.vectorize(zero_one)\n",
"\n",
"def logistic_loss(fx):\n",
" # assumes y == 1\n",
" return -np.log(fx)"
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"C:\\Users\\dmitryso\\AppData\\Local\\Temp/ipykernel_55820/331859503.py:10: RuntimeWarning: divide by zero encountered in log\n",
" return -np.log(fx)\n"
]
},
{
"data": {
"application/javascript": "/* Put everything inside the global mpl namespace */\n/* global mpl */\nwindow.mpl = {};\n\nmpl.get_websocket_type = function () {\n if (typeof WebSocket !== 'undefined') {\n return WebSocket;\n } else if (typeof MozWebSocket !== 'undefined') {\n return MozWebSocket;\n } else {\n alert(\n 'Your browser does not have WebSocket support. ' +\n 'Please try Chrome, Safari or Firefox ≥ 6. ' +\n 'Firefox 4 and 5 are also supported but you ' +\n 'have to enable WebSockets in about:config.'\n );\n }\n};\n\nmpl.figure = function (figure_id, websocket, ondownload, parent_element) {\n this.id = figure_id;\n\n this.ws = websocket;\n\n this.supports_binary = this.ws.binaryType !== undefined;\n\n if (!this.supports_binary) {\n var warnings = document.getElementById('mpl-warnings');\n if (warnings) {\n warnings.style.display = 'block';\n warnings.textContent =\n 'This browser does not support binary websocket messages. ' +\n 'Performance may be slow.';\n }\n }\n\n this.imageObj = new Image();\n\n this.context = undefined;\n this.message = undefined;\n this.canvas = undefined;\n this.rubberband_canvas = undefined;\n this.rubberband_context = undefined;\n this.format_dropdown = undefined;\n\n this.image_mode = 'full';\n\n this.root = document.createElement('div');\n this.root.setAttribute('style', 'display: inline-block');\n this._root_extra_style(this.root);\n\n parent_element.appendChild(this.root);\n\n this._init_header(this);\n this._init_canvas(this);\n this._init_toolbar(this);\n\n var fig = this;\n\n this.waiting = false;\n\n this.ws.onopen = function () {\n fig.send_message('supports_binary', { value: fig.supports_binary });\n fig.send_message('send_image_mode', {});\n if (fig.ratio !== 1) {\n fig.send_message('set_dpi_ratio', { dpi_ratio: fig.ratio });\n }\n fig.send_message('refresh', {});\n };\n\n this.imageObj.onload = function () {\n if (fig.image_mode === 'full') {\n // Full images could contain transparency (where diff images\n // almost always do), so we need to clear the canvas so that\n // there is no ghosting.\n fig.context.clearRect(0, 0, fig.canvas.width, fig.canvas.height);\n }\n fig.context.drawImage(fig.imageObj, 0, 0);\n };\n\n this.imageObj.onunload = function () {\n fig.ws.close();\n };\n\n this.ws.onmessage = this._make_on_message_function(this);\n\n this.ondownload = ondownload;\n};\n\nmpl.figure.prototype._init_header = function () {\n var titlebar = document.createElement('div');\n titlebar.classList =\n 'ui-dialog-titlebar ui-widget-header ui-corner-all ui-helper-clearfix';\n var titletext = document.createElement('div');\n titletext.classList = 'ui-dialog-title';\n titletext.setAttribute(\n 'style',\n 'width: 100%; text-align: center; padding: 3px;'\n );\n titlebar.appendChild(titletext);\n this.root.appendChild(titlebar);\n this.header = titletext;\n};\n\nmpl.figure.prototype._canvas_extra_style = function (_canvas_div) {};\n\nmpl.figure.prototype._root_extra_style = function (_canvas_div) {};\n\nmpl.figure.prototype._init_canvas = function () {\n var fig = this;\n\n var canvas_div = (this.canvas_div = document.createElement('div'));\n canvas_div.setAttribute(\n 'style',\n 'border: 1px solid #ddd;' +\n 'box-sizing: content-box;' +\n 'clear: both;' +\n 'min-height: 1px;' +\n 'min-width: 1px;' +\n 'outline: 0;' +\n 'overflow: hidden;' +\n 'position: relative;' +\n 'resize: both;'\n );\n\n function on_keyboard_event_closure(name) {\n return function (event) {\n return fig.key_event(event, name);\n };\n }\n\n canvas_div.addEventListener(\n 'keydown',\n on_keyboard_event_closure('key_press')\n );\n canvas_div.addEventListener(\n 'keyup',\n on_keyboard_event_closure('key_release')\n );\n\n this._canvas_extra_style(canvas_div);\n this.root.appendChild(canvas_div);\n\n var canvas = (this.canvas = document.createElement('canvas'));\n canvas.classList.add('mpl-canvas');\n canvas.setAttribute('style', 'box-sizing: content-box;');\n\n this.context = canvas.getContext('2d');\n\n var backingStore =\n this.context.backingStorePixelRatio ||\n this.context.webkitBackingStorePixelRatio ||\n this.context.mozBackingStorePixelRatio ||\n this.context.msBackingStorePixelRatio ||\n this.context.oBackingStorePixelRatio ||\n this.context.backingStorePixelRatio ||\n 1;\n\n this.ratio = (window.devicePixelRatio || 1) / backingStore;\n\n var rubberband_canvas = (this.rubberband_canvas = document.createElement(\n 'canvas'\n ));\n rubberband_canvas.setAttribute(\n 'style',\n 'box-sizing: content-box; position: absolute; left: 0; top: 0; z-index: 1;'\n );\n\n // Apply a ponyfill if ResizeObserver is not implemented by browser.\n if (this.ResizeObserver === undefined) {\n if (window.ResizeObserver !== undefined) {\n this.ResizeObserver = window.ResizeObserver;\n } else {\n var obs = _JSXTOOLS_RESIZE_OBSERVER({});\n this.ResizeObserver = obs.ResizeObserver;\n }\n }\n\n this.resizeObserverInstance = new this.ResizeObserver(function (entries) {\n var nentries = entries.length;\n for (var i = 0; i < nentries; i++) {\n var entry = entries[i];\n var width, height;\n if (entry.contentBoxSize) {\n if (entry.contentBoxSize instanceof Array) {\n // Chrome 84 implements new version of spec.\n width = entry.contentBoxSize[0].inlineSize;\n height = entry.contentBoxSize[0].blockSize;\n } else {\n // Firefox implements old version of spec.\n width = entry.contentBoxSize.inlineSize;\n height = entry.contentBoxSize.blockSize;\n }\n } else {\n // Chrome <84 implements even older version of spec.\n width = entry.contentRect.width;\n height = entry.contentRect.height;\n }\n\n // Keep the size of the canvas and rubber band canvas in sync with\n // the canvas container.\n if (entry.devicePixelContentBoxSize) {\n // Chrome 84 implements new version of spec.\n canvas.setAttribute(\n 'width',\n entry.devicePixelContentBoxSize[0].inlineSize\n );\n canvas.setAttribute(\n 'height',\n entry.devicePixelContentBoxSize[0].blockSize\n );\n } else {\n canvas.setAttribute('width', width * fig.ratio);\n canvas.setAttribute('height', height * fig.ratio);\n }\n canvas.setAttribute(\n 'style',\n 'width: ' + width + 'px; height: ' + height + 'px;'\n );\n\n rubberband_canvas.setAttribute('width', width);\n rubberband_canvas.setAttribute('height', height);\n\n // And update the size in Python. We ignore the initial 0/0 size\n // that occurs as the element is placed into the DOM, which should\n // otherwise not happen due to the minimum size styling.\n if (fig.ws.readyState == 1 && width != 0 && height != 0) {\n fig.request_resize(width, height);\n }\n }\n });\n this.resizeObserverInstance.observe(canvas_div);\n\n function on_mouse_event_closure(name) {\n return function (event) {\n return fig.mouse_event(event, name);\n };\n }\n\n rubberband_canvas.addEventListener(\n 'mousedown',\n on_mouse_event_closure('button_press')\n );\n rubberband_canvas.addEventListener(\n 'mouseup',\n on_mouse_event_closure('button_release')\n );\n rubberband_canvas.addEventListener(\n 'dblclick',\n on_mouse_event_closure('dblclick')\n );\n // Throttle sequential mouse events to 1 every 20ms.\n rubberband_canvas.addEventListener(\n 'mousemove',\n on_mouse_event_closure('motion_notify')\n );\n\n rubberband_canvas.addEventListener(\n 'mouseenter',\n on_mouse_event_closure('figure_enter')\n );\n rubberband_canvas.addEventListener(\n 'mouseleave',\n on_mouse_event_closure('figure_leave')\n );\n\n canvas_div.addEventListener('wheel', function (event) {\n if (event.deltaY < 0) {\n event.step = 1;\n } else {\n event.step = -1;\n }\n on_mouse_event_closure('scroll')(event);\n });\n\n canvas_div.appendChild(canvas);\n canvas_div.appendChild(rubberband_canvas);\n\n this.rubberband_context = rubberband_canvas.getContext('2d');\n this.rubberband_context.strokeStyle = '#000000';\n\n this._resize_canvas = function (width, height, forward) {\n if (forward) {\n canvas_div.style.width = width + 'px';\n canvas_div.style.height = height + 'px';\n }\n };\n\n // Disable right mouse context menu.\n this.rubberband_canvas.addEventListener('contextmenu', function (_e) {\n event.preventDefault();\n return false;\n });\n\n function set_focus() {\n canvas.focus();\n canvas_div.focus();\n }\n\n window.setTimeout(set_focus, 100);\n};\n\nmpl.figure.prototype._init_toolbar = function () {\n var fig = this;\n\n var toolbar = document.createElement('div');\n toolbar.classList = 'mpl-toolbar';\n this.root.appendChild(toolbar);\n\n function on_click_closure(name) {\n return function (_event) {\n return fig.toolbar_button_onclick(name);\n };\n }\n\n function on_mouseover_closure(tooltip) {\n return function (event) {\n if (!event.currentTarget.disabled) {\n return fig.toolbar_button_onmouseover(tooltip);\n }\n };\n }\n\n fig.buttons = {};\n var buttonGroup = document.createElement('div');\n buttonGroup.classList = 'mpl-button-group';\n for (var toolbar_ind in mpl.toolbar_items) {\n var name = mpl.toolbar_items[toolbar_ind][0];\n var tooltip = mpl.toolbar_items[toolbar_ind][1];\n var image = mpl.toolbar_items[toolbar_ind][2];\n var method_name = mpl.toolbar_items[toolbar_ind][3];\n\n if (!name) {\n /* Instead of a spacer, we start a new button group. */\n if (buttonGroup.hasChildNodes()) {\n toolbar.appendChild(buttonGroup);\n }\n buttonGroup = document.createElement('div');\n buttonGroup.classList = 'mpl-button-group';\n continue;\n }\n\n var button = (fig.buttons[name] = document.createElement('button'));\n button.classList = 'mpl-widget';\n button.setAttribute('role', 'button');\n button.setAttribute('aria-disabled', 'false');\n button.addEventListener('click', on_click_closure(method_name));\n button.addEventListener('mouseover', on_mouseover_closure(tooltip));\n\n var icon_img = document.createElement('img');\n icon_img.src = '_images/' + image + '.png';\n icon_img.srcset = '_images/' + image + '_large.png 2x';\n icon_img.alt = tooltip;\n button.appendChild(icon_img);\n\n buttonGroup.appendChild(button);\n }\n\n if (buttonGroup.hasChildNodes()) {\n toolbar.appendChild(buttonGroup);\n }\n\n var fmt_picker = document.createElement('select');\n fmt_picker.classList = 'mpl-widget';\n toolbar.appendChild(fmt_picker);\n this.format_dropdown = fmt_picker;\n\n for (var ind in mpl.extensions) {\n var fmt = mpl.extensions[ind];\n var option = document.createElement('option');\n option.selected = fmt === mpl.default_extension;\n option.innerHTML = fmt;\n fmt_picker.appendChild(option);\n }\n\n var status_bar = document.createElement('span');\n status_bar.classList = 'mpl-message';\n toolbar.appendChild(status_bar);\n this.message = status_bar;\n};\n\nmpl.figure.prototype.request_resize = function (x_pixels, y_pixels) {\n // Request matplotlib to resize the figure. Matplotlib will then trigger a resize in the client,\n // which will in turn request a refresh of the image.\n this.send_message('resize', { width: x_pixels, height: y_pixels });\n};\n\nmpl.figure.prototype.send_message = function (type, properties) {\n properties['type'] = type;\n properties['figure_id'] = this.id;\n this.ws.send(JSON.stringify(properties));\n};\n\nmpl.figure.prototype.send_draw_message = function () {\n if (!this.waiting) {\n this.waiting = true;\n this.ws.send(JSON.stringify({ type: 'draw', figure_id: this.id }));\n }\n};\n\nmpl.figure.prototype.handle_save = function (fig, _msg) {\n var format_dropdown = fig.format_dropdown;\n var format = format_dropdown.options[format_dropdown.selectedIndex].value;\n fig.ondownload(fig, format);\n};\n\nmpl.figure.prototype.handle_resize = function (fig, msg) {\n var size = msg['size'];\n if (size[0] !== fig.canvas.width || size[1] !== fig.canvas.height) {\n fig._resize_canvas(size[0], size[1], msg['forward']);\n fig.send_message('refresh', {});\n }\n};\n\nmpl.figure.prototype.handle_rubberband = function (fig, msg) {\n var x0 = msg['x0'] / fig.ratio;\n var y0 = (fig.canvas.height - msg['y0']) / fig.ratio;\n var x1 = msg['x1'] / fig.ratio;\n var y1 = (fig.canvas.height - msg['y1']) / fig.ratio;\n x0 = Math.floor(x0) + 0.5;\n y0 = Math.floor(y0) + 0.5;\n x1 = Math.floor(x1) + 0.5;\n y1 = Math.floor(y1) + 0.5;\n var min_x = Math.min(x0, x1);\n var min_y = Math.min(y0, y1);\n var width = Math.abs(x1 - x0);\n var height = Math.abs(y1 - y0);\n\n fig.rubberband_context.clearRect(\n 0,\n 0,\n fig.canvas.width / fig.ratio,\n fig.canvas.height / fig.ratio\n );\n\n fig.rubberband_context.strokeRect(min_x, min_y, width, height);\n};\n\nmpl.figure.prototype.handle_figure_label = function (fig, msg) {\n // Updates the figure title.\n fig.header.textContent = msg['label'];\n};\n\nmpl.figure.prototype.handle_cursor = function (fig, msg) {\n var cursor = msg['cursor'];\n switch (cursor) {\n case 0:\n cursor = 'pointer';\n break;\n case 1:\n cursor = 'default';\n break;\n case 2:\n cursor = 'crosshair';\n break;\n case 3:\n cursor = 'move';\n break;\n }\n fig.rubberband_canvas.style.cursor = cursor;\n};\n\nmpl.figure.prototype.handle_message = function (fig, msg) {\n fig.message.textContent = msg['message'];\n};\n\nmpl.figure.prototype.handle_draw = function (fig, _msg) {\n // Request the server to send over a new figure.\n fig.send_draw_message();\n};\n\nmpl.figure.prototype.handle_image_mode = function (fig, msg) {\n fig.image_mode = msg['mode'];\n};\n\nmpl.figure.prototype.handle_history_buttons = function (fig, msg) {\n for (var key in msg) {\n if (!(key in fig.buttons)) {\n continue;\n }\n fig.buttons[key].disabled = !msg[key];\n fig.buttons[key].setAttribute('aria-disabled', !msg[key]);\n }\n};\n\nmpl.figure.prototype.handle_navigate_mode = function (fig, msg) {\n if (msg['mode'] === 'PAN') {\n fig.buttons['Pan'].classList.add('active');\n fig.buttons['Zoom'].classList.remove('active');\n } else if (msg['mode'] === 'ZOOM') {\n fig.buttons['Pan'].classList.remove('active');\n fig.buttons['Zoom'].classList.add('active');\n } else {\n fig.buttons['Pan'].classList.remove('active');\n fig.buttons['Zoom'].classList.remove('active');\n }\n};\n\nmpl.figure.prototype.updated_canvas_event = function () {\n // Called whenever the canvas gets updated.\n this.send_message('ack', {});\n};\n\n// A function to construct a web socket function for onmessage handling.\n// Called in the figure constructor.\nmpl.figure.prototype._make_on_message_function = function (fig) {\n return function socket_on_message(evt) {\n if (evt.data instanceof Blob) {\n var img = evt.data;\n if (img.type !== 'image/png') {\n /* FIXME: We get \"Resource interpreted as Image but\n * transferred with MIME type text/plain:\" errors on\n * Chrome. But how to set the MIME type? It doesn't seem\n * to be part of the websocket stream */\n img.type = 'image/png';\n }\n\n /* Free the memory for the previous frames */\n if (fig.imageObj.src) {\n (window.URL || window.webkitURL).revokeObjectURL(\n fig.imageObj.src\n );\n }\n\n fig.imageObj.src = (window.URL || window.webkitURL).createObjectURL(\n img\n );\n fig.updated_canvas_event();\n fig.waiting = false;\n return;\n } else if (\n typeof evt.data === 'string' &&\n evt.data.slice(0, 21) === 'data:image/png;base64'\n ) {\n fig.imageObj.src = evt.data;\n fig.updated_canvas_event();\n fig.waiting = false;\n return;\n }\n\n var msg = JSON.parse(evt.data);\n var msg_type = msg['type'];\n\n // Call the \"handle_{type}\" callback, which takes\n // the figure and JSON message as its only arguments.\n try {\n var callback = fig['handle_' + msg_type];\n } catch (e) {\n console.log(\n \"No handler for the '\" + msg_type + \"' message type: \",\n msg\n );\n return;\n }\n\n if (callback) {\n try {\n // console.log(\"Handling '\" + msg_type + \"' message: \", msg);\n callback(fig, msg);\n } catch (e) {\n console.log(\n \"Exception inside the 'handler_\" + msg_type + \"' callback:\",\n e,\n e.stack,\n msg\n );\n }\n }\n };\n};\n\n// from http://stackoverflow.com/questions/1114465/getting-mouse-location-in-canvas\nmpl.findpos = function (e) {\n //this section is from http://www.quirksmode.org/js/events_properties.html\n var targ;\n if (!e) {\n e = window.event;\n }\n if (e.target) {\n targ = e.target;\n } else if (e.srcElement) {\n targ = e.srcElement;\n }\n if (targ.nodeType === 3) {\n // defeat Safari bug\n targ = targ.parentNode;\n }\n\n // pageX,Y are the mouse positions relative to the document\n var boundingRect = targ.getBoundingClientRect();\n var x = e.pageX - (boundingRect.left + document.body.scrollLeft);\n var y = e.pageY - (boundingRect.top + document.body.scrollTop);\n\n return { x: x, y: y };\n};\n\n/*\n * return a copy of an object with only non-object keys\n * we need this to avoid circular references\n * http://stackoverflow.com/a/24161582/3208463\n */\nfunction simpleKeys(original) {\n return Object.keys(original).reduce(function (obj, key) {\n if (typeof original[key] !== 'object') {\n obj[key] = original[key];\n }\n return obj;\n }, {});\n}\n\nmpl.figure.prototype.mouse_event = function (event, name) {\n var canvas_pos = mpl.findpos(event);\n\n if (name === 'button_press') {\n this.canvas.focus();\n this.canvas_div.focus();\n }\n\n var x = canvas_pos.x * this.ratio;\n var y = canvas_pos.y * this.ratio;\n\n this.send_message(name, {\n x: x,\n y: y,\n button: event.button,\n step: event.step,\n guiEvent: simpleKeys(event),\n });\n\n /* This prevents the web browser from automatically changing to\n * the text insertion cursor when the button is pressed. We want\n * to control all of the cursor setting manually through the\n * 'cursor' event from matplotlib */\n event.preventDefault();\n return false;\n};\n\nmpl.figure.prototype._key_event_extra = function (_event, _name) {\n // Handle any extra behaviour associated with a key event\n};\n\nmpl.figure.prototype.key_event = function (event, name) {\n // Prevent repeat events\n if (name === 'key_press') {\n if (event.key === this._key) {\n return;\n } else {\n this._key = event.key;\n }\n }\n if (name === 'key_release') {\n this._key = null;\n }\n\n var value = '';\n if (event.ctrlKey && event.key !== 'Control') {\n value += 'ctrl+';\n }\n else if (event.altKey && event.key !== 'Alt') {\n value += 'alt+';\n }\n else if (event.shiftKey && event.key !== 'Shift') {\n value += 'shift+';\n }\n\n value += 'k' + event.key;\n\n this._key_event_extra(event, name);\n\n this.send_message(name, { key: value, guiEvent: simpleKeys(event) });\n return false;\n};\n\nmpl.figure.prototype.toolbar_button_onclick = function (name) {\n if (name === 'download') {\n this.handle_save(this, null);\n } else {\n this.send_message('toolbar_button', { name: name });\n }\n};\n\nmpl.figure.prototype.toolbar_button_onmouseover = function (tooltip) {\n this.message.textContent = tooltip;\n};\n\n///////////////// REMAINING CONTENT GENERATED BY embed_js.py /////////////////\n// prettier-ignore\nvar _JSXTOOLS_RESIZE_OBSERVER=function(A){var t,i=new WeakMap,n=new WeakMap,a=new WeakMap,r=new WeakMap,o=new Set;function s(e){if(!(this instanceof s))throw new TypeError(\"Constructor requires 'new' operator\");i.set(this,e)}function h(){throw new TypeError(\"Function is not a constructor\")}function c(e,t,i,n){e=0 in arguments?Number(arguments[0]):0,t=1 in arguments?Number(arguments[1]):0,i=2 in arguments?Number(arguments[2]):0,n=3 in arguments?Number(arguments[3]):0,this.right=(this.x=this.left=e)+(this.width=i),this.bottom=(this.y=this.top=t)+(this.height=n),Object.freeze(this)}function d(){t=requestAnimationFrame(d);var s=new WeakMap,p=new Set;o.forEach((function(t){r.get(t).forEach((function(i){var r=t instanceof window.SVGElement,o=a.get(t),d=r?0:parseFloat(o.paddingTop),f=r?0:parseFloat(o.paddingRight),l=r?0:parseFloat(o.paddingBottom),u=r?0:parseFloat(o.paddingLeft),g=r?0:parseFloat(o.borderTopWidth),m=r?0:parseFloat(o.borderRightWidth),w=r?0:parseFloat(o.borderBottomWidth),b=u+f,F=d+l,v=(r?0:parseFloat(o.borderLeftWidth))+m,W=g+w,y=r?0:t.offsetHeight-W-t.clientHeight,E=r?0:t.offsetWidth-v-t.clientWidth,R=b+v,z=F+W,M=r?t.width:parseFloat(o.width)-R-E,O=r?t.height:parseFloat(o.height)-z-y;if(n.has(t)){var k=n.get(t);if(k[0]===M&&k[1]===O)return}n.set(t,[M,O]);var S=Object.create(h.prototype);S.target=t,S.contentRect=new c(u,d,M,O),s.has(i)||(s.set(i,[]),p.add(i)),s.get(i).push(S)}))})),p.forEach((function(e){i.get(e).call(e,s.get(e),e)}))}return s.prototype.observe=function(i){if(i instanceof window.Element){r.has(i)||(r.set(i,new Set),o.add(i),a.set(i,window.getComputedStyle(i)));var n=r.get(i);n.has(this)||n.add(this),cancelAnimationFrame(t),t=requestAnimationFrame(d)}},s.prototype.unobserve=function(i){if(i instanceof window.Element&&r.has(i)){var n=r.get(i);n.has(this)&&(n.delete(this),n.size||(r.delete(i),o.delete(i))),n.size||r.delete(i),o.size||cancelAnimationFrame(t)}},A.DOMRectReadOnly=c,A.ResizeObserver=s,A.ResizeObserverEntry=h,A}; // eslint-disable-line\nmpl.toolbar_items = [[\"Home\", \"Reset original view\", \"fa fa-home icon-home\", \"home\"], [\"Back\", \"Back to previous view\", \"fa fa-arrow-left icon-arrow-left\", \"back\"], [\"Forward\", \"Forward to next view\", \"fa fa-arrow-right icon-arrow-right\", \"forward\"], [\"\", \"\", \"\", \"\"], [\"Pan\", \"Left button pans, Right button zooms\\nx/y fixes axis, CTRL fixes aspect\", \"fa fa-arrows icon-move\", \"pan\"], [\"Zoom\", \"Zoom to rectangle\\nx/y fixes axis, CTRL fixes aspect\", \"fa fa-square-o icon-check-empty\", \"zoom\"], [\"\", \"\", \"\", \"\"], [\"Download\", \"Download plot\", \"fa fa-floppy-o icon-save\", \"download\"]];\n\nmpl.extensions = [\"eps\", \"jpeg\", \"pgf\", \"pdf\", \"png\", \"ps\", \"raw\", \"svg\", \"tif\"];\n\nmpl.default_extension = \"png\";/* global mpl */\n\nvar comm_websocket_adapter = function (comm) {\n // Create a \"websocket\"-like object which calls the given IPython comm\n // object with the appropriate methods. Currently this is a non binary\n // socket, so there is still some room for performance tuning.\n var ws = {};\n\n ws.binaryType = comm.kernel.ws.binaryType;\n ws.readyState = comm.kernel.ws.readyState;\n function updateReadyState(_event) {\n if (comm.kernel.ws) {\n ws.readyState = comm.kernel.ws.readyState;\n } else {\n ws.readyState = 3; // Closed state.\n }\n }\n comm.kernel.ws.addEventListener('open', updateReadyState);\n comm.kernel.ws.addEventListener('close', updateReadyState);\n comm.kernel.ws.addEventListener('error', updateReadyState);\n\n ws.close = function () {\n comm.close();\n };\n ws.send = function (m) {\n //console.log('sending', m);\n comm.send(m);\n };\n // Register the callback with on_msg.\n comm.on_msg(function (msg) {\n //console.log('receiving', msg['content']['data'], msg);\n var data = msg['content']['data'];\n if (data['blob'] !== undefined) {\n data = {\n data: new Blob(msg['buffers'], { type: data['blob'] }),\n };\n }\n // Pass the mpl event to the overridden (by mpl) onmessage function.\n ws.onmessage(data);\n });\n return ws;\n};\n\nmpl.mpl_figure_comm = function (comm, msg) {\n // This is the function which gets called when the mpl process\n // starts-up an IPython Comm through the \"matplotlib\" channel.\n\n var id = msg.content.data.id;\n // Get hold of the div created by the display call when the Comm\n // socket was opened in Python.\n var element = document.getElementById(id);\n var ws_proxy = comm_websocket_adapter(comm);\n\n function ondownload(figure, _format) {\n window.open(figure.canvas.toDataURL());\n }\n\n var fig = new mpl.figure(id, ws_proxy, ondownload, element);\n\n // Call onopen now - mpl needs it, as it is assuming we've passed it a real\n // web socket which is closed, not our websocket->open comm proxy.\n ws_proxy.onopen();\n\n fig.parent_element = element;\n fig.cell_info = mpl.find_output_cell(\"<div id='\" + id + \"'></div>\");\n if (!fig.cell_info) {\n console.error('Failed to find cell for figure', id, fig);\n return;\n }\n fig.cell_info[0].output_area.element.on(\n 'cleared',\n { fig: fig },\n fig._remove_fig_handler\n );\n};\n\nmpl.figure.prototype.handle_close = function (fig, msg) {\n var width = fig.canvas.width / fig.ratio;\n fig.cell_info[0].output_area.element.off(\n 'cleared',\n fig._remove_fig_handler\n );\n fig.resizeObserverInstance.unobserve(fig.canvas_div);\n\n // Update the output cell to use the data from the current canvas.\n fig.push_to_output();\n var dataURL = fig.canvas.toDataURL();\n // Re-enable the keyboard manager in IPython - without this line, in FF,\n // the notebook keyboard shortcuts fail.\n IPython.keyboard_manager.enable();\n fig.parent_element.innerHTML =\n '<img src=\"' + dataURL + '\" width=\"' + width + '\">';\n fig.close_ws(fig, msg);\n};\n\nmpl.figure.prototype.close_ws = function (fig, msg) {\n fig.send_message('closing', msg);\n // fig.ws.close()\n};\n\nmpl.figure.prototype.push_to_output = function (_remove_interactive) {\n // Turn the data on the canvas into data in the output cell.\n var width = this.canvas.width / this.ratio;\n var dataURL = this.canvas.toDataURL();\n this.cell_info[1]['text/html'] =\n '<img src=\"' + dataURL + '\" width=\"' + width + '\">';\n};\n\nmpl.figure.prototype.updated_canvas_event = function () {\n // Tell IPython that the notebook contents must change.\n IPython.notebook.set_dirty(true);\n this.send_message('ack', {});\n var fig = this;\n // Wait a second, then push the new image to the DOM so\n // that it is saved nicely (might be nice to debounce this).\n setTimeout(function () {\n fig.push_to_output();\n }, 1000);\n};\n\nmpl.figure.prototype._init_toolbar = function () {\n var fig = this;\n\n var toolbar = document.createElement('div');\n toolbar.classList = 'btn-toolbar';\n this.root.appendChild(toolbar);\n\n function on_click_closure(name) {\n return function (_event) {\n return fig.toolbar_button_onclick(name);\n };\n }\n\n function on_mouseover_closure(tooltip) {\n return function (event) {\n if (!event.currentTarget.disabled) {\n return fig.toolbar_button_onmouseover(tooltip);\n }\n };\n }\n\n fig.buttons = {};\n var buttonGroup = document.createElement('div');\n buttonGroup.classList = 'btn-group';\n var button;\n for (var toolbar_ind in mpl.toolbar_items) {\n var name = mpl.toolbar_items[toolbar_ind][0];\n var tooltip = mpl.toolbar_items[toolbar_ind][1];\n var image = mpl.toolbar_items[toolbar_ind][2];\n var method_name = mpl.toolbar_items[toolbar_ind][3];\n\n if (!name) {\n /* Instead of a spacer, we start a new button group. */\n if (buttonGroup.hasChildNodes()) {\n toolbar.appendChild(buttonGroup);\n }\n buttonGroup = document.createElement('div');\n buttonGroup.classList = 'btn-group';\n continue;\n }\n\n button = fig.buttons[name] = document.createElement('button');\n button.classList = 'btn btn-default';\n button.href = '#';\n button.title = name;\n button.innerHTML = '<i class=\"fa ' + image + ' fa-lg\"></i>';\n button.addEventListener('click', on_click_closure(method_name));\n button.addEventListener('mouseover', on_mouseover_closure(tooltip));\n buttonGroup.appendChild(button);\n }\n\n if (buttonGroup.hasChildNodes()) {\n toolbar.appendChild(buttonGroup);\n }\n\n // Add the status bar.\n var status_bar = document.createElement('span');\n status_bar.classList = 'mpl-message pull-right';\n toolbar.appendChild(status_bar);\n this.message = status_bar;\n\n // Add the close button to the window.\n var buttongrp = document.createElement('div');\n buttongrp.classList = 'btn-group inline pull-right';\n button = document.createElement('button');\n button.classList = 'btn btn-mini btn-primary';\n button.href = '#';\n button.title = 'Stop Interaction';\n button.innerHTML = '<i class=\"fa fa-power-off icon-remove icon-large\"></i>';\n button.addEventListener('click', function (_evt) {\n fig.handle_close(fig, {});\n });\n button.addEventListener(\n 'mouseover',\n on_mouseover_closure('Stop Interaction')\n );\n buttongrp.appendChild(button);\n var titlebar = this.root.querySelector('.ui-dialog-titlebar');\n titlebar.insertBefore(buttongrp, titlebar.firstChild);\n};\n\nmpl.figure.prototype._remove_fig_handler = function (event) {\n var fig = event.data.fig;\n if (event.target !== this) {\n // Ignore bubbled events from children.\n return;\n }\n fig.close_ws(fig, {});\n};\n\nmpl.figure.prototype._root_extra_style = function (el) {\n el.style.boxSizing = 'content-box'; // override notebook setting of border-box.\n};\n\nmpl.figure.prototype._canvas_extra_style = function (el) {\n // this is important to make the div 'focusable\n el.setAttribute('tabindex', 0);\n // reach out to IPython and tell the keyboard manager to turn it's self\n // off when our div gets focus\n\n // location in version 3\n if (IPython.notebook.keyboard_manager) {\n IPython.notebook.keyboard_manager.register_events(el);\n } else {\n // location in version 2\n IPython.keyboard_manager.register_events(el);\n }\n};\n\nmpl.figure.prototype._key_event_extra = function (event, _name) {\n var manager = IPython.notebook.keyboard_manager;\n if (!manager) {\n manager = IPython.keyboard_manager;\n }\n\n // Check for shift+enter\n if (event.shiftKey && event.which === 13) {\n this.canvas_div.blur();\n // select the cell after this one\n var index = IPython.notebook.find_cell_index(this.cell_info[0]);\n IPython.notebook.select(index + 1);\n }\n};\n\nmpl.figure.prototype.handle_save = function (fig, _msg) {\n fig.ondownload(fig, null);\n};\n\nmpl.find_output_cell = function (html_output) {\n // Return the cell and output element which can be found *uniquely* in the notebook.\n // Note - this is a bit hacky, but it is done because the \"notebook_saving.Notebook\"\n // IPython event is triggered only after the cells have been serialised, which for\n // our purposes (turning an active figure into a static one), is too late.\n var cells = IPython.notebook.get_cells();\n var ncells = cells.length;\n for (var i = 0; i < ncells; i++) {\n var cell = cells[i];\n if (cell.cell_type === 'code') {\n for (var j = 0; j < cell.output_area.outputs.length; j++) {\n var data = cell.output_area.outputs[j];\n if (data.data) {\n // IPython >= 3 moved mimebundle to data attribute of output\n data = data.data;\n }\n if (data['text/html'] === html_output) {\n return [cell, data, j];\n }\n }\n }\n }\n};\n\n// Register the function which deals with the matplotlib target/channel.\n// The kernel may be null if the page has been refreshed.\nif (IPython.notebook.kernel !== null) {\n IPython.notebook.kernel.comm_manager.register_target(\n 'matplotlib',\n mpl.mpl_figure_comm\n );\n}\n",
"text/plain": [
"<IPython.core.display.Javascript object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<img src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAA4QAAAEsCAYAAACbnn2RAAAAAXNSR0IArs4c6QAAFK1JREFUeF7t10EBAAAIAjHpX9ogNxsw/LBzBAgQIECAAAECBAgQIJAUWDK10AQIECBAgAABAgQIECBwBqEnIECAAAECBAgQIECAQFTAIIwWLzYBAgQIECBAgAABAgQMQj9AgAABAgQIECBAgACBqIBBGC1ebAIECBAgQIAAAQIECBiEfoAAAQIECBAgQIAAAQJRAYMwWrzYBAgQIECAAAECBAgQMAj9AAECBAgQIECAAAECBKICBmG0eLEJECBAgAABAgQIECBgEPoBAgQIECBAgAABAgQIRAUMwmjxYhMgQIAAAQIECBAgQMAg9AMECBAgQIAAAQIECBCIChiE0eLFJkCAAAECBAgQIECAgEHoBwgQIECAAAECBAgQIBAVMAijxYtNgAABAgQIECBAgAABg9APECBAgAABAgQIECBAICpgEEaLF5sAAQIECBAgQIAAAQIGoR8gQIAAAQIECBAgQIBAVMAgjBYvNgECBAgQIECAAAECBAxCP0CAAAECBAgQIECAAIGogEEYLV5sAgQIECBAgAABAgQIGIR+gAABAgQIECBAgAABAlEBgzBavNgECBAgQIAAAQIECBAwCP0AAQIECBAgQIAAAQIEogIGYbR4sQkQIECAAAECBAgQIGAQ+gECBAgQIECAAAECBAhEBQzCaPFiEyBAgAABAgQIECBAwCD0AwQIECBAgAABAgQIEIgKGITR4sUmQIAAAQIECBAgQICAQegHCBAgQIAAAQIECBAgEBUwCKPFi02AAAECBAgQIECAAAGD0A8QIECAAAECBAgQIEAgKmAQRosXmwABAgQIECBAgAABAgahHyBAgAABAgQIECBAgEBUwCCMFi82AQIECBAgQIAAAQIEDEI/QIAAAQIECBAgQIAAgaiAQRgtXmwCBAgQIECAAAECBAgYhH6AAAECBAgQIECAAAECUQGDMFq82AQIECBAgAABAgQIEDAI/QABAgQIECBAgAABAgSiAgZhtHixCRAgQIAAAQIECBAgYBD6AQIECBAgQIAAAQIECEQFDMJo8WITIECAAAECBAgQIEDAIPQDBAgQIECAAAECBAgQiAoYhNHixSZAgAABAgQIECBAgIBB6AcIECBAgAABAgQIECAQFTAIo8WLTYAAAQIECBAgQIAAAYPQDxAgQIAAAQIECBAgQCAqYBBGixebAAECBAgQIECAAAECBqEfIECAAAECBAgQIECAQFTAIIwWLzYBAgQIECBAgAABAgQMQj9AgAABAgQIECBAgACBqIBBGC1ebAIECBAgQIAAAQIECBiEfoAAAQIECBAgQIAAAQJRAYMwWrzYBAgQIECAAAECBAgQMAj9AAECBAgQIECAAAECBKICBmG0eLEJECBAgAABAgQIECBgEPoBAgQIECBAgAABAgQIRAUMwmjxYhMgQIAAAQIECBAgQMAg9AMECBAgQIAAAQIECBCIChiE0eLFJkCAAAECBAgQIECAgEHoBwgQIECAAAECBAgQIBAVMAijxYtNgAABAgQIECBAgAABg9APECBAgAABAgQIECBAICpgEEaLF5sAAQIECBAgQIAAAQIGoR8gQIAAAQIECBAgQIBAVMAgjBYvNgECBAgQIECAAAECBAxCP0CAAAECBAgQIECAAIGogEEYLV5sAgQIECBAgAABAgQIGIR+gAABAgQIECBAgAABAlEBgzBavNgECBAgQIAAAQIECBAwCP0AAQIECBAgQIAAAQIEogIGYbR4sQkQIECAAAECBAgQIGAQ+gECBAgQIECAAAECBAhEBQzCaPFiEyBAgAABAgQIECBAwCD0AwQIECBAgAABAgQIEIgKGITR4sUmQIAAAQIECBAgQICAQegHCBAgQIAAAQIECBAgEBUwCKPFi02AAAECBAgQIECAAAGD0A8QIECAAAECBAgQIEAgKmAQRosXmwABAgQIECBAgAABAgahHyBAgAABAgQIECBAgEBUwCCMFi82AQIECBAgQIAAAQIEDEI/QIAAAQIECBAgQIAAgaiAQRgtXmwCBAgQIECAAAECBAgYhH6AAAECBAgQIECAAAECUQGDMFq82AQIECBAgAABAgQIEDAI/QABAgQIECBAgAABAgSiAgZhtHixCRAgQIAAAQIECBAgYBD6AQIECBAgQIAAAQIECEQFDMJo8WITIECAAAECBAgQIEDAIPQDBAgQIECAAAECBAgQiAoYhNHixSZAgAABAgQIECBAgIBB6AcIECBAgAABAgQIECAQFTAIo8WLTYAAAQIECBAgQIAAAYPQDxAgQIAAAQIECBAgQCAqYBBGixebAAECBAgQIECAAAECBqEfIECAAAECBAgQIECAQFTAIIwWLzYBAgQIECBAgAABAgQMQj9AgAABAgQIECBAgACBqIBBGC1ebAIECBAgQIAAAQIECBiEfoAAAQIECBAgQIAAAQJRAYMwWrzYBAgQIECAAAECBAgQMAj9AAECBAgQIECAAAECBKICBmG0eLEJECBAgAABAgQIECBgEPoBAgQIECBAgAABAgQIRAUMwmjxYhMgQIAAAQIECBAgQMAg9AMECBAgQIAAAQIECBCIChiE0eLFJkCAAAECBAgQIECAgEHoBwgQIECAAAECBAgQIBAVMAijxYtNgAABAgQIECBAgAABg9APECBAgAABAgQIECBAICpgEEaLF5sAAQIECBAgQIAAAQIGoR8gQIAAAQIECBAgQIBAVMAgjBYvNgECBAgQIECAAAECBAxCP0CAAAECBAgQIECAAIGogEEYLV5sAgQIECBAgAABAgQIGIR+gAABAgQIECBAgAABAlEBgzBavNgECBAgQIAAAQIECBAwCP0AAQIECBAgQIAAAQIEogIGYbR4sQkQIECAAAECBAgQIGAQ+gECBAgQIECAAAECBAhEBQzCaPFiEyBAgAABAgQIECBAwCD0AwQIECBAgAABAgQIEIgKGITR4sUmQIAAAQIECBAgQICAQegHCBAgQIAAAQIECBAgEBUwCKPFi02AAAECBAgQIECAAAGD0A8QIECAAAECBAgQIEAgKmAQRosXmwABAgQIECBAgAABAgahHyBAgAABAgQIECBAgEBUwCCMFi82AQIECBAgQIAAAQIEDEI/QIAAAQIECBAgQIAAgaiAQRgtXmwCBAgQIECAAAECBAgYhH6AAAECBAgQIECAAAECUQGDMFq82AQIECBAgAABAgQIEDAI/QABAgQIECBAgAABAgSiAgZhtHixCRAgQIAAAQIECBAgYBD6AQIECBAgQIAAAQIECEQFDMJo8WITIECAAAECBAgQIEDAIPQDBAgQIECAAAECBAgQiAoYhNHixSZAgAABAgQIECBAgIBB6AcIECBAgAABAgQIECAQFTAIo8WLTYAAAQIECBAgQIAAAYPQDxAgQIAAAQIECBAgQCAqYBBGixebAAECBAgQIECAAAECBqEfIECAAAECBAgQIECAQFTAIIwWLzYBAgQIECBAgAABAgQMQj9AgAABAgQIECBAgACBqIBBGC1ebAIECBAgQIAAAQIECBiEfoAAAQIECBAgQIAAAQJRAYMwWrzYBAgQIECAAAECBAgQMAj9AAECBAgQIECAAAECBKICBmG0eLEJECBAgAABAgQIECBgEPoBAgQIECBAgAABAgQIRAUMwmjxYhMgQIAAAQIECBAgQMAg9AMECBAgQIAAAQIECBCIChiE0eLFJkCAAAECBAgQIECAgEHoBwgQIECAAAECBAgQIBAVMAijxYtNgAABAgQIECBAgAABg9APECBAgAABAgQIECBAICpgEEaLF5sAAQIECBAgQIAAAQIGoR8gQIAAAQIECBAgQIBAVMAgjBYvNgECBAgQIECAAAECBAxCP0CAAAECBAgQIECAAIGogEEYLV5sAgQIECBAgAABAgQIGIR+gAABAgQIECBAgAABAlEBgzBavNgECBAgQIAAAQIECBAwCP0AAQIECBAgQIAAAQIEogIGYbR4sQkQIECAAAECBAgQIGAQ+gECBAgQIECAAAECBAhEBQzCaPFiEyBAgAABAgQIECBAwCD0AwQIECBAgAABAgQIEIgKGITR4sUmQIAAAQIECBAgQICAQegHCBAgQIAAAQIECBAgEBUwCKPFi02AAAECBAgQIECAAAGD0A8QIECAAAECBAgQIEAgKmAQRosXmwABAgQIECBAgAABAgahHyBAgAABAgQIECBAgEBUwCCMFi82AQIECBAgQIAAAQIEDEI/QIAAAQIECBAgQIAAgaiAQRgtXmwCBAgQIECAAAECBAgYhH6AAAECBAgQIECAAAECUQGDMFq82AQIECBAgAABAgQIEDAI/QABAgQIECBAgAABAgSiAgZhtHixCRAgQIAAAQIECBAgYBD6AQIECBAgQIAAAQIECEQFDMJo8WITIECAAAECBAgQIEDAIPQDBAgQIECAAAECBAgQiAoYhNHixSZAgAABAgQIECBAgIBB6AcIECBAgAABAgQIECAQFTAIo8WLTYAAAQIECBAgQIAAAYPQDxAgQIAAAQIECBAgQCAqYBBGixebAAECBAgQIECAAAECBqEfIECAAAECBAgQIECAQFTAIIwWLzYBAgQIECBAgAABAgQMQj9AgAABAgQIECBAgACBqIBBGC1ebAIECBAgQIAAAQIECBiEfoAAAQIECBAgQIAAAQJRAYMwWrzYBAgQIECAAAECBAgQMAj9AAECBAgQIECAAAECBKICBmG0eLEJECBAgAABAgQIECBgEPoBAgQIECBAgAABAgQIRAUMwmjxYhMgQIAAAQIECBAgQMAg9AMECBAgQIAAAQIECBCIChiE0eLFJkCAAAECBAgQIECAgEHoBwgQIECAAAECBAgQIBAVMAijxYtNgAABAgQIECBAgAABg9APECBAgAABAgQIECBAICpgEEaLF5sAAQIECBAgQIAAAQIGoR8gQIAAAQIECBAgQIBAVMAgjBYvNgECBAgQIECAAAECBAxCP0CAAAECBAgQIECAAIGogEEYLV5sAgQIECBAgAABAgQIGIR+gAABAgQIECBAgAABAlEBgzBavNgECBAgQIAAAQIECBAwCP0AAQIECBAgQIAAAQIEogIGYbR4sQkQIECAAAECBAgQIGAQ+gECBAgQIECAAAECBAhEBQzCaPFiEyBAgAABAgQIECBAwCD0AwQIECBAgAABAgQIEIgKGITR4sUmQIAAAQIECBAgQICAQegHCBAgQIAAAQIECBAgEBUwCKPFi02AAAECBAgQIECAAAGD0A8QIECAAAECBAgQIEAgKmAQRosXmwABAgQIECBAgAABAgahHyBAgAABAgQIECBAgEBUwCCMFi82AQIECBAgQIAAAQIEDEI/QIAAAQIECBAgQIAAgaiAQRgtXmwCBAgQIECAAAECBAgYhH6AAAECBAgQIECAAAECUQGDMFq82AQIECBAgAABAgQIEDAI/QABAgQIECBAgAABAgSiAgZhtHixCRAgQIAAAQIECBAgYBD6AQIECBAgQIAAAQIECEQFDMJo8WITIECAAAECBAgQIEDAIPQDBAgQIECAAAECBAgQiAoYhNHixSZAgAABAgQIECBAgIBB6AcIECBAgAABAgQIECAQFTAIo8WLTYAAAQIECBAgQIAAAYPQDxAgQIAAAQIECBAgQCAqYBBGixebAAECBAgQIECAAAECBqEfIECAAAECBAgQIECAQFTAIIwWLzYBAgQIECBAgAABAgQMQj9AgAABAgQIECBAgACBqIBBGC1ebAIECBAgQIAAAQIECBiEfoAAAQIECBAgQIAAAQJRAYMwWrzYBAgQIECAAAECBAgQMAj9AAECBAgQIECAAAECBKICBmG0eLEJECBAgAABAgQIECBgEPoBAgQIECBAgAABAgQIRAUMwmjxYhMgQIAAAQIECBAgQMAg9AMECBAgQIAAAQIECBCIChiE0eLFJkCAAAECBAgQIECAgEHoBwgQIECAAAECBAgQIBAVMAijxYtNgAABAgQIECBAgAABg9APECBAgAABAgQIECBAICpgEEaLF5sAAQIECBAgQIAAAQIGoR8gQIAAAQIECBAgQIBAVMAgjBYvNgECBAgQIECAAAECBAxCP0CAAAECBAgQIECAAIGogEEYLV5sAgQIECBAgAABAgQIGIR+gAABAgQIECBAgAABAlEBgzBavNgECBAgQIAAAQIECBAwCP0AAQIECBAgQIAAAQIEogIGYbR4sQkQIECAAAECBAgQIGAQ+gECBAgQIECAAAECBAhEBQzCaPFiEyBAgAABAgQIECBAwCD0AwQIECBAgAABAgQIEIgKGITR4sUmQIAAAQIECBAgQICAQegHCBAgQIAAAQIECBAgEBUwCKPFi02AAAECBAgQIECAAAGD0A8QIECAAAECBAgQIEAgKmAQRosXmwABAgQIECBAgAABAgahHyBAgAABAgQIECBAgEBUwCCMFi82AQIECBAgQIAAAQIEDEI/QIAAAQIECBAgQIAAgaiAQRgtXmwCBAgQIECAAAECBAgYhH6AAAECBAgQIECAAAECUQGDMFq82AQIECBAgAABAgQIEDAI/QABAgQIECBAgAABAgSiAgZhtHixCRAgQIAAAQIECBAgYBD6AQIECBAgQIAAAQIECEQFDMJo8WITIECAAAECBAgQIEDAIPQDBAgQIECAAAECBAgQiAoYhNHixSZAgAABAgQIECBAgIBB6AcIECBAgAABAgQIECAQFTAIo8WLTYAAAQIECBAgQIAAAYPQDxAgQIAAAQIECBAgQCAqYBBGixebAAECBAgQIECAAAECBqEfIECAAAECBAgQIECAQFTAIIwWLzYBAgQIECBAgAABAgQMQj9AgAABAgQIECBAgACBqIBBGC1ebAIECBAgQIAAAQIECBiEfoAAAQIECBAgQIAAAQJRAYMwWrzYBAgQIECAAAECBAgQMAj9AAECBAgQIECAAAECBKICBmG0eLEJECBAgAABAgQIECBgEPoBAgQIECBAgAABAgQIRAUMwmjxYhMgQIAAAQIECBAgQOAB6HkBLXgQOl0AAAAASUVORK5CYII=\" width=\"900\">"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"plot_loss_functions(suptitle = 'Common loss functions for classification (class=1)',\n",
" functions = [zero_one_v(x), logistic_loss(x)],\n",
" ylabels = ['$\\mathcal{L}_{0-1}}$ (0-1 loss)',\n",
" '$\\mathcal{L}_{log}$ (logistic loss)'],\n",
" xlabel = '$p$')\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"To understand logistic loss, consider two cases of the expected output:\n",
"* If we expect output to be 1 ($y=1$), then the loss is $-log f_\\theta(x_i)$. The loss is 0 is the network predicts 1 with probability 1, and grows larger when probability of 1 gets smaller.\n",
"* If we expect output to be 0 ($y=0$), the loss is $-log(1-f_\\theta(x_i))$. Here, $1-f_\\theta(x_i)$ is the probability of 0 which is predicted by the network, and the meaning of log-loss is the same as described in the previous case"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Neural Network Architecture\n",
"\n",
"We have generated a dataset for binary classification problem. However, let's consider it as multi-class classification right from the start, so that we can then easily switch our code to multi-class classification. In this case, our one-layer perceptron will have the following architecture:\n",
"\n",
"<img src=\"images/NeuroArch.png\" width=\"50%\"/>\n",
"\n",
"Two outputs of the network correspond to two classes, and the class with highest value among two outputs corresponds to the right solution.\n",
"\n",
"The model is defined as\n",
"$$\n",
"f_\\theta(x) = W\\times x + b\n",
"$$\n",
"where $$\\theta = \\langle W,b\\rangle$$ are parameters.\n",
"\n",
"We will define this linear layer as a Python class with a `forward` function that performs the calculation. It receives input value $x$, and produces the output of the layer. Parameters `W` and `b` are stored within the layer class, and are initialized upon creation with random values and zeroes respectively."
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([[ 1.77202116, -0.25384488],\n",
" [ 0.28370828, -0.39610552],\n",
" [-0.30097433, 0.30513182],\n",
" [-0.8120485 , 0.56079421],\n",
" [-1.23519653, 0.3394973 ]])"
]
},
"execution_count": 23,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"class Linear:\n",
" def __init__(self,nin,nout):\n",
" self.W = np.random.normal(0, 1.0/np.sqrt(nin), (nout, nin))\n",
" self.b = np.zeros((1,nout))\n",
" \n",
" def forward(self, x):\n",
" return np.dot(x, self.W.T) + self.b\n",
" \n",
"net = Linear(2,2)\n",
"net.forward(train_x[0:5])"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"In many cases, it is more efficient to operate not on the one input value, but on the vector of input values. Because we use Numpy operations, we can pass a vector of input values to our network, and it will give us the vector of output values.\n",
"\n",
"## Softmax: Turning Outputs into Probabilities\n",
"\n",
"As you can see, our outputs are not probabilities - they can take any values. In order to convert them into probabilities, we need to normalize the values across all classes. This is done using **softmax** function: $$\\sigma(\\mathbf{z}_c) = \\frac{e^{z_c}}{\\sum_{j} e^{z_j}}, \\quad\\mathrm{for}\\quad c\\in 1 .. |C|$$\n",
"\n",
"<img src=\"https://raw.githubusercontent.com/shwars/NeuroWorkshop/master/images/NeuroArch-softmax.PNG\" width=\"50%\">\n",
"\n",
"> Output of the network $\\sigma(\\mathbf{z})$ can be interpreted as probability distribution on the set of classes $C$: $q = \\sigma(\\mathbf{z}_c) = \\hat{p}(c | x)$\n",
"\n",
"We will define the `Softmax` layer in the same manner, as a class with `forward` function: "
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([[0.88348621, 0.11651379],\n",
" [0.66369714, 0.33630286],\n",
" [0.35294795, 0.64705205],\n",
" [0.20216095, 0.79783905],\n",
" [0.17154828, 0.82845172],\n",
" [0.24279153, 0.75720847],\n",
" [0.18915732, 0.81084268],\n",
" [0.17282951, 0.82717049],\n",
" [0.13897531, 0.86102469],\n",
" [0.72746882, 0.27253118]])"
]
},
"execution_count": 24,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"class Softmax:\n",
" def forward(self,z):\n",
" zmax = z.max(axis=1,keepdims=True)\n",
" expz = np.exp(z-zmax)\n",
" Z = expz.sum(axis=1,keepdims=True)\n",
" return expz / Z\n",
"\n",
"softmax = Softmax()\n",
"softmax.forward(net.forward(train_x[0:10]))"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"You can see that we are now getting probabilities as outputs, i.e. the sum of each output vector is exactly 1. \n",
"\n",
"In case we have more than 2 classes, softmax will normalize probabilities across all of them. Here is a diagram of network architecture that does MNIST digit classification:\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Cross-Entropy Loss\n",
"\n",
"A loss function in classification is typically a logistic function, which can be generalized as **cross-entropy loss**. Cross-entropy loss is a function that can calculate similarity between two arbitrary probability distributions. You can find more detailed discussion about it [on Wikipedia](https://en.wikipedia.org/wiki/Cross_entropy).\n",
"\n",
"In our case, first distribution is the probabilistic output of our network, and the second one is so-called **one-hot** distribution, which specifies that a given class $c$ has corresponding probability 1 (all the rest being 0). In such a case cross-entropy loss can be calculated as $-\\log p_c$, where $c$ is the expected class, and $p_c$ is the corresponding probability of this class given by our neural network.\n",
"\n",
"> If the network return probability 1 for the expected class, cross-entropy loss would be 0. The closer the probability of the actual class is to 0, the higher is cross-entropy loss (and it can go up to infinity!)."
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"outputs": [],
"source": [
"def plot_cross_ent():\n",
" p = np.linspace(0.01, 0.99, 101) # estimated probability p(y|x)\n",
" cross_ent_v = np.vectorize(cross_ent)\n",
" f3, ax = plt.subplots(1,1, figsize=(8, 3))\n",
" l1, = plt.plot(p, cross_ent_v(p, 1), 'r--')\n",
" l2, = plt.plot(p, cross_ent_v(p, 0), 'r-')\n",
" plt.legend([l1, l2], ['$y = 1$', '$y = 0$'], loc = 'upper center', ncol = 2)\n",
" plt.xlabel('$\\hat{p}(y|x)$', size=18)\n",
" plt.ylabel('$\\mathcal{L}_{CE}$', size=18)\n",
" plt.show()"
]
},
{
"cell_type": "code",
"execution_count": 26,
"metadata": {
"scrolled": true,
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [
{
"data": {
"application/javascript": "/* Put everything inside the global mpl namespace */\n/* global mpl */\nwindow.mpl = {};\n\nmpl.get_websocket_type = function () {\n if (typeof WebSocket !== 'undefined') {\n return WebSocket;\n } else if (typeof MozWebSocket !== 'undefined') {\n return MozWebSocket;\n } else {\n alert(\n 'Your browser does not have WebSocket support. ' +\n 'Please try Chrome, Safari or Firefox ≥ 6. ' +\n 'Firefox 4 and 5 are also supported but you ' +\n 'have to enable WebSockets in about:config.'\n );\n }\n};\n\nmpl.figure = function (figure_id, websocket, ondownload, parent_element) {\n this.id = figure_id;\n\n this.ws = websocket;\n\n this.supports_binary = this.ws.binaryType !== undefined;\n\n if (!this.supports_binary) {\n var warnings = document.getElementById('mpl-warnings');\n if (warnings) {\n warnings.style.display = 'block';\n warnings.textContent =\n 'This browser does not support binary websocket messages. ' +\n 'Performance may be slow.';\n }\n }\n\n this.imageObj = new Image();\n\n this.context = undefined;\n this.message = undefined;\n this.canvas = undefined;\n this.rubberband_canvas = undefined;\n this.rubberband_context = undefined;\n this.format_dropdown = undefined;\n\n this.image_mode = 'full';\n\n this.root = document.createElement('div');\n this.root.setAttribute('style', 'display: inline-block');\n this._root_extra_style(this.root);\n\n parent_element.appendChild(this.root);\n\n this._init_header(this);\n this._init_canvas(this);\n this._init_toolbar(this);\n\n var fig = this;\n\n this.waiting = false;\n\n this.ws.onopen = function () {\n fig.send_message('supports_binary', { value: fig.supports_binary });\n fig.send_message('send_image_mode', {});\n if (fig.ratio !== 1) {\n fig.send_message('set_dpi_ratio', { dpi_ratio: fig.ratio });\n }\n fig.send_message('refresh', {});\n };\n\n this.imageObj.onload = function () {\n if (fig.image_mode === 'full') {\n // Full images could contain transparency (where diff images\n // almost always do), so we need to clear the canvas so that\n // there is no ghosting.\n fig.context.clearRect(0, 0, fig.canvas.width, fig.canvas.height);\n }\n fig.context.drawImage(fig.imageObj, 0, 0);\n };\n\n this.imageObj.onunload = function () {\n fig.ws.close();\n };\n\n this.ws.onmessage = this._make_on_message_function(this);\n\n this.ondownload = ondownload;\n};\n\nmpl.figure.prototype._init_header = function () {\n var titlebar = document.createElement('div');\n titlebar.classList =\n 'ui-dialog-titlebar ui-widget-header ui-corner-all ui-helper-clearfix';\n var titletext = document.createElement('div');\n titletext.classList = 'ui-dialog-title';\n titletext.setAttribute(\n 'style',\n 'width: 100%; text-align: center; padding: 3px;'\n );\n titlebar.appendChild(titletext);\n this.root.appendChild(titlebar);\n this.header = titletext;\n};\n\nmpl.figure.prototype._canvas_extra_style = function (_canvas_div) {};\n\nmpl.figure.prototype._root_extra_style = function (_canvas_div) {};\n\nmpl.figure.prototype._init_canvas = function () {\n var fig = this;\n\n var canvas_div = (this.canvas_div = document.createElement('div'));\n canvas_div.setAttribute(\n 'style',\n 'border: 1px solid #ddd;' +\n 'box-sizing: content-box;' +\n 'clear: both;' +\n 'min-height: 1px;' +\n 'min-width: 1px;' +\n 'outline: 0;' +\n 'overflow: hidden;' +\n 'position: relative;' +\n 'resize: both;'\n );\n\n function on_keyboard_event_closure(name) {\n return function (event) {\n return fig.key_event(event, name);\n };\n }\n\n canvas_div.addEventListener(\n 'keydown',\n on_keyboard_event_closure('key_press')\n );\n canvas_div.addEventListener(\n 'keyup',\n on_keyboard_event_closure('key_release')\n );\n\n this._canvas_extra_style(canvas_div);\n this.root.appendChild(canvas_div);\n\n var canvas = (this.canvas = document.createElement('canvas'));\n canvas.classList.add('mpl-canvas');\n canvas.setAttribute('style', 'box-sizing: content-box;');\n\n this.context = canvas.getContext('2d');\n\n var backingStore =\n this.context.backingStorePixelRatio ||\n this.context.webkitBackingStorePixelRatio ||\n this.context.mozBackingStorePixelRatio ||\n this.context.msBackingStorePixelRatio ||\n this.context.oBackingStorePixelRatio ||\n this.context.backingStorePixelRatio ||\n 1;\n\n this.ratio = (window.devicePixelRatio || 1) / backingStore;\n\n var rubberband_canvas = (this.rubberband_canvas = document.createElement(\n 'canvas'\n ));\n rubberband_canvas.setAttribute(\n 'style',\n 'box-sizing: content-box; position: absolute; left: 0; top: 0; z-index: 1;'\n );\n\n // Apply a ponyfill if ResizeObserver is not implemented by browser.\n if (this.ResizeObserver === undefined) {\n if (window.ResizeObserver !== undefined) {\n this.ResizeObserver = window.ResizeObserver;\n } else {\n var obs = _JSXTOOLS_RESIZE_OBSERVER({});\n this.ResizeObserver = obs.ResizeObserver;\n }\n }\n\n this.resizeObserverInstance = new this.ResizeObserver(function (entries) {\n var nentries = entries.length;\n for (var i = 0; i < nentries; i++) {\n var entry = entries[i];\n var width, height;\n if (entry.contentBoxSize) {\n if (entry.contentBoxSize instanceof Array) {\n // Chrome 84 implements new version of spec.\n width = entry.contentBoxSize[0].inlineSize;\n height = entry.contentBoxSize[0].blockSize;\n } else {\n // Firefox implements old version of spec.\n width = entry.contentBoxSize.inlineSize;\n height = entry.contentBoxSize.blockSize;\n }\n } else {\n // Chrome <84 implements even older version of spec.\n width = entry.contentRect.width;\n height = entry.contentRect.height;\n }\n\n // Keep the size of the canvas and rubber band canvas in sync with\n // the canvas container.\n if (entry.devicePixelContentBoxSize) {\n // Chrome 84 implements new version of spec.\n canvas.setAttribute(\n 'width',\n entry.devicePixelContentBoxSize[0].inlineSize\n );\n canvas.setAttribute(\n 'height',\n entry.devicePixelContentBoxSize[0].blockSize\n );\n } else {\n canvas.setAttribute('width', width * fig.ratio);\n canvas.setAttribute('height', height * fig.ratio);\n }\n canvas.setAttribute(\n 'style',\n 'width: ' + width + 'px; height: ' + height + 'px;'\n );\n\n rubberband_canvas.setAttribute('width', width);\n rubberband_canvas.setAttribute('height', height);\n\n // And update the size in Python. We ignore the initial 0/0 size\n // that occurs as the element is placed into the DOM, which should\n // otherwise not happen due to the minimum size styling.\n if (fig.ws.readyState == 1 && width != 0 && height != 0) {\n fig.request_resize(width, height);\n }\n }\n });\n this.resizeObserverInstance.observe(canvas_div);\n\n function on_mouse_event_closure(name) {\n return function (event) {\n return fig.mouse_event(event, name);\n };\n }\n\n rubberband_canvas.addEventListener(\n 'mousedown',\n on_mouse_event_closure('button_press')\n );\n rubberband_canvas.addEventListener(\n 'mouseup',\n on_mouse_event_closure('button_release')\n );\n rubberband_canvas.addEventListener(\n 'dblclick',\n on_mouse_event_closure('dblclick')\n );\n // Throttle sequential mouse events to 1 every 20ms.\n rubberband_canvas.addEventListener(\n 'mousemove',\n on_mouse_event_closure('motion_notify')\n );\n\n rubberband_canvas.addEventListener(\n 'mouseenter',\n on_mouse_event_closure('figure_enter')\n );\n rubberband_canvas.addEventListener(\n 'mouseleave',\n on_mouse_event_closure('figure_leave')\n );\n\n canvas_div.addEventListener('wheel', function (event) {\n if (event.deltaY < 0) {\n event.step = 1;\n } else {\n event.step = -1;\n }\n on_mouse_event_closure('scroll')(event);\n });\n\n canvas_div.appendChild(canvas);\n canvas_div.appendChild(rubberband_canvas);\n\n this.rubberband_context = rubberband_canvas.getContext('2d');\n this.rubberband_context.strokeStyle = '#000000';\n\n this._resize_canvas = function (width, height, forward) {\n if (forward) {\n canvas_div.style.width = width + 'px';\n canvas_div.style.height = height + 'px';\n }\n };\n\n // Disable right mouse context menu.\n this.rubberband_canvas.addEventListener('contextmenu', function (_e) {\n event.preventDefault();\n return false;\n });\n\n function set_focus() {\n canvas.focus();\n canvas_div.focus();\n }\n\n window.setTimeout(set_focus, 100);\n};\n\nmpl.figure.prototype._init_toolbar = function () {\n var fig = this;\n\n var toolbar = document.createElement('div');\n toolbar.classList = 'mpl-toolbar';\n this.root.appendChild(toolbar);\n\n function on_click_closure(name) {\n return function (_event) {\n return fig.toolbar_button_onclick(name);\n };\n }\n\n function on_mouseover_closure(tooltip) {\n return function (event) {\n if (!event.currentTarget.disabled) {\n return fig.toolbar_button_onmouseover(tooltip);\n }\n };\n }\n\n fig.buttons = {};\n var buttonGroup = document.createElement('div');\n buttonGroup.classList = 'mpl-button-group';\n for (var toolbar_ind in mpl.toolbar_items) {\n var name = mpl.toolbar_items[toolbar_ind][0];\n var tooltip = mpl.toolbar_items[toolbar_ind][1];\n var image = mpl.toolbar_items[toolbar_ind][2];\n var method_name = mpl.toolbar_items[toolbar_ind][3];\n\n if (!name) {\n /* Instead of a spacer, we start a new button group. */\n if (buttonGroup.hasChildNodes()) {\n toolbar.appendChild(buttonGroup);\n }\n buttonGroup = document.createElement('div');\n buttonGroup.classList = 'mpl-button-group';\n continue;\n }\n\n var button = (fig.buttons[name] = document.createElement('button'));\n button.classList = 'mpl-widget';\n button.setAttribute('role', 'button');\n button.setAttribute('aria-disabled', 'false');\n button.addEventListener('click', on_click_closure(method_name));\n button.addEventListener('mouseover', on_mouseover_closure(tooltip));\n\n var icon_img = document.createElement('img');\n icon_img.src = '_images/' + image + '.png';\n icon_img.srcset = '_images/' + image + '_large.png 2x';\n icon_img.alt = tooltip;\n button.appendChild(icon_img);\n\n buttonGroup.appendChild(button);\n }\n\n if (buttonGroup.hasChildNodes()) {\n toolbar.appendChild(buttonGroup);\n }\n\n var fmt_picker = document.createElement('select');\n fmt_picker.classList = 'mpl-widget';\n toolbar.appendChild(fmt_picker);\n this.format_dropdown = fmt_picker;\n\n for (var ind in mpl.extensions) {\n var fmt = mpl.extensions[ind];\n var option = document.createElement('option');\n option.selected = fmt === mpl.default_extension;\n option.innerHTML = fmt;\n fmt_picker.appendChild(option);\n }\n\n var status_bar = document.createElement('span');\n status_bar.classList = 'mpl-message';\n toolbar.appendChild(status_bar);\n this.message = status_bar;\n};\n\nmpl.figure.prototype.request_resize = function (x_pixels, y_pixels) {\n // Request matplotlib to resize the figure. Matplotlib will then trigger a resize in the client,\n // which will in turn request a refresh of the image.\n this.send_message('resize', { width: x_pixels, height: y_pixels });\n};\n\nmpl.figure.prototype.send_message = function (type, properties) {\n properties['type'] = type;\n properties['figure_id'] = this.id;\n this.ws.send(JSON.stringify(properties));\n};\n\nmpl.figure.prototype.send_draw_message = function () {\n if (!this.waiting) {\n this.waiting = true;\n this.ws.send(JSON.stringify({ type: 'draw', figure_id: this.id }));\n }\n};\n\nmpl.figure.prototype.handle_save = function (fig, _msg) {\n var format_dropdown = fig.format_dropdown;\n var format = format_dropdown.options[format_dropdown.selectedIndex].value;\n fig.ondownload(fig, format);\n};\n\nmpl.figure.prototype.handle_resize = function (fig, msg) {\n var size = msg['size'];\n if (size[0] !== fig.canvas.width || size[1] !== fig.canvas.height) {\n fig._resize_canvas(size[0], size[1], msg['forward']);\n fig.send_message('refresh', {});\n }\n};\n\nmpl.figure.prototype.handle_rubberband = function (fig, msg) {\n var x0 = msg['x0'] / fig.ratio;\n var y0 = (fig.canvas.height - msg['y0']) / fig.ratio;\n var x1 = msg['x1'] / fig.ratio;\n var y1 = (fig.canvas.height - msg['y1']) / fig.ratio;\n x0 = Math.floor(x0) + 0.5;\n y0 = Math.floor(y0) + 0.5;\n x1 = Math.floor(x1) + 0.5;\n y1 = Math.floor(y1) + 0.5;\n var min_x = Math.min(x0, x1);\n var min_y = Math.min(y0, y1);\n var width = Math.abs(x1 - x0);\n var height = Math.abs(y1 - y0);\n\n fig.rubberband_context.clearRect(\n 0,\n 0,\n fig.canvas.width / fig.ratio,\n fig.canvas.height / fig.ratio\n );\n\n fig.rubberband_context.strokeRect(min_x, min_y, width, height);\n};\n\nmpl.figure.prototype.handle_figure_label = function (fig, msg) {\n // Updates the figure title.\n fig.header.textContent = msg['label'];\n};\n\nmpl.figure.prototype.handle_cursor = function (fig, msg) {\n var cursor = msg['cursor'];\n switch (cursor) {\n case 0:\n cursor = 'pointer';\n break;\n case 1:\n cursor = 'default';\n break;\n case 2:\n cursor = 'crosshair';\n break;\n case 3:\n cursor = 'move';\n break;\n }\n fig.rubberband_canvas.style.cursor = cursor;\n};\n\nmpl.figure.prototype.handle_message = function (fig, msg) {\n fig.message.textContent = msg['message'];\n};\n\nmpl.figure.prototype.handle_draw = function (fig, _msg) {\n // Request the server to send over a new figure.\n fig.send_draw_message();\n};\n\nmpl.figure.prototype.handle_image_mode = function (fig, msg) {\n fig.image_mode = msg['mode'];\n};\n\nmpl.figure.prototype.handle_history_buttons = function (fig, msg) {\n for (var key in msg) {\n if (!(key in fig.buttons)) {\n continue;\n }\n fig.buttons[key].disabled = !msg[key];\n fig.buttons[key].setAttribute('aria-disabled', !msg[key]);\n }\n};\n\nmpl.figure.prototype.handle_navigate_mode = function (fig, msg) {\n if (msg['mode'] === 'PAN') {\n fig.buttons['Pan'].classList.add('active');\n fig.buttons['Zoom'].classList.remove('active');\n } else if (msg['mode'] === 'ZOOM') {\n fig.buttons['Pan'].classList.remove('active');\n fig.buttons['Zoom'].classList.add('active');\n } else {\n fig.buttons['Pan'].classList.remove('active');\n fig.buttons['Zoom'].classList.remove('active');\n }\n};\n\nmpl.figure.prototype.updated_canvas_event = function () {\n // Called whenever the canvas gets updated.\n this.send_message('ack', {});\n};\n\n// A function to construct a web socket function for onmessage handling.\n// Called in the figure constructor.\nmpl.figure.prototype._make_on_message_function = function (fig) {\n return function socket_on_message(evt) {\n if (evt.data instanceof Blob) {\n var img = evt.data;\n if (img.type !== 'image/png') {\n /* FIXME: We get \"Resource interpreted as Image but\n * transferred with MIME type text/plain:\" errors on\n * Chrome. But how to set the MIME type? It doesn't seem\n * to be part of the websocket stream */\n img.type = 'image/png';\n }\n\n /* Free the memory for the previous frames */\n if (fig.imageObj.src) {\n (window.URL || window.webkitURL).revokeObjectURL(\n fig.imageObj.src\n );\n }\n\n fig.imageObj.src = (window.URL || window.webkitURL).createObjectURL(\n img\n );\n fig.updated_canvas_event();\n fig.waiting = false;\n return;\n } else if (\n typeof evt.data === 'string' &&\n evt.data.slice(0, 21) === 'data:image/png;base64'\n ) {\n fig.imageObj.src = evt.data;\n fig.updated_canvas_event();\n fig.waiting = false;\n return;\n }\n\n var msg = JSON.parse(evt.data);\n var msg_type = msg['type'];\n\n // Call the \"handle_{type}\" callback, which takes\n // the figure and JSON message as its only arguments.\n try {\n var callback = fig['handle_' + msg_type];\n } catch (e) {\n console.log(\n \"No handler for the '\" + msg_type + \"' message type: \",\n msg\n );\n return;\n }\n\n if (callback) {\n try {\n // console.log(\"Handling '\" + msg_type + \"' message: \", msg);\n callback(fig, msg);\n } catch (e) {\n console.log(\n \"Exception inside the 'handler_\" + msg_type + \"' callback:\",\n e,\n e.stack,\n msg\n );\n }\n }\n };\n};\n\n// from http://stackoverflow.com/questions/1114465/getting-mouse-location-in-canvas\nmpl.findpos = function (e) {\n //this section is from http://www.quirksmode.org/js/events_properties.html\n var targ;\n if (!e) {\n e = window.event;\n }\n if (e.target) {\n targ = e.target;\n } else if (e.srcElement) {\n targ = e.srcElement;\n }\n if (targ.nodeType === 3) {\n // defeat Safari bug\n targ = targ.parentNode;\n }\n\n // pageX,Y are the mouse positions relative to the document\n var boundingRect = targ.getBoundingClientRect();\n var x = e.pageX - (boundingRect.left + document.body.scrollLeft);\n var y = e.pageY - (boundingRect.top + document.body.scrollTop);\n\n return { x: x, y: y };\n};\n\n/*\n * return a copy of an object with only non-object keys\n * we need this to avoid circular references\n * http://stackoverflow.com/a/24161582/3208463\n */\nfunction simpleKeys(original) {\n return Object.keys(original).reduce(function (obj, key) {\n if (typeof original[key] !== 'object') {\n obj[key] = original[key];\n }\n return obj;\n }, {});\n}\n\nmpl.figure.prototype.mouse_event = function (event, name) {\n var canvas_pos = mpl.findpos(event);\n\n if (name === 'button_press') {\n this.canvas.focus();\n this.canvas_div.focus();\n }\n\n var x = canvas_pos.x * this.ratio;\n var y = canvas_pos.y * this.ratio;\n\n this.send_message(name, {\n x: x,\n y: y,\n button: event.button,\n step: event.step,\n guiEvent: simpleKeys(event),\n });\n\n /* This prevents the web browser from automatically changing to\n * the text insertion cursor when the button is pressed. We want\n * to control all of the cursor setting manually through the\n * 'cursor' event from matplotlib */\n event.preventDefault();\n return false;\n};\n\nmpl.figure.prototype._key_event_extra = function (_event, _name) {\n // Handle any extra behaviour associated with a key event\n};\n\nmpl.figure.prototype.key_event = function (event, name) {\n // Prevent repeat events\n if (name === 'key_press') {\n if (event.key === this._key) {\n return;\n } else {\n this._key = event.key;\n }\n }\n if (name === 'key_release') {\n this._key = null;\n }\n\n var value = '';\n if (event.ctrlKey && event.key !== 'Control') {\n value += 'ctrl+';\n }\n else if (event.altKey && event.key !== 'Alt') {\n value += 'alt+';\n }\n else if (event.shiftKey && event.key !== 'Shift') {\n value += 'shift+';\n }\n\n value += 'k' + event.key;\n\n this._key_event_extra(event, name);\n\n this.send_message(name, { key: value, guiEvent: simpleKeys(event) });\n return false;\n};\n\nmpl.figure.prototype.toolbar_button_onclick = function (name) {\n if (name === 'download') {\n this.handle_save(this, null);\n } else {\n this.send_message('toolbar_button', { name: name });\n }\n};\n\nmpl.figure.prototype.toolbar_button_onmouseover = function (tooltip) {\n this.message.textContent = tooltip;\n};\n\n///////////////// REMAINING CONTENT GENERATED BY embed_js.py /////////////////\n// prettier-ignore\nvar _JSXTOOLS_RESIZE_OBSERVER=function(A){var t,i=new WeakMap,n=new WeakMap,a=new WeakMap,r=new WeakMap,o=new Set;function s(e){if(!(this instanceof s))throw new TypeError(\"Constructor requires 'new' operator\");i.set(this,e)}function h(){throw new TypeError(\"Function is not a constructor\")}function c(e,t,i,n){e=0 in arguments?Number(arguments[0]):0,t=1 in arguments?Number(arguments[1]):0,i=2 in arguments?Number(arguments[2]):0,n=3 in arguments?Number(arguments[3]):0,this.right=(this.x=this.left=e)+(this.width=i),this.bottom=(this.y=this.top=t)+(this.height=n),Object.freeze(this)}function d(){t=requestAnimationFrame(d);var s=new WeakMap,p=new Set;o.forEach((function(t){r.get(t).forEach((function(i){var r=t instanceof window.SVGElement,o=a.get(t),d=r?0:parseFloat(o.paddingTop),f=r?0:parseFloat(o.paddingRight),l=r?0:parseFloat(o.paddingBottom),u=r?0:parseFloat(o.paddingLeft),g=r?0:parseFloat(o.borderTopWidth),m=r?0:parseFloat(o.borderRightWidth),w=r?0:parseFloat(o.borderBottomWidth),b=u+f,F=d+l,v=(r?0:parseFloat(o.borderLeftWidth))+m,W=g+w,y=r?0:t.offsetHeight-W-t.clientHeight,E=r?0:t.offsetWidth-v-t.clientWidth,R=b+v,z=F+W,M=r?t.width:parseFloat(o.width)-R-E,O=r?t.height:parseFloat(o.height)-z-y;if(n.has(t)){var k=n.get(t);if(k[0]===M&&k[1]===O)return}n.set(t,[M,O]);var S=Object.create(h.prototype);S.target=t,S.contentRect=new c(u,d,M,O),s.has(i)||(s.set(i,[]),p.add(i)),s.get(i).push(S)}))})),p.forEach((function(e){i.get(e).call(e,s.get(e),e)}))}return s.prototype.observe=function(i){if(i instanceof window.Element){r.has(i)||(r.set(i,new Set),o.add(i),a.set(i,window.getComputedStyle(i)));var n=r.get(i);n.has(this)||n.add(this),cancelAnimationFrame(t),t=requestAnimationFrame(d)}},s.prototype.unobserve=function(i){if(i instanceof window.Element&&r.has(i)){var n=r.get(i);n.has(this)&&(n.delete(this),n.size||(r.delete(i),o.delete(i))),n.size||r.delete(i),o.size||cancelAnimationFrame(t)}},A.DOMRectReadOnly=c,A.ResizeObserver=s,A.ResizeObserverEntry=h,A}; // eslint-disable-line\nmpl.toolbar_items = [[\"Home\", \"Reset original view\", \"fa fa-home icon-home\", \"home\"], [\"Back\", \"Back to previous view\", \"fa fa-arrow-left icon-arrow-left\", \"back\"], [\"Forward\", \"Forward to next view\", \"fa fa-arrow-right icon-arrow-right\", \"forward\"], [\"\", \"\", \"\", \"\"], [\"Pan\", \"Left button pans, Right button zooms\\nx/y fixes axis, CTRL fixes aspect\", \"fa fa-arrows icon-move\", \"pan\"], [\"Zoom\", \"Zoom to rectangle\\nx/y fixes axis, CTRL fixes aspect\", \"fa fa-square-o icon-check-empty\", \"zoom\"], [\"\", \"\", \"\", \"\"], [\"Download\", \"Download plot\", \"fa fa-floppy-o icon-save\", \"download\"]];\n\nmpl.extensions = [\"eps\", \"jpeg\", \"pgf\", \"pdf\", \"png\", \"ps\", \"raw\", \"svg\", \"tif\"];\n\nmpl.default_extension = \"png\";/* global mpl */\n\nvar comm_websocket_adapter = function (comm) {\n // Create a \"websocket\"-like object which calls the given IPython comm\n // object with the appropriate methods. Currently this is a non binary\n // socket, so there is still some room for performance tuning.\n var ws = {};\n\n ws.binaryType = comm.kernel.ws.binaryType;\n ws.readyState = comm.kernel.ws.readyState;\n function updateReadyState(_event) {\n if (comm.kernel.ws) {\n ws.readyState = comm.kernel.ws.readyState;\n } else {\n ws.readyState = 3; // Closed state.\n }\n }\n comm.kernel.ws.addEventListener('open', updateReadyState);\n comm.kernel.ws.addEventListener('close', updateReadyState);\n comm.kernel.ws.addEventListener('error', updateReadyState);\n\n ws.close = function () {\n comm.close();\n };\n ws.send = function (m) {\n //console.log('sending', m);\n comm.send(m);\n };\n // Register the callback with on_msg.\n comm.on_msg(function (msg) {\n //console.log('receiving', msg['content']['data'], msg);\n var data = msg['content']['data'];\n if (data['blob'] !== undefined) {\n data = {\n data: new Blob(msg['buffers'], { type: data['blob'] }),\n };\n }\n // Pass the mpl event to the overridden (by mpl) onmessage function.\n ws.onmessage(data);\n });\n return ws;\n};\n\nmpl.mpl_figure_comm = function (comm, msg) {\n // This is the function which gets called when the mpl process\n // starts-up an IPython Comm through the \"matplotlib\" channel.\n\n var id = msg.content.data.id;\n // Get hold of the div created by the display call when the Comm\n // socket was opened in Python.\n var element = document.getElementById(id);\n var ws_proxy = comm_websocket_adapter(comm);\n\n function ondownload(figure, _format) {\n window.open(figure.canvas.toDataURL());\n }\n\n var fig = new mpl.figure(id, ws_proxy, ondownload, element);\n\n // Call onopen now - mpl needs it, as it is assuming we've passed it a real\n // web socket which is closed, not our websocket->open comm proxy.\n ws_proxy.onopen();\n\n fig.parent_element = element;\n fig.cell_info = mpl.find_output_cell(\"<div id='\" + id + \"'></div>\");\n if (!fig.cell_info) {\n console.error('Failed to find cell for figure', id, fig);\n return;\n }\n fig.cell_info[0].output_area.element.on(\n 'cleared',\n { fig: fig },\n fig._remove_fig_handler\n );\n};\n\nmpl.figure.prototype.handle_close = function (fig, msg) {\n var width = fig.canvas.width / fig.ratio;\n fig.cell_info[0].output_area.element.off(\n 'cleared',\n fig._remove_fig_handler\n );\n fig.resizeObserverInstance.unobserve(fig.canvas_div);\n\n // Update the output cell to use the data from the current canvas.\n fig.push_to_output();\n var dataURL = fig.canvas.toDataURL();\n // Re-enable the keyboard manager in IPython - without this line, in FF,\n // the notebook keyboard shortcuts fail.\n IPython.keyboard_manager.enable();\n fig.parent_element.innerHTML =\n '<img src=\"' + dataURL + '\" width=\"' + width + '\">';\n fig.close_ws(fig, msg);\n};\n\nmpl.figure.prototype.close_ws = function (fig, msg) {\n fig.send_message('closing', msg);\n // fig.ws.close()\n};\n\nmpl.figure.prototype.push_to_output = function (_remove_interactive) {\n // Turn the data on the canvas into data in the output cell.\n var width = this.canvas.width / this.ratio;\n var dataURL = this.canvas.toDataURL();\n this.cell_info[1]['text/html'] =\n '<img src=\"' + dataURL + '\" width=\"' + width + '\">';\n};\n\nmpl.figure.prototype.updated_canvas_event = function () {\n // Tell IPython that the notebook contents must change.\n IPython.notebook.set_dirty(true);\n this.send_message('ack', {});\n var fig = this;\n // Wait a second, then push the new image to the DOM so\n // that it is saved nicely (might be nice to debounce this).\n setTimeout(function () {\n fig.push_to_output();\n }, 1000);\n};\n\nmpl.figure.prototype._init_toolbar = function () {\n var fig = this;\n\n var toolbar = document.createElement('div');\n toolbar.classList = 'btn-toolbar';\n this.root.appendChild(toolbar);\n\n function on_click_closure(name) {\n return function (_event) {\n return fig.toolbar_button_onclick(name);\n };\n }\n\n function on_mouseover_closure(tooltip) {\n return function (event) {\n if (!event.currentTarget.disabled) {\n return fig.toolbar_button_onmouseover(tooltip);\n }\n };\n }\n\n fig.buttons = {};\n var buttonGroup = document.createElement('div');\n buttonGroup.classList = 'btn-group';\n var button;\n for (var toolbar_ind in mpl.toolbar_items) {\n var name = mpl.toolbar_items[toolbar_ind][0];\n var tooltip = mpl.toolbar_items[toolbar_ind][1];\n var image = mpl.toolbar_items[toolbar_ind][2];\n var method_name = mpl.toolbar_items[toolbar_ind][3];\n\n if (!name) {\n /* Instead of a spacer, we start a new button group. */\n if (buttonGroup.hasChildNodes()) {\n toolbar.appendChild(buttonGroup);\n }\n buttonGroup = document.createElement('div');\n buttonGroup.classList = 'btn-group';\n continue;\n }\n\n button = fig.buttons[name] = document.createElement('button');\n button.classList = 'btn btn-default';\n button.href = '#';\n button.title = name;\n button.innerHTML = '<i class=\"fa ' + image + ' fa-lg\"></i>';\n button.addEventListener('click', on_click_closure(method_name));\n button.addEventListener('mouseover', on_mouseover_closure(tooltip));\n buttonGroup.appendChild(button);\n }\n\n if (buttonGroup.hasChildNodes()) {\n toolbar.appendChild(buttonGroup);\n }\n\n // Add the status bar.\n var status_bar = document.createElement('span');\n status_bar.classList = 'mpl-message pull-right';\n toolbar.appendChild(status_bar);\n this.message = status_bar;\n\n // Add the close button to the window.\n var buttongrp = document.createElement('div');\n buttongrp.classList = 'btn-group inline pull-right';\n button = document.createElement('button');\n button.classList = 'btn btn-mini btn-primary';\n button.href = '#';\n button.title = 'Stop Interaction';\n button.innerHTML = '<i class=\"fa fa-power-off icon-remove icon-large\"></i>';\n button.addEventListener('click', function (_evt) {\n fig.handle_close(fig, {});\n });\n button.addEventListener(\n 'mouseover',\n on_mouseover_closure('Stop Interaction')\n );\n buttongrp.appendChild(button);\n var titlebar = this.root.querySelector('.ui-dialog-titlebar');\n titlebar.insertBefore(buttongrp, titlebar.firstChild);\n};\n\nmpl.figure.prototype._remove_fig_handler = function (event) {\n var fig = event.data.fig;\n if (event.target !== this) {\n // Ignore bubbled events from children.\n return;\n }\n fig.close_ws(fig, {});\n};\n\nmpl.figure.prototype._root_extra_style = function (el) {\n el.style.boxSizing = 'content-box'; // override notebook setting of border-box.\n};\n\nmpl.figure.prototype._canvas_extra_style = function (el) {\n // this is important to make the div 'focusable\n el.setAttribute('tabindex', 0);\n // reach out to IPython and tell the keyboard manager to turn it's self\n // off when our div gets focus\n\n // location in version 3\n if (IPython.notebook.keyboard_manager) {\n IPython.notebook.keyboard_manager.register_events(el);\n } else {\n // location in version 2\n IPython.keyboard_manager.register_events(el);\n }\n};\n\nmpl.figure.prototype._key_event_extra = function (event, _name) {\n var manager = IPython.notebook.keyboard_manager;\n if (!manager) {\n manager = IPython.keyboard_manager;\n }\n\n // Check for shift+enter\n if (event.shiftKey && event.which === 13) {\n this.canvas_div.blur();\n // select the cell after this one\n var index = IPython.notebook.find_cell_index(this.cell_info[0]);\n IPython.notebook.select(index + 1);\n }\n};\n\nmpl.figure.prototype.handle_save = function (fig, _msg) {\n fig.ondownload(fig, null);\n};\n\nmpl.find_output_cell = function (html_output) {\n // Return the cell and output element which can be found *uniquely* in the notebook.\n // Note - this is a bit hacky, but it is done because the \"notebook_saving.Notebook\"\n // IPython event is triggered only after the cells have been serialised, which for\n // our purposes (turning an active figure into a static one), is too late.\n var cells = IPython.notebook.get_cells();\n var ncells = cells.length;\n for (var i = 0; i < ncells; i++) {\n var cell = cells[i];\n if (cell.cell_type === 'code') {\n for (var j = 0; j < cell.output_area.outputs.length; j++) {\n var data = cell.output_area.outputs[j];\n if (data.data) {\n // IPython >= 3 moved mimebundle to data attribute of output\n data = data.data;\n }\n if (data['text/html'] === html_output) {\n return [cell, data, j];\n }\n }\n }\n }\n};\n\n// Register the function which deals with the matplotlib target/channel.\n// The kernel may be null if the page has been refreshed.\nif (IPython.notebook.kernel !== null) {\n IPython.notebook.kernel.comm_manager.register_target(\n 'matplotlib',\n mpl.mpl_figure_comm\n );\n}\n",
"text/plain": [
"<IPython.core.display.Javascript object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<img src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAyAAAAEsCAYAAAA7Ldc6AAAAAXNSR0IArs4c6QAAEtpJREFUeF7t1zENAAAMw7CVP+mxyOURqGTtyc4RIECAAAECBAgQIEAgEli0Y4YAAQIECBAgQIAAAQInQDwBAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAgIED8AAECBAgQIECAAAECmYAAyagNESBAgAABAgQIECAgQPwAAQIECBAgQIAAAQKZgADJqA0RIECAAAECBAgQICBA/AABAgQIECBAgAABApmAAMmoDREgQIAAAQIECBAg8HCeAS1AYC/AAAAAAElFTkSuQmCC\" width=\"800\">"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"def cross_ent(prediction, ground_truth):\n",
" t = 1 if ground_truth > 0.5 else 0\n",
" return -t * np.log(prediction) - (1 - t) * np.log(1 - prediction)\n",
"plot_cross_ent()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Cross-entropy loss will be defined again as a separate layer, but `forward` function will have two input values: output of the previous layers of the network `p`, and the expected class `y`:"
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"1.429664938969559"
]
},
"execution_count": 27,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"class CrossEntropyLoss:\n",
" def forward(self,p,y):\n",
" self.p = p\n",
" self.y = y\n",
" p_of_y = p[np.arange(len(y)), y]\n",
" log_prob = np.log(p_of_y)\n",
" return -log_prob.mean() # average over all input samples\n",
"\n",
"cross_ent_loss = CrossEntropyLoss()\n",
"p = softmax.forward(net.forward(train_x[0:10]))\n",
"cross_ent_loss.forward(p,train_labels[0:10])"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"> **IMPORTANT**: Loss function returns a number that shows how good (or bad) our network performs. It should return us one number for the whole dataset, or for the part of the dataset (minibatch). Thus after calculating cross-entropy loss for each individual component of the input vector, we need to average (or add) all components together - which is done by the call to `.mean()`.\n",
"\n",
"## Computational Graph\n",
"\n",
"<img src=\"images/ComputeGraph.png\" width=\"600px\"/>\n",
"\n",
"Up to this moment, we have defined different classes for different layers of the network. Composition of those layers can be represented as **computational graph**. Now we can compute the loss for a given training dataset (or part of it) in the following manner:"
]
},
{
"cell_type": "code",
"execution_count": 28,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"1.429664938969559\n"
]
}
],
"source": [
"z = net.forward(train_x[0:10])\n",
"p = softmax.forward(z)\n",
"loss = cross_ent_loss.forward(p,train_labels[0:10])\n",
"print(loss)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Loss Minimization Problem and Network Training\n",
"\n",
"Once we have defined out network as $f_\\theta$, and given the loss function $\\mathcal{L}(Y,f_\\theta(X))$, we can consider $\\mathcal{L}$ as a function of $\\theta$ under our fixed training dataset: $\\mathcal{L}(\\theta) = \\mathcal{L}(Y,f_\\theta(X))$\n",
"\n",
"In this case, the network training would be a minimization problem of $\\mathcal{L}$ under argument $\\theta$:\n",
"$$\n",
"\\theta = \\mathrm{argmin}_{\\theta} \\mathcal{L}(Y,f_\\theta(X))\n",
"$$\n",
"\n",
"There is a well-known method of function optimization called **gradient descent**. The idea is that we can compute a derivative (in multi-dimensional case call **gradient**) of loss function with respect to parameters, and vary parameters in such a way that the error would decrease.\n",
"\n",
"Gradient descent works as follows:\n",
" * Initialize parameters by some random values $w^{(0)}$, $b^{(0)}$\n",
" * Repeat the following step many times:\n",
"\n",
" $$\\begin{align}\n",
" W^{(i+1)}&=W^{(i)}-\\eta\\frac{\\partial\\mathcal{L}}{\\partial W}\\\\\n",
" b^{(i+1)}&=b^{(i)}-\\eta\\frac{\\partial\\mathcal{L}}{\\partial b}\n",
" \\end{align}\n",
" $$\n",
"\n",
"During training, the optimization steps are supposed to be calculated considering the whole dataset (remember that loss is calculated as a sum/average through all training samples). However, in real life we take small portions of the dataset called **minibatches**, and calculate gradients based on a subset of data. Because subset is taken randomly each time, such method is called **stochastic gradient descent** (SGD).\n",
" "
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Backward Propagation\n",
"\n",
"<img src=\"images/ComputeGraph.png\" width=\"300px\" align=\"left\"/>\n",
"\n",
"$$\\def\\L{\\mathcal{L}}\\def\\zz#1#2{\\frac{\\partial#1}{\\partial#2}}\n",
"\\begin{align}\n",
"\\zz{\\L}{W} =& \\zz{\\L}{p}\\zz{p}{z}\\zz{z}{W}\\cr\n",
"\\zz{\\L}{b} =& \\zz{\\L}{p}\\zz{p}{z}\\zz{z}{b}\n",
"\\end{align}\n",
"$$"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"To compute $\\partial\\mathcal{L}/\\partial W$ we can use the **chaining rule** for computing derivatives of a composite function, as you can see in the formulae above. It corresponds to the following idea:\n",
"\n",
"* Suppose under given input we have obtanes loss $\\Delta\\mathcal{L}$\n",
"* To minimize it, we would have to adjust softmax output $p$ by value $\\Delta p = (\\partial\\mathcal{L}/\\partial p)\\Delta\\mathcal{L}$ \n",
"* This corresponds to the changes to node $z$ by $\\Delta z = (\\partial\\mathcal{p}/\\partial z)\\Delta p$\n",
"* To minimize this error, we need to adjust parameters accordingly: $\\Delta W = (\\partial\\mathcal{z}/\\partial W)\\Delta z$ (and the same for $b$)\n",
"\n",
"<img src=\"images/ComputeGraphGrad.PNG\" width=\"400px\" align=\"right\"/>\n",
"\n",
"This process starts distributing the loss error from the output of the network back to its parameters. Thus the process is called **back propagation**.\n",
"\n",
"One pass of the network training consists of two parts:\n",
"* **Forward pass**, when we calculate the value of loss function for a given input minibatch\n",
"* **Backward pass**, when we try to minimize this error by distributing it back to the model parameters through the computational graph."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Implementation of Back Propagation\n",
"\n",
"* Let's add `backward` function to each of our nodes that will compute the derivative and propagate the error during the backward pass.\n",
"* We also need to implement parameter updates according to the procedure described above\n",
"\n",
"We need to compute derivatives for each layer manually, for example for linear layer $z = x\\times W+b$:\n",
"$$\\begin{align}\n",
"\\frac{\\partial z}{\\partial W} &= x \\\\\n",
"\\frac{\\partial z}{\\partial b} &= 1 \\\\\n",
"\\end{align}$$\n",
"\n",
"If we need to compensate for the error $\\Delta z$ at the output of the layer, we need to update the weights accordingly:\n",
"$$\\begin{align}\n",
"\\Delta x &= \\Delta z \\times W \\\\\n",
"\\Delta W &= \\frac{\\partial z}{\\partial W} \\Delta z = \\Delta z \\times x \\\\\n",
"\\Delta b &= \\frac{\\partial z}{\\partial b} \\Delta z = \\Delta z \\\\\n",
"\\end{align}$$\n",
"\n",
"**IMPORTANT:** Calculations are done not for each training sample independently, but rather for a whole **minibatch**. Required parameter updates $\\Delta W$ and $\\Delta b$ are computed across the whole minibatch, and the respective vectors have dimensions: $x\\in\\mathbb{R}^{\\mathrm{minibatch}\\, \\times\\, \\mathrm{nclass}}$"
]
},
{
"cell_type": "code",
"execution_count": 29,
"metadata": {},
"outputs": [],
"source": [
"class Linear:\n",
" def __init__(self,nin,nout):\n",
" self.W = np.random.normal(0, 1.0/np.sqrt(nin), (nout, nin))\n",
" self.b = np.zeros((1,nout))\n",
" self.dW = np.zeros_like(self.W)\n",
" self.db = np.zeros_like(self.b)\n",
" \n",
" def forward(self, x):\n",
" self.x=x\n",
" return np.dot(x, self.W.T) + self.b\n",
" \n",
" def backward(self, dz):\n",
" dx = np.dot(dz, self.W)\n",
" dW = np.dot(dz.T, self.x)\n",
" db = dz.sum(axis=0)\n",
" self.dW = dW\n",
" self.db = db\n",
" return dx\n",
" \n",
" def update(self,lr):\n",
" self.W -= lr*self.dW\n",
" self.b -= lr*self.db"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In the same manner we can define `backward` function for the rest of our layers:"
]
},
{
"cell_type": "code",
"execution_count": 30,
"metadata": {},
"outputs": [],
"source": [
"class Softmax:\n",
" def forward(self,z):\n",
" self.z = z\n",
" zmax = z.max(axis=1,keepdims=True)\n",
" expz = np.exp(z-zmax)\n",
" Z = expz.sum(axis=1,keepdims=True)\n",
" return expz / Z\n",
" def backward(self,dp):\n",
" p = self.forward(self.z)\n",
" pdp = p * dp\n",
" return pdp - p * pdp.sum(axis=1, keepdims=True)\n",
" \n",
"class CrossEntropyLoss:\n",
" def forward(self,p,y):\n",
" self.p = p\n",
" self.y = y\n",
" p_of_y = p[np.arange(len(y)), y]\n",
" log_prob = np.log(p_of_y)\n",
" return -log_prob.mean()\n",
" def backward(self,loss):\n",
" dlog_softmax = np.zeros_like(self.p)\n",
" dlog_softmax[np.arange(len(self.y)), self.y] -= 1.0/len(self.y)\n",
" return dlog_softmax / self.p"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Training the Model\n",
"\n",
"Now we are ready to write the **training loop**, which will go through our dataset, and perform the optimization minibatch by minibatch.One complete pass through the dataset is often called **an epoch**:"
]
},
{
"cell_type": "code",
"execution_count": 31,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Initial accuracy: 0.725\n",
"Final accuracy: 0.825\n"
]
}
],
"source": [
"lin = Linear(2,2)\n",
"softmax = Softmax()\n",
"cross_ent_loss = CrossEntropyLoss()\n",
"\n",
"learning_rate = 0.1\n",
"\n",
"pred = np.argmax(lin.forward(train_x),axis=1)\n",
"acc = (pred==train_labels).mean()\n",
"print(\"Initial accuracy: \",acc)\n",
"\n",
"batch_size=4\n",
"for i in range(0,len(train_x),batch_size):\n",
" xb = train_x[i:i+batch_size]\n",
" yb = train_labels[i:i+batch_size]\n",
" \n",
" # forward pass\n",
" z = lin.forward(xb)\n",
" p = softmax.forward(z)\n",
" loss = cross_ent_loss.forward(p,yb)\n",
" \n",
" # backward pass\n",
" dp = cross_ent_loss.backward(loss)\n",
" dz = softmax.backward(dp)\n",
" dx = lin.backward(dz)\n",
" lin.update(learning_rate)\n",
" \n",
"pred = np.argmax(lin.forward(train_x),axis=1)\n",
"acc = (pred==train_labels).mean()\n",
"print(\"Final accuracy: \",acc)\n",
" "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Nice to see how we can increase accuracy of the model from about 50% to around 80% in one epoch.\n",
"\n",
"## Network Class\n",
"\n",
"Since in many cases neural network is just a composition of layers, we can build a class that will allow us to stack layers together and make forward and backward passes through them without explicitly programming that logic. We will store the list of layers inside the `Net` class, and use `add()` function to add new layers:"
]
},
{
"cell_type": "code",
"execution_count": 32,
"metadata": {
"scrolled": true,
"slideshow": {
"slide_type": "skip"
}
},
"outputs": [],
"source": [
"class Net:\n",
" def __init__(self):\n",
" self.layers = []\n",
" \n",
" def add(self,l):\n",
" self.layers.append(l)\n",
" \n",
" def forward(self,x):\n",
" for l in self.layers:\n",
" x = l.forward(x)\n",
" return x\n",
" \n",
" def backward(self,z):\n",
" for l in self.layers[::-1]:\n",
" z = l.backward(z)\n",
" return z\n",
" \n",
" def update(self,lr):\n",
" for l in self.layers:\n",
" if 'update' in l.__dir__():\n",
" l.update(lr)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"With this `Net` class our model definition and training becomes more neat:"
]
},
{
"cell_type": "code",
"execution_count": 33,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Initial loss=0.6212072429381601, accuracy=0.6875: \n",
"Final loss=0.44369925927417986, accuracy=0.8: \n",
"Test loss=0.4767711377257787, accuracy=0.85: \n"
]
}
],
"source": [
"net = Net()\n",
"net.add(Linear(2,2))\n",
"net.add(Softmax())\n",
"loss = CrossEntropyLoss()\n",
"\n",
"def get_loss_acc(x,y,loss=CrossEntropyLoss()):\n",
" p = net.forward(x)\n",
" l = loss.forward(p,y)\n",
" pred = np.argmax(p,axis=1)\n",
" acc = (pred==y).mean()\n",
" return l,acc\n",
"\n",
"print(\"Initial loss={}, accuracy={}: \".format(*get_loss_acc(train_x,train_labels)))\n",
"\n",
"def train_epoch(net, train_x, train_labels, loss=CrossEntropyLoss(), batch_size=4, lr=0.1):\n",
" for i in range(0,len(train_x),batch_size):\n",
" xb = train_x[i:i+batch_size]\n",
" yb = train_labels[i:i+batch_size]\n",
"\n",
" p = net.forward(xb)\n",
" l = loss.forward(p,yb)\n",
" dp = loss.backward(l)\n",
" dx = net.backward(dp)\n",
" net.update(lr)\n",
" \n",
"train_epoch(net,train_x,train_labels)\n",
" \n",
"print(\"Final loss={}, accuracy={}: \".format(*get_loss_acc(train_x,train_labels)))\n",
"print(\"Test loss={}, accuracy={}: \".format(*get_loss_acc(test_x,test_labels)))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Plotting the Training Process\n",
"\n",
"It would be nice to see visually how the network is being trained! We will define a `train_and_plot` function for that. To visualize the state of the network we will use level map, i.e. we will represent different values of the network output using different colors.\n",
"\n",
"> Do not worry if you do not understand some of the plotting code below - it is more important to understand the underlying neural network concepts."
]
},
{
"cell_type": "code",
"execution_count": 34,
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"outputs": [],
"source": [
"def train_and_plot(n_epoch, net, loss=CrossEntropyLoss(), batch_size=4, lr=0.1):\n",
" fig, ax = plt.subplots(2, 1)\n",
" ax[0].set_xlim(0, n_epoch + 1)\n",
" ax[0].set_ylim(0,1)\n",
"\n",
" train_acc = np.empty((n_epoch, 3))\n",
" train_acc[:] = np.NAN\n",
" valid_acc = np.empty((n_epoch, 3))\n",
" valid_acc[:] = np.NAN\n",
"\n",
" for epoch in range(1, n_epoch + 1):\n",
"\n",
" train_epoch(net,train_x,train_labels,loss,batch_size,lr)\n",
" tloss, taccuracy = get_loss_acc(train_x,train_labels,loss)\n",
" train_acc[epoch-1, :] = [epoch, tloss, taccuracy]\n",
" vloss, vaccuracy = get_loss_acc(test_x,test_labels,loss)\n",
" valid_acc[epoch-1, :] = [epoch, vloss, vaccuracy]\n",
" \n",
" ax[0].set_ylim(0, max(max(train_acc[:, 2]), max(valid_acc[:, 2])) * 1.1)\n",
"\n",
" plot_training_progress(train_acc[:, 0], (train_acc[:, 2],\n",
" valid_acc[:, 2]), fig, ax[0])\n",
" plot_decision_boundary(net, fig, ax[1])\n",
" fig.canvas.draw()\n",
" fig.canvas.flush_events()\n",
"\n",
" return train_acc, valid_acc"
]
},
{
"cell_type": "code",
"execution_count": 35,
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"outputs": [],
"source": [
"import matplotlib.cm as cm\n",
"\n",
"def plot_decision_boundary(net, fig, ax):\n",
" draw_colorbar = True\n",
" # remove previous plot\n",
" while ax.collections:\n",
" ax.collections.pop()\n",
" draw_colorbar = False\n",
"\n",
" # generate countour grid\n",
" x_min, x_max = train_x[:, 0].min() - 1, train_x[:, 0].max() + 1\n",
" y_min, y_max = train_x[:, 1].min() - 1, train_x[:, 1].max() + 1\n",
" xx, yy = np.meshgrid(np.arange(x_min, x_max, 0.1),\n",
" np.arange(y_min, y_max, 0.1))\n",
" grid_points = np.c_[xx.ravel().astype('float32'), yy.ravel().astype('float32')]\n",
" n_classes = max(train_labels)+1\n",
" while train_x.shape[1] > grid_points.shape[1]:\n",
" # pad dimensions (plot only the first two)\n",
" grid_points = np.c_[grid_points,\n",
" np.empty(len(xx.ravel())).astype('float32')]\n",
" grid_points[:, -1].fill(train_x[:, grid_points.shape[1]-1].mean())\n",
"\n",
" # evaluate predictions\n",
" prediction = np.array(net.forward(grid_points))\n",
" # for two classes: prediction difference\n",
" if (n_classes == 2):\n",
" Z = np.array([0.5+(p[0]-p[1])/2.0 for p in prediction]).reshape(xx.shape)\n",
" else:\n",
" Z = np.array([p.argsort()[-1]/float(n_classes-1) for p in prediction]).reshape(xx.shape)\n",
" \n",
" # draw contour\n",
" levels = np.linspace(0, 1, 40)\n",
" cs = ax.contourf(xx, yy, Z, alpha=0.4, levels = levels)\n",
" if draw_colorbar:\n",
" fig.colorbar(cs, ax=ax, ticks = [0, 0.5, 1])\n",
" c_map = [cm.jet(x) for x in np.linspace(0.0, 1.0, n_classes) ]\n",
" colors = [c_map[l] for l in train_labels]\n",
" ax.scatter(train_x[:, 0], train_x[:, 1], marker='o', c=colors, s=60, alpha = 0.5)"
]
},
{
"cell_type": "code",
"execution_count": 36,
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"outputs": [],
"source": [
"def plot_training_progress(x, y_data, fig, ax):\n",
" styles = ['k--', 'g-']\n",
" # remove previous plot\n",
" while ax.lines:\n",
" ax.lines.pop()\n",
" # draw updated lines\n",
" for i in range(len(y_data)):\n",
" ax.plot(x, y_data[i], styles[i])\n",
" ax.legend(ax.lines, ['training accuracy', 'validation accuracy'],\n",
" loc='upper center', ncol = 2)"
]
},
{
"cell_type": "code",
"execution_count": 37,
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [
{
"data": {
"application/javascript": "/* Put everything inside the global mpl namespace */\n/* global mpl */\nwindow.mpl = {};\n\nmpl.get_websocket_type = function () {\n if (typeof WebSocket !== 'undefined') {\n return WebSocket;\n } else if (typeof MozWebSocket !== 'undefined') {\n return MozWebSocket;\n } else {\n alert(\n 'Your browser does not have WebSocket support. ' +\n 'Please try Chrome, Safari or Firefox ≥ 6. ' +\n 'Firefox 4 and 5 are also supported but you ' +\n 'have to enable WebSockets in about:config.'\n );\n }\n};\n\nmpl.figure = function (figure_id, websocket, ondownload, parent_element) {\n this.id = figure_id;\n\n this.ws = websocket;\n\n this.supports_binary = this.ws.binaryType !== undefined;\n\n if (!this.supports_binary) {\n var warnings = document.getElementById('mpl-warnings');\n if (warnings) {\n warnings.style.display = 'block';\n warnings.textContent =\n 'This browser does not support binary websocket messages. ' +\n 'Performance may be slow.';\n }\n }\n\n this.imageObj = new Image();\n\n this.context = undefined;\n this.message = undefined;\n this.canvas = undefined;\n this.rubberband_canvas = undefined;\n this.rubberband_context = undefined;\n this.format_dropdown = undefined;\n\n this.image_mode = 'full';\n\n this.root = document.createElement('div');\n this.root.setAttribute('style', 'display: inline-block');\n this._root_extra_style(this.root);\n\n parent_element.appendChild(this.root);\n\n this._init_header(this);\n this._init_canvas(this);\n this._init_toolbar(this);\n\n var fig = this;\n\n this.waiting = false;\n\n this.ws.onopen = function () {\n fig.send_message('supports_binary', { value: fig.supports_binary });\n fig.send_message('send_image_mode', {});\n if (fig.ratio !== 1) {\n fig.send_message('set_dpi_ratio', { dpi_ratio: fig.ratio });\n }\n fig.send_message('refresh', {});\n };\n\n this.imageObj.onload = function () {\n if (fig.image_mode === 'full') {\n // Full images could contain transparency (where diff images\n // almost always do), so we need to clear the canvas so that\n // there is no ghosting.\n fig.context.clearRect(0, 0, fig.canvas.width, fig.canvas.height);\n }\n fig.context.drawImage(fig.imageObj, 0, 0);\n };\n\n this.imageObj.onunload = function () {\n fig.ws.close();\n };\n\n this.ws.onmessage = this._make_on_message_function(this);\n\n this.ondownload = ondownload;\n};\n\nmpl.figure.prototype._init_header = function () {\n var titlebar = document.createElement('div');\n titlebar.classList =\n 'ui-dialog-titlebar ui-widget-header ui-corner-all ui-helper-clearfix';\n var titletext = document.createElement('div');\n titletext.classList = 'ui-dialog-title';\n titletext.setAttribute(\n 'style',\n 'width: 100%; text-align: center; padding: 3px;'\n );\n titlebar.appendChild(titletext);\n this.root.appendChild(titlebar);\n this.header = titletext;\n};\n\nmpl.figure.prototype._canvas_extra_style = function (_canvas_div) {};\n\nmpl.figure.prototype._root_extra_style = function (_canvas_div) {};\n\nmpl.figure.prototype._init_canvas = function () {\n var fig = this;\n\n var canvas_div = (this.canvas_div = document.createElement('div'));\n canvas_div.setAttribute(\n 'style',\n 'border: 1px solid #ddd;' +\n 'box-sizing: content-box;' +\n 'clear: both;' +\n 'min-height: 1px;' +\n 'min-width: 1px;' +\n 'outline: 0;' +\n 'overflow: hidden;' +\n 'position: relative;' +\n 'resize: both;'\n );\n\n function on_keyboard_event_closure(name) {\n return function (event) {\n return fig.key_event(event, name);\n };\n }\n\n canvas_div.addEventListener(\n 'keydown',\n on_keyboard_event_closure('key_press')\n );\n canvas_div.addEventListener(\n 'keyup',\n on_keyboard_event_closure('key_release')\n );\n\n this._canvas_extra_style(canvas_div);\n this.root.appendChild(canvas_div);\n\n var canvas = (this.canvas = document.createElement('canvas'));\n canvas.classList.add('mpl-canvas');\n canvas.setAttribute('style', 'box-sizing: content-box;');\n\n this.context = canvas.getContext('2d');\n\n var backingStore =\n this.context.backingStorePixelRatio ||\n this.context.webkitBackingStorePixelRatio ||\n this.context.mozBackingStorePixelRatio ||\n this.context.msBackingStorePixelRatio ||\n this.context.oBackingStorePixelRatio ||\n this.context.backingStorePixelRatio ||\n 1;\n\n this.ratio = (window.devicePixelRatio || 1) / backingStore;\n\n var rubberband_canvas = (this.rubberband_canvas = document.createElement(\n 'canvas'\n ));\n rubberband_canvas.setAttribute(\n 'style',\n 'box-sizing: content-box; position: absolute; left: 0; top: 0; z-index: 1;'\n );\n\n // Apply a ponyfill if ResizeObserver is not implemented by browser.\n if (this.ResizeObserver === undefined) {\n if (window.ResizeObserver !== undefined) {\n this.ResizeObserver = window.ResizeObserver;\n } else {\n var obs = _JSXTOOLS_RESIZE_OBSERVER({});\n this.ResizeObserver = obs.ResizeObserver;\n }\n }\n\n this.resizeObserverInstance = new this.ResizeObserver(function (entries) {\n var nentries = entries.length;\n for (var i = 0; i < nentries; i++) {\n var entry = entries[i];\n var width, height;\n if (entry.contentBoxSize) {\n if (entry.contentBoxSize instanceof Array) {\n // Chrome 84 implements new version of spec.\n width = entry.contentBoxSize[0].inlineSize;\n height = entry.contentBoxSize[0].blockSize;\n } else {\n // Firefox implements old version of spec.\n width = entry.contentBoxSize.inlineSize;\n height = entry.contentBoxSize.blockSize;\n }\n } else {\n // Chrome <84 implements even older version of spec.\n width = entry.contentRect.width;\n height = entry.contentRect.height;\n }\n\n // Keep the size of the canvas and rubber band canvas in sync with\n // the canvas container.\n if (entry.devicePixelContentBoxSize) {\n // Chrome 84 implements new version of spec.\n canvas.setAttribute(\n 'width',\n entry.devicePixelContentBoxSize[0].inlineSize\n );\n canvas.setAttribute(\n 'height',\n entry.devicePixelContentBoxSize[0].blockSize\n );\n } else {\n canvas.setAttribute('width', width * fig.ratio);\n canvas.setAttribute('height', height * fig.ratio);\n }\n canvas.setAttribute(\n 'style',\n 'width: ' + width + 'px; height: ' + height + 'px;'\n );\n\n rubberband_canvas.setAttribute('width', width);\n rubberband_canvas.setAttribute('height', height);\n\n // And update the size in Python. We ignore the initial 0/0 size\n // that occurs as the element is placed into the DOM, which should\n // otherwise not happen due to the minimum size styling.\n if (fig.ws.readyState == 1 && width != 0 && height != 0) {\n fig.request_resize(width, height);\n }\n }\n });\n this.resizeObserverInstance.observe(canvas_div);\n\n function on_mouse_event_closure(name) {\n return function (event) {\n return fig.mouse_event(event, name);\n };\n }\n\n rubberband_canvas.addEventListener(\n 'mousedown',\n on_mouse_event_closure('button_press')\n );\n rubberband_canvas.addEventListener(\n 'mouseup',\n on_mouse_event_closure('button_release')\n );\n rubberband_canvas.addEventListener(\n 'dblclick',\n on_mouse_event_closure('dblclick')\n );\n // Throttle sequential mouse events to 1 every 20ms.\n rubberband_canvas.addEventListener(\n 'mousemove',\n on_mouse_event_closure('motion_notify')\n );\n\n rubberband_canvas.addEventListener(\n 'mouseenter',\n on_mouse_event_closure('figure_enter')\n );\n rubberband_canvas.addEventListener(\n 'mouseleave',\n on_mouse_event_closure('figure_leave')\n );\n\n canvas_div.addEventListener('wheel', function (event) {\n if (event.deltaY < 0) {\n event.step = 1;\n } else {\n event.step = -1;\n }\n on_mouse_event_closure('scroll')(event);\n });\n\n canvas_div.appendChild(canvas);\n canvas_div.appendChild(rubberband_canvas);\n\n this.rubberband_context = rubberband_canvas.getContext('2d');\n this.rubberband_context.strokeStyle = '#000000';\n\n this._resize_canvas = function (width, height, forward) {\n if (forward) {\n canvas_div.style.width = width + 'px';\n canvas_div.style.height = height + 'px';\n }\n };\n\n // Disable right mouse context menu.\n this.rubberband_canvas.addEventListener('contextmenu', function (_e) {\n event.preventDefault();\n return false;\n });\n\n function set_focus() {\n canvas.focus();\n canvas_div.focus();\n }\n\n window.setTimeout(set_focus, 100);\n};\n\nmpl.figure.prototype._init_toolbar = function () {\n var fig = this;\n\n var toolbar = document.createElement('div');\n toolbar.classList = 'mpl-toolbar';\n this.root.appendChild(toolbar);\n\n function on_click_closure(name) {\n return function (_event) {\n return fig.toolbar_button_onclick(name);\n };\n }\n\n function on_mouseover_closure(tooltip) {\n return function (event) {\n if (!event.currentTarget.disabled) {\n return fig.toolbar_button_onmouseover(tooltip);\n }\n };\n }\n\n fig.buttons = {};\n var buttonGroup = document.createElement('div');\n buttonGroup.classList = 'mpl-button-group';\n for (var toolbar_ind in mpl.toolbar_items) {\n var name = mpl.toolbar_items[toolbar_ind][0];\n var tooltip = mpl.toolbar_items[toolbar_ind][1];\n var image = mpl.toolbar_items[toolbar_ind][2];\n var method_name = mpl.toolbar_items[toolbar_ind][3];\n\n if (!name) {\n /* Instead of a spacer, we start a new button group. */\n if (buttonGroup.hasChildNodes()) {\n toolbar.appendChild(buttonGroup);\n }\n buttonGroup = document.createElement('div');\n buttonGroup.classList = 'mpl-button-group';\n continue;\n }\n\n var button = (fig.buttons[name] = document.createElement('button'));\n button.classList = 'mpl-widget';\n button.setAttribute('role', 'button');\n button.setAttribute('aria-disabled', 'false');\n button.addEventListener('click', on_click_closure(method_name));\n button.addEventListener('mouseover', on_mouseover_closure(tooltip));\n\n var icon_img = document.createElement('img');\n icon_img.src = '_images/' + image + '.png';\n icon_img.srcset = '_images/' + image + '_large.png 2x';\n icon_img.alt = tooltip;\n button.appendChild(icon_img);\n\n buttonGroup.appendChild(button);\n }\n\n if (buttonGroup.hasChildNodes()) {\n toolbar.appendChild(buttonGroup);\n }\n\n var fmt_picker = document.createElement('select');\n fmt_picker.classList = 'mpl-widget';\n toolbar.appendChild(fmt_picker);\n this.format_dropdown = fmt_picker;\n\n for (var ind in mpl.extensions) {\n var fmt = mpl.extensions[ind];\n var option = document.createElement('option');\n option.selected = fmt === mpl.default_extension;\n option.innerHTML = fmt;\n fmt_picker.appendChild(option);\n }\n\n var status_bar = document.createElement('span');\n status_bar.classList = 'mpl-message';\n toolbar.appendChild(status_bar);\n this.message = status_bar;\n};\n\nmpl.figure.prototype.request_resize = function (x_pixels, y_pixels) {\n // Request matplotlib to resize the figure. Matplotlib will then trigger a resize in the client,\n // which will in turn request a refresh of the image.\n this.send_message('resize', { width: x_pixels, height: y_pixels });\n};\n\nmpl.figure.prototype.send_message = function (type, properties) {\n properties['type'] = type;\n properties['figure_id'] = this.id;\n this.ws.send(JSON.stringify(properties));\n};\n\nmpl.figure.prototype.send_draw_message = function () {\n if (!this.waiting) {\n this.waiting = true;\n this.ws.send(JSON.stringify({ type: 'draw', figure_id: this.id }));\n }\n};\n\nmpl.figure.prototype.handle_save = function (fig, _msg) {\n var format_dropdown = fig.format_dropdown;\n var format = format_dropdown.options[format_dropdown.selectedIndex].value;\n fig.ondownload(fig, format);\n};\n\nmpl.figure.prototype.handle_resize = function (fig, msg) {\n var size = msg['size'];\n if (size[0] !== fig.canvas.width || size[1] !== fig.canvas.height) {\n fig._resize_canvas(size[0], size[1], msg['forward']);\n fig.send_message('refresh', {});\n }\n};\n\nmpl.figure.prototype.handle_rubberband = function (fig, msg) {\n var x0 = msg['x0'] / fig.ratio;\n var y0 = (fig.canvas.height - msg['y0']) / fig.ratio;\n var x1 = msg['x1'] / fig.ratio;\n var y1 = (fig.canvas.height - msg['y1']) / fig.ratio;\n x0 = Math.floor(x0) + 0.5;\n y0 = Math.floor(y0) + 0.5;\n x1 = Math.floor(x1) + 0.5;\n y1 = Math.floor(y1) + 0.5;\n var min_x = Math.min(x0, x1);\n var min_y = Math.min(y0, y1);\n var width = Math.abs(x1 - x0);\n var height = Math.abs(y1 - y0);\n\n fig.rubberband_context.clearRect(\n 0,\n 0,\n fig.canvas.width / fig.ratio,\n fig.canvas.height / fig.ratio\n );\n\n fig.rubberband_context.strokeRect(min_x, min_y, width, height);\n};\n\nmpl.figure.prototype.handle_figure_label = function (fig, msg) {\n // Updates the figure title.\n fig.header.textContent = msg['label'];\n};\n\nmpl.figure.prototype.handle_cursor = function (fig, msg) {\n var cursor = msg['cursor'];\n switch (cursor) {\n case 0:\n cursor = 'pointer';\n break;\n case 1:\n cursor = 'default';\n break;\n case 2:\n cursor = 'crosshair';\n break;\n case 3:\n cursor = 'move';\n break;\n }\n fig.rubberband_canvas.style.cursor = cursor;\n};\n\nmpl.figure.prototype.handle_message = function (fig, msg) {\n fig.message.textContent = msg['message'];\n};\n\nmpl.figure.prototype.handle_draw = function (fig, _msg) {\n // Request the server to send over a new figure.\n fig.send_draw_message();\n};\n\nmpl.figure.prototype.handle_image_mode = function (fig, msg) {\n fig.image_mode = msg['mode'];\n};\n\nmpl.figure.prototype.handle_history_buttons = function (fig, msg) {\n for (var key in msg) {\n if (!(key in fig.buttons)) {\n continue;\n }\n fig.buttons[key].disabled = !msg[key];\n fig.buttons[key].setAttribute('aria-disabled', !msg[key]);\n }\n};\n\nmpl.figure.prototype.handle_navigate_mode = function (fig, msg) {\n if (msg['mode'] === 'PAN') {\n fig.buttons['Pan'].classList.add('active');\n fig.buttons['Zoom'].classList.remove('active');\n } else if (msg['mode'] === 'ZOOM') {\n fig.buttons['Pan'].classList.remove('active');\n fig.buttons['Zoom'].classList.add('active');\n } else {\n fig.buttons['Pan'].classList.remove('active');\n fig.buttons['Zoom'].classList.remove('active');\n }\n};\n\nmpl.figure.prototype.updated_canvas_event = function () {\n // Called whenever the canvas gets updated.\n this.send_message('ack', {});\n};\n\n// A function to construct a web socket function for onmessage handling.\n// Called in the figure constructor.\nmpl.figure.prototype._make_on_message_function = function (fig) {\n return function socket_on_message(evt) {\n if (evt.data instanceof Blob) {\n var img = evt.data;\n if (img.type !== 'image/png') {\n /* FIXME: We get \"Resource interpreted as Image but\n * transferred with MIME type text/plain:\" errors on\n * Chrome. But how to set the MIME type? It doesn't seem\n * to be part of the websocket stream */\n img.type = 'image/png';\n }\n\n /* Free the memory for the previous frames */\n if (fig.imageObj.src) {\n (window.URL || window.webkitURL).revokeObjectURL(\n fig.imageObj.src\n );\n }\n\n fig.imageObj.src = (window.URL || window.webkitURL).createObjectURL(\n img\n );\n fig.updated_canvas_event();\n fig.waiting = false;\n return;\n } else if (\n typeof evt.data === 'string' &&\n evt.data.slice(0, 21) === 'data:image/png;base64'\n ) {\n fig.imageObj.src = evt.data;\n fig.updated_canvas_event();\n fig.waiting = false;\n return;\n }\n\n var msg = JSON.parse(evt.data);\n var msg_type = msg['type'];\n\n // Call the \"handle_{type}\" callback, which takes\n // the figure and JSON message as its only arguments.\n try {\n var callback = fig['handle_' + msg_type];\n } catch (e) {\n console.log(\n \"No handler for the '\" + msg_type + \"' message type: \",\n msg\n );\n return;\n }\n\n if (callback) {\n try {\n // console.log(\"Handling '\" + msg_type + \"' message: \", msg);\n callback(fig, msg);\n } catch (e) {\n console.log(\n \"Exception inside the 'handler_\" + msg_type + \"' callback:\",\n e,\n e.stack,\n msg\n );\n }\n }\n };\n};\n\n// from http://stackoverflow.com/questions/1114465/getting-mouse-location-in-canvas\nmpl.findpos = function (e) {\n //this section is from http://www.quirksmode.org/js/events_properties.html\n var targ;\n if (!e) {\n e = window.event;\n }\n if (e.target) {\n targ = e.target;\n } else if (e.srcElement) {\n targ = e.srcElement;\n }\n if (targ.nodeType === 3) {\n // defeat Safari bug\n targ = targ.parentNode;\n }\n\n // pageX,Y are the mouse positions relative to the document\n var boundingRect = targ.getBoundingClientRect();\n var x = e.pageX - (boundingRect.left + document.body.scrollLeft);\n var y = e.pageY - (boundingRect.top + document.body.scrollTop);\n\n return { x: x, y: y };\n};\n\n/*\n * return a copy of an object with only non-object keys\n * we need this to avoid circular references\n * http://stackoverflow.com/a/24161582/3208463\n */\nfunction simpleKeys(original) {\n return Object.keys(original).reduce(function (obj, key) {\n if (typeof original[key] !== 'object') {\n obj[key] = original[key];\n }\n return obj;\n }, {});\n}\n\nmpl.figure.prototype.mouse_event = function (event, name) {\n var canvas_pos = mpl.findpos(event);\n\n if (name === 'button_press') {\n this.canvas.focus();\n this.canvas_div.focus();\n }\n\n var x = canvas_pos.x * this.ratio;\n var y = canvas_pos.y * this.ratio;\n\n this.send_message(name, {\n x: x,\n y: y,\n button: event.button,\n step: event.step,\n guiEvent: simpleKeys(event),\n });\n\n /* This prevents the web browser from automatically changing to\n * the text insertion cursor when the button is pressed. We want\n * to control all of the cursor setting manually through the\n * 'cursor' event from matplotlib */\n event.preventDefault();\n return false;\n};\n\nmpl.figure.prototype._key_event_extra = function (_event, _name) {\n // Handle any extra behaviour associated with a key event\n};\n\nmpl.figure.prototype.key_event = function (event, name) {\n // Prevent repeat events\n if (name === 'key_press') {\n if (event.key === this._key) {\n return;\n } else {\n this._key = event.key;\n }\n }\n if (name === 'key_release') {\n this._key = null;\n }\n\n var value = '';\n if (event.ctrlKey && event.key !== 'Control') {\n value += 'ctrl+';\n }\n else if (event.altKey && event.key !== 'Alt') {\n value += 'alt+';\n }\n else if (event.shiftKey && event.key !== 'Shift') {\n value += 'shift+';\n }\n\n value += 'k' + event.key;\n\n this._key_event_extra(event, name);\n\n this.send_message(name, { key: value, guiEvent: simpleKeys(event) });\n return false;\n};\n\nmpl.figure.prototype.toolbar_button_onclick = function (name) {\n if (name === 'download') {\n this.handle_save(this, null);\n } else {\n this.send_message('toolbar_button', { name: name });\n }\n};\n\nmpl.figure.prototype.toolbar_button_onmouseover = function (tooltip) {\n this.message.textContent = tooltip;\n};\n\n///////////////// REMAINING CONTENT GENERATED BY embed_js.py /////////////////\n// prettier-ignore\nvar _JSXTOOLS_RESIZE_OBSERVER=function(A){var t,i=new WeakMap,n=new WeakMap,a=new WeakMap,r=new WeakMap,o=new Set;function s(e){if(!(this instanceof s))throw new TypeError(\"Constructor requires 'new' operator\");i.set(this,e)}function h(){throw new TypeError(\"Function is not a constructor\")}function c(e,t,i,n){e=0 in arguments?Number(arguments[0]):0,t=1 in arguments?Number(arguments[1]):0,i=2 in arguments?Number(arguments[2]):0,n=3 in arguments?Number(arguments[3]):0,this.right=(this.x=this.left=e)+(this.width=i),this.bottom=(this.y=this.top=t)+(this.height=n),Object.freeze(this)}function d(){t=requestAnimationFrame(d);var s=new WeakMap,p=new Set;o.forEach((function(t){r.get(t).forEach((function(i){var r=t instanceof window.SVGElement,o=a.get(t),d=r?0:parseFloat(o.paddingTop),f=r?0:parseFloat(o.paddingRight),l=r?0:parseFloat(o.paddingBottom),u=r?0:parseFloat(o.paddingLeft),g=r?0:parseFloat(o.borderTopWidth),m=r?0:parseFloat(o.borderRightWidth),w=r?0:parseFloat(o.borderBottomWidth),b=u+f,F=d+l,v=(r?0:parseFloat(o.borderLeftWidth))+m,W=g+w,y=r?0:t.offsetHeight-W-t.clientHeight,E=r?0:t.offsetWidth-v-t.clientWidth,R=b+v,z=F+W,M=r?t.width:parseFloat(o.width)-R-E,O=r?t.height:parseFloat(o.height)-z-y;if(n.has(t)){var k=n.get(t);if(k[0]===M&&k[1]===O)return}n.set(t,[M,O]);var S=Object.create(h.prototype);S.target=t,S.contentRect=new c(u,d,M,O),s.has(i)||(s.set(i,[]),p.add(i)),s.get(i).push(S)}))})),p.forEach((function(e){i.get(e).call(e,s.get(e),e)}))}return s.prototype.observe=function(i){if(i instanceof window.Element){r.has(i)||(r.set(i,new Set),o.add(i),a.set(i,window.getComputedStyle(i)));var n=r.get(i);n.has(this)||n.add(this),cancelAnimationFrame(t),t=requestAnimationFrame(d)}},s.prototype.unobserve=function(i){if(i instanceof window.Element&&r.has(i)){var n=r.get(i);n.has(this)&&(n.delete(this),n.size||(r.delete(i),o.delete(i))),n.size||r.delete(i),o.size||cancelAnimationFrame(t)}},A.DOMRectReadOnly=c,A.ResizeObserver=s,A.ResizeObserverEntry=h,A}; // eslint-disable-line\nmpl.toolbar_items = [[\"Home\", \"Reset original view\", \"fa fa-home icon-home\", \"home\"], [\"Back\", \"Back to previous view\", \"fa fa-arrow-left icon-arrow-left\", \"back\"], [\"Forward\", \"Forward to next view\", \"fa fa-arrow-right icon-arrow-right\", \"forward\"], [\"\", \"\", \"\", \"\"], [\"Pan\", \"Left button pans, Right button zooms\\nx/y fixes axis, CTRL fixes aspect\", \"fa fa-arrows icon-move\", \"pan\"], [\"Zoom\", \"Zoom to rectangle\\nx/y fixes axis, CTRL fixes aspect\", \"fa fa-square-o icon-check-empty\", \"zoom\"], [\"\", \"\", \"\", \"\"], [\"Download\", \"Download plot\", \"fa fa-floppy-o icon-save\", \"download\"]];\n\nmpl.extensions = [\"eps\", \"jpeg\", \"pgf\", \"pdf\", \"png\", \"ps\", \"raw\", \"svg\", \"tif\"];\n\nmpl.default_extension = \"png\";/* global mpl */\n\nvar comm_websocket_adapter = function (comm) {\n // Create a \"websocket\"-like object which calls the given IPython comm\n // object with the appropriate methods. Currently this is a non binary\n // socket, so there is still some room for performance tuning.\n var ws = {};\n\n ws.binaryType = comm.kernel.ws.binaryType;\n ws.readyState = comm.kernel.ws.readyState;\n function updateReadyState(_event) {\n if (comm.kernel.ws) {\n ws.readyState = comm.kernel.ws.readyState;\n } else {\n ws.readyState = 3; // Closed state.\n }\n }\n comm.kernel.ws.addEventListener('open', updateReadyState);\n comm.kernel.ws.addEventListener('close', updateReadyState);\n comm.kernel.ws.addEventListener('error', updateReadyState);\n\n ws.close = function () {\n comm.close();\n };\n ws.send = function (m) {\n //console.log('sending', m);\n comm.send(m);\n };\n // Register the callback with on_msg.\n comm.on_msg(function (msg) {\n //console.log('receiving', msg['content']['data'], msg);\n var data = msg['content']['data'];\n if (data['blob'] !== undefined) {\n data = {\n data: new Blob(msg['buffers'], { type: data['blob'] }),\n };\n }\n // Pass the mpl event to the overridden (by mpl) onmessage function.\n ws.onmessage(data);\n });\n return ws;\n};\n\nmpl.mpl_figure_comm = function (comm, msg) {\n // This is the function which gets called when the mpl process\n // starts-up an IPython Comm through the \"matplotlib\" channel.\n\n var id = msg.content.data.id;\n // Get hold of the div created by the display call when the Comm\n // socket was opened in Python.\n var element = document.getElementById(id);\n var ws_proxy = comm_websocket_adapter(comm);\n\n function ondownload(figure, _format) {\n window.open(figure.canvas.toDataURL());\n }\n\n var fig = new mpl.figure(id, ws_proxy, ondownload, element);\n\n // Call onopen now - mpl needs it, as it is assuming we've passed it a real\n // web socket which is closed, not our websocket->open comm proxy.\n ws_proxy.onopen();\n\n fig.parent_element = element;\n fig.cell_info = mpl.find_output_cell(\"<div id='\" + id + \"'></div>\");\n if (!fig.cell_info) {\n console.error('Failed to find cell for figure', id, fig);\n return;\n }\n fig.cell_info[0].output_area.element.on(\n 'cleared',\n { fig: fig },\n fig._remove_fig_handler\n );\n};\n\nmpl.figure.prototype.handle_close = function (fig, msg) {\n var width = fig.canvas.width / fig.ratio;\n fig.cell_info[0].output_area.element.off(\n 'cleared',\n fig._remove_fig_handler\n );\n fig.resizeObserverInstance.unobserve(fig.canvas_div);\n\n // Update the output cell to use the data from the current canvas.\n fig.push_to_output();\n var dataURL = fig.canvas.toDataURL();\n // Re-enable the keyboard manager in IPython - without this line, in FF,\n // the notebook keyboard shortcuts fail.\n IPython.keyboard_manager.enable();\n fig.parent_element.innerHTML =\n '<img src=\"' + dataURL + '\" width=\"' + width + '\">';\n fig.close_ws(fig, msg);\n};\n\nmpl.figure.prototype.close_ws = function (fig, msg) {\n fig.send_message('closing', msg);\n // fig.ws.close()\n};\n\nmpl.figure.prototype.push_to_output = function (_remove_interactive) {\n // Turn the data on the canvas into data in the output cell.\n var width = this.canvas.width / this.ratio;\n var dataURL = this.canvas.toDataURL();\n this.cell_info[1]['text/html'] =\n '<img src=\"' + dataURL + '\" width=\"' + width + '\">';\n};\n\nmpl.figure.prototype.updated_canvas_event = function () {\n // Tell IPython that the notebook contents must change.\n IPython.notebook.set_dirty(true);\n this.send_message('ack', {});\n var fig = this;\n // Wait a second, then push the new image to the DOM so\n // that it is saved nicely (might be nice to debounce this).\n setTimeout(function () {\n fig.push_to_output();\n }, 1000);\n};\n\nmpl.figure.prototype._init_toolbar = function () {\n var fig = this;\n\n var toolbar = document.createElement('div');\n toolbar.classList = 'btn-toolbar';\n this.root.appendChild(toolbar);\n\n function on_click_closure(name) {\n return function (_event) {\n return fig.toolbar_button_onclick(name);\n };\n }\n\n function on_mouseover_closure(tooltip) {\n return function (event) {\n if (!event.currentTarget.disabled) {\n return fig.toolbar_button_onmouseover(tooltip);\n }\n };\n }\n\n fig.buttons = {};\n var buttonGroup = document.createElement('div');\n buttonGroup.classList = 'btn-group';\n var button;\n for (var toolbar_ind in mpl.toolbar_items) {\n var name = mpl.toolbar_items[toolbar_ind][0];\n var tooltip = mpl.toolbar_items[toolbar_ind][1];\n var image = mpl.toolbar_items[toolbar_ind][2];\n var method_name = mpl.toolbar_items[toolbar_ind][3];\n\n if (!name) {\n /* Instead of a spacer, we start a new button group. */\n if (buttonGroup.hasChildNodes()) {\n toolbar.appendChild(buttonGroup);\n }\n buttonGroup = document.createElement('div');\n buttonGroup.classList = 'btn-group';\n continue;\n }\n\n button = fig.buttons[name] = document.createElement('button');\n button.classList = 'btn btn-default';\n button.href = '#';\n button.title = name;\n button.innerHTML = '<i class=\"fa ' + image + ' fa-lg\"></i>';\n button.addEventListener('click', on_click_closure(method_name));\n button.addEventListener('mouseover', on_mouseover_closure(tooltip));\n buttonGroup.appendChild(button);\n }\n\n if (buttonGroup.hasChildNodes()) {\n toolbar.appendChild(buttonGroup);\n }\n\n // Add the status bar.\n var status_bar = document.createElement('span');\n status_bar.classList = 'mpl-message pull-right';\n toolbar.appendChild(status_bar);\n this.message = status_bar;\n\n // Add the close button to the window.\n var buttongrp = document.createElement('div');\n buttongrp.classList = 'btn-group inline pull-right';\n button = document.createElement('button');\n button.classList = 'btn btn-mini btn-primary';\n button.href = '#';\n button.title = 'Stop Interaction';\n button.innerHTML = '<i class=\"fa fa-power-off icon-remove icon-large\"></i>';\n button.addEventListener('click', function (_evt) {\n fig.handle_close(fig, {});\n });\n button.addEventListener(\n 'mouseover',\n on_mouseover_closure('Stop Interaction')\n );\n buttongrp.appendChild(button);\n var titlebar = this.root.querySelector('.ui-dialog-titlebar');\n titlebar.insertBefore(buttongrp, titlebar.firstChild);\n};\n\nmpl.figure.prototype._remove_fig_handler = function (event) {\n var fig = event.data.fig;\n if (event.target !== this) {\n // Ignore bubbled events from children.\n return;\n }\n fig.close_ws(fig, {});\n};\n\nmpl.figure.prototype._root_extra_style = function (el) {\n el.style.boxSizing = 'content-box'; // override notebook setting of border-box.\n};\n\nmpl.figure.prototype._canvas_extra_style = function (el) {\n // this is important to make the div 'focusable\n el.setAttribute('tabindex', 0);\n // reach out to IPython and tell the keyboard manager to turn it's self\n // off when our div gets focus\n\n // location in version 3\n if (IPython.notebook.keyboard_manager) {\n IPython.notebook.keyboard_manager.register_events(el);\n } else {\n // location in version 2\n IPython.keyboard_manager.register_events(el);\n }\n};\n\nmpl.figure.prototype._key_event_extra = function (event, _name) {\n var manager = IPython.notebook.keyboard_manager;\n if (!manager) {\n manager = IPython.keyboard_manager;\n }\n\n // Check for shift+enter\n if (event.shiftKey && event.which === 13) {\n this.canvas_div.blur();\n // select the cell after this one\n var index = IPython.notebook.find_cell_index(this.cell_info[0]);\n IPython.notebook.select(index + 1);\n }\n};\n\nmpl.figure.prototype.handle_save = function (fig, _msg) {\n fig.ondownload(fig, null);\n};\n\nmpl.find_output_cell = function (html_output) {\n // Return the cell and output element which can be found *uniquely* in the notebook.\n // Note - this is a bit hacky, but it is done because the \"notebook_saving.Notebook\"\n // IPython event is triggered only after the cells have been serialised, which for\n // our purposes (turning an active figure into a static one), is too late.\n var cells = IPython.notebook.get_cells();\n var ncells = cells.length;\n for (var i = 0; i < ncells; i++) {\n var cell = cells[i];\n if (cell.cell_type === 'code') {\n for (var j = 0; j < cell.output_area.outputs.length; j++) {\n var data = cell.output_area.outputs[j];\n if (data.data) {\n // IPython >= 3 moved mimebundle to data attribute of output\n data = data.data;\n }\n if (data['text/html'] === html_output) {\n return [cell, data, j];\n }\n }\n }\n }\n};\n\n// Register the function which deals with the matplotlib target/channel.\n// The kernel may be null if the page has been refreshed.\nif (IPython.notebook.kernel !== null) {\n IPython.notebook.kernel.comm_manager.register_target(\n 'matplotlib',\n mpl.mpl_figure_comm\n );\n}\n",
"text/plain": [
"<IPython.core.display.Javascript object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<img src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAoAAAAHgCAYAAAA10dzkAAAAAXNSR0IArs4c6QAAIABJREFUeF7snQd4VUX6xt8kpDdqKCH0YuhFAUFQ7B0URXZt2Cu2v11XwYJtV3DdFRDWhnVRsaxrV5CiKCg1FJGWAgECpPfk/7yTPSHl3uQm5yY3ufedffK4JGfOmfOb75x5zzfffONXWlpaChUREAEREAEREAEREAGfIeAnAegzfa0bFQEREAEREAEREAFDQAJQhiACIiACIiACIiACPkZAAtDHOly3KwIiIAIiIAIiIAISgLIBERABERABERABEfAxAhKAPtbhul0REAEREAEREAERkACUDYiACIiACIiACIiAjxGQAPSxDtftioAIiIAIiIAIiIAEoGxABERABERABERABHyMgASgj3W4blcEREAEREAEREAEJABlAyIgAiIgAiIgAiLgYwQkAH2sw3W7IiACIiACIiACIiABKBsQAREQAREQAREQAR8jIAHoYx2u2xUBERABERABERABCUDZgAiIgAiIgAiIgAj4GAEJQB/rcN2uCIiACIiACIiACEgAygZEQAREQAREQAREwMcISAD6WIfrdkVABERABERABERAAlA2IAIiIAIiIAIiIAI+RkAC0Mc6XLcrAiIgAiIgAiIgAhKAsgEREAEREAEREAER8DECEoA+1uG6XREQAREQAREQARGQAJQNiIAIiIAIiIAIiICPEZAA9LEO1+2KgAiIgAiIgAiIgASgbEAEREAEREAEREAEfIyABKCPdbhuVwREQAREQAREQAQkAGUDIiACIiACIiACIuBjBCQAfazDdbsiIAIiIAIiIAIiIAEoGxABERABERABERABHyMgAehjHa7bFQEREAEREAEREAEJQNmACIiACIiACIiACPgYAQlAH+tw3a4IiIAIiIAIiIAISADKBkRABERABERABETAxwhIAPpYh+t2RUAEREAEREAEREACUDYgAiIgAiIgAiIgAj5GQALQxzpctysCIiACIiACIiACEoCyAREQAREQAREQARHwMQISgD7W4bpdERABERABERABEZAAlA2IgAiIgAiIgAiIgI8RkAD0sQ7X7YqACIiACIiACIiABKBsQAREQAREQAREQAR8jIAEoI91uG5XBERABERABERABCQAZQMiIAIiIAIiIAIi4GMEJAB9rMN1uyIgAiIgAiIgAiIgASgbEAEREAEREAEREAEfIyAB6GMdrtsVAREQAREQAREQAQlA2YAIiIAIiIAIiIAI+BgBCUAf63DdrgiIgAiIgAiIgAhIAMoGREAEREAEREAERMDHCEgA+liH63ZFQAREQAREQAREQAJQNiACIiACIiACIiACPkZAAtDHOly3KwIiIAIiIAIiIAISgLIBERABERABERABEfAxAhKAPtbhul0REAEREAEREAERkACUDYiACIiACIiACIiAjxGQAPSxDtftioAIiIAIiIAIiIAEoGxABERABERABERABHyMgASgjQ4vKSlBSkoKIiMj4efnZ+NMqioCIiACIiACItBYBEpLS5GZmYlOnTrB39+/sS7bpK4jAWijO5KSkhAXF2fjDKoqAiIgAiIgAiLgKQKJiYno3Lmzpy7v0etKANrAn56ejpYtW4IGFBUVZeNMqioCIiACIiACItBYBDIyMowD58iRI4iOjm6syzap60gA2ugOGhANh0JQAtAGSFUVAREQAREQgUYkoPEbkAC0YXAyIBvwVFUEREAEREAEPERA47cEoC3TkwHZwqfKIiACIiACIuARAhq/JQBtGZ4MyBY+VRYBERABERABjxDQ+C0BaMvwfNmAuIS+qKgIxcXFthiqsgiIgAiIgOcIBAQEoEWLFj6XysyXx2/L2hQDaOO581UDKigowN69e5GTk2ODnqqKgAiIgAg0BQJhYWHo2LEjgoKCmkJzGqUNvjp+V4QrAWjD1HzRgJj8+vfffwe/Gtu1a2deGEqCbcOIVFUEREAEPESAMzn8oD9w4ICZzendu7fPJEX2xfG7qplJANp48HzRgPLy8rBz50507doV/GpUEQEREAERaN4EOJuze/dudO/eHSEhIc37ZlxsvS+O3xKALhqHK4f5ogFZAtCXXhSu2IKOEQEREIHmSsAX3+u+OH5LALrxCfVFA/LFF4UbTUanEgEREIEmR8AX3+u+OH5LALrx0fNFA/LFF4Ujk+nWrRvuuOMO8+NKWbJkCcaPH4/Dhw+b7QNVRKCpE6hq44z1Xbx4MSZOnOiw6bt27TJTiL/99huGDBlS79tz13nq3QAfrOiL73VfHL8lAN34cPuiATXXF8VJJ51kBqXZs2e7xQIYNB0eHu5yHCQDrQ8dOoT27dtr0YxbesDzJykuKUZBcYHnG9JALTim9zG4ddqtuPW2W80V9u3bh1atWiE4ONjhFXfv2o34PvH48ecfMXjIYJdadf0115u9WP/9wb/Lj+diBD5fbdu2NelJVJwTCAoIQoB/gG1EzfW9bufGfXH8lgC0YzFV6vqiATXXF4UrApAr4jj4aNBx40MCmFWG3pReoqikCKlZqUjNTkVJaYl7YTWhs50/8nxMuXYK/nzdn11qVUpiCiaMmoA3v3wTfQf0danO9DumIysjC3995a8uHe/tBxUVFqFFoOui19/PH+3D26N9RHu08He9XlWOzfW9bscefHH8lgC0YzESgGiOL4qpU6fi9ddfr9R7XMnMqSZOy37xxRd46KGHsH79enz55Zfo0qUL7rrrLvz000/Izs5GfHw8nnrqKZx66qnl53A0PTZ//nx89tln5hyxsbH429/+hvPPP9/UqToF/Nprr5np4/fee8/8NzExESeccAJeffVVk4+LhYm22Y433njDpN259tprjRcmPT0dH330kUNLTktLw6233oply5YZj2PPnj3x4IMP4k9/+lP58Uzl89xzz4Ht5XXplbzhhhsMA5akpCTcfffd+Oqrr5Cfn2/u/5///CdGjhwJsqTHpuL12f61a9eae2Sh2B4wYIARfWx7//79sXTpUjz//PPm/nbs2IHWrVvjvPPOw7PPPouIiIjytq1YscK095dffjGephEjRuDdd9/Fp59+ijvvvBMpKSmVPFCTJk0ynlhep6ELue3P2Y99WftAEcgS4BfQ5Dy677/xPl6e9TK+WPNFpZQet195O6Kio/D43x9H4q5E/G3637BhzQbk5uSie+/umPbgNIwaN6oc49nHnY1Lr7sUl15/qfnd0I5D8fwrz2P8WePNvzf+thFP3PsEdv6+Ez379sS1t1+L/7vm//Du1+8aAciPqcfveRy/LP8FaQfS0CG2AyZfOblcUM7961zM+9u8St02/4P56BTXCeeMOKf8PDxg9crVmP34bGxL2IboltE4d/K5uOW+W8o/1q698Fr07tfb2MbitxcjMDAQF11xEW68+0anZrFp7Sa8+NSL2Lphq3nW+vTvg7tn3I34QfHldTLTM811l3y5BFmZWYjrFofbHroN404bZ45Z+/Nac46EtQkIDA7EgCED8PTcpxHVMgpV+fH4S069BOPPHF/eLjJ98OkHseL7FVj1wypccdMVuP6u62vkZjWO97lw3kIk7UoyTM6beB4WzFlg3hP79+/Hf/7zn/L74P117twZM2fOxNVXX12NSXN8r9t93iUAtROILRvyRQOq6UVBseSsUMBUTC9Q07H+/v4IDQ0tP5WjYznou1oomM466ywjSh577DFTjTkMKZIoAAcNGoS//vWv6NGjh4nPowCi+Bs9erRpM8UjxdzWrVuNOGRxJAD5gqWgOe644/Diiy/ilVdeMakVKHYcCcDrr78eJ554ohGXvOfLLrsMQ4cOxVtvvWWu8eSTTxrRtGDBAiPCXnjhBbz99tumzc4EYHJyMt555x0jVqOioowgpXCisKKAY7nvvvuM+Js1a5YRnUzqvWXLFjNwZGVlYfDgwUbAcrDo0KEDfv31V8TFxeH44493WQCuWbMGN910E6655hrQs3rMMceY6Xeem+wowG+++WacfPLJeOmll8oG07VrMWrUKDNAUZDSE/v9999jypQpRuRRGLPdF198sTn+4MGDpp0U8GTSUIXtP5hzECmZKSgsKTT3w/91iuiEliEtG00AhgWGuXQtCn+y+u9//4tTTjnFYGHsKfuSQvr000/HunXr6mXjVgwgn0nG+7H//vKXv5j+vP322424t2IACwsL8cQTT+Dcc88107krV64EbZ4fAZMnTza2Rvvge5S/Y+GzQpFfMZaQNt2nTx9je9OmTTO2et111+GWW27B9OnTTT1+dPC6/GD685//jB9//NEcz4+x0047zaFpfPfdd+Zaw4cPN3/nM07RxDynkZGRoOAfM2YMMjMzzbPCj6mEhATzMcb3SU32yvt1FCfMMBTGUFrtZlxlTEyMeQfwHnjuTp061ciNbZ0zZ46510cefwQDxwxE2uE0rPtlHa684UokbUrChDMmmI8762Pyk08+MVz4AVnxg8sCIwEY1VCvjyZ9XuUBtNE9EoCV80XVlBD67LPPNmLEKhzQne0kQlFkeZMsscbBvmLhIFyX4mgK2BJlFFMTJkyo8XT0YlHQ0LvG4kgAPvzww3j88cfN3zlAchDhIHzmmWc6FIBXXXUVtm/fbgYWFgohClS+pFk4YNMTxx8WelQoUikSnQlARzdxzjnnGAFJkcvBjOL3H//4hxF8VcvLL79srkfvKAfjqsVVDyBFNwfkmsqiRYsMU6tvOUDt2bMHy5cvd1iNgpHtIlMWCuK///3vhmFDJCOnjR3OO2yEX15RnrkmY66ig6PR7YVudTE/txyb9UAWwoNc+/ChPVOE/Otf/zLXZr8++uij5uOGIsNRccXGLQHI8z3wwANGZFj5QOfOnWv6s6ZFIBRtqampeP/9900THNlT1UUg9Ex/8MEH2Lx5c3k/81nhhwztjB9PfL75fPCjzir0HlOgPv300y7xZ33GOPIji6KVHnAKPV6XArRqqc1eXRWA9KBTYNZUqnLjhw/fHxTYtNO03DRjp1ZM6pSTp+Cyyy/Dow8+aphdcMEF5uPWEtpVryUBKAHo0kOig44SkAD0DgHIQZEvVKtQvM2YMcN4A+gh4PRJbm4u/u///s94+FgcCcB///vf5d4pHhMdHW08gVdccYVDAciXekXvJgdXTmnS88CBjS9sTp2OG1c23cRy4YUXmr87E4AcxDjgcWqZnhNO4fKHAwDb9/PPPxtPID019LJULRRZmzZtMtd1VFwVgNxRgN66ioXePHoV6UXhs0OuHHjoCeIHQb9+/Qw/sndUKCzoXaVXlf1Fbwp50QPlzsIBNSM/A8mZycgpLNvukPFVHSM6ol14O+QW5iLiqaPT1u68dk3nqosAZF/T20axxWlRflQNGzasXGjU18YtAUivMr2I9KJZhf9mn1QUgBSF9GCzz/gMMR6Ux9AOWVwRgLR5PksVxYt1LZ6XXnkKQApYhipYhSK4TZs2xhPvqHCa9JFHHjH3QE58dvhRyo8jPgd81nk+XsNRqc1eXRWAb775Ji69tGya3So1cWO7GbbBdlf0fDMe9UD2AezN2os35r6BxW8txmcrP0NwbjDie8Xj22+/xdixYx3eiwSgBGBjvce85joSgJUFYFOdAqbB1eQBrJqahS9/Th3RY9arVy8zHX3RRReZc1iriF1JkUEBx+M5yDmLAWQ8nVUo6ijUKEAsAfjDDz9Uemlbf3cmADlo8YfXHThwoBFW9DBwOpV1NmzYYKa8nQlAitzVq1c7FYCcnmWc4ccff1zebgpZisaKMYBVV1xzEOU08I033ohLLrnEeBfp6eMUoMWfU3H0vDgTgLwgj2FfnHHGGUYM0lvE6Wl3layCLCRnJCOzINOckkH2HSI6mEB7a7Ul+8cShu66rivncXUKmOei2KJIYKwpOXHnHsZVWtOddm2cNsWY2ZoEIEXolVdeaaZWGT5AjzhjT1etWmWmT10VgLR5euYqCjnWpyecHmP2v6Pnm1OtfAbJwFHhrARXG9PeyIdCme2kx5H3x483vgOcCcDa7JXeek5ZUyxbhSKVHzkVp4CrptapjRu9+AzvqCoArWtwdfrmPZsxrO8wzF0018R5Ln5zMTZt3uTUgywBKAHoyjtIx1QgIAHYfLYMYtxT3759zUvdKs5y81E4MUbJ8izRQ8X4Pgq5xhKAbCOngO+55x7jeWShh4LTxRRXzgQgF1Ywpsia+qO3kNO//GEdvugpvjh16mgKmPGOt912m4npcjQFzGk3evIsDw7bxTgpBt3XJAA5hcdYPnojOWXHwukrMrYEIKe0GH/lbAqYdRj7xOky9iePpVB3R6FXjx6/I3llgtwPfogJjzHiLzAg0B2XaPRz0F4pFujxpXhi7JxV7Nq4NQVM77kVrztv3jwj8C0PIMUPvb30PFmFsamc8rcEIL2UjEFlbKJVXJ0Cvv/++82CJGsKuOpHR20CkIKUU8mXX365uTSns+lNpH1RANILzilkZ1PAtdkrudPzas0acLzgM33vvffWKABd4UbvPb2GfIaclYsnX4zAsED8vOpnnDHxDFw17SoTsxobGYvQwKMx1qwvASgB2OgvqOZ+QQnA5iMAOdBw0OHXNYOgKW7oXXOUnJkeBw5CnHJi/AxFCsUNvV+NKQC5CISDEcUcvWcUrwsXLjSDEr0Gjgq9DRRbXDlLrwkXkfCeKy4coceD8XO8F4o3ekHowaM3jlN0FAf0HjEwnUHkHNAZmE7vCAUX46LoVeG/OX3F89AbU5MAtDw2PJYilYtSGEPGaWpLAG7bts1cm+2gkOAqYopNekwYz8bCZ45t4vQxV/7Sm2in5BflIyUrBWk5aeWnaRvW1kz3BrdwnO/OzvUas+7XX39tWNNbzQVGjFG1il0b50cRRQgXWPC8fF64CITxmJYApI1xipX2x2Npu/zw4P+3BCBDAigcGW/H6VpO9dImHC0CoeBiDC4XY/HjpeoikLoKQNos42HZTtoVP7bo/WabrATvfG4oWPkccTaAIprvBMb11mavtG8+J7x/Pot8j3zzzTfmg64mD6Ar3PihxmfkmWeeMc8jhT6fKYpHq7D/6VHnh+OKDSsQEH009rOqjUsASgA25rvJK64lAdh8BCBf1pyOYuwQp8cqpoGpOgXMwYxijyuBKTzo9eKChYoDTENPAfMBocihoLPSwFDEcuqWQfxc6euocAUo206vC4PzWYfTZBVTx9ArSHHHGD3GOFJQcTDhgMXCKS8OUhxA2AbGOjEWikH1LFxMwEE7Ny8Xk/48CcxdtnXzVrz1Sdnq5UvPvxTxA+Lx8MyjgoO/f3XOq1jw4gIz2B53/HE4/6Lzcc/N92DNjjUmPQnLqhWr8PwTz2Pjuo1mBfbg4YMxe8Hs8r/zmHtuugdLvl6C5ZuWO01K7MoLxor144peFmfeEVfO1RSP4cDP6VF62P744w+zgMgq7rBxPh+0G3rIaCMUOIzJtAQgvb38Oz9WKJqYiogC7/PPPy8XgPz4oCeLq3YpKin4+WxV3VGE3jgKND6//Hjjs0zvl5Wzsz5TwGwnnw+GRdDzR+HHBVAVd/jh88TfcRUtQ1woAhljy4VVLGwX0xZx1Ts9ofT68eOLU8+0c65W5ip13jcXiPGDruoq4KpTwK5w47X5DPJ8fCfwPcXQCApsq9C+yZHTzlyA58jLHRUcZfqmuKAY+5L2YeGehUgrPPox1BTtumqb3pn0DkJa1H0s8sXxuyo7rQK2YeG+aEC++KVow0TcWtWazuX0tLXa2K0XcOFkjC9iAmTmwvNUEuRbptxi8tbd/XjZ6mi7JTIoErFRsYgIavyFHXbbrvoi4IwAF7TQc8/pfy6ksUrVOFfz+yLgYPJB3LjiRuzOdrzopamSrsviqIr34IvjtwSgG63YFw1IAtCNBlTLqeiJ49QY44joFeDqRE5L0wvCmL7GLBVXGFpJkLkogVNJjJdrjHL40GEs+XYJbrrmJixfvRy9+vSyfVl6Dij8GiKNjO3G6QQiUA8C/FBkKikuvmG6HXp/q+5uRO8ghaCV3qgwvxBJiUnYXLQZ+civx1U9V+WqIVfVK07XF8dvCUA32qkvGpAEoBsNqJZTMSidCyc2btxoVgYzkTWnnyqmhWno1jjKMUbR1CmyE1qFtGpU4cSpQU7Xc6rRyo3Y0Pev84tAcyNgLaLhwjXGIFrJwGu6D198r/vi+C0B6Man2RcNyFteFFYiacvzU/XfbjSTZnkq8uCKWK6MtbwEgf6BRvgZr59f43j9miU8NVoEmhkBb3mv1wW7L47fXi0AuaSfeaYY9MzAV644dJb4kiC45RaX6DOdBIN0ubKLeZ+4Gs2V4osG5A0vCvYbVxpyBaC1upRTrPS0ceUp84E5+nG2g4IrttKcjjFJkDOSkV1YtrUfkyAzHUpMWEylvWWb0z2prSIgAs4JeMN7va7964vjt9cKQO58wHxOFIFMbcEVUsxAzzxU1v6tFW+eucYYW8VVVEyVQEHAFWvcwcBZio2q8HzRgJrzi4Kr+NjP7DcWrjLlhwK9WfwdVwo7K1wpa+0Wwj1OrR0WLKHIPHhWfru6voiayvHZBdnG40cByMIkyEyA3D6ivRGBKiIgAt5JoDm/1+vbI744fnutAOTye251xESxVmGgPJfcM+VF1UJPH49lgKxVmGeNHkHGXrlSfNGArBcF47GsBLCusPLkMWyzlW+O7aDgo/ePoo7CjYVTnhR21tZpVf/LeJqKuegciUWKQXoQmYiZeb9YGJDNH3oPm+q0aV5hnhF+3PfW8IGf2fKMufCaaxJkT9qbri0CzY0AU2NZsYP8MPaF4ovjt1cKQCavZc4z5mpjglOrMDEpE4462td05cqVJjkuvX1MpMn9FZleg6KR+zC6UnzRgJhbjOKHIsfVqXJXWDbUMfTUVRT0bDNTI1Cs1aVQIFoCji9L5i/LK8gz8XGFBYVGQFqFwpJ5wFiYioGreekdpDik4AwMCkRQYJD5N1+2nppaZv67gzkHzY9V2oS2MXF+zT0Jcl36Vsc2PgF+FPG9zfeJo8LnhM+H9RHFZ85Z4QpX63nmeV091mzpl1O217OjUvG8tR3LZ7iicKrLtphsQ8X3R8W28L1R8UO7LseSA3k4KnyXccy0CsOmuHLY0fHkMHjw4PJjmYibORsdFbaXCbatwvAqa8bF0fHW1oT8G50xFbfGrHo8z2vNsjCPK3M0VqxfVyv2xfG7KiOvyAPIZLacnmMm9NGjR5ffIxN7MmM6DdZR4RJ5Zpenh4gJb88//3yzbN7yClWtY3mFrN/TgJholUl2uTejrxS+LPigUgTyJdJUPVvsD76I+bLgtk9srzu8lkXFRTiQcwCHcg/BSiLc3PueufC49VnVLaKa+32p/Z4hQEHDdypFHn/oXee+1Pyp+Fw6ax297dw2jYX1awrPYGJoftSx8DrO3vf8Oz3zVigHxSeTWDsrfKdb4UO8H+6W46xwdyHOiliFxzoTdWTABM1WYRucCWG+r7j9o1V4b7xHR4UimCFMViEzsnNUKK779Olj2khRyfZyrKy437JVj7sCURxahVkIli1b5vC85MBdSazCuHpn2zVy3KgoOJlE/MMPP3TKmILWEtlXXHGF2VnGGWNXrF4CEPAqAUivHrensgq30qKRVNwD0/obYwO5LyV3WuDG8hQ1zDTPjdOtfVSrGhG373G0Ub2vCUA+dHwh1PS15soD6O5j+DKxXj5c1GMVvjCdifq6tIG58DLzM5Gen17+4gnwP7q9Uo3nKkWZWLT++79pZ37RVlyJXFLs4Iv9fwtuKx5rrkWno83FuFzZyx0wmqvHj31e0yBQja8TjwhxNrdjaTeWR4QMnHl7eG+NcSwFH70y/C9/qvYLBZUVGkHBw7AMZ3Gz/GCznmGeq6IAqfqcUXRYHneel+9yZ4Xiq2J4Bp0Hzgo/bq39sHkvbK+zQmFihYjwGB7rzC4p1BiCYhW2wVnfWSEl1rG8t5q8phRrVuHshzOxSK+eJbB5PPdi5m4/jtrMtnKnE6twx5GKe4FXZEK+FUU14/IdzcBZNsnZEavccMMNZpcYZ4WC1hKA3K2FYpG7HNW3SAB6iQCszxQwDZOeP04bW4ULQ7hq2Noeq6phyQNYmQhfRM5eMPV9KOtTj/3P/TYZ05mWlmZeEkygTI+fO0pBcQH+venfmLN6TvmesfHt4nHXqLtwQpcT3OYB5YuTW2FxyjopKcn8ly9x66XMJNDWBw5DF6yt2xzd48svv1yeL5AvSm5X5awwtyBjZa2Biy97eratuEdPeXh535xqJwuLB//Ll7810NFjwbhdZ4V50EaNGmX+zO3zHH3AWXVpPwwLYfnoo49w//33Oz0v94Y9++yzzd/p4WC4ibPCD1F6N1i4/zS3H3NWuK8u9+1l+eWXX8zCNmeF2/VxqzEWrmDnwOys3HzzzbjtttvMnznVZm1l5uh4zopw+0MWvgu597Szwj15rZyMtFUurLMK7Yb9ZNkSt2vjxzaL0i45Rdrof+DHsafCUBr9ZitcUALQSwQg+5SLQBgPwFXAVuH+lBMmTHC4CIQvZH4FcfWwVbgfJaeQ+fVmTSnUZKAyIE8+vjBfwm+//bbZcJ4BzCycLuE2aZdccontVbnc9uztDW/jkSWPYNeR/52/VU88cfITmNx/slkl29CFHyn8SuZ+nxR/lqeDu4JQADqLxfnvf/9rYltZKBy5P7Czwj1/6Q1n4TTQ1KlTyw/lFBT3kLV+rrnmGgwcOLB8ELcrDjn9xCl6Tp9Z04P02lOU8veO4rm4z7ElSriYi557Z6XisRR4FELOCr0g5557rvkz91/mfrPOCvd7pY2xMGzk4osvdnosZxQs/vRwWMLRUQUuRLv11lvNnygWKwqqqseTkSXUVq9ebWYvnBUmz37sscfMn629e50dS2FJriz0sHTt2tXpebmP75tvvmn+Ti8W46c5vUl7YT1fWVDgFJD+0GQJaPz2IgFopYEvORaDAAAgAElEQVThC4gDJT0g3OyesQ18EXGwpLDji52FngF+PXPzbGsKmJ4FTkmsWrXKJaOVAbmEqUEOsrwjVrwPF15QCFKg2J3upXfi022f4qHvHsLG/RtN+7ki9pETH8E1Q69pMitja4t/qZrkumpHsD6n7DiFZg3UH3/8MV544QUjOOmBrDo1RW/X6aefbk5FocZpG2ci8JNPPinfhYAi1BI2Vjt4fUvgfffdd+XeN3r12I8sfB7pQaooQim8rLio2hjwHLVxqMhFx5bRqA+HBnnQdVIRaCACGr+9SADSRuj943QQ4yS4bRZz/FnbZtGrQS/RkiVLys2JX9sUjPQ00LNCr8IzzzxTHiRcm93JgGoj5L6/my3J0tLK42w43cQpSooXTtVNmzat0qq2+l75h90/4P5v7sePST+aUzA+7v4x92PayGng3re+VDi1Tg8QxSCfEf6X04hWEL2zmFiLEafhTzvtNPNPfow5m/pkrFfFDev5ocYPN4o+BuFbq0F9ib3uVQREoGEJaPz2MgHYsOZS/ewyoIYnzhgwTkvSo0vhwRg5q9AbRW+vO1Zgr923Fg9++yA+314WhBzaIhR3jLoD94y+B61Cy3L6qVQmQPunB9FZYfyXteqai3Mo4KsW9p0VaC++IiACItBYBDR+SwDasjUZkC18Titz2pFCj6KPix2shSZcGUgvVMXVdnZbsP3Qdvzl+7/g3Y3vmlNxx4vrhl2Hv4z7CzpGdrR7etUXAREQARFoggQ0fksA2jJLXzaglMwU/HVlWaD43aPvNsmD3VEYXM+VkBV3aGFwO6cPp0yZYqZ8Wazrbz7oPI9Xbe0pLC7E0t1LUVRSZA7904A/4bHxj6FX6161VdXfRUAEREAEmjEBXx6/rW7zijyAnrJBXzSgw7mH8cyKZ/D3VX9HblFZdn5Ol94+8nbcO+beOk+X0ttHD5+VyZ8rCpn6glODTIfBhTpDhgwp72JH17fb/2f3PhtPnvwkhnQ4eh2751R9ERABERCBpkvAF8fvqr0hAWjDPn3JgHIKc4zoo/g7knfEUDu+c1nS7YoLJu4bcx9uG3lbrQsmuKiDiUeZT42pOaxcYlwVyhXdTKthpQXhNRxdf0zcGFw15Cpbq3Lj28bjuFjn6TNsmIeqioAIiIAINFECvjR+O+sCCUAbxukLBsRp0gW/LsBjPzyGfVll2wENiBmAmSfPxLl9ynKm1TVlCr1+TAnCvGwsnOJ1llne0fUHxgzEzFNm4pze57gtCbMNM1BVERABERCBZkbAF8bv2rpEArA2QjX83ZsNiNuevbfxPbNA4o/DfxgK3Vp2w+PjHzexclW3QHOWNJnHXzLgkvKkyZzuZYJd7srAXGNMu3PLLbdUS+Hi6PrdW3Y3158yYEq169voRlUVAREQARHwMQLePH672pUSgK6ScnCcNxoQp2aZCoUpUdalrjN3HRMeY1bFXj/8egQFBNVILL8oH/N/nY/Hf3gc+7P3m2MHtx+Mp055CuM6jcPkyZPBXSq4Cwvj/azdFKyTOrp++/D25vrXDb+u1uvb6E5VFQEREAER8BEC3jh+17XrJADrSqzC8d5mQCv2rMAD3z6AZXuWmbuMCo7CvaPvxe2jbkdEUNnqW1dLVkEWZv80G8+tfA4Z+Rll5zschYwPMxB6MNRs5H3mmWdWOp07r+9qO3WcCIiACIiA7xHwtvG7Pj0oAVgfav+r4y0GtD51vdn27D/b/mPuLKRFCG497lbcf8L9aBPWxgYhIC0nDU8vfxov/vwi8ovzzbnGtB2DORfNwcD2ZXvKbkjdYK7PWELr+tNGTAMXlNi9vq3Gq7IIiIAIiIBXEvCW8dtO50gA2qDX3A1o5+GdeGTJI3hr/VsoRSkC/AJw9dCrzZ63naM62yBTvWpSRhIeW/oYXvntFRSXFsMPfrhs0GXmwDfXv9ng13frzehkIiACIiACzZpAcx+/3QFfAtAGxeZqQKlZqXjihycwb808FJYUGgKT+082Cyz6tOljg0jlqgkJCbjnnnvw1ltvmb2WWbYe3GoWlixKWFTp4Ia4vttuRCcSAREQARHwKgLNdfx2ZydIANqg2dwMKD0v3cTkzfpplsmrx3J6z9NNSpfhnYbbIFG9KtO6nHXWWWav2Kuvvhr/+te/Kh20JmWNSS3D8si4R9x+fbfejE4mAiIgAiLgVQSa2/jdEPAlAG1QbS4GlFuYi3/8/A88veJpHMo9ZO54ZOxIszJ3fPfxNgg4rvrtt99iwoQJyM7OxogRI8yq3zZt7MUSur2ROqEIiIAIiIDPEmgu43dDdpAEoA26Td2AuMftq7+9ihlLZyA5M9ncKXe+YBLlCX0nNEgS5cWLF5s9ewsKCnDKKafgo48+Kt+/1wZqVRUBERABERABtxFo6uO32260hhNJANqg3FQNiEmUP0j4AA9//zC2pW0zd9glugtmnDQDlw+6vMGSKL/yyitm717u9HHhhRfi7bffLt/j1wZmVRUBERABERABtxJoquO3W2+ylpNJANqg3dQMiEmUv97xtcnl9+veX82dtQ1ri4fHPowbj70RwS2CbdxtzVU53RsfH4/ExEQT8zdv3jyT7FlFBERABERABJoagaY2fnuCjwSgDepNyYBWJa0ywu/7Xd+bO2Li5ruPvxt3HX8XIoMjbdyl61W3bNmCd999F48++miDTC+73hIdKQIiIAIiIALOCTSl8dtT/SQBaIN8UzCghAMJJonyR1s+MnfCrdpuPvZmPDj2QbQLb2fj7mqvWlxcjLVr12L4cPeuIK79yjpCBERABERABOpPoCmM3/VvvXtqSgDa4OhJA9p9ZDemL52ON9a9Acb8+fv548rBV+LREx9F15ZdbdyVa1W5yOPKK680W7p99tlnOPXUU12rqKNEQAREQAREwMMEPDl+e/jWyy8vAWijJzxhQPuz92PmspmYs3oOCooLTOsvOOYCPHHyE+jXrp+Nu3G9ak5ODiZNmoQvvvgCgYGBZrHHRRdd5PoJdKQIiIAIiIAIeJCAJ8ZvD96uw0tLANrokcY0oIz8DDz/4/P4249/Q1ZBlmn1+G7jTS6/kZ1H2riLulU9fPgwzj33XKxcuRJhYWHGA3jGGWfU7SQ6WgREQAREQAQ8SKAxx28P3maNl5YAtNEzjWFAeUV5mPPLHDy57Emk5aaZ1g7rOAxPn/I0Tu1xaqMutti3b58Re+vXrzdbuzHB8/HHH2+DoKqKgAiIgAiIQOMTaIzxu/Hvqm5XlACsG69KRzekATGJ88J1C/HokkeRmJForst9ep8Y/wQm9ZtkYv4as+zfvx+jR4/GH3/8gQ4dOuDLL7/EoEGDGrMJupYIiIAIiIAIuIVAQ47fbmlgI5zEqwTgSy+9hOeeew579+5F//79MXv2bIwdO9Ypxvz8fDz22GN48803Qe9W586d8dBDD5k8dq6UhjAg5vJbvGUxHv7uYWw+uNk0IzYyFtNPmo6pQ6aihb9ncusxuTMXfaxYsQJff/01evbs6QoiHSMCIiACIiACTY5AQ4zfTe4ma2mQ1wjA9957D5dffjkoAseMGWMSES9YsAAJCQno0qWLQwzcrzY1NRVPPPEEevXqBXq5ioqKjKfLleJuA/pu53cml9/PyT+by7cObY0HTngAtxx3C0IDQ11pUoMeU1hYCMYAxsTENOh1dHIREAEREAERaEgC7h6/G7KtDXVurxGAI0eOxLBhwzBnzpxyVtyZYuLEiXjqqaeq8eMKVu5Zu2PHDrRu3bpefN1lQKtTVhvh982Ob0w7wgLDcNeou3D36LsRHRJdr7a5oxI9fe+88w7mz5+PgIAAd5xS5xABERABERABjxNw1/jt8Rux0QCvEIDMSccVqYsWLcIFF1xQjuP22283iYqXLl1aDdHNN9+Mbdu24dhjj8XChQsRHh6O888/H48//jhCQ13zttk1oC0Ht+Av3/8F7ye8b9oX6B+IG4bfgIfHPYz2Ee1tdKv9qu+//z7+/Oc/g16/v//975g2bZr9k+oMIiACIiACItAECNgdv5vALdhuglcIwJSUFMTGxpr4tIrTtzNnzsTrr7+OrVu3VgN15plnYsmSJSaB8SOPPIKDBw+CovDkk0/GK6+84hAsYwb5YxUaUFxcHNLT0xEVFVWnzqDXb+SCkSaJsx/8cNmgyzDjpBno3qp7nc7TEAf/61//wvXXXw/G/V188cVGIAcHN9w+wg1xDzqnCIiACIiACDgjIAEIeJUAZG66imlJnnzySSNeuEdt1XL66adj2bJlZvFHdHTZNCtz2jGhcXZ2tkMv4PTp0zFjxoxq56qPAKTwGzF/BDpFdsKTJz+Jge0HNoknlYto7r33XtOWa6+9FnPnztX0b5PoGTVCBERABETAXQQkAL1EANZnCtha0bp9+/Zye9q8eTP69etnpoZ79+5dzc7c6QHkyZnQOSIowl32bOs8XH384IMP4umnnzbnue+++0zspJ+fn63zqrIIiIAIiIAINDUCEoBeIgBpWFwEMnz4cLMK2CoUc1zp62gRyMsvv4w77rjDrPyNiCgTYR9//DEuvPBCZGVluRQH6E0GxGnyIUOGIC8vz4hACkAVERABERABEfBGAt40fte3f7xiCpg3b6WB4ZQlp4Ep8Lh6ddOmTejatSseeOABJCcn44033jCsKPK4SnjUqFFmWpcxgJzyPPHEE009V4q3GdCnn35qpsSvu+46V25fx4iACIiACIhAsyTgbeN3fTrBawQgb57ev2effdYkgh4wYABmzZqFcePGGS5Tp07Frl27zMIPqzA2kKtbuXikTZs2mDx5sskJ2FirgOvTYe6sw1hHLqBxNN3tzuvoXCIgAiIgAiLQlAhIAHrRFLAnDKs5GxDjHC+77DIkJSVh+fLl6NGjhycQ6poiIAIiIAIi0OgEmvP47S5YXuUBdBcUV8/THA2ouLjYbJH38MMPm3i/Vq1agUmxR4wY4ept6zgREAEREAERaNYEmuP47W7gEoA2iDY3A+KU91VXXYWffvrJ3PVpp51m4h0ZI6kiAiIgAiIgAr5CoLmN3w3RLxKANqg2FwNiihfm92PCa6ayYdLq559/HldffbXSvNjof1UVAREQARFongSay/jdkHQlAG3QbU4GdMMNN5iV0WeddRbmzZtndjBREQEREAEREAFfJNCcxu+G6h8JQBtkm7IBFRUVmS3quLqZhW1lmhfu76vkzjY6XVVFQAREQASaPYGmPH43FlwJQBukm6oBbdiwwcT6cYHHV199JcFno49VVQREQAREwPsINNXxuzFJSwDaoN3UDKiwsNDs4vH444+D/79ly5b4+eeflefPRh+rqgiIgAiIgPcRaGrjtycISwDaoN6UDGjdunXG6/fbb7+ZOzr//PMxZ84cdOrUycYdqqoIiIAIiIAIeB+BpjR+e4quBKAN8k3BgAoKCsxex9zBhHF/rVu3xosvvog//elPmvq10beqKgIiIAIi4L0EmsL47Wm6EoA2eqApGBCTOQ8dOhTM8Tdx4kTj9evQoYONu1JVERABERABEfBuAk1h/PY0YQlAGz3gSQNibj9rNe+qVauwc+dOXHLJJfL62ehPVRUBERABEfANAp4cv5sKYQlAGz3hCQNiIufbbrsNAwYMwLRp02y0XlVFQAREQAREwDcJeGL8bmqkJQBt9EhjG1BKSgomTZpktnILCgrCjh07EBsba+MOVFUEREAEREAEfI9AY4/fTZGwBKCNXmlMA1q+fDkuuugipKammvQub7/9ttnVQ0UEREAEREAERKBuBBpz/K5byxrvaAlAG6wbw4AY68eFHbfffrtZ5Ttw4EAsXrwYPXv2tNFyVRUBERABERAB3yXQGON3U6crAWijhxragCj+uIfv/PnzTSsnT56MV155BeHh4TZaraoiIAIiIAIi4NsEGnr8bg50JQBt9FJjGNDzzz+Pe+65x+zwcffdd2uVr43+UlUREAEREAERIIHGGL+bOmkJQBs91FAGxG3cAgMDTcvoBVy/fj0GDx5so6WqKgIiIAIiIAIiYBFoqPG7ORGWALTRW+42IIq9F154wUzzctFHVFSUjdapqgiIgAiIgAiIgCMC7h6/myNlCUAbveZOA8rJycF1111nVveycDu3W2+91UbrVFUEREAEREAEREAC0LENSADaeDbcJQC5i8eFF16ItWvXIiAgAIz7Y5Jna6cPG01UVREQAREQAREQgSoE3DV+N2ewEoA2es8dBvT1119jypQpOHToENq1a4dFixbhxBNPtNEqVRUBERABERABEaiJgDvG7+ZOWALQRg/aNaB3330Xl156KUpKSnDsscfiww8/RFxcnI0WqaoIiIAIiIAIiEBtBOyO37Wdvzn83asE4EsvvYTnnnsOe/fuRf/+/TF79myMHTu21n5YsWKF8bpxf11Ow7pa7BrQvn37MHz4cJxxxhlg20NCQly9tI4TAREQAREQARGoJwG743c9L9ukqnmNAHzvvfdw+eWXGyE1ZswYzJs3DwsWLEBCQgK6dOniFHp6ejqGDRuGXr16mW3WGlMAslG8ZkxMjOL9mtRjocaIgAiIgAh4MwEJQMBrBODIkSONkOO2aVaJj4/HxIkT8dRTTzm1Y8bf9e7d2yy++OijjxpdAHrzA6Z7EwEREAEREIGmSEAC0EsEYEFBAcLCwswCigsuuKDc1rh/Lj16S5cudWh/r776qvEY/vjjj3jiiSckAJviU6o2iYAIiIAIiICbCUgAeokATElJQWxsLBjLN3r06HIzmTlzJl5//XVs3bq1mun8/vvvOOGEE7Bs2TL06dMH06dPr1UA5ufngz9WoQFx0QankZW02c1Pp04nAiIgAiIgAg1EQALQywTgypUrcfzxx5eby5NPPomFCxdiy5YtlUyouLgYo0aNwjXXXIMbb7zR/M0VAchjZsyYUc0cJQAb6AnVaUVABERABESgAQhIAHqJAKzrFPCRI0fQqlUrE/dnFaZi4VZs/N1XX32Fk08+uZrJyQPYAE+hTikCIiACIiACjUxAAtBLBCDthotAmFKFMX1W6devHyZMmFBtEQjFHlcHVyys99133+H9999H9+7dER4eXqs5yoBqRaQDREAEREAERKDJEdD47UUC0EoDM3fuXDMN/PLLL2P+/PnYtGkTunbtigceeADJycl44403HBqiK1PAVSvKgJrcM60GiYAIiIAIiECtBDR+e5EAZG/Ti/fss8+aRNBM6jxr1iyMGzfOGMLUqVOxa9cuLFmyRAKw1kdDB4iACIiACIiA9xKQAPQyAdjYpioDamziup4IiIAIiIAI2Ceg8VsC0JYVyYBs4VNlERABERABEfAIAY3fEoC2DE8GZAufKouACIiACIiARwho/JYAtGV4MiBb+FRZBERABERABDxCQOO3BKAtw5MB2cKnyiIgAiIgAiLgEQIavyUAbRmeDMgWPlUWAREQAREQAY8Q0PgtAWjL8GRAtvCpsgiIgAiIgAh4hIDGbwlAW4YnA7KFT5VFQAREQAREwCMENH5LANoyPBmQLXyqLAIiIAIiIAIeIaDxWwLQluHJgGzhU2UREAEREAER8AgBjd8SgLYMTwZkC58qi4AIiIAIiIBHCGj8lgC0ZXgyIFv4VFkEREAEREAEPEJA47cEoC3DkwHZwqfKIiACIiACIuARAhq/JQBtGZ4MyBY+VRYBERABERABjxDQ+C0BaMvwZEC28KmyCIiACIiACHiEgMZvCUBbhicDsoVPlUVABERABETAIwQ0fksA2jI8GZAtfKosAiIgAiIgAh4hoPFbAtCW4cmAbOFTZREQAREQARHwCAGN3xKAtgxPBmQLnyqLgAiIgAiIgEcIaPyWALRleDIgW/hUWQREQAREQAQ8QkDjtwSgLcOTAdnCp8oiIAIiIAIi4BECGr8lAG0ZngzIFj5VFgEREAEREAGPEND4LQFoy/BkQLbwqbIIiIAIiIAIeISAxm8JQFuGJwOyhU+VRUAEREAERMAjBDR+e5kAfOmll/Dcc89h79696N+/P2bPno2xY8c6NK4PP/wQc+bMwdq1a5Gfn2+Onz59Os444wyXjVEG5DIqHSgCIiACIiACTYaAxm8vEoDvvfceLr/8clAEjhkzBvPmzcOCBQuQkJCALl26VDO6O+64A506dcL48ePRsmVLvPrqq/jrX/+KVatWYejQoS4ZqQzIJUw6SAREQAREQASaFAGN314kAEeOHIlhw4YZr55V4uPjMXHiRDz11FMuGR69gJdccgkeeeQRl46XAbmESQeJgAiIgAiIQJMioPHbSwRgQUEBwsLCsGjRIlxwwQXlRnb77bebKd6lS5fWanglJSXo1q0b7r33Xtx66621Hs8DZEAuYdJBIiACIiACItCkCGj89hIBmJKSgtjYWKxYsQKjR48uN7KZM2fi9ddfx9atW2s1PMYOPv3009i8eTNiYmIcHs9YQf5YhQYUFxeH9PR0REVF1XoNHSACIiACIiACIuB5AhKAXiYAV65cieOPP77csp588kksXLgQW7ZsqdHa3nnnHVx77bX4+OOPceqppzo9lotEZsyYUe3vEoCef5jVAhEQAREQARFwlYAEoJcIQDtTwFw8ctVVV5np43POOadG25EH0NVHS8eJgAiIgAiIQNMlIAHoJQKQJsZFIMOHDzergK3Sr18/TJgwwekiEHr+rr76avC/XCxS1yIDqisxHS8CIiACIiACnieg8duLBKCVBmbu3LlmGvjll1/G/PnzsWnTJnTt2hUPPPAAkpOT8cYbbxjLo+i74oor8MILL+DCCy8st8bQ0FBER0e7ZJ0yIJcw6SAREAEREAERaFIENH57kQCkZdH79+yzz5pE0AMGDMCsWbMwbtw4Y3RTp07Frl27sGTJEvPvk046yeHq4CuvvBKvvfaaS4YqA3IJkw4SAREQAREQgSZFQOO3lwnAxrYuGVBjE9f1REAEREAERMA+AY3fEoC2rEgGZAufKouACIiACIiARwho/JYAtGV4MiBb+FRZBERABERABDxCQOO3BKAtw5MB2cKnyiIgAiIgAiLgEQIavyUAbRmeDMgWPlUWAREQAREQAY8Q0PgtAWjL8GRAtvCpsgiIgAiIgAh4hIDGbwlAW4YnA7KFT5VFQAREQAREwCMENH5LANoyPBmQLXyqLAIiIAIiIAIeIaDxWwLQluHJgGzhU2UREAEREAER8AgBjd8SgLYMTwZkC58qi4AIiIAIiIBHCGj8lgC0ZXgyIFv4VFkEREAEREAEPEJA47cEoC3DkwHZwqfKIiACIiACIuARAhq/JQBtGZ4MyBY+VRYBERABERABjxDQ+C0BaMvwZEC28KmyCIiACIiACHiEgMZvCUBbhicDsoVPlUVABERABETAIwQ0fksA2jI8GZAtfKosAiIgAiIgAh4hoPFbAtCW4cmAbOFTZREQAREQARHwCAGN3xKAtgxPBmQLnyqLgAiIgAiIgEcIaPyWALRleDIgW/hUWQREQAREQAQ8QkDjtwSgLcOTAdnCp8oiIAIiIAIi4BECGr8lAG0ZngzIFj5VFgEREAEREAGPEND4LQFoy/BkQLbwqbIIiIAIiIAIeISAxm8JQFuGJwOyhU+VRUAEREAERMAjBDR+SwDaMjwZkC18qiwCIiACIiACHiGg8dvLBOBLL72E5557Dnv37kX//v0xe/ZsjB071qlxLV26FHfddRc2bdqETp064d5778WNN97osjHKgFxGpQNFQAREQAREoMkQ0PjtRQLwvffew+WXXw6KwDFjxmDevHlYsGABEhIS0KVLl2pGt3PnTgwYMADXXXcdbrjhBqxYsQI333wz3nnnHUyaNMklI5UBuYRJB4mACIiACIhAkyKg8duLBODIkSMxbNgwzJkzp9zI4uPjMXHiRDz11FPVDO++++7DJ598gs2bN5f/jd6/devW4ccff3TJUGVALmHSQSIgAiIgAiLQpAho/PYSAVhQUICwsDAsWrQIF1xwQbmR3X777Vi7di041Vu1jBs3DkOHDsULL7xQ/qfFixdj8uTJyMnJQWBgYK3GKgOqFZEOEAEREAEREIEmR0Djt5cIwJSUFMTGxppp3NGjR5cb2syZM/H6669j69at1YyvT58+mDp1Kh588MHyv61cudJMH/N8HTt2rFYnPz8f/LFKenq6mV5OTExEVFRUkzNwNUgEREAEREAERKA6AQrAuLg4HDlyBNHR0T6JyK+0tLS0ud+5JQAp4I4//vjy23nyySexcOFCbNmyxaEAvOqqq/DAAw+U/40C8oQTTjCLSDp06FCtzvTp0zFjxozmjkvtFwEREAEREAERAIwDp3Pnzj7JwisEYGNNAVf1APLLoWvXrtizZ4/PfkHYeWqsLzB5UOtHUfzqx82qJX7iZ4+AvdqyP8/yo+8rMzPTZADx9/e315hmWtsrBCDZcxHI8OHDzSpgq/Tr1w8TJkxwugjk008/NauErXLTTTeZmEEtAmkca1YMhj3O4id+9gjYqy37Ez97BOzVlv3Z48faXiMArTQwc+fONdPAL7/8MubPn29y/NFLx6ne5ORkvPHGG4aalQaGKWCYCoaij6uAlQbGvlG5egY9wK6Scnyc+ImfPQL2asv+xM8eAXu1ZX/2+HmVAOTN0Pv37LPPmhg+5vibNWsWuNqXhQs+du3ahSVLlpRT4+rgO++8szwRNFPDKBG0faNy9Qx6gF0lJQFoj5T4iV9DELB3Tr3/xM8eAfu1vcYDaB9F3c/AmEDmGKR3MTg4uO4n8PEa4mfPAMRP/OwRsFdb9id+9gjYqy37s8fP6zyA9nHoDCIgAiIgAiIgAiLg/QTkAfT+PtYdioAIiIAIiIAIiEAlAhKAMggREAEREAEREAER8DECEoA+1uG6XREQAREQAREQARGQAKynDXDF8XPPPWdWHPfv3x+zZ8/G2LFj63k236rmaKHKi90AACAASURBVEeV9u3bY9++fb4FwsW7/eGHH4ytrVmzxtgb96yeOHFieW0mNOUONUx9dPjwYZMT85///KexSxWgNn7MEMAtIysWMvzpp598Hh8XuX344YdmN6XQ0FCz1eYzzzyDvn37yv5csA5X+Mn+nIOcM2cO+MMMHix8pz3yyCM466yzzL/17nPBCGs4RAKwHvysnIMUgdw7eN68eViwYIFJKs29gVVqJkAB+P777+Obb74pPzAgIADt2rUTOgcEPv/8c7PP9bBhwzBp0qRqApADMrc9fO2118A9rp944gkjergHdmRkpM8zrY0fB+DU1FS8+uqr5ayCgoLQunVrn2d35plnYsqUKTjuuONQVFSEhx56CBs2bDDvuvDwcMNH9ufcTFzhJ/tzzo+bNXBs6NWrlzmIH2r8GP7tt9+MGJTt2XtFSQDWgx+9AxyM+WVilfj4eOOV4RefSu0C8KOPPjK7rqjUjYCfn18lAcgvYG5ldMcdd4B5LFmYHoEeVb4cmehc5SiBqvz4Fw7A3NaRNqlSM4EDBw4gJiYGzKHKHKuyv7pZTFV+sr+68ePR/DCjCLz66qv17qs7vko1JADrCLA++w7X8RJefzg9gHyAo6OjTf5ECuqZM2eiR48eXn/vdm+wqoDZsWMHevbsiV9//RVDhw4tPz23QGzZsmW1qU2712/u9Z0JQIo/ev3I7MQTTzQeVQodlcoEtm/fjt69exsvIJPty/7qZiFV+VkCUPZXO8fi4mIsWrQIV155pfEAhoSE6N1XO7Yaj5AArCPAlJQUxMbGmik5xsNYhQKG7mlOu6nUTIBTcjk5OWa6klNvnLJkjBG37WvTpo3w1UCgqoBZuXKlCUPgNof0BFrl+uuvx+7du/Hll1+KZwUCjgQgQzoiIiLMlpHcIvIvf/mLme5kzKUSvB+FR28fPywYZ7ps2TLzB9mf64+XI36sLfurmSE/Nri9a15ennlO3377bZx99tmyPddNz+mREoB1hGgJQL74aJRWocdg4cKFRsio1I1Adna2+ZK79957cdddd9Wtso8d7UwA0i47duxYToP7WycmJuKLL77wMUI1364jAVi1BhfaUAy+++67uPDCC8XvfwRuueUWfPbZZ1i+fDk6d+5cSQDK/mo3E0f8HNWS/VWmwlm3PXv2mDCNDz74wMTbMwSB/+bHr2yvdttzdoQEYB3ZaQq4jsBcPPy0004zgb4V4ypdrOpTh2kK2F53uyIAeQVOc1577bXlcZX2rtr8a0+bNs3ESHJxUffu3ctvSFPArvWtM37Oasv+nHM99dRTjcOAMc8Kf3HN/iQA7XGqVJsxa8OHDwdXAVulX79+ZnpEi0DqDpqLFvggc9qSS/xVnBNwtgjkzjvvNB5UFn6kMH5Ni0Cqc3RFAKalpZkwD6bVueKKK3zaHDltSfHC1ENLliwxwrhisRaByP4cm0lt/BzVkv3V/MidcsopiIuLM6v2GfYi26v/K0oewHqws9LAzJ0710wDc6CYP3++iWHj1JFKzQTuvvtunHfeeSZlzv79+00MIF36jPUQv+rssrKywOBxFi70eP755zF+/HizGo4MKfT44cEXIgdoxqNysFYamDKWNfEjQy5KYnodTqEz39iDDz5oppw2b97s82l0br75ZhNz9fHHH1fK/ccFXMwLyCL7c/6+q40fbVP255wfn0Xm/KPgy8zMNGEZTz/9tAlt4ayRbM+e2pAArCc/ev+effZZk5iXq+FmzZpl0iKo1E6AecU4lXTw4EGT+2/UqFF4/PHHQS+qSnUCFHMUfFULV8Mx95+VDJX5KCsmgqZdqsCIYWf8GHLA9E1cVciYIopAHkt75KDj64UeU0eFHxtMn8Mi+3NuJbXxy83Nlf3V8JBdc801+Pbbb804y4+OQYMGmalfij/Znv23kwSgfYY6gwiIgAiIgAiIgAg0KwISgM2qu9RYERABERABERABEbBPQALQPkOdQQREQAREQAREQASaFQEJwGbVXWqsCIiACIiACIiACNgnIAFon6HOIAIiIAIiIAIiIALNioAEYLPqLjVWBERABERABERABOwTkAC0z1BnEAEREAEREAEREIFmRUACsFl1lxorAiIgAiIgAiIgAvYJSADaZ6gziIAIiIAIiIAIiECzIiAB2Ky6S40VAREQAREQgcYjwF2bnnvuOaxZs8bsyMF9obl7Tk2FW3veddddZntU7tfLfcpvvPHGxmu0ruQSAQlAlzDpIBEQAREQARHwPQKff/45VqxYgWHDhpk9s2sTgDt37jTbo1533XW44YYbTF3uifzOO++Y+ipNh4AEYNPpC7VEBERABERABJosAe5tXJsA5F69n3zyCTZv3lx+H/T+rVu3Dj/++GOTvTdfbJgEoI1eLykpQUpKCiIjI+Fs028bp1dVERABERABLyJQWlqKzMxMMy3q7+/fYHeWl5eHgoICp+dnO6qOWcHBweBPTcUVAThu3DgMHToUL7zwQvmpKBonT56MnJwcBAYGNth968R1IyABWDdelY5OSkpCXFycjTOoqgiIgAiIgK8RSExMROfOnRvktin+unfriH2pR5yePyIiAllZWZX+/uijj2L69Om2BWCfPn0wdepUPPjgg+XnWrlyJcaMGWMcJh07dmyQ+9ZJ605AArDuzMprpKeno2XLlti57e+Iigyt95kK8lcDQcPrXd9Oxdz8X+AfdJydU5TXzchfg5B6nOtw/lqEuVhvf94GtAweWmN7U3I3oU3IYKfHJOZsQUzIAId/35W9DbFh8Q7/tj1zO7qEH1Ptb1sz/kCPiD6Vfr85Y6f5d++IXpV+vz59F+Ije1b63brDu9E/ulul361NS8TAll0r/e7Xg0kY0vroB8eaA8nm70PbVPhdajKGt4str7dmX4r5/8Njjv7u15S9GN6hU/kxvyb/75iKv0tKwbCOR4/5LXGvOX5Y7NGX99rdezE0rvLLfO3OvRja9ejv1u0oqzekwu/W//G/33WvXHf973sxuEfl323YmoJBvY62w2r0xq3JGNS7+u83JCRjUF8Hx29KxsD46r/n+TatTzKnHdDvKKOqnbxpfSIGDHD+d3Oe3/ag/0DXBvWENbvQb3CXOj93Cat3oN+wynZR55MASFi13VTrN7yy3dXnXPWps+nHbeg3ovJzUJ/zNLc6Obk5mHrfn3DkyBFER0c3SPMzMjLMuZ2NSxmZueje5zZQhEZFRZW3wV0eQArAq666Cg888ED5uRkHeMIJJ5hFJB06dGiQ+9ZJ605AArDuzMprWA9a2t75iIoKq/eZCvJXAS4KoHpfxEnFnPyfEBA0yi2nTc/7BSHBdT/Xobw1CHexXmruWrQMObbG9ibnbEDbUOcicXd2AjqEOhaIO7LK4lY6h1UXiNsyt6FbeL9q196c8Tt6RVQXhhvT/0DfyMrCcO2RnegfVVkU/npoFwa17FHpvGsO7saQVt0r/e6XA3swvE3lwf/n1EQc2/bo737el4gRMZW90j/vTcKI9kd/90tyMkZ0rCxUfklMwohOR3/3y54kHBdb+ZjVu5NxXFwFcbkzGcd2qSyK1uxIxrHdKv/u1+3JGN698u9++z0Zw3tW+d3WZAzrVfl3azcnYWif6qJqfUIihhxT/ffrNiRiSLxjEbZ+QyIG93f8tw1r92DQAOfe/A2/7cagwTV7+zes3oWBQ1wTdRt/3oEB9RByG1dtx4BjK9tFfR/ejSu3YcCIynZX33PVtd6G5Vsw4Pjeda3W7I/Pyc3G5NsmgM6DiuLLnTdW27iUkZGDNh2vq1cbNAXszp7y/LkkAG30QW0Pmqun9rQAZDvdIQKbkgDkPTkTgRSALDWJwMYWgGxPRRFIAchSUQS6KgBZr6IIdCQAzTEVRKAjAchjKorAqgKQf19TRQQ2hADkdaqKQAlAV98uzo+TALTPsK5n8AUByEUgn376KRISyt6zLDfddBPWrl2rRSB1NZgGPl4C0AZgtwpAtsNHvYD0ALK44gWkB5Clob2AzgQgr+2qF9CRB5D16+sFpABkqegFrOoB5N+regGrCkAeU9ULWFUAmmOqeAFdFYCsW9ELaMcDaHg58AJSALJU9QLSA2h+78AL2NAeQF63ob2A3uIBNKx80AvYHAUgYwW3by8LGeDijueffx7jx49H69at0aVLFzPVm5ycjDfeeMMcY6WBYQoYpoLhyl+uAlYaGBtio4GqSgDaAOsuAcgmeNoL6A4PIO+jKXkB7UwDOxKAvL+6TANTALI05DQwBaARXDVMA1MAstQ0DUwBaI6pYRqYApClrtPAFIAsFaeBOQVsfteI08C1CUC2x5emgekBZNE0sI1BoI5VG1MA7k15GVFR1WPTMzJy0bHT9S5PAS9ZssQIvqrlyiuvxGuvvWYWfOzatQs8zipMBH3nnXeWJ4KmV1CJoOtoLI1wuASgDcgSgNXheYsA5J019jSw4gCB3xQHWOsbyVu8gPIAHl2AUWun1+EAa1xylwCsw6V1aDMjIAFoo8O8SQASgzu8gBSALHVdDNIQC0HYjqYaB8i2VVwMwoUgxgNVYTGI4gDLHk5OAbMoDrCMhwSgjZe2h6s2Rw+gh5Hp8g1IQALQBly3C0C2RXGAtfaI4gC9Ow6QBuDKamBfjgMkI3esBvbkQhDeg695ASUAa32964BGJCABaAO2OwUgm6E4wKaVDsab4gBpX7Wlg2nIOEBev+pikKYWB2gEiY+lg1EcoI0BoB5VG1MA7kp2HgPYLdb1GMB63KaqNBMCEoA2OkoC0DE8xQFW5qJ8gGU87KwGVj7AKjalfIA23tyeqyoB6Dn2unJ1AhKANqzCmwQgMbgrKbTiAF0TgDyqtqTQigMsY+nNcYC8v7omhfamOEBz/z6SFFoC0MaAq6puJyABaANpgwhAtkdxgLX2iifjANm4uqaDqZoKxoiaeu4K4gv5AMnHThwg6zvbFaS2dDB2U8Hw2o2RD9AIJzfsCqI4wFpfN247QALQbSh1IjcQkAC0AdHdApBNURyg4gDrkw7GlXyAtC/FAQI1CUAj3hQHaOOtWPeqvrQQpDEF4LakBYh0sEVpZkYO+nS+1uU8gHXvUdVoLgQkAG30lASgc3iKA6zMRnGAZTwUB1j9mdG+wL6zL7AEoI0BV1XdTkAC0AZSbxOARKE4wMoGsSNrc4MmhObVGioO0Hj89iXWuC8wj6ltWzhuCcfi7fsClxYVoiQ/D5s27cWgob3Aje8dlQ2/7cagwXE1vjk4BcwycEiXWt8wFIAsDREHWFJUhOKcTKC0FAGh4fAPCnbYHk9OA9MDaO7fB+IAJQBrfRx0QCMSkAC0AbvBBCDbpDjAWnvGl+MAjXA7sKdJ7wvMNq7ZkVzvfYFZvzHiAAvSDiBry3pkJfyKkrxcHDqUg9gB/RERPwRhPfrCv0VgJVukAGRxRQS6IgB5rvp4AbkQxAgnB3GARTlZyN6+CRlbfkNh+mEApWgRFoGIPoMR2WcgAqNbV7onTwpANsRXpoElAGt9reuARiQgAWgDdkMIQDZHcYDeEQdYWlqK1KQ8bPk1A39szMTezMPo1qE9jhkejd6DohAW2aLeC0GcCUD+vqZ9gfn32uIAzbkTk2rcF5jHcG/guu4LzHpNJR8g29IrOANpX32EgsNpaBERCf+QUJSWFCNt9160bBmGyP7D0fbk8+AfHFJNBHpaABrh6CAdTOGRNOz/djFyknaadreIiDbeTIpCegOD23ZAu5MnIrRjZe+kJ0WgBKCNgahKVWtc2pD4utMYwIFxVyoG0H3Im+2ZfFYAPvXUU/jwww+xZcsWhIaGYvTo0XjmmWfQt29flztTArBmVL4cB9g9tC9WfnEAq74+iOyMIiP20oszEVocgcKCUnToEooz/hSLg2331zoFbDxpB3djSKvulYBX9QAacZea6FEBaARol9jydlb1ADoSgPwdk0IP73m0nvldA+8LvO77VWi14SsU5+UguGNcpSnf1H3paBvZAgWpKYg+dizannJ+pb+7Og3ckB5ARwKwpCAfez97Gzl7tiOkY1f4t2hRyWZKS0qQt283glq1Q8fzLkdgVKvyv0sAuvzqr/eBjekBlACsdzf5TEWfFYBnnnkmpkyZguOOOw5FRUV46KGHsGHDBiQkJCA8PNwlA/BGAcgbVxxg5e6vTxzg+u9ykfBZC0RGB6Jlu0AjHvbnHUbb4DYoLipF8o4ctI4JQt8/+6Nlx8BaRWB98wEaUeiGOEDjFdyT5PE4QLbDUVJobgs35JjO1Z5bZ6lgjMBcMB/R+zYjpGv1eD8KwPYx0SjKOIKS/FzE/ukmBLVtX35+VwUgK7giAq04wP5DuyB/fzKyd25FUcZh+PkHILhDZ4R374sW4VHV7q+qBzDrj03Y9993ERwTC//AIIfvMYrA3KQdaDvubLQadkKTEYBsiLfHAUoAujS06qBGIuCzArAq3wMHDiAmJgZLly7FuHHjXMLfoAKQLVAcYK390BTjADMOF+KvM9Yi3C8abTseDbq3BCBvqqSkFLu2ZGHAyJbIj8lCeHorcM1Bm06h6DUoCjv896K+6WAqTgF7WgDy+k0xDrAw/RA2znoabdpFV4uHY5stAchp/Lzd29HmpHPQatT4SgKQ/3DnNPCGZRvRLn0DcnZuQXFerok9LC0tMVPSgVGt0erYsYgaMAJ+/v5HRVuVOMB9n78HisDQTt1qfHbyD+5Di8godL74xnIvoSc9gGysL0wDSwDW+krXAY1IQALwf7C3b9+O3r17Gy/ggAEDXOqChhKAvLjiAJtvHODqpYfw1oLf0a9fe/gHVF5JWtELmLDmCA7tzUN4Zz+0i4o0NldUVIqoVkEIH1iMY/t2xR8b0nE4NR8tAv1R0D4Xp47pg5i4sPLpSEcLQXie5hYHaDxyjTgNnLtnBzbPmQW07oCOHSsviLAefksE5iXvQkS/oWh/1uRK7wVXvYCueAAp+NYvmIuI3BQEte1gVuxaq5DpsSs4fACl+floM/ZMtBwyGpzqzUn8A8W52di1dR+OGTMEQW07IvGdf6A4JxtBrWNqfIcVZWcaz2bclJvRIqLMs6h9gV167ds6SALQFj5VdjMBCUCujystxYQJE3D48GEsW7bMKeL8/HzwxyoUgHFxcUjbOx9RDhJu2ukrCcCmJQDZl53Dqn8YONoR5PN39uLrL3ajf/+O1UyAArB1YGvs2JiJ5B3ZKC4G4ka2QI/YsgGbnsHUPblIWJuGltGhaNcpBCHhLczv96ZlokPrKAw6oR2OP6eTEYWKAzyKuC7TwHmJO5Hy1lwc8Y9Ex05tHD6q5QIwaSciBxyLmDMvajABmL7uJ2xf9BZi+vZ2mqql4NB+ZnNBVL9hyNm51YhC/iI9LROtOrRFaFwP5KXsNu+z4DZHp6sd3VxRVjpKCgvKBGB42ceHJQIHjOhh59VV77ryANYbXaWKlmNizZ63EOFgXMrKyMHwLpdqEYh7cDfrs0gAArjlllvw2WefYfny5ejcuXockdXD06dPx4wZM6p1uLcJQN6gr8UB8p53ZyegQ+hghw90XeIAP3trL779eg869mqB1kFtK52PAtA/PQIJvxxBSKg/crOL0XFoACLatkC74NYoyCvGltVHcCg1H8UtijB8TEdEtSmL5eLAvmPvIbTICMbIMzpizPmdsPpgYqVUMDyu6kIQ8zvFAVbqh6LMDKS8/iIOpmWiY8+uTgVgTNtI5O3ZjranXYCWw4/Gy7GCqx5AHluTF5C5+lIWzUfq1u3wb9keMR2jHbanpKQI6b/9CP+gEIR0jENgyzZmmjg1MQ1tWgai8PABFGVnAP4BiOw9sMaBKW9fojlHpwlTK08pr9wGTwpANtqb4wAb0wMoAdistVmjNN7nBeC0adPw0Ucf4YcffkD37pVXWVbtgUb3ALIBigOs9UFoanGAP36Vhv++sw+RPXLRJrhdNQGYltAC+5NyERIWYBaEDBrTGhktMo0ATP4jGzs3ZRjRt39/Nrr3boXu/Y8G/+/NSUdodigK80tw8R190bZTaLPNB0gww7sfXfnrbAqYxzVEPsC0bz/Fni8+Q/uBAyuJIKvD6AFsHVhItyxiL70FgS0rTxW7Kx8gPXtJ78xBi8hopB0qcCoAGbeXuWkNAlvHIHrgceV2tT/pkPn/7Tq2RPaOBBSkpSIyfjgCI50IycIC5KcmIea0SYiKH1rJPhUHWOvrxtYBEoC28Kmymwn4rACkN4Xib/HixViyZImJ/6tracgYQLZF08BNaxrY0RQw+6nqNPDBffl45emdyPLPRNcOlWOxKPiWfZOCYP9gFOSVIKZzKPoMiUJq/hG0DmiJ9SsOGS9geFQgDh7JRmRIKIae2A74XyghBWD74Gjs2ZqJsRNjMfLMTg4FINulOMCjT7Sj1cAFB1Oxad4/gcxDaB9/DPwCAsor8P1QnJmOgzsT0f3siWg95lSHrwdXvYA1eQDzD+xF0rtzEdiqLdIO5joUgGxP1tZ1yN+fgqB2nRAVP6Tyh0XSIcR0aoXiwgJkbvwF/qGhCIvrXWl6lxWK8/OQvz8J4d3j0f6MixFQJb8hj/GkCPT2aWAJwLqOsjq+IQn4rAC8+eab8fbbb+Pjjz+ulPsvOjra5AV0pXi7ACSDgKBRrqCo8RhfzAf4xbv78OV/dqNHj7YIiziai62ooAQ/fJMCv/xABIcGIP7YlohuE4R9eYcRXhCF9SvSzO8Dg/yRlp6D8MBgDD0pBn7/W/hJAdghpCX27cpGXN8oTLypl+IAK1hfXeIAWS0veTc2L3wN0UUZ8AsMMomgUVKM4uws8//T2/TBkClT4BfQAgWHDiB39+9mtxD+m+lZfj/gh8FDa15xy23hahKAjMdLfOuf5pyHs+BQAHJhR0bCGrNVXUjHLgjvcYxDAchfcnGI2fLNz88czwUlLFz0AX9/hHXtg5gTzzUeR0dFAtD2K8/pCRpTAK7c/Z7TGMDRXS9RDGDDdXOzObPPCkBn+3y++uqrmDp1qksd6M0C0Awk+T+5TQDyfCHBdROTh/Lc7wFkO9qGVp72qtjZ7ooDpHfv1VfXY9uqfEQERiG6dQsEtPBDbnYJVq84gOCAQPQf0QrtYst2mKAAzM0sRsovJQgJDzALPLKOFKIkrBijxlVOkEwR6Lc/CJ16RODCW3u7JAB5DV+PAySDIfEOcgX+sgU9WqQjO2EtCjOOGCEW1qMPwvsOxLb9wICerXFo2ZfI+n0TihljZ9yxpfALDEaGfzTiJ12E0FjnItCVfYH3f7MY6etXIcu/bJq5ahwgRWL6xtW8LCKPGYKgVpUXrnAamB5AI2pTkxAa1xOtho9F1vZNZrqXK4m5MCSiV3+ExHavliC64jPgaQHItnhrHKAEoEtDqw5qJAI+KwDdwbdRBCAbqjjAWrurqcUBssFFhSX46sfNOLI+Gkk7chhKhpAQfxwuTUfmjiCzJVzFD5GkjENI/rnELPYICQ1AeloBWh/TAsf0rjyNTAFYsDsAw05uj/EXxxk29d0XmHVHtC87hzlPcjJGdDwqkqpuCWeOqUdCaNZzlA+Qv/dkHCCvv35DIgb3L7tnsq/YJ+tXbUXxT58hKjPJxN4FRB7tM+4gcmDbH/ALi0K/K65FaOeaRWBNXsDcpJ3Y+8lC47VLzw92KAAPr15mpomjB1bOBch2W3GAFIG5yTsR1W84Yk6Z6PS5Ydtzdv+OvH1JZjUwt8EL79oHwe07Y9NP2z22EIQN9uZpYAnAWl/lOqARCUgA2oDd0AKQTVMcoPu9gO72ALKfHKWD4e83Z/yOnuF9kZNVjOLCUjDMbNPqdPxr1hYUprdAZMsWiGwdhHYdQ5ATloWcHUFI2p5lpnzDIgLRdnAAYqssPth98BACM0Jw4S290bl3WQqPppgOpuKWcI4EIH9XdV9g/q4x8wFaApD/tURgxVfC4R+/w87FixBzTDz8g6rvrGH2e964CR3i+6LT5OudetZqmwbmNTM2rsbBH/6LwykH0bZ7HPxDwoDSEhRlHkFhZrqJSeS0bVhXx/HKFIFtYyKMx6/9mZMdrgQ2sYTbN+LQqu9MGhk/ejP9/FFaXGT2DQ6L62l2CNm6Yb/HRKAEoI1BiXaUkQGGMmkK2B5HX6gtAWijl31BABKP4gCPGkld0sFYArBXRFm81pGDBfjszRTsTMjCvoPpyN4fYBZ8MFl0cIg/gmNKEdMqEr+vzURAC2DQmDbIjcg1dduHlE3v8fjNWw5ixPGdcc7VPcoTTbsqAHmOETFHPX4/702q5gE0x9TgBazqAeTxq3cn47i4CnsA70w27W0u+wJX9AJavc1Yv+SF/8SBxH3o0KeX0zdF6p5URPvloNOFVyKsu+O9xF0RgLwA9/Dd+t8vEV2chpKCPOONbBHREhHHDDZbu6Wt+MIkebbi+io2iulgovyzENSqLWInXetwgUfmtvU48P2nZocRTglzutsqTCrNlcYUgYdaDcPAsa4lxLfxCnVYVQLQHlFrXPp+5weIiKq+rWlWRjbGd5+kGEB7mL2itgSgjW70dgFoBiQ3xQHyXPVZDNKc4gB5j93C+1WyKHoAWWL9euGDuXuwfVMWYnuEIh0ZCM6Jwr49uUhLzUNedjEyswvQKrYFRo3thNTEPOTlFCOyVSAyW2SjTWA0Mg4VoKigFD0HRqPdWX44vutRUUIByDK8zdGcdu7OB1haXIyCnTuRt3EDdm3ZhtjoaATGdkbowIFYXxKA47pUjq9bszO5mgA0orDbUaFoxwPIc9V1X2DWcRQH6EgA5u7ejpR35yPdLxIdOlXO51ixk/fvPYLI/ANmpXCbMac7FjWrd5nfu7IryIZVfwDZR9CrZ0uTnobTvgEhYWC+wINLPjGxgC3CI8wWdpaAK87NwYHf/0CbuI6IOeUChHWpLlg57Zu0aL7JFRgSUzmutFz0FhWaZNLZbQdiyJSLbbwd61+VApDFG+MAeKPiHgAAIABJREFUG3MKWAKw/jboKzUlAG30dKMJQLbRR+MAeeuuisCmFAfIqbZD+wvM1O+uvN3w39sBn7+1F517hpoVvkwIzdI2uI0Revm5xcjNKkJqRiZufGAoQiNaYMvqw0j4+TCSDx9B6+AotO0Ygv6j2qD30JbYmJ2EIa0q561syDhAZGej18+/IG/jRpQWFOJQSSnahoWZbcf8Q8Owv0t3FJ14Mo7rdXQXiaoCkPfbGHGAvE5dVwM7EoA52zdj77//hfTgtsYT1z7G8apZCsCSA4nodsppaDv+PKdvFFe9gDzBxp93YMCw6gmqS4oKceTX5chM+BWFR9JMzCB3A+EK5oySSAT0OhaDTq+csNpqUObWdUj9ahFCOnSp5Pmr2uD8tFQcOVKAwBEXYuCYeBtvyPpX9VYvoARg/W1CNd1PQALQBtPGEIBsnuIAm08cYNeweGxbn4V1K49g55ZsFOSXILMoE1kpQQgK9kP8sdFmhS+LtS+wZYIF+cVY80sqBg/rgFFntkdsj3AEhQRgVdJODGjVDeHRgQj4397Caw7urlUA8rxVvYBVVwKbYxxMA1ecAi4pKMCGl+ag3Y6dxuPnHx6OlPQMdIqKMosmSjIzUZS6Dwf79MPgq64uz6fnigDk9e14AR15AOsrAFmvYhyg2S7u3ZcR1LY9DhzOdyoAWW/f2rXocd4ktD7+5AYVgNbJKbzL9gJmvKi/2T+Ywm7TLzsw4FjHCe0PLP0M6et/QmhszQnvuSikIG0fsrudjEGnj7bxhqx/VQnA+rPTFHD92flaTQlAGz3uKwKQiBQHeNRQnMUBbs3YiuRl7bH00wMoLChB6/ZBCA0LQPKhNGxbXmJSeLSPCzGJn4OCy7yA9ADy2KQ/snEgOQ8HD+YgPCQYXY6JNNO/8ce2QvCxeQiJCsCglhW8azYEIO+kLnGAuWvX4o95L6N9z97wDylLW2MJwHJBkpWFA3sS0eeWaQjuVbZIgQKQpaHjAHmNoX0qTz/X1QNoRGOF1cD8d2lRIZLfmovCg6k44hfpVABSjB3clYh+102rNR2MK1PAvLYzD2Btr6uNq7Y7FYCp3yxG5pa1CO3keOs769yc6s9LTUROt5Mw6MxxtV2yQf4uAVh/rBKA9WfnazUlAG30uC8IQOLxtjhA3lNyzga35wP8fNkmfP1aDmJatkHrmKMrRpOPHMSWFfkIaRGC/NwStO8Sgr5DInEg/wii/Vph62/pOLQv3ySALikpRVFQ4f+zdx7gbVxXvv+j9w4CBDspkmJXlyzbknuJS2y5xXYcp9hJnGyyyctuNutk9yV5m7IbbxKnujsuiZ3EsWU7jnuRZVuS1UVK7J0ESRC9AwMM8L57YZAECYpgUbE4158+k8DcmYszg5kfz/mfc2jtP1IGxutgUFGvRtE2HjaVZGZ/TvcCnggdIA1lP/oHjB48BEttpr5xOgSONbeg6IILobvuholvVS46wANdw1hVXEC9m3x+quVJrpnAZNsTqQP0HdwN++vPwZuQI7/EMuNuQWFpqBd+RSHWfOXrWVvKpSflUg8wvS0BQDKyhYGPd8s6HgA697wJ14fv0CSP4414KAA26Eeg7AI0XbRxEXfIhU89U3WAJzME/Hrv81CoZiaBBP1BXFpxLZcEsvDL84yZyQHgIk7lSQVAsk5OB5jT2SJaQK10/XG3XWoAJOD25K+GcejIGOpqM0GBZZP44J1R8GJiqv8jfXybztEhJPZjpIOBp48HlT4V3vW6YpCaEli7oYCun3gHrd1BGDfy8Pk712V8plzCwPvaj6Jm1I+4g+jFAKHJhNY8NTZUTsIcCQGTka0eYCIUwvg9/wt7OIKkRoNC1WRf4ukAONrTi6RUhqbv/3Cilt7xwsAeZwRdx1x4990BxKIs8nUqrKjRobpej9Gwh+5j3YopvYI7Uh7FXPoCk+3m6wXMpgMkXkDHmy9i6J23AbEUpopSmo1LCivHfR7E3Q5IC0vhKtsCnkqPplWTGdbZLsCl0AHOBYAUHLOEgSOjg7C+8DhEKm3WLOL0fsMjA5CXVsKp30jPQcPGSc9zTl/AJdroTPQCcgC4RBcHt5slsQAHgIsw48kCQLJETgd4eusARwYieORng0go/LAYzDOuqqNtNtg6eNAYRPA6Y1hRp4S5WIZdO0Yh4UshUwjAxhMI+liYmvhYUTxZ/Jl4Asf8Pnzte+uhzZNM7Pt4AJiMxeB/7W2Edu+DY3QEWhmpF5gEEkm4pCKUXnoplOdvBU+YKgMymw6QJaHd//05eGIxbDz+cQEwbrdjPMKg6Yf/NScAaoIy7Hh5EF5XBFK5EL5IBHq5HAE/A5lciPXnWCC2JLG+KjO0e6jDekIBkNhiej1AAoG+wx+i9403oeGFQSp6JwEIlWooqhqg3bAVIq0eS9EXeOqFs9RhYAKtY6/8mXYHIe3k+ELRjOs05nWBlIMxX3YDlCvqub7Ai3g+ZJvKAeASG5Tb3aIswAHgIsy3nACQmGm56QDJZ86Xrcp6hUzXAXa2BPD4L4agLgtDJ83s3EF2YHU7MHyIj5CfpfBQWC6DRi/GoX125BmVNPTrd8WgN0ugb0zCLJ9s9UXeazk6jg2f0uGayxozAJD8MjUbmISB12qL4Nv+DwR2vAeBVgubVIACZarFGAlZjg4NgOcPoeSqK6G68nIKa7PVA9xgssDxq18j7nTCrlTNAECyT5IMQv9IGeiH3WjG6m9+a3KNWXSAr+3sQt9OP2KxBMwFcnr8UZcPBbpUUonPHYXfF0PxKgU+ta0+w/6zASDZ6EToAKcevHlfN6q0cZCSKwScSdcMkSZVn5GM0x0AyRpjfg/G33iO1hsUKFQQqXXg8QVgo2GaVcwT8KFbfz5067fS83Kq28KdaaVgOABcxAOXm7rkFuAAcBEmXS4ASEzE6QAzL5TpANjTGsRjvxiCvDBEe/5qxXkZE+xRJ4LuBGytAoxbo7CUSqExiNFxzA2lTAoSJtaZJKhqUkMqF9DewHmSFLSRMdgRQOnlItxwTVPGfrPpAPmdfTD/9XUIdDoINGpYg14UyLUZ80aGh5CX4EF/150Ql5fNAECycbotXGDnTvie2w5HngmFmmn7SWcDx2Ng+vsxfuFlWHvZZZlrnFIPkADe/b85gJHuABrrTROewjQApic6bCHqFfw/395EPYLpkQ0AyXsnUgc4AXiHB9HUMHuIN1cAJPvLJRnkROgAybHjQT98rQcR6CC9j900nM0XSSAtKIG6bi0UFXUT5+VUAiCF6vfbz6h6gCcTAF/qfmlWDeBVlVdxGsBFPPvPlKkcAC7iTJ5MACTL5MLAp28YOOCL4/4fDYCNJ8HT+WcAIDl/BALFYQ06D/uRVyildf+62rzQWgQoLzfAkC+hGkEyCACqExq4bFFaAJr8v/AsMW76dD0KViiPWw7m0P0PwtjcA0lVSuyfDQCtAS+Mw+NQXnge1Ns+SQGQjGw6QNbthvOhhzHe14dEUXEGBBIdoEWhANPXC3FZOQYuvoI0PJ7RFSSdCTw+EsQzD7chwEZRUTgJuAQAySBeQDJIOPxYqx0bLsnHtk+kOqkcDwKXshxMtpZwFEYOp4ptzwaBBADp+6exDnCqHdloBIxrnLaBE0gVEBsmgTy9HQFAMjgd4CIeFFOmcgC4NHbk9rI0FuAAcBF25ABwYcZbSEcQcqRcC0KTbU9FIsjrz9rx9gsOaMrDMMhnhoEJAEZHFFDrRbjtX8rhtjN4+lf9CPIDqCia1A0SL1lrlx3+AR6ioQTYRAKJeBKqEgEsRjVKa9W48KYiaAwSTPcAJqIMjn7vhzCKFRDmTXavmA6BBADN/gh4chny7v72rGHgdD1Apq8fnr/+FeM9PTCa8iFQpXoQ20ZGoE8kKPxpb/wURPn5WdvCpQGw66gLL/6xCwJdEoW6zMLK072Ag70+FNTLcdunJsPe5JinQgeYvtIJBC6FFzAXDyA55lLrABfyjT2VXkDOAzj/M5Z+LnEewPnbbrnN4ABwEWd8uQEgMdWZpgMkn8koW5P1KhgIttLXc9UBuh0x/Pk+K9o67KisNEEqE0zsl3gGO/vHoRZqcMVthWjYlAql/uNJK956bRB1deaJAtEj/SG0NjuhkEghkfHhd8dgKpJBVQMw4QQSNjFKa1W46o5ytDMjdD9pHWAiGMLR//wv5Cl0EOgnw7XZAJDndMOkVMP03X+jmra5+gLHHQ60vPEmCvsGkAj46XHtRD/W2ISmCy+GQJMCuuP1BZ43ANbJcdvNuQEgOfaJ1gFyALiIG+YCpnIAOH+jcQA4f5st1xkcAC7izJ8SACTrXablYObrASSmOtnlYOyjUTz2aCtGuuOQJtUQSfg0LJxgk4AuhPVXyHDZ+U0TGiuS4fvAb5sRHJDAkC+l3UKad7np9gyfQTIkhEonRvUaDW0PR8qjGAQaWLsCuPBTxVh/sTnDC0iSPJr/74/A8weQXz5ZNzBbGHi0swP55RUwfvPr9FswHQDJa2kdYPprsm9oGOv1RiR8PtqGjK/R4MDYODYUTmbrTgdAMjddDsZmTYWAQ8koJHIhCrSTZWWmegBJCNg6GEDlWWpcd0VmO7KPuw6Q2CPXcjAnSgc4n9veqfQAUludQTrAkxkC3t75yqwawG3Vn+A0gPP5Epyh23IAuIgTe7IBkCyV0wGevjrA9KUUjyex42Abgl158LnjtOtHabUc1atUGEIfKpWZmrY9fZ049HIQ/i4prD0h2IbCkCkFYIUs8s0qlKxUUvgDG4fTbUWeWIVxJx/yPDVu+ddqtAQy+wL7Xn0bA395FqbGSdAkAEhGOhmECP9tLS0ou+3TUGxN9Y49ng5wKgCSnzcWTALfvsHhGQBIttlQPFnDLw2AJLxNQsA9rW7wtZlh4Kk6QJIEQtrgNVykpwWyp9YDJPs+0WHgj7sOkNjoeEWh53Pb43SA87HW8bflAHDpbMntafEW4ABwETbkAHDhxjtTdIDEAkXyhhmGaBnvANNbhNb9Pqr1I8kdFXUKiKqcWK23gBkaQzLOQqBSQFZZgmOhARgDpfjLb/rQ3+aHqViGqCKMEoMRfCYM2Ugn5MNtiPnskAnEYHhijMvKsfXfL8FohSCjFEzMZkfrPffCEOdDVFI04W1MewEJhMX6B+CQiFD3r9+i2cLpMVdfYOoVHBo+LgCSbbKFgdM6wIEuL17+czfs3iBqqowT6yPzCAQqeRL4PAzOu7wY68625NwVZKkSQcg6shWFTtuICwMv/Hu/kJmcB3B+Vks/lzgP4Pzsthy35gBwEWd9OQIgMddS6QDJvqSSs+Z1BuYbBs4lBEwWsFQ6QAoxgxH84ZFW+AcVNFtXKueDZQHW7oDOfRB1Ci8M6jh4fB6tuyYpssCzrgB1l3wSzz00hN6jfhSUy2kmcD4rgO7IW5A4hpAQSeATimnoVR5LgHV7ULKxBP6bzkGytiIDAg+8+xbyX9mNhNdHk0H4GjVGgl6YGR5IwWah0YChi7eArSyfV1/g2QCQevxyDAOTbduPOPHsX9ogZYWQK0TU2xePJzA84oNZr8S6s/Nx1vmF1H65toUjAEgGpwNMfaWWygNI97Wrk8sEntedKvvGnAdwCYzI7WLJLMAB4CJMudwAkJjqdKgHSNahyAEcc8kEJvtayrZwrnEGT//eio4eB/LLBTAqUtnAQq8DqoNvIDk+DjdPgrKmYhraTUSiYGxOePwuFF1+GZqTjdjzhgtltUoa8hXtfhYG5xii+gJAkKqHF4hFIIqJwMYSqCmJQJanxujNF2N13WTfVlIQmjdgRXnLACJtnUgEgnBFwzAYTJA21EF+zlkQlxRj79jQcQGQAp/VinQ2cDYApK9lCQNPDQGTbaa3hduxrw8jPX7ER4FohAVfwKPZwZeevwKFpaoJz2CuAEiOMVs9QPLe6prMriJHWoaoPVfXZr5OXjvRHkByjFx1gBTA9vYuaV/g+d72TiUAUludITpADgDne+Vx259IC3AAuAjrngoAJMvldICnrw7wze12vPW8A+U1cnjirlQ9QDYO7Z6/Q2wbAKvSwROMQihWo+k8Ey36TIbNNowkCX9eehO271DQjiDa4DBke7aDr7EgKZpsAUcAkPXykVckw4pVGjDtPXBuaULTrbdnXM20K4i+BOy4A3GXG0edY0gadFhXPdndJA2AiUgETGc3Wvt6qIexvqYe4vJy8ASCrABIDrRQHeDURR7otWJtcQFiMRYCIR/N/aP07XXlU3oAd6V6AGfTAa5ZUQDGOojo8ABI+7u+8QAazj0LImNmO76l6AucXvdc9QApsBwaOOX1ACk4ftidtS/wfG97nA5wvhbLvv3JBMBn2l+HXKWYsZCQP4gbay7lkkCW5pR+rPfCAeAiTh8HgIswHoCPuw6QtGgjHUH4fB7VAQYDcTzwowEw0QTyLBI4o04KgLKew9DuegE8NgZeEogn44jFxdBUFcCyvhQCpZwacqy9FXk1Tdgt2oqOw340Rj6EaPAIYKrIMLTTHYSUL0H1Oh00RjGI5s8Rj6D++/8Bvlw2sS0BwHWG0oy5e21DWG+cfG2vtR913VaEP9iDuM1G+9y6wmHo1RqIS0qgvOgCNKtVGR5AssPF6gDTiyIAuL5sEvbI6we7rRkASF7L6gXcdRBFvYcR6esGAVjwAI8vDEN+HuTV9dCefzmE6lRpmvkCIJ3TMjSjL/BUCOTqAS7u+z+f2ZwHMHdrpZ9LHADmbrPluiUHgIs488sVAInJlqsOMBJOoKMliCN7fBgdYoizDFKLF1u3VEMq4+OJe4dhLpLQzF8CgCZ3EPodf4bINYa4UoekQEgBMBliIRfFoS3RQbm6BkKdGjbHCNQBPtRf/Cxeez0G/gtPQ8yMQ2xKJXKQ3rmRIIsYPwZztRQrK1PhZTYYgn3EivjXb8Xq6nU5AyDJBD781FMQvbcHRq0ewrw88MQiWP0+5PMENFEEQiHsmzYgdtYmbCyvnNx3lkQQ8uZ8dIBk+4UCIDM2gtYHH4CO8UNksoAvV1Ab2exeIOiHhg1CVrESedd9GgKlmgPARdzn0lNPZRiYA8DcTyAHgLnbarlvyQHgIq6A5QiAxFynmw4wFkuiqz2KsZEYcWBBqxOgtkEChVJAO4KQkUsyyGyJIGQ+KQotDtThucdt6DoWgoDPg1LDB5LAqNMNuVADU5EEo/0RlFTKaNav19mL/H1vQ2WzIsnEEBUo6PbEUxWLs1BrxFDwQ0hIFBDU1SMkCiEvHEXRt76ARH4Rjt79ADxH+2HjKSHhi2mIVJcnoRnCYUUIJmkqe5f1B5Dw+mH74jVYXbU2AwDJL1O9gFM9gJFjbfD84UnYhTwUFKR63JIM4dG+PuhcXsTdbiT8fvBEIgQbG1B28SWQb9wAgVo9wwNI5i5EB0gAkIypXsCpHkCyHsY6hPZ33kORIE5tJzIXINx+DOPtHTDXN4DHT7XPSw+bwwezWobIYC80Wy6B/uIrKQCSwekAF37DO5UASFZ9JkAgFwJe+PXHzVx6C3AAuAibnioAJEvmdIApHWDzwTBee8mHgT4GpP4ej/AVjwdTvhDnnKfABZep4IwdWTQAdtiP4c0/GNFxJIjSKlKweRI63IwD4rgBve1hkMLO1Y1KGMxiyDoPgHdoB2J+QBZ0IMyf1OOQjFexWEALP4uYAHzaMoTEIhhYB/KvvwJlZ5cj3N4Dz84DsJt00Im0tKh0WjM4FvZMACAzMAxRQT6sn7kMq02TxZ8plE0LAxMATEFhCTxPPo3I/kOwW4woVGgo/DG9ffB2dBIShIKETwV8sC43wkYjtBoNJCtXQnfLzTgYCNL9LJUOcDoAUlgz6+B59UWEW1vgsrsAkRh6pQxxjxvMyDDExaXway3Q82KI2ceQCIcAHh8hoQzGsjIkYgz4AiEsd3x9QV7AuULAZI1nchiY9AoODXYh5nbQ62Fo0A++sQSNWzI7syziFjqvqRwA5mau9HPpqda3ZtUA3lp3EacBzM2cZ/RWHAAu4vRyALgI4300dTE6wLZDTfjz4y5Eo0kUFIloCJaMeCyJcVscAT+Liz+hxuar+2FQrD/uYufKBH7pzRa8+YQSxRWp8O7UQQCQDAKBB97zQq4UYPVGOZRv/BUuqxuJuAD5zCAYkRzgCRCPJRBjElQ7SKBOI47QcjCJeJwmQ0TUhbT1m9YoAmMdR6hYB3N1XcYx0wCYYBgwvUPQ33otOusLMkrBZANA8hqBwDViPRw/+yX17o2J+RQA42M2RJqPAmIxggI+VOJU4gnr9SKkkMO0fgMFRNnqVdB99nbst44sqh5g+gNlDQO39aPs8PsIHt4PkdkCvlIFm8sPi16NyGAfwq1HKZxGYwnIpGKAeAGFIgoqkUAIUrkUYnMBTWIx33IH5CsbuDBwjl9X8oeAv/0wPAffA+Oy0z8MyB9W9PUwD2XnnQftmi3gC1NZ6SdrcACYm6U5AMzNTtxWAAeAi7gKljMAErOdSh1g78gB3P+/JQgEEigpE2c9i14PC6c9juvudOHszZPauGwbEwAkI1sYmDz4/ucnzRjuSmDlSkvWYxEIVItMaD3oh88dQ2lhEpa9z8Ib5kGkVCDP1wMhG0GEJ0M0kgSPn4REKqBt4hRJP+TJIJISGTymEkRkRUgmkqipEwFd7Qg5nUDjCphW1mUUTR5z2aAZC0DaUA3DF27B4UgKRNN9gY8HgKuTCjh+/mvav3cUcQpO+vY+sE4H+Dod/NHoBACSvr8hvgDxtWuQz+ODdbth+PKXcEQgnAGA5JhLoQM89MobyHvvNYgsBeBLU4ktY05fCgD7exDu7kAiFEI8FISsuAx8WSqRhoxgKEpc5JAiDp5MhoIv/ytUjes4AMzxXudt/hCOD14Fjy+ASJcHPgFrAoCJBMY7+6BWAJrGTTCeexnd5mQNAoBkNGzO9HKfrOMvxXFOZgiY8wAuxRk7s/fBAeAizu8pB0Cy9mXaF/jFlw/jmSdEqG2wUE/abKO7I4LK1R7c8qUEdLKFeQHDoQTu+d4QQgk3CvPzjwuAjjEGPk8MomQU2rf/CoiAuFQJRTwMbWAQPDaOaFICoUyEJBLgxVhIGC/kwhiYggq4jRZIRUp4HAyKKxW0hZzvvX0I+bxQl5eDr1bSAtKJUAQeNoL8pjXQ3XothHotXdcBx8AMACSvT9cBTvUACvQ6jIyNQN3SDp5UCp5YnAGAxAMo1OvgrqqCRakC09UF1ScuR0ddfQYAUuBcQD3A6TpAAtzN994Lg3MUktLyCXunAZCUfAkc2o9kjAEbZwGVBgpz5nkhEChDHKzfi8Kvfw+aTVs4HWAO9zrG7YD1uUfoHwRifSrJaOoYH3JCrxMh7nMh/7JPQVG+Moe9Lt0mH3cvIAeAS3ctcHtavAU4AFyEDU8lAJJlL2cd4L3/68TRo2OorC447hl0O+M0RHzH3XaUWhYGgKEgi//9j2GEk24UTAON9MHTHkCHjYFQyENNkxzHfvZX6IODcIoMEPKEUPGDkPlGIGHDEPAJ/iUhiMUoFDLaPKC6DsFkPLXLsBjEubL2PCOQiMF2pAX5m0ihZx5tIcfT69Gq0aC8ZhMUOgl0Jgn1Dk4HQApl2XSAySRWvLobkX0HIa5agZGhAaiOdoKv1UwkVVAvoEhENYCS+jo4tFoUqNQ0DCzfuB5dm8+mS11qHSAbDMD2m5/DEYzCUpJKTiGDACAZeXwWnrdfoz+TEG88yYOiJLPcDQFAaYJBMhyC+bP/BP1FV9Dt51sO5ng6QLK/M60tnOvATjjffw2yoooMb3P6HBAAJEPF80JZ2QDzJddn3W4Rt9XjTuUAcG7Lpp9LTxx9G3KVcsaEkD+A2xsu5DSAc5vyjN+CA8BFnGIOAOfXxm02Uy9EB/jjH9gxPGJD6bQactOPQXSAHheLL/y7A5WlCwNAUu/v/p+Noq3Dhurq2UPA5NiuPiVqVhOtoAy7HtqLlbad8ElkEMu0tNOFtTcISTwIGS+KRDwKaciHcFKCSGkdTKaUtjAYj0AQk4CNJ7Bmq4G2SRs9dhQF526F7Npr0LzLidYP3egbdUEjVEIi5aOkVo3GzQY48lxYo8+sGzhbPcCG8TA8jz4BvloFWzAA1dEO+jOBKjL80Qjk4Sj4EglkGzdgjGFSANjdDcWWc6HZtu2E1ANkfT6M/e4XcEZYWIoyAZ9AYJ4I8L7zOhJMFHyR6CMALMs49UE/sXMEQo0O6rPOg/lTn18QAFJoXEb1AIeffQiMwwZJ3ux/WBEI1Kl5SLJxFN90F4TymZCxiNsqB4CLNB4HgIs04DKazgHgIk72cgdAYrpTpQP87S9cOHRkFFUrj+8BdDriNCnkzu/aUWyeGwDJZ8qmA/xwpw+P3t+DohU8aOXZw8Djfjt8VhVuvqsAoWACLzw6jMboPvD7j4CvzkdCqsDIYISuRyzmQRSyQxFwoke0EvpyA/LV4QkA5EVTusa15xkgFPEx2tmOYH4J+uRXYLAjAKVWBLVehHHGBzWrhHucoSCYf6EQ5WcpMyBwNgAkmcCBl19H4PW34IxFwR+2QsETgq9S0rI1QY8bcpkc0vo6CM2pcOCIyw2jwwXtbZ+GfN3arABItluMDpB09LD9/l6Mj9hgqVyR8Q0lAGgSAf79u8EGg0gEA0iIpVCUlNEOJqQOUCISQtgfBHR50OabIVtRDfPNd5zWAEgW17i6ZM670UJawpGd5toRZPDp34J4YLOFf9OLS4eB2XAQxTd+GSJ1Sn5wMsbHXQd4MkPAnAfwZFyRH+9jcAC4iPN3WgAgWf8y1AG+tyOEBx/ow4rqBKSSmb1c06e1qy2Ks7bIcdltqfInC60HGPSz+MOvx9BybBw1NWYKZVMH6f7R2m7HmrUFuPmrhVQH+PBPByEVxaEeeheK0X5IY0n4Q0DQG4dUnECML8VIUIlxRSXOWuGEWJCYAEDGLYClVI7KVWr6WqCjH//w1yMIczxoAAAgAElEQVQor0VhlQJCYer4o2EPzB/VA3SORmhSSeWNUly6sXZieQQAychWD5AI+0O79yL03i7YP9wLKSm3IpfRsF5YKoaushqi4sKJMN9YewdMBYUwfv1r4MvlJ6weoHfHWxj82zMwNzVlJr44fTArpQge2AOSAR33ehAXSmgWsEQspNuShBBSK9Ar1UIbcEB97kXQX3zVBACSH05mPUByvKZVk6HsbLec06Uv8MgLTyBs7YM0f/b1Ug+gMkFL7hR/6i4IJJPdZxZxO8156sc5DMwBYM6nmdvwJFiAA8BFGPlUAyBZ+nLVAZIM33t+4sSozY7KaktWHRLJAA74E/jSPxtR1yilRaEXCoDE1rYRBg8/0IbxXhUtBaPSCmhhZ5+HpV49Y3UQt9+5CjpjKmty+2Oj2PuOB6WVUoR9fchz+cA6nBjuiyAo0sGvK4Zz0I1G6QAqyiYTWSIhFv5wBGs2maHNkyAZi6P3sBMvM+tQsbpiohbgdAAkvw91BKBqAO78Sqa3c7Z6gOm2cIloFIdefw2mZ19GrK8fPKGQJoSEEwko84wQmc1IsCxcDIPK22+HbPVq+hlJSzgylloHGHPYYX/0fjgGh5FXWQmBRErXlNYBahwjiHQcgyi/ALLaBjhG7dArU9uQsC9PKIJtaBQ6fhz5t30JkqLJEPFCdIDkM66qz/6HxpmkA/Qd2w/bW9shKyibNcOXAGDCNYyS8y9G3pZPLOIOurCpHAAe325cCHhh19VynMUB4CLOOgeAe5YkBExOwUJ0gC1HInjwwW4EfQZYCkVQqvgUBKORBMZG44gzSVyxTY1PfFKdahO2SAAk62yzHYOzrRwHd/ngHI/Rq8dUIMbas9WQVQ1DIkv1BWbZJPo6QnjxiTEM9YYhzYugpNAMgYCHwe4Q2g/6aOHqfK0TFzAfgK9WIy6UIhxMwaSmLIn6+lSoOdo3jH2uMhwQrEDj6sy+uVM9gGRbv4vBiNeHr31vAzTGVB0/Cmpz9AUm3rTmRx+D6XArYqOjSEQZGk6NxOOQsCzNChVVVMB+wzasveSyDOBeaF9gsq71JZOfJ10PkPV5EWo+BN/bb8B9+BDECRYChRKiwiKIjCY4HB5oIgHE3U4KptKqWticfuTrU95SMkhvYNIppOjcrTBec3NGt5D5AiDZ33LRAcaDfli3/wExr4t6Acn3ZvogtQFdNjfqbv8SZJa5w9aLuMVmncoBYG4A+IcjO2ZNAvn8qvO5JJClvjA/hvtb9gD4+9//Hvfccw9GR0dRX1+Pe++9F1u2bMnpVC53ACRGWqq2cAQAyZBK5pdYsvfIIXzwVjm626MIhRJUBkYgq6hEjK0XKXHWufKJMjG5AiBZx2xt4UhLOAp9kiYwkVTIViJLgScZXZ5WuFpKcHCXFyP9EQp0LkcM/lAIaoUCKp2IZgknkkmwsSTCrA8l1gMocPchLpKDr9eioEIJ1sggYNPDPRwAXy7BgHoVxhJJmCrEyJPoM67PqRAYi7Lo7HNiy10mXNhUM7HdXADof+V1+P/xGhxaJQry8pHweBF3OOH2eqAlfXZJ0V82AfWN16FtRQU2Wia9YdkAkBx4ITrAVTIBXM/9BcxAH/gKFZzBCNSRAGKjVgp1AqkMocIKlF9+OYRaHbzvvoW4YxweVoA8cx6SyQRYrwfJBAt5XRNGVpyFNauqM+x1ugIgWWSuOkCybcPazMznuW5aueoAQ0M9GH9rO2I+N8RaIwSK1B9QbDgExmMHTyBEQFePVdd/cq5DnpD3P846wJMZAuYA8IRcfmfUTpc1AP7lL3/BZz7zGRAIPOecc/DAAw/g4YcfRmtrK0pK5v7L9rQBQHJJLkMdIPnYrsgB4pyCw7qGev3SvYArV4ohFk/rEbuEfYHzZatm3AhIh48/PNaM7t0KmvFrMIloqDjGJDE2FIXN5UPjKjO2XmVERa2SgmpPawDNPT3IbxuBuK8NEtaPLpcBh0bViLAq8JVyiPPzMO7kgYSGjXV81FSZM2ofTgXASCgOj51BwxcUOL8+s0ZbtjAwCQGzfj+cv/gtaPKFQkK7gqSH1e9DoTLlWYtZR2jh6P6brsfG0slM4+kASLZdSD1A0sqt5Vf3Qu92QFJWQbORx1w+WHRqkBA1eZ8ZHYZXrkfjN/6FAmDMaUfoWDP6duwELxKCQaOE2FIIZdNayFbWo7nHjjXVmaHb2foCk3UfaRnC6tqZod7FegDJvk8HHSAFx/WTtRVne5pFbMPwHN6N0GA32HAAPPKfSAxZQSnUjRvRN8pH46bMBJ2T+WT8uHoBOQA8mVcJd6y5LLCsAXDTpk1Yu3Yt7rvvvgk71dbW4tprr8VPf/rTuWyH0wEAySKXqw4wfYIIBJK+wLmMXL2As3kAyTGIFzAbAO58xYW//KkXZSV5UKqFIOVjIqEE/T8BwXG/E2GbAhdfTyAwb2K5bb4uVCprEPMF8M4jrXh/RwgSCQ8CCwsz1WLxYe0JonW/B5AksKJGh5Jq5YTXcSoAjg+FoDVKUf05CdabMx/Qs+kA6/ud8DzxFEQV5RgJBzIAkCwyDYEEEGODQxi9+kqsO++CDHMvNAw8NQQcOnwQPY8+DHNNDdXwpUcaAsnvJGkl2tcD15otWHvdtoltErEYjrT0YNWKQvAVk7Y53DY8AwDJpPl6AQkAkrEcdIBpo5KC3DG3HaQ4NPkrS6hUQ2IqnAinH93ViYaNmSWHcvkOLsU2HADObsX0c4nzAC7FlXZm72PZAiDDMJDL5XjmmWewbdvkg+Qb3/gGDh8+jHfffXfOM88B4NKFgImxF6IDJPNOBwAkod7f/WgINo8T2jwewuMajA1FQLKHiYeShH1lJgYamQa6PDG++L1yyJWpXqppALT2hvDUr/ohlvLpNuMRN4wSA90mGmZx5H0XPIEIFBIJ6jbqoDakSsWkATAWTWCkN4iLby5Gsimcc1/gmpYB+Lb/HZKqFbAGvLMCIF1Hdw9sF12ItVdcuSQASHaShkDH449g5PBhWGqn9T3+yAuYPiAzaoUTYqz+9ndToemPxqEOK9ZWZmokCQA26MUI93ZSDyLZXmwpQmdQjDX1M0Oos3kAKTROqwdI6uCFB3sRbG/G8JFjyDOqIDYVQLmyCbKSSlqjMD1aDg0syAPIRsII9bUj6hwHWBZClRry8hp0dvnmHQIma8k1DDznzY/siwPAXMyUsc3J9AA+cujdWTWAd6w5j9MAzvvsnXkTli0AjoyMoLCwEB988AHOPjvV0YCMn/zkJ3j88cfR0dEx42xHo1GQf+lBALC4uBjO0YegVk/2Ij3Zl8mp9ACSz3qqdYAnAgDJ55qPF/Dofj/++PtRmAvEOHLEjoBdDIEQkMkFVJcYiyXhC4Qhk0qh0ohw53fL0bAhFWolAEhG/ysafPCKHWW1CurdmwqA5P3RgRCONTsRDSRQWadDZVNqPvHUDDhcSNrFqGjQ4MovlKM1YqXvZesLHPAy8IxHKZh2x+1YPzYG7zPPQ7Kyks6ZDoETHsBkEkxXD7S33YJjBZbj6gDJfuYTBiafwfaL/4Hd4UJ+eWaIcqoHkOyXlH5xONxo/Pf/S5ND0mM6AJJ6dp63X8Hwng+h4af0oXQIRfBKtKi5/kbISjO9pLkCIBsKwvnmCwh0tCAZj8MbAfQ6ZSpcyuNDXlGDvEu2QahKnaNcAZBsS3SAxB7+1oNw79sBknRBss3J+kmPaKFSBa+oAIKazWjclKlvnOvecyYBIPmsH7e+wBwAznWFcu+fTAssewDctWsXNm/ePGHzH//4x3jyySfR3p5qPD51/OAHP8APf/jDGa+fFgBIVrWMdYDk4+cSBiYhYDIWUw6GzJ8eBt7zjgfbH7MhGExgaNAHg0EJgXBaBmUyCYc7hFhQiE9/owRXf2ayiHWLswPbv5eAbTgyoe9LKhiUlxho4ggBQgIFY4NhHD3kAD8uQnmDitYDJNrDID+CNasLcMFNRVBpU57B6W3hXGNhvPhqB5guAYI+hkKFnxfF5uIESttfgqrCQmv7zQaApENHwh+A4Z/uArHi8RJBcgHARCSMQ+29WFNSBJ5SifFf/xyJUAgOkQwW7WRG7wwA9LjhcHnR+O/fh0A++YcXAUAyiBeQJCw4n3sKofaj8AoUMJdM1jIkx7V39QByFWo+ewdkZSnwTY+5dIDJeAzj/3gGgaMHIM4vhEAmh23MC7MpBXuJaATR0UEoqhpgvuoW8CVSCoBk5KoD9DbvhWPHS1QHKTaYaOIFGeQaiPs9iLkdCChLseqzXwBflDrfuQwCgGTkogOca3+n0gNI1vZxDANzADjXVcW9fzItsGwBcCEh4NPVA0gumJPtBXQ4WLS0ROH3J5BAJ8oqG1BdLaKJDYsZH7cwsM8Th9sRQ8t+P1562g63MwaBlIFClr09VjAWhGtIgPOvNuJrP6qipvKTotEPNmPfs3EkE4RLBNQ7R8rYxPkMioq1KK9TThSf7hiwA24JVm81Uq8QKfcSKfLjvKaVGckhUwFwrD+I157sR2evE6UWLZRaMZ3bN+5GyM7gbMdOlKs90K1biZGgb2YY2OeFcdQO2ZpV0H72MzjQ2YnGWJImjvAVcohLS7F/zHbceoDks+4fsGKVAAgfOYxw8xHq8TOqlBAXFoH1+8AMD8FtyM8AQDJvKgRGB/ogrVyJ4U2XYV1lZsJG2gvoff9tuF//O8RFpbD7osg3TAJlGqTGj7bCVF0Jy+1fzQjXHg8AydwqaQBjzz4GkcFE4S89MiCQiSI6NkwBUFW3hm6SqxewpkKF4T/fT1utSYzZu86Q0LC9sxtVN90OdW1q/7mOM8kLyHkAZ571tDSJCwHn+o1YvtstWwAkp5wkgaxbt45mAadHXV0drrnmmo9VEsjJBEDi4Xr++QB274nA5UqF1eJxJyQyAyorxbjmOiVqanL3SEz/6n1cAHBfx1EM7CtEy/4A1fmFAixaDwdowoexKAmRhAeJQDHjzuILBcH4RFhRr8T/+Z9qSOV8PPuQFR/usSLilCDGJKDUTOravMEwEiEh7QqyolFFAa+9z45CvR6f+8+aCeA+6OpHkzZTkJ8GwEgwjr/9pgvjQ0HwLCwKFbqMdQ0HPND221Ha/zZKy0TwFWpRqElpDyksMQxsXV3IsxRCfe0nwfT2YeCDXTCSWoFU4CiCqLAAw+XlWHP5FRO6vOkhYLKvg6+/iYIPd4H1uCHQamFnWFpv0BCPgdT/Y71uBIpXwFKS2YliMhs4AmbECuONn0aHzIR1KzI1fwQAVxcbMPbwbxAP+CE2W2Bz+GYAIFmLbcQObcwP802fg7xqUnc4VxjY0vs+/McOQlaSae+pAEj2H7H2Uy2g5YYvUA9urgBYnByC/a0XICtZQcPJs42xlmMwN9TCePZltC8y0TdKjBbwxZP1H7PN5QDw1D3wT6YH8KED70GunPmHaCgQwBfXbeE0gKfuMjhtjrysATBdBub++++nYeAHH3wQDz30EI4dO4bS0rlrbJ0uSSAnCwDD4QQeesiHXbvCMJsFMBoFEx4nj28Y1hETtFo+vvhlDerqj/8Qmu0bsNB6gCdTB9jVGsYzf7Cjf9iJgnwjVBrisUvitWedcDvj9HeNOQH1tJsv2WZ0NAiZWELBueksLfgCoK89BENNDB4bi/EOAbTGVEszMoLxCERxCS0BU79JB41BhCPN41h1lRKfuiHT8zMbBIq71HjlsT4UVigwxvhRIMvs3WoNemGRaeA50IoN4hbkyYJwRcMwKNW0JzBZil2nQeWVVyP03ntgunshMOgwLhShQKujJVpYux1OtwfFF10MzdVXZ4XAaF8fOu/7PQwSMUQFkyHZUbcPBRo1iK4utH8vQkwMhvWbIFCn6s+RQQDQJBEgNmKFvHE19NfdjCODjqwAWCeMwPbkAxCZC8EXiymk2gZHoJcK6L5IqziBUoVxhw8avw3acy+G/sIrJi7J4wHgkeZB6D94mgKX2DCZyU2BckoYmPxO6uiRxI3iz3+L1i/MFQDju7ZDHXdCapm9HRupd2hrPoqExwZdWTkVCJJwsUhrhKpmNdQN6yGQZtcln0kASOz8cfICcgB42rAPtxAAyxoAyRVAvH8/+9nPaCHohoYG/PKXv8TWrVtzujhOOwAkqz6BOsDX3wjhscd8KC8TQi7P9EzE2GGAX4jurhiKikW4+3s6SKWzey98XhYdHTEQqBSLeaiqEsNgTD2gF+IFJABIxonWARLAe+CeEdoFRFfqh0o8GaLbt9OLnvYQEnFAJE+gqHQybMvGE7BZGUTDCQikLMQCMapWKTHcHUbAF4e5SApVCQNHlwgxhqVeQAI/BADlAhk8DgaWUhnUOjFEEj6aviDC2dWThZ6pd20WL+DAX1n0H/OhoCLlkbSGvBkQSACwQK6FyxaGmB/Dtk8I0HtkP0r4EvBVSkhrVqJZI4dk+0vI6xmAeEUFBbwRnx8FqsnQ6sjIKIyhELQ33AjFpk30WFO9gI5HH4btzTeg0mhBsI4nl0OUZ8J4gocCbUo/F3c5EenqQECqgE4hA0+mANnYZXfSOnQF69ZDd+W1FOAOdVmzAiCGeqB/9wVISirAepyIWocQsI1DmpYnCIUQafUQF5bANTyC4gsuhvHyyUoAZB2zQeCRIwPQv/8UDc+K9MaM+wQBQDLSWsC4zwOiF6QAKJPnrAM8+Iv/gVYQgsQ0qRGdeiACf+HBHoSH+xANhmFYuxlirQ5JlqUdPNhQAIqKWpguuR5C+UwPEKcDzOn2fkI24gDwhJiV2+kCLbDsAXCBdqPTTicAJOs5kTpAhkniv/7LhdGxOMrLJstbpO1HAJAnKEI0mkR/Xwxf+2ct1q2XzjBvIJDAq68EsWdXBA47Sx/uxDOm0wmwfoMUV16lAF9+cN4dQciBToQXcHom8Huve/HsE3ZU1sngjzugFJknPuNgT4RqAUkY1++LQ60VQ6kWIMGCdgSJhFloDWKwgihUcgUaNqlx+H0PnU/a1/EVcZSWajHcEwITTUKuFNBQMoFA1iukrePqNmpx6c2FiFV7UK/OTF6YDQBb7gsj6GVgsMhmBUDyhjauQCgQw63frkcbM46NpkkPFNM/gM6f3wtzQSH4H3k2pwMg2cdYWzvyKythvOsrFBIJAK4358Pz9xfgeuJxcpGCibMQxmMUWIjXKqbRwbBuA4Q6Pb0Woj2dcJbVoLy0CDHrMH1NZM5Hv8qMNWdvonPSIysE7vwQhh3P024g0cE+olFAhCeCUq2kEetkjEEiFKBAGYEIZTffDt3WSzOu1ePpANkdf4M2OAbplP7C6clTvYBEAyg25qPw1q9M1M7LxQt46L7fQxUYhKwoe8Hm6PgIgr1tAI+PaJSFeeM5GWHfRIxBZHQQmoaNyLt4W9Z2bmeSF5DzAGbeZtPPJS4EvJin+/KYywHgIs7zcgLAnp4YfvRjJwosM71/xIRpACQ/t7VGcdnlCtx6W6bwnsDfww94ceBABHqDACaTgGrYiG6OJJWM21jU14txyxf6YLGkPEjzGScaAMk6f/X/rLCPxVBUJoGXsWcAYDiYwL73vPTzkJ+ZBIPScjX4PB7GhiMQiHhQqoQYtwdR06iHpViGfe+6aI1AUih6bDyIFVVaGPIlsPaFqdeP9AWOsgwECRG0RjG++pNaVNSrcNjTlxUAib2magGJDrDtkShIBrCpKBUSnO4BpK8FvVBF5dT7eOu/NeBYaCwDAAOvv4WBZ56FubFxAigIAJKR4QW0jSMvFIbxrrsgLivDvoEhrDx2DN4Xnke0twcJlgUbYyAUiigUkRIqbCSCpEwOzfoNEJeWgRnoh6OoAmu+dFfG6T/YbcW68mmav2xewNZBmF/7E3x73wdfmgr3BkNRKOWTsgSaZONxgQlHUPWdH0HZtC4nACQbHfn7a1C1vAGJpXhGBm4aAAncRoZ7kXfZ9dCsnqwykAsAHnn+NSi63oYkvxj8KQWxybFJIWx/6wHEgwEKuIxYC8uatTO+JvGAl9q16IYv0izi6YMDwPncWZZu25PqAdx3HA3gBk4DuHRn9eO7Jw4AF3HulhMAtrYy+Ol/u1C5ItXLNttIQ2BXJ4OzNkvxxS9nas3+9owfLzwXwIoqUdbwMKmV19nOYOO5dtx+R3TeXsATAYDkc6a9gMRL97O7h4jjBfq8lBd0OgT2d4XR3RqiniaeJIqtFxfCZo2i7ZCfhnX9njgk2jjWbcqnodyDO920Wwh5z+UL0n2edZ6FFoMO+uL0PVfUh4RbisbNWtzw1VTiAQFA4hkzB4rQddgDa08ICTYJv9KP88+uQnEVKUOTCsH/8almWN+Jo2RlqkPGbACYHBWgvF6Lq+6spNvtHRuagEDv9hcx/OrryG9oyDj1M8LAHi+M4+Mw3HEnpHV1iFmtaP/5L6CNhBFuaQYpjsiSYsxTCjgTqImRhA2NFvI165GMRuAorcaaO7+4MADssKJg53a4Xn0eQqN5wjs2FQLJMeNeN5gYi7IvfRO6cy7KHQAPdMPU9ibCfZ2QFpdnFKMmAGgyKBEZ7qMavvzrPw+hQjWx71wAsHl3B+Lv/w3qpBfSwvIJ7yHZCdEV+lsPUe8fkUcyujLw5BqYLJPt+ygoJpMID/Ug7/yroV1zzhkNgOTDfVy8gBwALuKBy01dcgtwALgIk56WAEg+zwnQAfb1EQ+gC6Y8AZTK7Nq+NAC2HmNw1dUK3HTz5IOPaP7+3w9dtCCyxTKZ5Trd/C4ni3A4ia99exBlZfPzAp5oHSBZ+8/uHiS6fhjN2QGQ9CIe6A6j82gI4WgUqzfqaR9g8k+hFkJrEGHlKgUiIi+04jwM9YTQ1RKAziii0Gh3BrH+HDP19qXHiM8Fx2Act3+1HrXrU1BNav/96ZmjcOwTIuiLQfpRwekxrx8GuQpltWpcdHMx1Hox3mruQutjYQqcWmPKCzYdAvvsLoi8Ylx++wpUr9XTbaYCoP/l1zC4/QWwFWUTvYHJNjMA0OmC0eOB8YtfgqSqCr7XyLztUEYiiPV0A2IJ4jxeBgCS/TDBIIREm2cpgMhghO66m9BhLsP6skmPH/EAkjHVC5g1BNw2CPaP90MxPggE/RBIRODLFAhF41DIxEhGwiBlVIRqLUSGPHiFKqz+t/+cESo9Xhi4ziLB+Mt/Q2SwB3yJBIKPCj47hsZI2xaYalbCdPmNM3R8udYDPPLGHugGdyFqH4FIo4dQraOJHqS+oL/9CAQKFeQllZDkF8E+4pkBgMROoaFe6DeeD8PZmeFt8h6nA1zEjX8RUzkAXITxuKlLbgEOABdh0tMNAOmDNPrhCQFAlk1SD2BPdxyVlTM1gNQ7wQ4jEi2AdTiOb/6LDg2NkyG3fXsj+N1vPKismt2DmPZctLUyuPG2MVx6+fp5n50T4QWcqgP888Pj2PeeHytqU3q66R7A9IJbDwUhNwVQUVoA0iXEPhpFzWoVdHmpWonOqJMCIMnubf7Qi5CfaAaFsDtDWHe2GTpTCgAJ6A11hSAvj+Er31oHiSyVcfzu82N45cU+FJpJZjCp6ZfyypK2cNqEGmP9IVQ0qmlXkGPhYUQ/lGLPy6OQKYXQ5kkwGvHRRBCyr4AnBpctAlWDAJ/7wloIxSnAnwqAkWNtcD/8GOxaFQqnJD9MB8DY2BgcbAINd99Ns20dv/sdRptboHQ6KHQl/X7wpDLEWDYDAploFOTPAqLLk9XUwvLd7+OIK5QBgGRNxwsD044ovT607R9G7NnfQSQQQJhgoGKdUPAZJMBCJhaBJ5HS5BNpYRnYSAgudxCrv/vDjP7D5Fhz1QNsKNci2HEU/pb9iHmc1GZClRZORTEaLrlwogvI9Is4Fy9gy/5+rCyRwntkDwJdR8EG/CDJH0S/GLWPQVFZnyoQTTrGWN3ZAXCwB4azL4F+04VZv0dcGHjet5dFT+AAcNEm5HawhBbgAHARxlxOAEjMtHNnGA895IXFIoBGMynET5swGhtGe7sQ9Q35+Nfv6DJCxe/tDOORBz2oqZu7PAzREF51vQ2fvDZTl5XLqTrRANjREsIj9459VPoF8DFuaOVGqLRC8D9yjPq9pDh0HJ/7ZiH4JQPo22nBy0+PY0WdfALU0gBIIdIVQ8dhPzxOBrFkDOvONUGuEMLnjiESTKB0pRxV2+LYWJmqVTc+HMafft4DHz+IcktmKRICgGTo+RqM9gZxyadLwNaF0KQuw8F3bDj4zjh8LgbeeAh6iYImlihUQur1k5wFbJ6SeEAAkAySDELKwTh/ez/GujqRX1uXVQdI6gWSGoHjm8/C2ptupnPtv/oVbM3NkDud4KtUiNvGkIwyiJMw8JReuUwkSmgXIokEmquuhfkrX8eBXmvOALi2ogAH94zhg7etYMJRVPf8GUpSNkaqRSQUhTAagEScoIkwlpIC8KUpgGfsY3CHE1h99w9z9gCSeVP7ApOMYNJ5hLhwSbbv0aOjaGqYvYRLrgBIWsKREfN5EHONU/0fucjsbzxHjyXSpbKQswEgKVMTdYzBctVtUJSv5AAwl5vHSdiGA8CTYGTuEDlbgAPAnE01c8PlBoAEFv70Jz9IORiFnIf8fCEkklSLMqcrgZGROIqKHbjrn2pQUprpJfxwTwS//60H1SuP3y2E7KutNYYbbh3DeRfFTrkOkJx1a6hlQgc4Psrg1z+y4sD7pOdrEkIxD0kBA4NBhYISCSRSPlz2GM65RIurbjFhINwOsacKD//3IC36rNGn7EIAkAziBSSDZAkf2+9FBBEYiwTQiLU0DNy0WYe69Wr0YxArVam+r+/9fYx6AKXlpEYfD3mSVMg2PQgEmqU6jA2EYCyQ4eZvVeGwZ4j2Bfa7GfS0eLC/3UpbwTWUWrCiQQu9RYp948NYb8ysfznVCxjt6kHXgw/BEEtCWGgBX5rK8h7x+mAmSRVjNkhra9B7wQXYWJlaq+tPf0Rgxw4ERsegNJlSNQMddjChMIRC0t9LVi8AACAASURBVCeZT68f4tliwYOsuATGz94B9fkXUQAkY3oYOFsiiIaR46VnuyGRCKE3SoGDL6PQ0w5GmwKxeDyBgD8GhUaAplUWjHqEGPMJEBi1IVFQivM+cxUKi5QZnVSIB5CM1bWZ3UbIa1MBcPqdoeXw4JIAINlvGgKnHsPx7j9oj2BZUQVtEUcAkIy0DpDYMzIySMPPhTfcOSORJL2vM8kDSD7Tx0EHeFIB8MP3Zy8EvelcrhD09C/uMvydA8BFnPTTEQDJxzlRYeDUgzSJt98JY8c7IQwNx2mJkySS0GoEWLNGggsu6kBx+cYZViXavv/6oZMK103m2TWAHg8Lvy+B79yth8Z8eN4ASKEjciCneoBkW9IbONe+wGPDDJ56cBx9nWHa85e0gSPlWmJsFPGoiDS0QGmlFFfdnIdLthlpZm9voA2Fsnr8/U827H7DjcIyKWSKlPc07QUkD+xxa5RqAK/9fCG8GivKlVVQaYS0/RvJKt7T00kcZKjLq8RLjw/REG9+qRxjEfesABjyxxD0xvHp79Sglz9GAXDq2GcfxDrDJPDttQ0dFwDJ3P27PoD47Z3Ic3lpGzhSx8cdDsFgNEFaVwv1lVfioM9HD7OxoAjh5mY47rsP/qEhKHQ68EQiWv4l4PWCFwqBBrpJKSCWBWMpgtZggOGWz0C+KlXkeroXMJsO8GDHMHa9Moq4K4nC0pTu1NvXgbLulyFRqMFKU9nopGTKqAeI6srhDosRizLgJRIQF5cjAj6aGsy48JISGPNS3kEy5uoLnO32QQCQjNm8gLnqAEkYOBsAkkSQsX88NQF5pOBz2guYiMfAjI9SD6f50hsgL021G8w2OB3grKY5YW9wAHjCTMvteAEW4ABwAUZLT1mOAJj+7JFIAu0dMQT8CRrqLS0TwpIvRCi6BwLxWVmt+vSffPjHS0FUrxTT4s/TB9EZdrQx2LRZhq9+TQNfdP9pA4Aa4Wo89PNRkC4gFSulNNwb9CcwPsbA4fZCKlAjkUxCKuPjc98sQv3aVAFeAoBF8gaq9fv7H2048qGPdv/QGcXwsW5I4jp4HDEoNQJccoMZa87Voc3XhUplDdX/te73omW3ByMDYbjCPhgUWjhGo7SkTGWjCraoZ1YADAfj8LtiuPXfVqJfaJsTAMl6p0PgVA9g+nztbW9Fvd0HovcjCRA90Sgaz9oMYX7+RBh139AwBUDi8XM+9CDGX30VMpEYQt1kGzp/JAqlRIyEzweeWIyARgdTcQlMX/1nqh8kI5cwsG0kiPvvPYCyIi3kio88z8kk2Ja3UeJsQUIoQVymgycmw+5BNcQKBQoUIYgQg7SsEpLiMgwRLxojgClfjmuur4TBkILAuXSAq+pnegfJvKXyAmYDQLJ/xu2AY8dLCA/1IhmLwuuJQK2RUvuT2oOGcy6Foix76Hfq9+5M8gJyHsDUmZ2oA7jEHkDSNOGee+6hTRPq6+tx7733YsuWLVnv9Tt27MAFF1ww4722tjbU1GQWsF/EI5ibukgLcAC4CAMuZwCczWwEAMnIBoEeN4sH7vPiaEsU+QVC6PWklEUqhOz1JGjySMUKEe76Jy3NFF5IRxBy7BPhAXR0r8QjvxyDpVhMIW/qIIkgZJCi0MQ7WLdaidu+ZpmAoTQEkgLRxw74cWSPD9b+CDxRL/QKPWrXqtCwUYPiFSnoIQBYIqzCq0+Ponm3G3wBD9o8MTysF8qkBq37PPC5YqhZq4GoOAaTdGYImOxH5JVBKOTjtrtr0Bq2LhgAyb6IDpAZGET4wCEM7d4DI19I9WgE+obKSxGvrcHG8snC1GkAJHNjNhva//u/IWhphoS0YSOeQACBQBDyBJtKyigqhDsUAXveJVhz7bUT5s0FAHs7PHj0vsNoakglRaTHqN2DqnAfFNbDEITc2OurgDWohFkWgqHQAElhCcSWIjrHZvfCpFdhoN+PNWvz8ImrU+V2pgMguVaJq5bUMDwZYeDZAJCsjdQBJKHe0EAXBlv7UVxpgdRSSrOD5+oHnLYRB4CLeAAsYOrH1QOYbptKIPCcc87BAw88gIcffhitra0oKUlpVaeONAB2dHRArZ6sB5uXlwfBlELuCzAhN2UJLcAB4CKMyQFgduMdzwvodrHY/lwAhw5G4fWyHwEgoFLxUN8gwbbrlRllYhYCgUsNgORTPvzoERx6R0s7gGQb6WxgEhYO+ll89XslyLOkMnnTAJieR0K6PnccnZ5uSOQ81Joza+sRANz7YghdbwphLpFCrkyFzMcjbhglBrhsURz5wEXB0NQogK5AOMMLOBJyI9ovwllX5GPrtalSKqQo9NQwMAkBk5FLGLhxyAHf9hfBer1wSiXIN+bRsC3rIp4zBuMlJWi84w4IVKkQ7FQAJL/HXS4ce+ABqPbuRSIYBE8sQhR8KEx5FFYEWi2U525Fd2U91pdNPlBy0QH2d3vxyO8Oob7WRDOs02PU4YNFrwYvFgEz1IfmzhhIlwylQYaa9dXgT0lCIQCYb1TD542CYRL49GfroDdIKQAS6KvT8xHsaEGo8xjdh0ChhF1eiNrN61IZujEGPImEZhbTtm8nWAc4/Ro8ureXvtSwdu4e5lPnnkkASD//5tlD3ou41S/Z1JMJgA/v/mBWDeCdm8+ZlwZw06ZNWLt2Le67774JW9TW1uLaa6/FT3/60xn2SQOg2+2GVptZD3bJjMntaNEW4ABwESY8XQGQfKQTqQOcy2THA8CJh/NoHO1tDCLhJMQSHqqrRSgqTvW/nToWAoBk/lJD4C9/fgQDbTqUrpjZ3o4cLw2A0WgCY0MMvvSdIpRWpmBxOgBO/Xyd/k6UKVLZvelBsoLv+cEhmgiiN01mTRMAJEMv1FMv4EhfCHmFUpjX82GWTXoBCbC0dthRYNBh21dWIK8otY7pAEhey0UHuP/D3Sh6/jWQAojCwgKMBFIdQAqVqb/sE5EoxtvbUbj1PGhvvYWeQwKAZJAwcHqQtnCN4RCCe3Yj2tEBZzAMSKUoXrsOsqZVkKyoxMH+Eawvyez2MZcO0O9j8Ouf7aWHqS43TByPACAZBAJHrQH0dLqp59FYKEVNVWb2NNmOQKDZoMLggB+f3FaJugYDzbw9/JdnoR9uAUtqCipUtFwM+dnT2wupIAmhVg8hsQWPRzNzlfVrMCwuA08sOWE6wIwL5qNfCAQuBADJ9Ib12dvOZTvObK8d3dVJ32rYmPKenuzR8n47B4BTQsBzAeDQ0FCGd04ikYD8mz4YhoFcLsczzzyDbdsme2Z/4xvfwOHDh/Huu+/OmJMGwLKyMkQiEdTV1eE//uM/soaFT/Z1wh1v0gIcAC7iauAAMLvxcgHAXM1+ugDg7353BO37tKhYmekBJLDlccbRN+hCyC1DLJpEIgl87psF2HKpjiZ8zBcAD7zrxh8f7kRdvTkjK5XYLO0FJPq+tv1eOKwRGGsFqK7Mox1KQn4WfheDmJLBTZ+rx4rGyQ4RuQAgOcZ0HeDh+x+A6WgXxFUrJgDd6vdlFoQeGUMey8Jw15ch/igkNN0LSABwQ2EKCBMMQzt+HBp1YEPlJDAc6LPOCYBk/vR6gO+8MoA3X+2jYWA+f6YX0DrkR8cxF3R6CUwlUhSZZ3ol0l5AEga+4upyNK3Og+/AbvT85SkYLCYIdQb6+UmJlVDHUQRGRkgcFjKNGoraVbTjB6kHyPp9UNQ0Ybx4M1atnd0jNd9yMHN9ZxYCgGSfZ5IXkPMATmoA5wLA6dfT97//ffzgBz+YcZmNjIygsLAQH3zwAc4+++yJ93/yk5/g8ccfBwnzTh/ktZ07d2LdunWIRqN48skncf/994OA4datW+e6lLn3T5IFOABchKE5AJwdAMk7syWDzMfkpwsAvvRGM155QoPyaikEH7XCY+NJdB4LwTrAIBKLUE8mExJDKOKhrEqG8moZrvucGflFklkhMJsH8N2/2/H8X3vR0Jg/w1RpACRvMNEEjn3oBl8Xh0qgIFVdoNaJUL9Jj0R1ENpC8Yy+wGTe9DDw1BDwdACMO11o/9FPkRSLUVA4WdtuBgB6feD3D6Dkmmug/sTldN3ZAJC8noZA8vP+ASs2FE96/AgAkjHVC5iLDtDtiODh+w+BH+HDUqiASJzKtCZeQLNWhe52N4YGfGhYnYekiKXvWQyZvaoJABq1SoxYg7juxipUlEox8uiv4bCOI796Ut8Y6m5FdGQIArUWoRADSTwMsdFEoY8CYjSCyMgQAsVrsObWT896uRMAbGywIDzUh2BPG+JeFyAUQVZYBkVlLe0AMlsmcLadcgDIeQDJdZF+Ls0FgLl6ANMAuGvXLmzePNnX+sc//jEFu/b29pxu6VdffTX9frz44os5bc9tdOItwAHgImzMAeDsxjvVXsClDgEHfCx+9IOjYAJ6FJdLaCJAe0sYA90RKNQCWv8vGA0g4pWiZpWCJov0d0VRXCHBZ75WAJekm2YDTx8EAMmYGgZ+72UHnnu6GwUrRdCLU8V+02MqAHpdDNr3e2HIl8CfCEEtVsJglqJpiwENm/ToTFgzAJDsY746QKa3nxaAtmuVKNROrmU6AJJ9j7W2o3DTJuhuS0HPdACkr03xAmYDQLrGaV7AXHSAZN47u3thaw5jdDhISxOJxQLY3UGoJBLI5EL4vQwMJim0OhnGiD4wCwAKkyJIZUJ85vN1iHe3YHz7n+AV62gP4/w8NRLhEPxH9gICAfgSGYLBCOQiUF2gqnH9RN9fxjkOtyeC/9/em4C3VV17338PsmVblufZjoc4cZyYDM7MFLjAhYbSEKYXChRooS0dboevT3spfL30bUtboKW9HRjCDC1QhjKWqUCAMCTOnNjO4Die5FG2bMu2Bkvy+6ytyBosWcORZEle+3n8QHT23mef/z4656e111p7xfduQ0KqNSLctRzccQB53Q3QdZyAxWyyBm5YzKIv2lEkc90mdFqKRa7E2YJBbP3Odz9A0iHSl4HD6QP4yA7PPoBfO9N3H8BAloDd3e8EjE8//TQoEphLZCjAAChhHiIZAOmyIt0P0FfpI8UK+PaHB/HvZ7NgmpwS0Hdg1xhkSfFIlsfBoJ/CyIgOxaXpWFafJvL3kYWw9YgOF1+diwWb+t0CIGngagWkXUGe/UsnZIUTKEh39lWz+QFODabhyJ5hGCYsqNuYickMHbITs8TyL+0golAmQl5jxqL8Qiizk1FVp0RGbrLffoDGtnYM/ukBDKSnoiTbPhYCQCo2P0ABgI1NKDnjTGRdY90FxJMfoKsFkOq6WgG9+QHuPtYlckquqbL7GNK+wJQ6J2dKgZajGgF8SUnxmEyexEXnVKPhs140fNqDwhIFNNrxGQComzDh+HE1Lt26GBtOL4bm43eh+eBNpFQuQm/fqABAQ28XJo41ITHDujcvAWBaahJMIxooalcKSyAVCpDpO9yEhDMvxcovnD/jVjeNa3Howb/A0t+B/JpFSDi1M4loa7FgUqOGRT+B7E2b0TVV6hMAUtu5tgKyH+DsT7VoBEC6IgoCoeVcigK2FfLr27Jli9sgEHcqXHHFFRgaGsL777/v66Of64VYAQZACQJHIwDa8veNj1kgo6XKikTk53tOzByoPHNtAaRxB9sKSDuCqA4vwruvaLBv5xh6uiaRrqSdLCAsgOn5erENHiWAtpWeToNI6Lz5BxNIksd5tAI6WgBpafex355ES7saS2pmLgO39arRvd+6hVxFTRpq6jNFQujcpGz0tU+gs2UMw/1GxKdNoWqxNeWKIlOGZRtykLx6AnUjgP7oCVjGJxCfkowTeWlYue5sEZlrKzY/QKqj/v2f0DekRnHlQqfbwdEKSNDSd+gwKq+7HmlnnjFdbzY/QFsld8vA7gBwcUYuWpo1OHpwEONjk9CMT2DDujIsWpqFwpI0sbxEELh6oXMQyb6jKtRXl0CnM+HtV1pxtGkIOpMR1eXZAtQpPY9mSA+93ozswkR89YYVYglZ89E70Gx/GymV1dMAqO9qg671mAj8oEIASDdAslkHxZLlSMotmL52XfsJjC0+Gyu3fHHG10izczsGP3gNo4nZKCyxB644VjQO9oGSRmqXfhHLT59pPXb33ZxrABQQ+ukxDgTx8OCMVgC0pYEhPz5aBn7ooYewbds2NDY2ory8HLfddhtUKhWefPJJceWUI5ACQChfIFkQyfL3m9/8Bi+++CIuu+yyQF8r3C7ICjAAShA0mgBwcnIK770/ge3bdVCpTGLXCgKXrKx4rFktx+bNqSiYZYcOf2WaLR+gv31FigXQtiWcQW/B3T/tREuzDlk5iQL+cvJlMCXTTidxIh8gbe2m01lEAmj9xBR+8MtyjGe0+gSApE/TnlE8ue0IFAlK5JfKndKb7N7di/6jFuQUJKN2TSYUGTIBgMauZLQf0QofxSR5PDQjepStSkFVcS5GaJ/hThWqzZ+jMEWDrAQ5kJiIKZMZGpMe+dWLkbH1i0iutkaDOgaCaN98Bx0vvARz5QKUKO3Rxo4AaOrrx4BOB/1NX8HaJctmBUA66K8f4L/eP4bevRPQ9OuRnEKaJ6BvaAwpkCElLRGrzyjC+rOLceBEt0cApPMa9Cbs/bgNH7/XAq3WjLT0dCSlypGXnyr8A5Gsx5o6axoa7YEGDLz6LORllehTj1stgN2dIB9AKwBag03GR7SQJ1iQXrcGielWv0ICYn1nK8aWno+VF1/odMuTj2DX3/4K09goRswpKCi0B+o4VqQ+KNHzePlGrLx8i09fGwbAyPYDjFYApJuPrH933323SARdV1eH++67bzqg48Ybb0RbW5sI8qBC9QgSCQpTUlIECBIkbt682af7mCuFRwEGQAk6RwsAEvw99ZQW/35vAgpFHAoLrHv4Uj66wUELevtMWFSdhFtvzUBxcfCsgXNtBQy2BZBuFRsEPnRvD9pa9CitcE6b0NXfj2FVOgZ6jSJIg3Y3oe3yLv1KPmrOH0b9kuUz7jh3foBU6fUPDqPhNR3MGlpSjhN/BJZHDmmQoUzBsnWZUGZbcw2Oj5qwa0cvUhKTkXIqbyBBn7wQWLW6CLJRNZQ7/wUMDkBeX4TqaruVrHt4AHFd/cgrKkbWjV9GUmW5EwBSrr+hx54SqV7yF9eI3T+EFtpRFKelw6xWwzI6hvRLLkbz4mqsK3JI/XJqRxDHi/bXD7C7Q4vHHzgAo96MpYsp2tkKXj1DoyjKTMeoxgDt6CQ2XVSG+DyLRwA09vdibO/nGG88CP3oONoGppCWlQ1lRQUqN62BYmE1Djad2v93SalI9dL96J+ET96QxRr9TbvEaQ/tQZwsaTrZ8nh/P9ILC5G2dOV0lPTkyJCA69FVW7Biw2lOc27oU0H19/shy86DekjvEQCpkU7VhvSa5egv2BDSZeBYiQQmzSLZDzCaAVDCq5KbRqgCDIASJibSAZAujfwA3/9kGR59dBQlxYlQKp13saA6BCnNR4xYtTIZP/pR1ozUI4FKNNcASOMONgTaAPCVZwbx4ZvDTomh1X2T2L9nECZ9MlLSyDcwHuNjZphNQEl5MlLyJ3DdzUuxcGmaWwikZWCCRs2AUfgPpikT0TLeCtOJQrQ2jkGvM4ukxLs+70V+VSLKTvmbUWcdx8ZwpFGD/HzrUiiVsZFJIM2MdWcUI3v/20jtOoKBuHzIcuJQv774lP3KOpTu8WFkqwYhr61B9q1fRcPAqWjcXGti4cnePjQ/+hjy+oaE5Tg+RQ71mBZxej1y8guRds7ZUJyzCbt7e2cAILV3zQfoqx8gXe+b/ziB5n1qJGQBJVl2axkBIJXiLCXUfbTEnoC687KRnJIwEwI/2omCfdsx2deDhKwcJCoz0T+oRa48DqbBAcQrlMj5wlakLV0hIHDlEivEDn/6AYbef0Pk91Pr4lCYm47xIwdhHOgVUcAUFKLXGZG7oh6ybGuQzJTZBF1HKzJWn4newjXiM8d9gWn3DtWzDyAprwjxsiT09wx7hEB9TydSq2qgLj4zpABIYwwWBLIfoOcnZlgB8MNZgkA2+R4EEujzn9tFvgIMgBLmKBoAcHxsJ351TxU6O02oqrL7eLle9tiY1RL4kx9nY8kSq1VJaollADx5TI8H7+1Gdq4MCmUCJsbN2PPpGLTjOuRkWyGM4GVIbULVkhQsXJKCxiP9yMyNx3d+shKZOc5zcaD7CIYPFeHAp8MY6jdO+xUqa3Q475wlWLDICo2DvQY8+usTMKSMozTHHpSxf8cghkYmkJNljzgdG55EnNKMjafJkf/Zi7DIkjExlYIxgwEbzimFzMFXsXtiBAVmmdixI/fbNyOpqmLmvsAdrViu0UF/8BDMg4OALAkdOZk47axNSCyw+r41qFROACg+c7ECuloAqY4nP8Chfh2ee7BJLPtqLXoUZzqnbiEIJAA0my1QtY3hC1dUwagwOgGgeXwMh/7wO2QbtEgqqxTbuFHpU4+iMEcp5snYqxIWvYJrb8ERtWkaAAnmNB++g9FdO6BRDyOntEhYBCeON8GsHUGCIgOpi2oxDAUK8tKF1dCo7oO8pBIFl14LWUbWjF1BJkc06Hr6T4iXJVtBdBYA1LW3IHP9JvSkLYsaABQwyX6Abh+fDIBS3yrcPpgKMABKUDMaALCxsQF33VuC4qJEpKbOtP45Xv6hwwZceaUCl19m3c5LaolVP0DSJTt5JZ55aAC7dmixoCoZvV0GHD2sQ1q2QcCfLC4Nw0MmkQh65fp0pKUniCX3Q4f7cdUNi7Hx/KxpeYcHJ7HtgcMYPJaKFEWCgMP4BGBizIyOHg0KsjJxwZVFWH46RSBb8OS9J3Hs5ACWLLIHiOz9cBAj4zpkZ1pBUeyvrDZCuTAei+SdWNj0KQzZJZg0TmF0Qo+K9QqUZdn9+QgAi1IyYTjWgsyrtiJt0+kzAbDXujxK+wLbyq6eLqwrsP+bAFDUmWUZmACQymx+gHSc0sFkjafi5SeOoawqHT3DWo8ASPU7Toxi47klSC2PcwLAsb27cOKpx1FQW4u4BLuLAwEgFRsE6k8eR+amC9FZsGwaAIWW5M/XfgJH3t2OXF2fiPClzyghNP3FIw4jWh0y0lMQn5KG1KrFyN70BchOBYq42xau/81/YPRgA1LKqz0CoFk3gclhNYqv+BpaeuMYAH18IPES8CgyMjLwCFsAfbxj5m81BkAJcx8NALirYTfuua8YtbWyGdusuV46LQNfcH4qbrjB2coiQSLMtRWQloCppCVv8HoZfbr9ok6m3Lps56nYloEpN+BLTw5gf8MYWpr0ojpZA8f1OliMyeL/a1emISvXbu070tKPirJ8fOP2cjEftPz+zF9U+HxnD4qrEpCrsKYRsRUCOYK9jKQMXHlrORYsTsPu7YN45tFjWLK4YNqKd3inRgBfXLoZaYkpoJ1CpizAaadnI653D6qad8KQUwr9uEn40BWsTkJxuh1CCQCpZHeqkXHZJVCcd/YMAKTju3o7ZwVAquNqBZSSD9AVAKl/RyugzQJIn3e0jmLDOcVIK493AsD+px9Gz/6DKKytnTGlNisgHZgc6ENcshyDZ18ltnuzLQPbGol9gSeNWF6db/X/S0iEoacThq42nGzpQVV1MeQLqsTSruOWhu4AkII7el+2RkxqJuUoLHLemYSsjHpVOxQ1p6HwS9cJq6WvSaE5H2Dk+gGyBdDrY5grhFEBBkAJYkcDADY1G/CLXx1HeWWBCPyYrRxuNGLr1jRcdWVwLIB0rrkGQBpDqPwAqW/y2dv3+Rj+8H9VIkCDUsBMJepRXp6F/GKrL6BjOanqR7osG9+/q0qkIGk9MoEn7+tETkESdLJhZCbN3KO2X6+G9mQq6s/KwhdvKBVw98CfD6DzsBGLqvJE0Edvhw7H9o8gXmlCojEZkwYLFtQoULpIgbH2A1h86CMYMgoworGgZGEakiqmUCh3ho7uMQ2yVUPI+srVSF2/WgAglTWn/AA9ASB97moFdLUAijou+wL74gdYLsvCP7Y1IU0hQ1p6Ero1ozMAkPouzEhH18lRXLi1CqaMSTFuWzqY7j/fA8vEOAbjUlCY7bL7x6llYKpPW7iZ9RMovuWHaOoanQGAVIcgcGWtPcjFNrcHD3VixbKZn9NxAkAqjn6A9O/Rw7sx+MEbME+MYXRShvyyAkxZzCKnIFn/0qpqkH/RFSIptOhnd1vUWAHZD9D9kzacAPjoB559AL96LvsASnj1x0xTBkAJUxkNAEipSG7/2WFoRvOwoMyzDyDV6+gw4Qc/zBLBIMEqwQLAgQETmlsOIjG+DukZ8ahamITEU1uyeRtrKAGQzk155H57WycMBgtycmUYhxrK5Jn5+6juiY5+yORxuOO3a0Swzb+e7cOOtzRYuDQVg4ZBtwA4YBhEvFYJk3EKN99eDWW2TCR7fuKxQ9C2yAWEJsQDrc1ajGqNyM1LQXFVmvijc/SN9qNuzzuYHB6DMS0bS9ZmQ5s0MQMAezrakJuiRN6PvoOEDCsoue4L7GoBFHXcLAM7AiDVCdQPcHVZMV7/+3EcP6xBaaXnZWC5WSau9cvfWAZFepJTPsCeB/8A09AghhIV0wBIy7jm0REMDg4jJ0uJhPQMWCbGMDU5ieJv/H9obNf4BYB0jd4g0BUAqQ1tA6dt2ouunbuQSdHbcXFIys5D+tJ6pNeucNpFJJoAkK6N/QBnPpkYAL09rfl4OBVgAJSgdjQAIF3eS//ci7/9owDVC2WQy2f6AdIy49Gjk6haKMMdt2eLBNHBKlL9AHt6THj37Qns2a3HwKBaDEsuz0F5ZRLOPicVG85I8Rq1HAoApHHkpqyalum5R/qx8yMtqmtTMGIcEJ9TPkDX0tI0gcVn6fCVr64Wh/72py60NI2jtDJlVgBUWLIw0G3ADT+uQnFFqmh7UNOCtP4SHN8/KiKHtRojDh0eQFpCKvLLUpCeRcv+QNvAENIPH8AS9QHkLS9HXk0Oek4t+dqsgJYJTK3n8wAAIABJREFUHfpajqPski8iY4s9V5c7AKRzh9MPMN+owBvPtojlcpPc5BQJTGNp7RxCkjERZ5xfig3nWNPbOCaEHv73vzDy4bsYTi9AQaYCxp4uGHu7YRkfg15vENCVmpEBxMdDue4sEQhyqLkrLABI3z31oB473j2MkvxUyFNkWFhbAkWGNe2MY2EA9P2pFKl+gAyAvs8h1wy9AgyAEjSOFgDUDO3EAw/L0bAnD4VFCcjJjp+GprFxCzo7TMjNTcA3v5kRtAhgR1kDtQJ2tE/ioQdG0HZyEgVFicjJicekpQdGQxH6ekwwmYHNlyhwydb0WSEwlH6Atus8emgCj/6xF1l5iVBmJAoIdAXAEY0Jo8MmXPB1AzasXCGaPveACk17x7Cg2gqAVNwtA3cNqzGlUeDGn1ShsMwKB4dHTqAmfbHTHdzXqcMrbx7F+NEkkRuQAIMSJScW63Du1EnIWw+JBNCy/Bz0WfTIj0+DaWAQU3o9BmrLYbr0P7G6xN6nKwDSycLtB0i7ghw5MIiP/tWBDtUIktMSUJKrhNlkgXbUCK1ej/84twJnXlCGhETrDxxHADR2d6HviQcxNGYANGrItUPCxy8+JRVxCQkYH9Mh2WyAZWwUyo3noPjm7wsLIBV3foDi8wCWgV0tgGq1Dp981oPWthH0do8iM9MK9pmZyVhWm40Na8ltIyGsEBisVDDi/pzDSGA6fyRCIAOghBcuNw26AgyAEiSNFgCkSxxU78QLr9agYbceGo35VJoSsqZBJIG+/HIFamqCk/7FVdJAAJCSV9/z2yEcPTKJxTWy6Z0wDKZuJCZYrTyDg2YMayy45dZM1K+daTFxHEcorICOFkCyTr36zCA+emcE2bmJiFdqppeBKfpXMzCJ4SEzzr4oCzWbB7BAYU0O/Om7Q3jt6T5U1aYKiPW0DHyktR+l+bm46SdVIt+dDQDpv64QuH/4JEoMZRjqMwgApG3gulL7sFxRjvGdezH2+R5MdvZgaGwEWakZkBUXCJ+/1A2rsWe0D6tzrLn/BOxFgB+gbVs4yvV3/PAQPv64A5lJcsQnxKN8oRI1y3OgtmixZqHzvsA0fvIDFBHR29+F+vmnMd7Tg7T8AsQl2e/18eFRJE9NIimvEPHJcqSvPwu5F1/hlA/Q8V4Khh9gf/8EXn29FX39OuTkyKFQyDDQO4LcvHQMjxjFUn7d0mxs/s9yJCXZITCarIDsBzjz5RJOAHzsvU+QmmZPC2UbzcT4GG46j30AJbz6Y6YpA6CEqYwmAKSE0EhaK3L9HT5shH0vYBlqHABLghwemwYCgAf2G/DH+zRYUJ7otGztCIB0wuPHjDhtRTK+/f3sWaOcQw2ANBbyBXznFQ12faxF74AGaclW532TaQoZ2YlYf04Gzr04Gx2Go+Lz0tQ6aNSTeOiudgEpeUXJbgGQ+m1sHsBl1y3Gxv+0Jhu2FXdWQALAZcpqp3p7h9qwPLNKfDZlMsGo6kVjz0nU5lZCVlI0vQ9ww0CHEwDaIHC2QBBRJ4R+gK77Ajcc78JphQUiiIb27KWyt0WF1ZUuewA77AtMuQA7fnkbtC0nkCxPEsu+olgsMEzFQ1m2ACkV1TCPjcIyOYmim76D5l5dSJaB6cfCP148jpMnR1BWZrdeO+YD1OtN6O2bwAXnlWHdarsrQTQBIMkbKiugWa/DRNcJkXcxLj5B+E3KixaI/7eV+W4BZAAMxdsstvpkAJQwn9EIgBIuV1JTfyHwqSdG8d67E1iy1NkqSQBIxWYFHBk2Y1RrwX///7koKvYc5BIKAKRxOFoBbQLRjiDbdx7BYK8FGUllyC9KQu1KBbLz7ONrHWue3hf4k3cG8fbzaqQq4hGXqUWW3J4KhvYSVrXpoVyoxy3fqRe7g/gCgFTHEQIJAKnYIJD+f4+6XXy2Msu6/y8VXwGQ6obTD9ARAve0qrCmwhn2vAGg7mgT+v/+KIYTUpEdZwIBIWXbpmXg4Tg5IE+15wNsa0HuF6/EyZTSkABgR6cWzz1/DFnZcqTI7fPpmhC6r38CGcokfOXLNZDJrGAz3wHQYjJh5ODnGG3eC9puD5gS80hL+vKCUmStPhupZQutWu2IvH2B2QIo6TXCjYOsAAOgBEGjDgDpWpPWSrjiwJv6C4B//qMGBw4YsbB6JtQ5WgGNhil0dk7iR7flYuEiz0vY4fADdFWnfbwJhSlWXz/X4giAtET82b+H8PFbQ1D1DUOWHIf05AwYdGbIkuJRVavA5i8Xoje5HdWKJU5dubMAUgVvVkBbJwSBrgBIx1yXgR0tgHR8LvwAp8fcemqbOgcI9AaAY/t3Y/ClZzCcboXr2dLB6NtakHXexWjPWiTqBtsPcMen3fjwIxUqKpxT0rgCoNFoRt+ADtdcsQjlC+ypmUINgeQHSKVujf2HQaDf/GBaAClFjvqTdwQAJqSkQZaRI3w4qZgNOhiH+pEgT0X+OV9CWmVNREIgA2CgdxK3C4UCDIASVI0mAKTLtC0DS7jkgJv6C4CPbBvBjo91qHGzLZ0jANJS9sCAGT+5Ixdl5Z4tgDTwUFgB3VkAbSL5CoC2+hTJ27xvDLsbO6CMzxcJpGtWpKN8cRoSEuPQPHrcLQBSe3d+gLMtA3sCQPrc1Qroix+ggEIvy8CUCoZKIPkAXZeBfbECUiCIgNmFJRg/fADq555EckUV+obGZgfAk8eRfdGlUK4/O6h+gDQWygk4MADsaujDAgeos82HIwSSW0B7hxZXXb4Iixba90AONQDSWIIVDBJMP8Cxlkb0/vtFyJTZSEyd6dtGehn6VUhUZKBk602iTqRZARkAA36FcMMQKMAAKEFUBkDfxfMXAD/ZoRMRwIsWy2bk+3MEQIoQLimV4Uc/zfGavibSAJDUIz9A13JMewwVaUtnfO4OAMXL2k00cKAWQHcAKOCur3PWhNC+AKDoO5B9gU904rQ4M4ztJzFlNCJOLsfxRAXq19Y7+X3OZgU0aQbRu+3PiEtMhNqc6BEAKSmzaViDgmu/DnlZhd8ASNfoLR/g6GiCWwsgtXUEwMlJM3r7dLjmyvBaAIMJgKKvIEQDE9z1vPkMJtqPI6XIHqTk+iWh/I66nnYUnLcVytpV8xoAH3/XcxDIjRdwEIjvb6/YrckAKGFuGQD9E88fCNRqLbjrF4MYHLSgsirR6UVv8wOkdDAdbSZcd1MGzjnPugfubCUUAEjn82QFJAsgFV+WgR3HHSwApD69WQED9QMUwBeEbeG87Qts7FZh9O030XvwMDIT4hAXF0+b80JjiUPxipXIvOhiJGbnCPm8LQMPvv4StJ99hJH0fBTm2bfBo7ZiX+CpKWSODUBevQQFV39NbL92sKkz6H6AyvQMPP/icWRnyyF38AF0BUDyAUxPT8INX66ZEQlMdU9bucDbLY9Y2RaO/P06X9yGhORUJKbNvlORrqcDiqolKLzwKgGAVOo2Wpfz57qE0wLIADjXsx3552cAlDBHUQmAdL1R4gdIyZ8ff3QUtEtJWZkMyXJr5Cb5zPX292JwIBfrNspx4y2ZbhNcu05tJPsBuo7VHwgMth8gjcXdMnC4/QAne7ox9MzfMNnbg6HUdBQXWndXIWuQZUyLgdaTKKhdipyrr0NiVrZXACQroPofT6Ov+QiQnY+C4oLpHxYWvQ79x1qQV74AeZddi+TiMnEuAkAqrn6A9Fmg6WCW1ZbiuReOoa1t1CkK2AaA9N/MLAV6eidw/rmlWL92ZkLxUC8DR5ofoGGwD10vPSKWfxOS5bM+tQ0DPUjOL0LJlhtFvUhaBmYAlPDC5aZBV2BeAmBbWxt+8Ytf4P3330dvby+Ki4tx3XXX4fbbb0eSQ34wb2pHGwDS9USTHyCNd/8+A1755xja2ydhoi1e46bEtKSmD2HD6eXYclk6Ul32251t3kJhBQymH6Bt7P4AILUJ9TJwOP0A6Xp2t3Wh6oN3oGs8jKSqhegdGUPxqe3pbBr1qDXI0vQj/YxNyLpkq/jY1Qro6AdIxycH1dC8/Sp69h9CJrmMUjoYiiJNSoJGno2lV12D5BJny5q/VsDZloAFkOzvQH5ejsgDSMmgc3JSkJZmtXLTj5vWFjUSZUlYWpuNiy8sn5EMWvQRRfsCi/tT4jLw5OgwOl94CAlJciQqnINnXL/vVgtgLQovvJIB0EMeQF4C9vZ2nx/H5yUAvvXWW3juuedwzTXXoLq6GocPH8Ytt9yC66+/Hvfee6/PM88A6LNUoqI/S8COPRuNU2hqNKKjYxJmM6BUxqN8cRPKytb7N4AIDAShC2A/wC6sLbEncSZN9nzegOI3XkZ8RiYSFAr0aEbFXDtCIH2WZ5kELGbkf+O7PlkBqQ+yIO7/uAELp8ZAlr/4pGQkl5XjyEQSVi2Z6V/mLwDSObz5AdKuIJQMesdnPWhrH4VeZzqVnH0K5kkjzjqzDBvXF0HushOI7YafbwBIc9b39vMYO9mElKIKj9/7KbMJ+t5OFJx/GdJrrBH489UC+MRbnn0Ab7iIfQD9fnnEYIN5CYDu5vGee+7B/fffj9bWVp+nmQHQZ6mmKwYKga5nGtE3iI/kyRv8GkQoLIA0APYDtE6DayQwfdagUmFdkR3w3AWCUD1HCNz3z5eR/8lHSFpYPb1MS8DnCoCFSgWMba3IueZ6pJ620usysO1m2XdUhfpq51yCdGx/cxdWLXaG0VABoA1G+wd06OkZh8lkgTwlERPDw1i/wTkFC+W/03e1wjjYD0qH0tE1gbj8Cixfb013MluJFT/AsZNHBASSBZAifV0LQaK+r1Okhynd+lUkyK27A0WSH2A4l4AZAL19M/g4A+Cpe+COO+4AWQZ3797t8a4wGAygP1shACwrK8NgzzYoldZ9PCO9iCVgKlHiBzibngSBgQAg9ZnmAzj26faL02fK18w6raqJQx4BkBr6mw7GdjJ/loFjzQ9w//MvIG/XZ0heaN/RxBUASSf6LHuoH9lXXI20+rV+ASC1d4VATwBIdYPpB+i6L7DjDXZonzVB9/IVVj/E8ZYmaHZuh76vC2QCJycI8oYdnZSh4pz/QGb9mYhPdE4Q7nrDEgTW1XuOnnV3g0eaH+CUxYKhne9Ds38H4mXJkGXmIj5RZvUJ1U/AMNQPWXoG8s/dMp0M2nZdkWIFZACM9Dfk/BofAyCAEydOoL6+Hr/73e9w8803e7wD7rzzTvz85z+fcTyaAJAGH21+gJ4mJBAApL5CYQVkP0D7LHnLB0g1vaWDGf/8M5x86kkU1J3m0QIoAFA9hOxRDXKvvREpS60pdbxFA9tG6s4K6A4Aqb6/VkBaAqayYpmzNXEaSPZ3wBsEEgBqjxzAwL9fhsVoQHKuda9iKrTU2Xe8HRnyKWTUn4HcTZudtkELBgBSH5GWD5Csn6NNezHauFsAH23lN4Up4RsoL6lA1qozkVI0MzqaAdB+R9BewLwEPL9Az9PVxhQAegI0x4tvaGjAmjV2i053dzc2bdok/h5++OFZ74pYsAAyAEYeANKcsB+gsx+gSa1G892/RV5WFhKzrClb3PkB9p5oRV5uDgq++V+IT7Va4SMBAAU0HuqUBIBLq5RQPfMAzPoJJBfMXK7u6x5GTno8TCNDKNh8NRSLZuaUtD3QArEABhMARV8SA0EcH86WSSN0qpOgvI2gvYCz8pCcX+xxP/D5CIBP/cuzD+D1m9kHkBGQYirJfh4jRa1Wg/5mKxUVFZDLrb+iCf7OPfdcrF+/Ho8//jji4+P9UiIafQDnGgDp/PPND5CuOZBl4NmWgKlPX7aFo4TQVLzlA6Q67raFoy3h6BEx2KND13EtGnv7sSS3AEWVChQsSBMv3GDkA6TzU05ARz/A/dseRl5zI5IqKkSUrg0CbX6AFh2lbjmG8q2XI+PcC6a/u1IAkDoJtx+gu4cOLQOXT6nQ/86LSClbKHISuhYCQCpK0xDSFi5F4SXXegSgWPED9OsB7VA5UvwAw7kEzAAY6N0yf9rFFAD6M20qlUrA3+rVq/H0008j4dSekv70EdUASBfKfoBep3u++wF+eLwVYzvi0N40gnGtCcMGHTKTUiFPTUDpIiVO/2IpTsSpsS7f6q9GJZBAEHcAuLvpKCp3bIf+SDPiMzORmJOL3tFxFKWmwDSohnlci9QV9WhbdQbWLK5yAkD6x+pKu9WM0sHQlnCOhZaAqUSqH6Bpx4vIMKohL/ac8FlYAdMAi1GP0i9/BzJlpsd7OhArYKT5AXr9ws5SIRKsgAyAUmaQ2wZbgXkJgLZl3wULFuDJJ590gr/CU8lmfRE6WgGQro39APf4FAhCWhEERkMgCI01mPkAtRojHv7zfphV8cgulCNNKUP3xKiwCGZZFFB365BXkoKii1NxXt1Cp69MMPwAqcPdzcexuK0FugP7YB4chGZchyxFGmS5eUipXwPF+tOxr3sQayqc4U6KFTBS/AD3/e7XyJTpkZxX5PFxRACYmymDaXQYpdfciqTs/KACoLindragbo1zVLIvz0d3dYK5DOzvGBgArYqRDyAvAft798Rm/XkJgLTce9NNN7mdUX9WxBkAA/tSBGsJmM4eK4EgdC2R5gf4wfMd+Pe/2lC3NB8JifYlSNX4CIpTM0XSYloWTl4Yh1v/a43T8mPQALBdhbVlJTCPjWGyswPN7SogKRkr169F/Kk0H3taVWEFQNOIBvrONhGYEZ+cjBZtElatmbl3M82pFD/AfX/5IzJ0vZCXeI7eJQDMTp3ClGkSZdd+2216FNu3NBALIANgYM84T63YAhhcPbk3aQrMSwCUJpm9NQNg4EoGCwKjJR8gKWXzA6QfGb1dRpxonoBuwoKkpDjEFXZj7WnLkZBg3e7OVubKD3B0yIhn723GgGEci0pzncZkA0BhTRidxNiIEdVXZ+CClfaULUFbBj4FgI4D2HNShTUL7Ba/YAMgncudFfDA7maU9R/BxJGDMGlHpoc0YkpE+cbTkbnxXCS47FMrBQAPvPQ6FMe2Q15aiTgPLioEgFMDnSg9/QzkX3ilRx9AAXK7rDlOA0kHEysWQHH9c7gvcDgB8OnXPAeBXHcJB4EE/vaKnZYMgBLmMuoBkK6d/QC93gHB9AOUTyzFW8+rceTQOMbHzCC7GoVhGRO1qF1ShAsuz0NppTWBrS8Q6EsgiACa4ZN+BYIc3TOE1x9pRWm1Aj36URSn2H3LHAGQYLbz2ChKzk/DVV+yR6ESAFJZV2D3DfSWEJrquwaC7PYBAKmdKwTSEjCVYPkBmkZH0LjtAUDVipwFpUjMyBKBGZSWxDSswWC7CkWrViH/S9c4QaA3AKQxekoHc/DzZpi3PwulzAh50QK3cDc5MoTBzj4s+crXkVbpW1LoQABQgFMQloHncgmYrmGul4EZAL0+brlCGBVgAJQgdjQDIF02+wGG1w/wcFcj3nsiBy1NEygsTUJ6RoJDjrsBjPWkI7dQhiu/XuwEgf4khBaWnpETqElf7HRn+wuAjZ+r8fZTbShblA7VxMgMAKTOaRmYSufxURSclYJrLj/N6ZzBXga2de5qAXQHgPRZMP0A1W+8AO2uHRhNzUVhYfaMp0avahAZukFkbtiEnAu2TB+Xmg/wwJvbkdn2CSZHNEjKLUBCqsK6Z7DRAOPQgFj61eYvx8qrr5rV+mcbEC8DH2EL4PgY2AIo4cUfQ00ZACVMJgNg4OLREjCVhCT/tnJzd8Zo8QN87ImD2PuOAlU1ciTKnNN6aIxqa7qVE+morkvFDT8oQ3y8dTk4lAAoLFCZ9ghaWyqYlgMavPpQK4orU9Fr1DoBILWxWQFpzF3HtCi5MA3la5WzRgOTBZCKt23hXPcFdrUCEgBSCeUysOMS8KRmEL2P/QlxiTIMTiagMFc5EwD7RpGTOAlMWVByw3eR6BCN680K6C0h9KIcM4Z3fwRdxwlYDHpxbloSTsovQsaKjWjTZ2P5Ks/74zoOlgGQAZCCQBgAA393xVJLBkAJs8kAKEG8IOYDpFEEAoGh2BGExuJuV5DRYRP+9/+qMGLUYEFJoVvhCAITDTkY6jfihh+WoWKxNbExASCVijTnQIPm0ePic1+Wgf3NB1gjK8Mz9zRj0mCBTmnd/tDdMjBFChv1Zlz5vSU4PhWidDA+LAPTErCAQodoYCkWQOrLBoFj+3dB/cqzSC5fiP5BrUcApF0pMnVq5G/5MhR19dNzLBUAaUcQAm1DnwqTtBfw1BQSFemQl1QiXibDod1t4lynrfScLsY2GPYDPCKkmCs/wHAuAf/tZc8+gNdeyj6A0t5esdGaAVDCPMYEANL1z1M/QLp0XyFQqh/g4b3jePSPvcip0EIp9wyASlk+TjSN4wtX5+Osi3Km705/rIC0BExF6jLwZ29045PXVCgsT4N6amwGAObLlOg+ocVpZ+Tj/C9Xuk0ITeMIhh8g9UPRwLbiyzJwsPwAR3d+jKG3X4a8whrk0jcw4hECMycGkHvRZVDWb/QZAKmiJyug677Anh5XBIG+ACC1D8QKGOx8gALC1tktzxIew343nUs/QAZAv6eLG4RQAQZACeJGOwDSpbMfYHj8APd9Poan/tqHvOoxKGQFbu86sgASALY2jeO8rXn4jy326Ft/AFC85IPgB2jQmfHO0204ukcDXbIB1aU5Ih2MxTyFkUED2ruHsWJFES76ShXSMpJmACCNIxb8ALX7dmLw1eeQXFEt/Ow8AmDvMDImBoQFML1utRMA0j+k7gs826Mq1AAo7inOByjhbWFtygAoWULuIIgKMABKEJMBUIJ4p5aAqYf54AfYfHACj9zXC2XJKLLS3FsASYshwwAGWhS45NoCnH6BPdgglAAoLFBu/ADpc4LAfdv7sP3DdiRpkzBlmQLi4qDMTkJC1RQu31InEkQL2OvtFP+dbVeQaPMDFJaq7CT0PP4XxCfLkZiR6RkAT3QiJz0ZxTd8B7JM50CRYCwDMwBKe97YWrMFcAy8BByceynae2EAlDCDDIASxDvVNFj5AKm7SPADpHGoJg7N8APU6yz431+ooB0xI7VgxKMVsLN/APH6DNz83+UoKEmeFngu/ABXZtl3f/ik/SSG2g1YmJyPxKR4sRdwo64Xa3KdkxSHal9gX9LBhNIPUP3qc9Du/Rzy8ir0D43PWAKeMpnQd6gRqF2N+ptunvHFCAcA0kl9WQZmP8C58wMMpwXwmZd2IDVNMeNepCCQay47EyMjI1AqZwY0SX+qcw/RogADoISZihkAJA3YD9CnO0HKtnAfvKnBq38fQnqxFnmZM62Ak0YLGpsHcOY5Zbj8a0Uz0nr4YwUMlh+goygNAx1YnWMHvl19nT4BoLAKeskHKOoUl06fzl0+QDo4V36AFAmsfulv0HeehCw3H4N6oDAvQwRkmEeHMTk4IHwE+2vOcbsriDcAFFbYOnvOREfd2Q/Qp6+mX5XmygrIAOjXNHHlECvAAChB4FgAQLp89gMMjx+g0WDBC0+o8eF2FXIys5FbIIMsKR5m8xSG+iehGTQhb/EEvvatFVBmWZdVHYs/AEjtguEHOBsA0jFXCHS1AIo6PV2zAiDVaejsmhUAqY67dDCOqWCoTrB3BXFMB2MaHsLwh29j4vgRaHr6kKlMwxSmROLn1EW1yNp0IRrbtVhZawdZm35S8wESBFI08GyF/QB9f5gzALIF0Pe7JXZrMgBKmFsGQAninWo63/IBEgS+8uZhnGjIQb/KKPbTBeKQlSfDyvXpOOOCTAwkHA/7vsDCAuXiB0ifOS4Du1oAPQEgfR4MP0DqxzEn4FzlA6RxrFpsh7pJdR+aPt6FqgKl2AtYvqASspx8cUcfONTpFgDpmDcroLd8gAyA0p83th4YABkAg3c3RW9PDIAS5o4BUIJ4Dk3nix+go1rHBxuhVy0C+QbKZHEoq5JDoUwUVVrHmj0CIB2Xkg+Q2vu7K4ht3ASAVAJZBnZcAqY+QrUtXCj9AB3n72BTJ1YumWnpIwAU4OzBChjqSGA6N/sB+vZcmgsIDOcS8LPPf4zUVDc+gBNjuPrKs9gH0LfbJKZrMQBKmN5YAUCSgJeBw7MMbLvd2sebUJiywu3d5wkAqbI/y8DsB2iV11M+QAGzC+25Benf+46qUF/t/JnjMrCvEBgIAFLf880PcK5yAZLWDIAcBCLh9R8TTRkAJUwjA6AE8RyazrUFkIbia0JoqislECScAEjnYj9AOwSurnSBveOqsAMgjSbUVkBfLIDi3tjVirp65yhuX77RnA/QF5Xc12ELYODaccvgK8AAKEFTBkAJ4rkAIP1zPuQDdARA+n9/rYD+WAD9BUBhgWI/QLcWQNLG0Q+Q/j3bMrA7C6Boc6iTAdDhu3/402PzakcQBsDgvDO4l+AowAAoQUcGQAniuTSdayugvxZAGn6mfM2sArjLB+jYIJBl4GDkA6QxsB8gsM9HC6DQq7nLLwCkNoEsAwcjEITO7YsVcL7nAySdwr0MzAAYvHcG9yRdAQZACRrGEgCSDOwHyH6AjhZAuif2qNudIoHps1jKB0jXw36Ac7cMTBZAKnPlCxjLAPjccx95DAL5P//nbA4CkfDuj5WmDIASZpIBUIJ4EWYBpOH4awWcCwsgjTOUy8CBACCNaT7lA3S8df1dBuZ8gDOfGfNpGTicFkAGwOC9n2K1JwZACTPLAChBPDcASB+xH6BdmNnSwbimgqFWzaPHUa1YMmNS/AkEocbsB+g+Epi0YT9A6+3FgSCBPfsYAAPTjVuFRgEGQAm6MgBKEM9N00jwA6RhpSVv8HphvkQCUyfsB2jfvcJ1RxDSJxbyAdJ1uOYEjJZ8gALmAogGjhUApOsP5zIwA6DXRytXCKMCDIASxI41ACQp2A+Q/QADWQamJWAqa3Id9gru7ZyxIwjVibR9gWlM7AfIfoASXgU+Nw0nAP7jbx969AG86tpN7APo86zFbkUGQAlzywAoQbwItADSkCLND5DGVJpaN0Ni5HQ+AAAZEklEQVQt9gNUYW2ZPa/fnpMqhHNfYMcJ8dcPkNpyOhjnW3q++AEyAAb3ncG9SVOAAVCCfgyAEsTzAID0cbD8AKkv+anl3JFhMw7s02NPgx7DGjPkKXFYsUqOVavlKCqWTY8mFABIneemrHIrFqWCocL5AO3bqjV0dmFdscO/O7qERrPtC0zHXSGQtoVbU+Gc/Jl2BZGSEJrOw36A1ls5VpaBY3UJmC2AwX0/xWJvDIASZpUBUIJ4HpqGwg+w6bABzzw1gu4uE5LlQEpKPCaNU9COWZCdnYCLt6Tj3PNTERcXJyyAVHz1A6S6cxEN7CkfII3HXTCIu0AQqsv5AIOTD5C0jCQ/QBrPod1tPuUDFDDHfoCo27go+A80lx7ZAhhyifkEfijAAOiHWK5VYxEA6RpjyQ+wu7MeD/5Fg5FhCyqrEpGQEDc9jVNTU+jtMUM3YcG1N2bijLNTxbFQWAE9WQDpfIEkhKZ2/iwDh2tfYBoX+wHanxQUDBJIQmjqIRhJoX1JCB0oAAbTCjhf8gGGEwCff+oDjz6AV15/LvsASnj3x0pTBkAJM8kAKEG8MFgAh3UNePrRGnz+yQSWLE0SFj53paNtEpnZCfjvn+UiNTU+4gCQxsx+gF1OS8Ckye529gOc7RsYagtgMAFQ9DUPtoVjAAz+O4N7DFwBBsDAtQMDoATxZgFAOhQMP8DmE3tw329SoMzIR1ZWgsfB0nJw64lJfP07WVi7PiUkAEgnj1Q/QBrbMmX1tD57h9rE/0dbPkAaM/sB2m9zBkDfn0/kB0gl1MvADIC+zwnXDL0CDIASNI5pACRdktZKUCfwpsHyA9zdoMfvf9+KpcsKPVr/bKNsbjJg6xVKfPHSdPYDBEAQ6C0dDG0JR2V1jkPql75OpyVgOr7LTToYx1QwVIfzATp/Xw7t75C8BEw9hhoCYyUQRGi14wgDYOCPbW4ZhQowAEqYtFgFQJIkFvwAG3bpcd99rVhWV+R1lo80GbHl8nRcsjVd1GU/QO8AKMBtoGMGANLns/kBCijs6ZqzfIB0ftdo4H3HVVaYXegcNbzvqPtdQVwjgamtv+lgZksFI4AkCBAYagCkcQYLAueDHyBbAL0+irlCGBVgAJQgNgOgBPFmaRosC+CJFiN+9asWFBQUQJEe7/GMZvMUjh2dxE23ZOLMTZEZCEKDZz9A9gN0vIkP7WvH8hX2nVbc3eDRBIACJmPcDzCsAPjYe56DQG46j4NAQvP6iqpeGQAlTBcDoATxvAAgHZbqB2ixTOGu3xzGseYE1CzxbAXs7TEhITEOt/0sF5mnfAVDYQGka4oWP0BhJXNZBt6jbheztjKrcnr2XC2AwrrnsgxMS8BU1uV73haOloBFnSLOB2gTN1gWQOrPl2jgQFLBBNMCGAkASGMIpR8gA2Bo3hnca2AKMAAGpptoFfMASBcZ5X6Ae/fo8Ze/tEIuz0NxSeKM2aYE0T3dJmy5QolLLrUu/1LhfIDsB0j3AS0BU6mvdl4a3t/c5TYhtADkJXaIpX+Hcl9g6n+urYDBWgKeawCk84faD5ABUMILl5sGXQEGQAmSxjIAkiyx4AdIuf7eeHs/3nilFNoRC7Jz4pGSEofJSUCtNou8gJv+IxVXXK2ETOacJiYUVsC5zgcoXrIjJ1CTvtjpzg80ITR1wn6AdinZD1DCA/XUEjD1ULeuSlpHAbZmAAxQOG4WlQrMewA0GAxYv349Dhw4gH379mHlypU+TyQDoM9S+V0xWH6AdOIRfQNUHfXYvUuHfbv1MBinkJAALFmSjLUbU1C3PNkpQbRtsJEGgDQu9gNkP0DHLxP7Afr9aJm1QUwB4LZ3kZqaNuN6JybGceUtF7APYHBvnajsbd4D4Pe+9z0cP34cb775JgOgyy08lxZAGkqwIJAAkArtCzwxYYFON4UkGURgiKfk0FQ/FABI/YbTD5DOV61Y4jSzniyAVMkxHyD9O1h+gNRXMNLBNETIvsB0PZ6WgV2XgKmupx1B6Nhs0cBz4QcoLHD19tQ+vrzZYmUZONT5AMO6BMwA6MutO6/rzGsAJOj74Q9/iBdffBHLli1jAHQHgPRZlPsB2qyABID+lGjxA6Rr8ndbONclYAE0wye9AiDVo2AQx0AQ+szdMrBjKphgAaA4V4ezFdB1RxAxxpMqrFng7Le3p1WFNRX2z1xTwVA7SgfjLhUMHZPiBzgbBEoFQOo7EvwABTiusQcI+fNdc6w7l5HANI5QWgEZAAO9K7hdKBSYtwDY19eH1atX4+WXX0Zubi4qKyu9AiAtF9OfrdAScFlZGQZ7tkGptKYPibUyl1bAYFkAAwVAahcKKyD7Adq/Jd7yAQrg6+zCumKH6GAXAKQ60bwtHAEglRXLnINHbCoFywroSyQwnXOuo4FjOR8gA2CsvSGj+3rmJQBSYMDmzZtxxhln4I477kBbW5tPAHjnnXfi5z//+YwZZwAMzZeAAdBZ19axZvEB+wH65gdIWjlaAV0tgHTcHyugLxZA6tPfQBDR5lAnA6DD7T6XVsCYsQA++BZSU9z4AOrGceU3LmIfwNC8tqKq15gCQE+A5jgjDQ0N+PTTT/Hcc8/ho48+QkJCgs8AyBbA8N/bwYJARz9Af64iFBZAOn+wrYCzLQHT+WLJD1BYBQNYBg42ANI42A/Q/m1iP0DvT5awWgAZAL1PyDyvEVMAqFarQX+zlYqKClx99dV47bXXnAIAzGZKCZKAa6+9Fk888YRPt0WsRwGTCGIJmAr7AXq9J/p0+0WdTPmaWeuqJg4FHQDphOwH6OLzx36ATvdhqHcFIQCkwn6Anr/+DIBeH6NcIYwKxBQA+qpbR0eHSOJsK93d3bjwwgvxwgsviJQwpaXufXFc+58PADgNgfMUAOn6Q2EFDLYF0F8ApPrRkg+Qxsp+gB1YXud52zdOB+Pr09+3eqFaBmYA9E1/rhUeBeYlALpK66sPIANgeG5Kx7PQEjAVqdvCUR+0DOxvJHAkAiCNif0A2Q/Q8XvCABjcZxMDYHD15N4iUwEGQMBnH0AGwLm5idkP0Fl3CgbxFwCph2D6AVJ/rulgfNkXmNoFIx+gsAqyH+D0jeErAFIDX6KBKRKYCucDXBTUh15YLYB/ed1zEMi3v8hBIEGd2ejsjAFQwrzNqyVg0mmeLgNzPkDf9gUWUDbQgdU59iTCu/o64Us+QGq7rsC+xNmgUmFdkUPqF5dUMIECoABXN/kA6fPVlXYfwkjMB0hj9LQMTAAojq/wvExMx9kP0PcHfiisgAyAvuvPNUOvAAOgBI3nCwCSRJwPcA/SfEwkTcEg0RAIQvPKfoDWB0AkpIPhfIAzH8axlg6GAVDCC5ebBl0BBkAJkjIAShDPj6bBWgKmU7IfoLPw/gCgsDBlVk134MuOIFTZ1QrougQs6vR0zWoBFBa/ABNCU1vOB2idtlBbAMWPip0tQYkEFn19egx16+z3nB+PDclV2QIoWULuIMIVYACUMEEMgBLE87NpsCCQ8wEGb19gmkL2A7TeyJwP0P6FjiUApKuq2xg8P8BwWgD/8b+vePQBvOq/tvjtA/jXv/4V99xzD3p6esTWqX/4wx9w1llneXySf/jhh2Kr1cbGRhQXF+PHP/4xvvnNb/r55OfqoVSAAVCCuvMOAEkr9gP0esdEWz5AuiDXvYHnYl9gGkcs+QHS9fi7K4i3HUGoT/YD9PoVDFqFYFsBoxUAaeOE66+/HgSBtIPWgw8+iIcffhhNTU1YsGDBDL1PnjyJuro63HLLLfjGN76BTz75BN/61rfwzDPP4PLLLw/a/HBH0hRgAJSg33wCQJKJ/QDZD9BxCdidBZA+8yUaONzLwI5LwGLcLoEg9Bn7Ac58GM71vsA0olhaBo5WAKT8uPX19bj//vunb5La2lpceuml+PWvfz3jxvnJT36CV199Fc3N1u0rqZD178CBA/jss88kvHW5aTAVYACUoObIyAgyMzNx8tj/QpmeIqGn6GhqNOwGklbPyWB1hgZx3vggWCBHDXsgD6AfjWE/Un1s168/hMzkVbNq1a1rRI58hcc6nRNHkC+v83i8bfwYSlJrZxxv0bZgQZrzUi9VOjp6AlWKxTPqN4+exCJFtdPnB0faUJu+0OmzAxprpOmyjIrpz/cPduK0THvUrwApdRdWZjtHo+4ZUGFVjv2zPX0q0cfqPHvk7Z7ebqzOt/97b3ePtU5h8fT59qq6nf/d1Y36Ivtxqrivswf1JUX2MbZb+1lV5vDZyR6sKrf/m44faO3BSpfPDp7owcpK53oHj/dgRZXzZ4eOdotzLK92HsvhoyosX+T8GdU71KTC8pqZn9Oxw40qnFbr/ljjwS7ULXXe8cRxkhoPdqKuzvNxqtu4rwPLTvMt2X3TnjYsXTHTwuPtIdC0uxVL653vC29tPB1v2tmCpavt91yg/QTSrvGzY1i6zvl7EEg/tjYTugnc+JNrMDw8jIyMDCldeWxrM0w8/ttnkJqSOqOebQydnZ1QKpXTx5OTk0F/rsVoNCI1NRXPP/88tm7dOn34e9/7Hvbv3w9a6nUtZ599NlatWoU//vGP04f++c9/4qqrrsLExARkMllIrp079U8BBkD/9HKq3dXVhbKy2dMuSOiem7ICrAArwArEoAIEX77uOOXv5ev1elRWVqK3t9djU4VCgbGxMafj//M//4M777xzRhvaKaukpEQs455++unTx++66y6xberRo0dntFm8eDFuvPFG/PSnP50+9umnn4rlY+qvqMj5B5S/18j1g6MAA6AEHS0Wi7iZ09PTnfYVduySfo0RJLr+2pJw2rA15bGHTerpE7Hm4deczhitukfruKNZcyljn5qaglarFUER8fHxIbvZCQLJcuep0Dji4uKcDnuyANoAkABu48aN021+9atf4amnnsKRI0fcAuBNN92E2267bfoYAeSZZ54pgkgKCwtDdu3cse8KMAD6rlVANW3meFoudjS3B9RZmBvx2MMs+CkQoaUhvl/Cq3203uvROm4bRPG9Ht77PJCz8RJwIKpFRxsGwBDPEz+gQyywh+6jVfdoHTe/0Pk+91cBvtf9VWzu6lMQyOrVq0UUsK0sXboUW7Zs8RgE8tprr4koYVu59dZbhc8gB4HM3Ty6npkBMMRzwQ+5EAvMADg3Ars5K9/r4Z8K1jz8mkf7j51AFLOlgXnggQfEMvBDDz2Ebdu2iRx/5eXlYqlXpVLhySefFN3b0sBQChhKBUPQR1HAnAYmEPVD14YBMHTaip4NBoP4hURfEHcRViE+vaTueeyS5AuoMWsekGySG0Wr7tE6bn42Sr5lw94BWf/uvvtu4cNHOf7uu+8+ULQvFQr4aGtrw/bt26fHRdHBP/jBD6YTQVNqGE4EHfZpm/WEDICRNR88GlaAFWAFWAFWgBVgBUKuAANgyCXmE7ACrAArwAqwAqwAKxBZCjAARtZ88GhYAVaAFWAFWAFWgBUIuQIMgCGXmE/ACrACrAArwAqwAqxAZCnAABhZ88GjYQVYAVaAFWAFWAFWIOQKMACGXGL3J6DoPcqtRJtj79u3DytXrpyjkfh22i996Usih1N/fz+ysrJw/vnn47e//a3IaB/JhSLTfvGLX+D9998XWyPReK+77jrcfvvtSEpKiuShi7FRtv033nhDaE/jpT1EI7FQhOA999wjIgSXLVuGP/zhDzjrrLMicahOY/roo4/EuPfs2SPGTvuV0gb3kV4os8BLL70kdmFISUkRW3TR97GmpibSh477779f/NF3kwrdLz/72c/whS98IeLH7jhAmgPa6oz2xKX7nQsrEG0KMADO0YzRQ+P48eN48803owIAKeSf8j/RHo6U7+lHP/qRUI62B4rk8tZbb4FyWF1zzTWorq7G4cOHRV6q66+/Hvfee28kD12MjfbnzMzMBO07/cgjj0QkANpyhBEE0l6fDz74IB5++GGRBHbBggURrTF9/2iLqvr6elx++eVRA4AXXXQRrr76aqxduxYmk0n8oDl06JDQPC0tLaI1pwTBCQkJ4vtIhfaTJQinH8IEg9FQGhoacNVVV4ndnc4991wGwGiYNB7jDAUYAOfgpqCXzg9/+EO8+OKL4oEXDRZAV5leffVVYSkhS6ZMJpsDFQM/Jb1syALR2toaeCdhbvn444/j+9//fkQCIFmyCaBIU1upra0V9wdZSaKl0N6o0WIBdNV0YGAA+fn5oNxrttxs0aI7jTM7O1tA4Ne+9rWIH/bY2Ji43+kHzy9/+UuxesMWwIifNh6gGwUYAMN8W/T19YktdV5++WXk5uaisrIy6gBwaGgItK0PWQJ37NgRZgWln+6OO+4AWQZ3794tvbMw9RCpABjIPqFhkszv00QzALa0tGDRokXCCkhJeqOlmM1mPP/887jhhhvEc5C2F4v0QmMlYKVVkXPOOYcBMNInjMfnUQEGwDDeHFNTU9i8ebNYJiMIIR+YaAJAyuT+5z//GRMTE9iwYQNef/115OTkhFFB6ac6ceKE+PX+u9/9DjfffLP0DsPUQ6QCYHd3N0pKSsQyKvmh2cpdd90llvaOHj0aJoWknyZaAZCeK7Qnq0ajwccffyxdiDD0QKBKLiV6vR4KhQJ///vfxbMx0suzzz4rrH7041EulzMARvqE8fhmVYABMAg3yJ133omf//zns/ZEPiPkL0f+UuR4Tj4wcw2Avo57zZo14trUajXI+tfe3i6uNyMjQ0AgvTjDXfwdO42PYGXTpk3ij3zU5qoEMvZIB0C6t+mFbisUvPLUU0+JIIVoKdEKgN/+9rdFoBBZ40tLS6NCbrIcd3R0CJcGcoWh7yMtX0eyBbCzsxP0LHznnXewYsUKoTNbAKPiduNBelCAATAItwaBEf3NVioqKoTTNjlAOwITLYEQDF577bXCYhLO4uu46Zeua6GghLKyMgG1ji/+cI3f37ET/JGzNvmrEUzFx8eHa6gzzuPv2KmDSAVAXgKes9tInPi73/2ucCehH5W0mhCthbIKLFy4UAQQRWohnbdu3Sqe17ZCz296ntPzhPyhHY9F6nXwuFgBmwIMgGG8F+gX7+jo6PQZCUouvPBCvPDCCwJMouXXO10A/RqmCM8PPvhA/AqO5EK+igR/5Hv59NNPR+VDOlIBkOad7l3SlpzibYUsObQsyUEgoflm0LIvwR8FrWzfvl34/0VzOe+888QPSrrPI7VotVqx+uFYbrrpJixZsgTkHhNNvpeRqjGPK7wKMACGV2+ns831ErCvl75r1y7Q35lnnilyAFL0LOXtorxpjY2NSE5O9rWrsNezLfsSrD755JNO8FdYWBj28fh7QvrRQMvuFHVNUZI2Hy9KoUG+U5FQbGlgHnjgAWENfuihh7Bt2zZxb5SXl0fCED2OgSI6KYCCyqpVq/D73/9e/FggJ/9ITmHzrW99S/jNvfLKK065/8gtg/ICRnKh3HmU84+Aj6CK/Op+85vfiMCsCy64IJKHPmNsvAQcVdPFg3VRgAFwDm+JaAFActimvIWUtHp8fFzkAqQ8ZBTIQgEAkVzIokC/0t0VsqJEernxxhvdugZEmuWVrH933323+FFAlhCKkIyGdCRkPSPgcy0U6RnJ1ihPfrePPfYY6J6J5EKpXt577z1xrxCwLl++XFjQog3+SGMGwEi+03hs3hRgAPSmEB9nBVgBVoAVYAVYAVYgxhRgAIyxCeXLYQVYAVaAFWAFWAFWwJsCDIDeFOLjrAArwAqwAqwAK8AKxJgCDIAxNqF8OawAK8AKsAKsACvACnhTgAHQm0J8nBVgBVgBVoAVYAVYgRhTgAEwxiaUL4cVYAVYAVaAFWAFWAFvCjAAelOIj7MCrAArwAqwAqwAKxBjCjAAxtiE8uWwAqwAK8AKsAKsACvgTQEGQG8K8XFWgBVgBVgBVoAVYAViTAEGwBibUL4cVoAVYAVYAVaAFWAFvCnAAOhNIT7OCrACrAArwAqwAqxAjCnAABhjE8qXwwqwAqwAK8AKsAKsgDcFGAC9KcTHWQFWgBVgBVgBVoAViDEFGABjbEL5clgBVoAVYAVYAVaAFfCmAAOgN4X4OCvACrACrAArwAqwAjGmAANgjE0oXw4rwAqwAqwAK8AKsALeFGAA9KYQH2cFWAFWgBVgBVgBViDGFGAAjLEJ5cthBVgBVoAVYAVYAVbAmwIMgN4U4uOsACvACrACrAArwArEmAIMgDE2oXw5rAArwAqwAqwAK8AKeFOAAdCbQnycFWAFWAFWgBVgBViBGFOAATDGJpQvhxVgBVgBVoAVYAVYAW8KMAB6U4iPswKsACvACrACrAArEGMKMADG2ITy5bACrAArwAqwAqwAK+BNAQZAbwrxcVaAFWAFWAFWgBVgBWJMAQbAGJtQvhxWgBVgBVgBVoAVYAW8KcAA6E0hPs4KsAKsACvACrACrECMKcAAGGMTypfDCrACrAArwAqwAqyANwUYAL0pxMdZAVaAFWAFWAFWgBWIMQUYAGNsQvlyWAFWgBVgBVgBVoAV8KYAA6A3hfg4K8AKsAKsACvACrACMaYAA2CMTShfDivACrACrAArwAqwAt4U+H+2B8dXN0nDLgAAAABJRU5ErkJggg==\" width=\"640\">"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"%matplotlib nbagg \n",
"net = Net()\n",
"net.add(Linear(2,2))\n",
"net.add(Softmax())\n",
"\n",
"res = train_and_plot(30,net,lr=0.005)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"After running the cell above you should be able to see interactively how the boundary between classes change during training. Note that we have chosen very small learning rate so that we can see how the process happens.\n",
"\n",
"## Multi-Layered Models\n",
"\n",
"The network above has been constructed from several layers, but we still had only one `Linear` layer, which does the actual classification. What happens if we decide to add several such layers?\n",
"\n",
"Surprisingly, our code will work! Very important thing to note, however, is that in between linear layers we need to have a non-linear **activation function**, such as `tanh`. Without such non-linearity, several linear layers would have the same expressive power as just one layers - because composition of linear functions is also linear!"
]
},
{
"cell_type": "code",
"execution_count": 38,
"metadata": {},
"outputs": [],
"source": [
"class Tanh:\n",
" def forward(self,x):\n",
" y = np.tanh(x)\n",
" self.y = y\n",
" return y\n",
" def backward(self,dy):\n",
" return (1.0-self.y**2)*dy"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
" Adding several layers make sense, because unlike one-layer network, multi-layered model will be able to accuratley classify sets that are not linearly separable. I.e., a model with several layers will be **reacher**.\n",
"\n",
"> It can be demonstrated that with sufficient number of neurons a two-layered model is capable to classifying any convex set of data points, and three-layered network can classify virtually any set.\n",
"\n",
"Mathematically, multi-layered perceptron would be represented by a more complex function $f_\\theta$ that can be computed in several steps:\n",
"* $z_1 = W_1\\times x+b_1$\n",
"* $z_2 = W_2\\times\\alpha(z_1)+b_2$\n",
"* $f = \\sigma(z_2)$\n",
"\n",
"Here, $\\alpha$ is a **non-linear activation function**, $\\sigma$ is a softmax function, and $\\theta=\\langle W_1,b_1,W_2,b_2\\rangle$ are parameters.\n",
"\n",
"The gradient descent algorithm would remain the same, but it would be more difficult to calculate gradients. Given the\n",
" chain differentiation rule, we can calculate derivatives as:\n",
"\n",
"$$\\begin{align}\n",
"\\frac{\\partial\\mathcal{L}}{\\partial W_2} &= \\color{red}{\\frac{\\partial\\mathcal{L}}{\\partial\\sigma}\\frac{\\partial\\sigma}{\\partial z_2}}\\color{black}{\\frac{\\partial z_2}{\\partial W_2}} \\\\\n",
"\\frac{\\partial\\mathcal{L}}{\\partial W_1} &= \\color{red}{\\frac{\\partial\\mathcal{L}}{\\partial\\sigma}\\frac{\\partial\\sigma}{\\partial z_2}}\\color{black}{\\frac{\\partial z_2}{\\partial\\alpha}\\frac{\\partial\\alpha}{\\partial z_1}\\frac{\\partial z_1}{\\partial W_1}}\n",
"\\end{align}\n",
"$$\n",
"\n",
"Note that the beginning of all those expressions is still the same, and thus we can continue back propagation beyond one linear layers to adjust further weights up the computational graph.\n",
"\n",
"Let's now experiment with two-layered network:"
]
},
{
"cell_type": "code",
"execution_count": 39,
"metadata": {},
"outputs": [],
"source": [
"net = Net()\n",
"net.add(Linear(2,10))\n",
"net.add(Tanh())\n",
"net.add(Linear(10,2))\n",
"net.add(Softmax())\n",
"loss = CrossEntropyLoss()"
]
},
{
"cell_type": "code",
"execution_count": 40,
"metadata": {
"scrolled": false,
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [
{
"data": {
"application/javascript": "/* Put everything inside the global mpl namespace */\n/* global mpl */\nwindow.mpl = {};\n\nmpl.get_websocket_type = function () {\n if (typeof WebSocket !== 'undefined') {\n return WebSocket;\n } else if (typeof MozWebSocket !== 'undefined') {\n return MozWebSocket;\n } else {\n alert(\n 'Your browser does not have WebSocket support. ' +\n 'Please try Chrome, Safari or Firefox ≥ 6. ' +\n 'Firefox 4 and 5 are also supported but you ' +\n 'have to enable WebSockets in about:config.'\n );\n }\n};\n\nmpl.figure = function (figure_id, websocket, ondownload, parent_element) {\n this.id = figure_id;\n\n this.ws = websocket;\n\n this.supports_binary = this.ws.binaryType !== undefined;\n\n if (!this.supports_binary) {\n var warnings = document.getElementById('mpl-warnings');\n if (warnings) {\n warnings.style.display = 'block';\n warnings.textContent =\n 'This browser does not support binary websocket messages. ' +\n 'Performance may be slow.';\n }\n }\n\n this.imageObj = new Image();\n\n this.context = undefined;\n this.message = undefined;\n this.canvas = undefined;\n this.rubberband_canvas = undefined;\n this.rubberband_context = undefined;\n this.format_dropdown = undefined;\n\n this.image_mode = 'full';\n\n this.root = document.createElement('div');\n this.root.setAttribute('style', 'display: inline-block');\n this._root_extra_style(this.root);\n\n parent_element.appendChild(this.root);\n\n this._init_header(this);\n this._init_canvas(this);\n this._init_toolbar(this);\n\n var fig = this;\n\n this.waiting = false;\n\n this.ws.onopen = function () {\n fig.send_message('supports_binary', { value: fig.supports_binary });\n fig.send_message('send_image_mode', {});\n if (fig.ratio !== 1) {\n fig.send_message('set_dpi_ratio', { dpi_ratio: fig.ratio });\n }\n fig.send_message('refresh', {});\n };\n\n this.imageObj.onload = function () {\n if (fig.image_mode === 'full') {\n // Full images could contain transparency (where diff images\n // almost always do), so we need to clear the canvas so that\n // there is no ghosting.\n fig.context.clearRect(0, 0, fig.canvas.width, fig.canvas.height);\n }\n fig.context.drawImage(fig.imageObj, 0, 0);\n };\n\n this.imageObj.onunload = function () {\n fig.ws.close();\n };\n\n this.ws.onmessage = this._make_on_message_function(this);\n\n this.ondownload = ondownload;\n};\n\nmpl.figure.prototype._init_header = function () {\n var titlebar = document.createElement('div');\n titlebar.classList =\n 'ui-dialog-titlebar ui-widget-header ui-corner-all ui-helper-clearfix';\n var titletext = document.createElement('div');\n titletext.classList = 'ui-dialog-title';\n titletext.setAttribute(\n 'style',\n 'width: 100%; text-align: center; padding: 3px;'\n );\n titlebar.appendChild(titletext);\n this.root.appendChild(titlebar);\n this.header = titletext;\n};\n\nmpl.figure.prototype._canvas_extra_style = function (_canvas_div) {};\n\nmpl.figure.prototype._root_extra_style = function (_canvas_div) {};\n\nmpl.figure.prototype._init_canvas = function () {\n var fig = this;\n\n var canvas_div = (this.canvas_div = document.createElement('div'));\n canvas_div.setAttribute(\n 'style',\n 'border: 1px solid #ddd;' +\n 'box-sizing: content-box;' +\n 'clear: both;' +\n 'min-height: 1px;' +\n 'min-width: 1px;' +\n 'outline: 0;' +\n 'overflow: hidden;' +\n 'position: relative;' +\n 'resize: both;'\n );\n\n function on_keyboard_event_closure(name) {\n return function (event) {\n return fig.key_event(event, name);\n };\n }\n\n canvas_div.addEventListener(\n 'keydown',\n on_keyboard_event_closure('key_press')\n );\n canvas_div.addEventListener(\n 'keyup',\n on_keyboard_event_closure('key_release')\n );\n\n this._canvas_extra_style(canvas_div);\n this.root.appendChild(canvas_div);\n\n var canvas = (this.canvas = document.createElement('canvas'));\n canvas.classList.add('mpl-canvas');\n canvas.setAttribute('style', 'box-sizing: content-box;');\n\n this.context = canvas.getContext('2d');\n\n var backingStore =\n this.context.backingStorePixelRatio ||\n this.context.webkitBackingStorePixelRatio ||\n this.context.mozBackingStorePixelRatio ||\n this.context.msBackingStorePixelRatio ||\n this.context.oBackingStorePixelRatio ||\n this.context.backingStorePixelRatio ||\n 1;\n\n this.ratio = (window.devicePixelRatio || 1) / backingStore;\n\n var rubberband_canvas = (this.rubberband_canvas = document.createElement(\n 'canvas'\n ));\n rubberband_canvas.setAttribute(\n 'style',\n 'box-sizing: content-box; position: absolute; left: 0; top: 0; z-index: 1;'\n );\n\n // Apply a ponyfill if ResizeObserver is not implemented by browser.\n if (this.ResizeObserver === undefined) {\n if (window.ResizeObserver !== undefined) {\n this.ResizeObserver = window.ResizeObserver;\n } else {\n var obs = _JSXTOOLS_RESIZE_OBSERVER({});\n this.ResizeObserver = obs.ResizeObserver;\n }\n }\n\n this.resizeObserverInstance = new this.ResizeObserver(function (entries) {\n var nentries = entries.length;\n for (var i = 0; i < nentries; i++) {\n var entry = entries[i];\n var width, height;\n if (entry.contentBoxSize) {\n if (entry.contentBoxSize instanceof Array) {\n // Chrome 84 implements new version of spec.\n width = entry.contentBoxSize[0].inlineSize;\n height = entry.contentBoxSize[0].blockSize;\n } else {\n // Firefox implements old version of spec.\n width = entry.contentBoxSize.inlineSize;\n height = entry.contentBoxSize.blockSize;\n }\n } else {\n // Chrome <84 implements even older version of spec.\n width = entry.contentRect.width;\n height = entry.contentRect.height;\n }\n\n // Keep the size of the canvas and rubber band canvas in sync with\n // the canvas container.\n if (entry.devicePixelContentBoxSize) {\n // Chrome 84 implements new version of spec.\n canvas.setAttribute(\n 'width',\n entry.devicePixelContentBoxSize[0].inlineSize\n );\n canvas.setAttribute(\n 'height',\n entry.devicePixelContentBoxSize[0].blockSize\n );\n } else {\n canvas.setAttribute('width', width * fig.ratio);\n canvas.setAttribute('height', height * fig.ratio);\n }\n canvas.setAttribute(\n 'style',\n 'width: ' + width + 'px; height: ' + height + 'px;'\n );\n\n rubberband_canvas.setAttribute('width', width);\n rubberband_canvas.setAttribute('height', height);\n\n // And update the size in Python. We ignore the initial 0/0 size\n // that occurs as the element is placed into the DOM, which should\n // otherwise not happen due to the minimum size styling.\n if (fig.ws.readyState == 1 && width != 0 && height != 0) {\n fig.request_resize(width, height);\n }\n }\n });\n this.resizeObserverInstance.observe(canvas_div);\n\n function on_mouse_event_closure(name) {\n return function (event) {\n return fig.mouse_event(event, name);\n };\n }\n\n rubberband_canvas.addEventListener(\n 'mousedown',\n on_mouse_event_closure('button_press')\n );\n rubberband_canvas.addEventListener(\n 'mouseup',\n on_mouse_event_closure('button_release')\n );\n rubberband_canvas.addEventListener(\n 'dblclick',\n on_mouse_event_closure('dblclick')\n );\n // Throttle sequential mouse events to 1 every 20ms.\n rubberband_canvas.addEventListener(\n 'mousemove',\n on_mouse_event_closure('motion_notify')\n );\n\n rubberband_canvas.addEventListener(\n 'mouseenter',\n on_mouse_event_closure('figure_enter')\n );\n rubberband_canvas.addEventListener(\n 'mouseleave',\n on_mouse_event_closure('figure_leave')\n );\n\n canvas_div.addEventListener('wheel', function (event) {\n if (event.deltaY < 0) {\n event.step = 1;\n } else {\n event.step = -1;\n }\n on_mouse_event_closure('scroll')(event);\n });\n\n canvas_div.appendChild(canvas);\n canvas_div.appendChild(rubberband_canvas);\n\n this.rubberband_context = rubberband_canvas.getContext('2d');\n this.rubberband_context.strokeStyle = '#000000';\n\n this._resize_canvas = function (width, height, forward) {\n if (forward) {\n canvas_div.style.width = width + 'px';\n canvas_div.style.height = height + 'px';\n }\n };\n\n // Disable right mouse context menu.\n this.rubberband_canvas.addEventListener('contextmenu', function (_e) {\n event.preventDefault();\n return false;\n });\n\n function set_focus() {\n canvas.focus();\n canvas_div.focus();\n }\n\n window.setTimeout(set_focus, 100);\n};\n\nmpl.figure.prototype._init_toolbar = function () {\n var fig = this;\n\n var toolbar = document.createElement('div');\n toolbar.classList = 'mpl-toolbar';\n this.root.appendChild(toolbar);\n\n function on_click_closure(name) {\n return function (_event) {\n return fig.toolbar_button_onclick(name);\n };\n }\n\n function on_mouseover_closure(tooltip) {\n return function (event) {\n if (!event.currentTarget.disabled) {\n return fig.toolbar_button_onmouseover(tooltip);\n }\n };\n }\n\n fig.buttons = {};\n var buttonGroup = document.createElement('div');\n buttonGroup.classList = 'mpl-button-group';\n for (var toolbar_ind in mpl.toolbar_items) {\n var name = mpl.toolbar_items[toolbar_ind][0];\n var tooltip = mpl.toolbar_items[toolbar_ind][1];\n var image = mpl.toolbar_items[toolbar_ind][2];\n var method_name = mpl.toolbar_items[toolbar_ind][3];\n\n if (!name) {\n /* Instead of a spacer, we start a new button group. */\n if (buttonGroup.hasChildNodes()) {\n toolbar.appendChild(buttonGroup);\n }\n buttonGroup = document.createElement('div');\n buttonGroup.classList = 'mpl-button-group';\n continue;\n }\n\n var button = (fig.buttons[name] = document.createElement('button'));\n button.classList = 'mpl-widget';\n button.setAttribute('role', 'button');\n button.setAttribute('aria-disabled', 'false');\n button.addEventListener('click', on_click_closure(method_name));\n button.addEventListener('mouseover', on_mouseover_closure(tooltip));\n\n var icon_img = document.createElement('img');\n icon_img.src = '_images/' + image + '.png';\n icon_img.srcset = '_images/' + image + '_large.png 2x';\n icon_img.alt = tooltip;\n button.appendChild(icon_img);\n\n buttonGroup.appendChild(button);\n }\n\n if (buttonGroup.hasChildNodes()) {\n toolbar.appendChild(buttonGroup);\n }\n\n var fmt_picker = document.createElement('select');\n fmt_picker.classList = 'mpl-widget';\n toolbar.appendChild(fmt_picker);\n this.format_dropdown = fmt_picker;\n\n for (var ind in mpl.extensions) {\n var fmt = mpl.extensions[ind];\n var option = document.createElement('option');\n option.selected = fmt === mpl.default_extension;\n option.innerHTML = fmt;\n fmt_picker.appendChild(option);\n }\n\n var status_bar = document.createElement('span');\n status_bar.classList = 'mpl-message';\n toolbar.appendChild(status_bar);\n this.message = status_bar;\n};\n\nmpl.figure.prototype.request_resize = function (x_pixels, y_pixels) {\n // Request matplotlib to resize the figure. Matplotlib will then trigger a resize in the client,\n // which will in turn request a refresh of the image.\n this.send_message('resize', { width: x_pixels, height: y_pixels });\n};\n\nmpl.figure.prototype.send_message = function (type, properties) {\n properties['type'] = type;\n properties['figure_id'] = this.id;\n this.ws.send(JSON.stringify(properties));\n};\n\nmpl.figure.prototype.send_draw_message = function () {\n if (!this.waiting) {\n this.waiting = true;\n this.ws.send(JSON.stringify({ type: 'draw', figure_id: this.id }));\n }\n};\n\nmpl.figure.prototype.handle_save = function (fig, _msg) {\n var format_dropdown = fig.format_dropdown;\n var format = format_dropdown.options[format_dropdown.selectedIndex].value;\n fig.ondownload(fig, format);\n};\n\nmpl.figure.prototype.handle_resize = function (fig, msg) {\n var size = msg['size'];\n if (size[0] !== fig.canvas.width || size[1] !== fig.canvas.height) {\n fig._resize_canvas(size[0], size[1], msg['forward']);\n fig.send_message('refresh', {});\n }\n};\n\nmpl.figure.prototype.handle_rubberband = function (fig, msg) {\n var x0 = msg['x0'] / fig.ratio;\n var y0 = (fig.canvas.height - msg['y0']) / fig.ratio;\n var x1 = msg['x1'] / fig.ratio;\n var y1 = (fig.canvas.height - msg['y1']) / fig.ratio;\n x0 = Math.floor(x0) + 0.5;\n y0 = Math.floor(y0) + 0.5;\n x1 = Math.floor(x1) + 0.5;\n y1 = Math.floor(y1) + 0.5;\n var min_x = Math.min(x0, x1);\n var min_y = Math.min(y0, y1);\n var width = Math.abs(x1 - x0);\n var height = Math.abs(y1 - y0);\n\n fig.rubberband_context.clearRect(\n 0,\n 0,\n fig.canvas.width / fig.ratio,\n fig.canvas.height / fig.ratio\n );\n\n fig.rubberband_context.strokeRect(min_x, min_y, width, height);\n};\n\nmpl.figure.prototype.handle_figure_label = function (fig, msg) {\n // Updates the figure title.\n fig.header.textContent = msg['label'];\n};\n\nmpl.figure.prototype.handle_cursor = function (fig, msg) {\n var cursor = msg['cursor'];\n switch (cursor) {\n case 0:\n cursor = 'pointer';\n break;\n case 1:\n cursor = 'default';\n break;\n case 2:\n cursor = 'crosshair';\n break;\n case 3:\n cursor = 'move';\n break;\n }\n fig.rubberband_canvas.style.cursor = cursor;\n};\n\nmpl.figure.prototype.handle_message = function (fig, msg) {\n fig.message.textContent = msg['message'];\n};\n\nmpl.figure.prototype.handle_draw = function (fig, _msg) {\n // Request the server to send over a new figure.\n fig.send_draw_message();\n};\n\nmpl.figure.prototype.handle_image_mode = function (fig, msg) {\n fig.image_mode = msg['mode'];\n};\n\nmpl.figure.prototype.handle_history_buttons = function (fig, msg) {\n for (var key in msg) {\n if (!(key in fig.buttons)) {\n continue;\n }\n fig.buttons[key].disabled = !msg[key];\n fig.buttons[key].setAttribute('aria-disabled', !msg[key]);\n }\n};\n\nmpl.figure.prototype.handle_navigate_mode = function (fig, msg) {\n if (msg['mode'] === 'PAN') {\n fig.buttons['Pan'].classList.add('active');\n fig.buttons['Zoom'].classList.remove('active');\n } else if (msg['mode'] === 'ZOOM') {\n fig.buttons['Pan'].classList.remove('active');\n fig.buttons['Zoom'].classList.add('active');\n } else {\n fig.buttons['Pan'].classList.remove('active');\n fig.buttons['Zoom'].classList.remove('active');\n }\n};\n\nmpl.figure.prototype.updated_canvas_event = function () {\n // Called whenever the canvas gets updated.\n this.send_message('ack', {});\n};\n\n// A function to construct a web socket function for onmessage handling.\n// Called in the figure constructor.\nmpl.figure.prototype._make_on_message_function = function (fig) {\n return function socket_on_message(evt) {\n if (evt.data instanceof Blob) {\n var img = evt.data;\n if (img.type !== 'image/png') {\n /* FIXME: We get \"Resource interpreted as Image but\n * transferred with MIME type text/plain:\" errors on\n * Chrome. But how to set the MIME type? It doesn't seem\n * to be part of the websocket stream */\n img.type = 'image/png';\n }\n\n /* Free the memory for the previous frames */\n if (fig.imageObj.src) {\n (window.URL || window.webkitURL).revokeObjectURL(\n fig.imageObj.src\n );\n }\n\n fig.imageObj.src = (window.URL || window.webkitURL).createObjectURL(\n img\n );\n fig.updated_canvas_event();\n fig.waiting = false;\n return;\n } else if (\n typeof evt.data === 'string' &&\n evt.data.slice(0, 21) === 'data:image/png;base64'\n ) {\n fig.imageObj.src = evt.data;\n fig.updated_canvas_event();\n fig.waiting = false;\n return;\n }\n\n var msg = JSON.parse(evt.data);\n var msg_type = msg['type'];\n\n // Call the \"handle_{type}\" callback, which takes\n // the figure and JSON message as its only arguments.\n try {\n var callback = fig['handle_' + msg_type];\n } catch (e) {\n console.log(\n \"No handler for the '\" + msg_type + \"' message type: \",\n msg\n );\n return;\n }\n\n if (callback) {\n try {\n // console.log(\"Handling '\" + msg_type + \"' message: \", msg);\n callback(fig, msg);\n } catch (e) {\n console.log(\n \"Exception inside the 'handler_\" + msg_type + \"' callback:\",\n e,\n e.stack,\n msg\n );\n }\n }\n };\n};\n\n// from http://stackoverflow.com/questions/1114465/getting-mouse-location-in-canvas\nmpl.findpos = function (e) {\n //this section is from http://www.quirksmode.org/js/events_properties.html\n var targ;\n if (!e) {\n e = window.event;\n }\n if (e.target) {\n targ = e.target;\n } else if (e.srcElement) {\n targ = e.srcElement;\n }\n if (targ.nodeType === 3) {\n // defeat Safari bug\n targ = targ.parentNode;\n }\n\n // pageX,Y are the mouse positions relative to the document\n var boundingRect = targ.getBoundingClientRect();\n var x = e.pageX - (boundingRect.left + document.body.scrollLeft);\n var y = e.pageY - (boundingRect.top + document.body.scrollTop);\n\n return { x: x, y: y };\n};\n\n/*\n * return a copy of an object with only non-object keys\n * we need this to avoid circular references\n * http://stackoverflow.com/a/24161582/3208463\n */\nfunction simpleKeys(original) {\n return Object.keys(original).reduce(function (obj, key) {\n if (typeof original[key] !== 'object') {\n obj[key] = original[key];\n }\n return obj;\n }, {});\n}\n\nmpl.figure.prototype.mouse_event = function (event, name) {\n var canvas_pos = mpl.findpos(event);\n\n if (name === 'button_press') {\n this.canvas.focus();\n this.canvas_div.focus();\n }\n\n var x = canvas_pos.x * this.ratio;\n var y = canvas_pos.y * this.ratio;\n\n this.send_message(name, {\n x: x,\n y: y,\n button: event.button,\n step: event.step,\n guiEvent: simpleKeys(event),\n });\n\n /* This prevents the web browser from automatically changing to\n * the text insertion cursor when the button is pressed. We want\n * to control all of the cursor setting manually through the\n * 'cursor' event from matplotlib */\n event.preventDefault();\n return false;\n};\n\nmpl.figure.prototype._key_event_extra = function (_event, _name) {\n // Handle any extra behaviour associated with a key event\n};\n\nmpl.figure.prototype.key_event = function (event, name) {\n // Prevent repeat events\n if (name === 'key_press') {\n if (event.key === this._key) {\n return;\n } else {\n this._key = event.key;\n }\n }\n if (name === 'key_release') {\n this._key = null;\n }\n\n var value = '';\n if (event.ctrlKey && event.key !== 'Control') {\n value += 'ctrl+';\n }\n else if (event.altKey && event.key !== 'Alt') {\n value += 'alt+';\n }\n else if (event.shiftKey && event.key !== 'Shift') {\n value += 'shift+';\n }\n\n value += 'k' + event.key;\n\n this._key_event_extra(event, name);\n\n this.send_message(name, { key: value, guiEvent: simpleKeys(event) });\n return false;\n};\n\nmpl.figure.prototype.toolbar_button_onclick = function (name) {\n if (name === 'download') {\n this.handle_save(this, null);\n } else {\n this.send_message('toolbar_button', { name: name });\n }\n};\n\nmpl.figure.prototype.toolbar_button_onmouseover = function (tooltip) {\n this.message.textContent = tooltip;\n};\n\n///////////////// REMAINING CONTENT GENERATED BY embed_js.py /////////////////\n// prettier-ignore\nvar _JSXTOOLS_RESIZE_OBSERVER=function(A){var t,i=new WeakMap,n=new WeakMap,a=new WeakMap,r=new WeakMap,o=new Set;function s(e){if(!(this instanceof s))throw new TypeError(\"Constructor requires 'new' operator\");i.set(this,e)}function h(){throw new TypeError(\"Function is not a constructor\")}function c(e,t,i,n){e=0 in arguments?Number(arguments[0]):0,t=1 in arguments?Number(arguments[1]):0,i=2 in arguments?Number(arguments[2]):0,n=3 in arguments?Number(arguments[3]):0,this.right=(this.x=this.left=e)+(this.width=i),this.bottom=(this.y=this.top=t)+(this.height=n),Object.freeze(this)}function d(){t=requestAnimationFrame(d);var s=new WeakMap,p=new Set;o.forEach((function(t){r.get(t).forEach((function(i){var r=t instanceof window.SVGElement,o=a.get(t),d=r?0:parseFloat(o.paddingTop),f=r?0:parseFloat(o.paddingRight),l=r?0:parseFloat(o.paddingBottom),u=r?0:parseFloat(o.paddingLeft),g=r?0:parseFloat(o.borderTopWidth),m=r?0:parseFloat(o.borderRightWidth),w=r?0:parseFloat(o.borderBottomWidth),b=u+f,F=d+l,v=(r?0:parseFloat(o.borderLeftWidth))+m,W=g+w,y=r?0:t.offsetHeight-W-t.clientHeight,E=r?0:t.offsetWidth-v-t.clientWidth,R=b+v,z=F+W,M=r?t.width:parseFloat(o.width)-R-E,O=r?t.height:parseFloat(o.height)-z-y;if(n.has(t)){var k=n.get(t);if(k[0]===M&&k[1]===O)return}n.set(t,[M,O]);var S=Object.create(h.prototype);S.target=t,S.contentRect=new c(u,d,M,O),s.has(i)||(s.set(i,[]),p.add(i)),s.get(i).push(S)}))})),p.forEach((function(e){i.get(e).call(e,s.get(e),e)}))}return s.prototype.observe=function(i){if(i instanceof window.Element){r.has(i)||(r.set(i,new Set),o.add(i),a.set(i,window.getComputedStyle(i)));var n=r.get(i);n.has(this)||n.add(this),cancelAnimationFrame(t),t=requestAnimationFrame(d)}},s.prototype.unobserve=function(i){if(i instanceof window.Element&&r.has(i)){var n=r.get(i);n.has(this)&&(n.delete(this),n.size||(r.delete(i),o.delete(i))),n.size||r.delete(i),o.size||cancelAnimationFrame(t)}},A.DOMRectReadOnly=c,A.ResizeObserver=s,A.ResizeObserverEntry=h,A}; // eslint-disable-line\nmpl.toolbar_items = [[\"Home\", \"Reset original view\", \"fa fa-home icon-home\", \"home\"], [\"Back\", \"Back to previous view\", \"fa fa-arrow-left icon-arrow-left\", \"back\"], [\"Forward\", \"Forward to next view\", \"fa fa-arrow-right icon-arrow-right\", \"forward\"], [\"\", \"\", \"\", \"\"], [\"Pan\", \"Left button pans, Right button zooms\\nx/y fixes axis, CTRL fixes aspect\", \"fa fa-arrows icon-move\", \"pan\"], [\"Zoom\", \"Zoom to rectangle\\nx/y fixes axis, CTRL fixes aspect\", \"fa fa-square-o icon-check-empty\", \"zoom\"], [\"\", \"\", \"\", \"\"], [\"Download\", \"Download plot\", \"fa fa-floppy-o icon-save\", \"download\"]];\n\nmpl.extensions = [\"eps\", \"jpeg\", \"pgf\", \"pdf\", \"png\", \"ps\", \"raw\", \"svg\", \"tif\"];\n\nmpl.default_extension = \"png\";/* global mpl */\n\nvar comm_websocket_adapter = function (comm) {\n // Create a \"websocket\"-like object which calls the given IPython comm\n // object with the appropriate methods. Currently this is a non binary\n // socket, so there is still some room for performance tuning.\n var ws = {};\n\n ws.binaryType = comm.kernel.ws.binaryType;\n ws.readyState = comm.kernel.ws.readyState;\n function updateReadyState(_event) {\n if (comm.kernel.ws) {\n ws.readyState = comm.kernel.ws.readyState;\n } else {\n ws.readyState = 3; // Closed state.\n }\n }\n comm.kernel.ws.addEventListener('open', updateReadyState);\n comm.kernel.ws.addEventListener('close', updateReadyState);\n comm.kernel.ws.addEventListener('error', updateReadyState);\n\n ws.close = function () {\n comm.close();\n };\n ws.send = function (m) {\n //console.log('sending', m);\n comm.send(m);\n };\n // Register the callback with on_msg.\n comm.on_msg(function (msg) {\n //console.log('receiving', msg['content']['data'], msg);\n var data = msg['content']['data'];\n if (data['blob'] !== undefined) {\n data = {\n data: new Blob(msg['buffers'], { type: data['blob'] }),\n };\n }\n // Pass the mpl event to the overridden (by mpl) onmessage function.\n ws.onmessage(data);\n });\n return ws;\n};\n\nmpl.mpl_figure_comm = function (comm, msg) {\n // This is the function which gets called when the mpl process\n // starts-up an IPython Comm through the \"matplotlib\" channel.\n\n var id = msg.content.data.id;\n // Get hold of the div created by the display call when the Comm\n // socket was opened in Python.\n var element = document.getElementById(id);\n var ws_proxy = comm_websocket_adapter(comm);\n\n function ondownload(figure, _format) {\n window.open(figure.canvas.toDataURL());\n }\n\n var fig = new mpl.figure(id, ws_proxy, ondownload, element);\n\n // Call onopen now - mpl needs it, as it is assuming we've passed it a real\n // web socket which is closed, not our websocket->open comm proxy.\n ws_proxy.onopen();\n\n fig.parent_element = element;\n fig.cell_info = mpl.find_output_cell(\"<div id='\" + id + \"'></div>\");\n if (!fig.cell_info) {\n console.error('Failed to find cell for figure', id, fig);\n return;\n }\n fig.cell_info[0].output_area.element.on(\n 'cleared',\n { fig: fig },\n fig._remove_fig_handler\n );\n};\n\nmpl.figure.prototype.handle_close = function (fig, msg) {\n var width = fig.canvas.width / fig.ratio;\n fig.cell_info[0].output_area.element.off(\n 'cleared',\n fig._remove_fig_handler\n );\n fig.resizeObserverInstance.unobserve(fig.canvas_div);\n\n // Update the output cell to use the data from the current canvas.\n fig.push_to_output();\n var dataURL = fig.canvas.toDataURL();\n // Re-enable the keyboard manager in IPython - without this line, in FF,\n // the notebook keyboard shortcuts fail.\n IPython.keyboard_manager.enable();\n fig.parent_element.innerHTML =\n '<img src=\"' + dataURL + '\" width=\"' + width + '\">';\n fig.close_ws(fig, msg);\n};\n\nmpl.figure.prototype.close_ws = function (fig, msg) {\n fig.send_message('closing', msg);\n // fig.ws.close()\n};\n\nmpl.figure.prototype.push_to_output = function (_remove_interactive) {\n // Turn the data on the canvas into data in the output cell.\n var width = this.canvas.width / this.ratio;\n var dataURL = this.canvas.toDataURL();\n this.cell_info[1]['text/html'] =\n '<img src=\"' + dataURL + '\" width=\"' + width + '\">';\n};\n\nmpl.figure.prototype.updated_canvas_event = function () {\n // Tell IPython that the notebook contents must change.\n IPython.notebook.set_dirty(true);\n this.send_message('ack', {});\n var fig = this;\n // Wait a second, then push the new image to the DOM so\n // that it is saved nicely (might be nice to debounce this).\n setTimeout(function () {\n fig.push_to_output();\n }, 1000);\n};\n\nmpl.figure.prototype._init_toolbar = function () {\n var fig = this;\n\n var toolbar = document.createElement('div');\n toolbar.classList = 'btn-toolbar';\n this.root.appendChild(toolbar);\n\n function on_click_closure(name) {\n return function (_event) {\n return fig.toolbar_button_onclick(name);\n };\n }\n\n function on_mouseover_closure(tooltip) {\n return function (event) {\n if (!event.currentTarget.disabled) {\n return fig.toolbar_button_onmouseover(tooltip);\n }\n };\n }\n\n fig.buttons = {};\n var buttonGroup = document.createElement('div');\n buttonGroup.classList = 'btn-group';\n var button;\n for (var toolbar_ind in mpl.toolbar_items) {\n var name = mpl.toolbar_items[toolbar_ind][0];\n var tooltip = mpl.toolbar_items[toolbar_ind][1];\n var image = mpl.toolbar_items[toolbar_ind][2];\n var method_name = mpl.toolbar_items[toolbar_ind][3];\n\n if (!name) {\n /* Instead of a spacer, we start a new button group. */\n if (buttonGroup.hasChildNodes()) {\n toolbar.appendChild(buttonGroup);\n }\n buttonGroup = document.createElement('div');\n buttonGroup.classList = 'btn-group';\n continue;\n }\n\n button = fig.buttons[name] = document.createElement('button');\n button.classList = 'btn btn-default';\n button.href = '#';\n button.title = name;\n button.innerHTML = '<i class=\"fa ' + image + ' fa-lg\"></i>';\n button.addEventListener('click', on_click_closure(method_name));\n button.addEventListener('mouseover', on_mouseover_closure(tooltip));\n buttonGroup.appendChild(button);\n }\n\n if (buttonGroup.hasChildNodes()) {\n toolbar.appendChild(buttonGroup);\n }\n\n // Add the status bar.\n var status_bar = document.createElement('span');\n status_bar.classList = 'mpl-message pull-right';\n toolbar.appendChild(status_bar);\n this.message = status_bar;\n\n // Add the close button to the window.\n var buttongrp = document.createElement('div');\n buttongrp.classList = 'btn-group inline pull-right';\n button = document.createElement('button');\n button.classList = 'btn btn-mini btn-primary';\n button.href = '#';\n button.title = 'Stop Interaction';\n button.innerHTML = '<i class=\"fa fa-power-off icon-remove icon-large\"></i>';\n button.addEventListener('click', function (_evt) {\n fig.handle_close(fig, {});\n });\n button.addEventListener(\n 'mouseover',\n on_mouseover_closure('Stop Interaction')\n );\n buttongrp.appendChild(button);\n var titlebar = this.root.querySelector('.ui-dialog-titlebar');\n titlebar.insertBefore(buttongrp, titlebar.firstChild);\n};\n\nmpl.figure.prototype._remove_fig_handler = function (event) {\n var fig = event.data.fig;\n if (event.target !== this) {\n // Ignore bubbled events from children.\n return;\n }\n fig.close_ws(fig, {});\n};\n\nmpl.figure.prototype._root_extra_style = function (el) {\n el.style.boxSizing = 'content-box'; // override notebook setting of border-box.\n};\n\nmpl.figure.prototype._canvas_extra_style = function (el) {\n // this is important to make the div 'focusable\n el.setAttribute('tabindex', 0);\n // reach out to IPython and tell the keyboard manager to turn it's self\n // off when our div gets focus\n\n // location in version 3\n if (IPython.notebook.keyboard_manager) {\n IPython.notebook.keyboard_manager.register_events(el);\n } else {\n // location in version 2\n IPython.keyboard_manager.register_events(el);\n }\n};\n\nmpl.figure.prototype._key_event_extra = function (event, _name) {\n var manager = IPython.notebook.keyboard_manager;\n if (!manager) {\n manager = IPython.keyboard_manager;\n }\n\n // Check for shift+enter\n if (event.shiftKey && event.which === 13) {\n this.canvas_div.blur();\n // select the cell after this one\n var index = IPython.notebook.find_cell_index(this.cell_info[0]);\n IPython.notebook.select(index + 1);\n }\n};\n\nmpl.figure.prototype.handle_save = function (fig, _msg) {\n fig.ondownload(fig, null);\n};\n\nmpl.find_output_cell = function (html_output) {\n // Return the cell and output element which can be found *uniquely* in the notebook.\n // Note - this is a bit hacky, but it is done because the \"notebook_saving.Notebook\"\n // IPython event is triggered only after the cells have been serialised, which for\n // our purposes (turning an active figure into a static one), is too late.\n var cells = IPython.notebook.get_cells();\n var ncells = cells.length;\n for (var i = 0; i < ncells; i++) {\n var cell = cells[i];\n if (cell.cell_type === 'code') {\n for (var j = 0; j < cell.output_area.outputs.length; j++) {\n var data = cell.output_area.outputs[j];\n if (data.data) {\n // IPython >= 3 moved mimebundle to data attribute of output\n data = data.data;\n }\n if (data['text/html'] === html_output) {\n return [cell, data, j];\n }\n }\n }\n }\n};\n\n// Register the function which deals with the matplotlib target/channel.\n// The kernel may be null if the page has been refreshed.\nif (IPython.notebook.kernel !== null) {\n IPython.notebook.kernel.comm_manager.register_target(\n 'matplotlib',\n mpl.mpl_figure_comm\n );\n}\n",
"text/plain": [
"<IPython.core.display.Javascript object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<img src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAoAAAAHgCAYAAAA10dzkAAAAAXNSR0IArs4c6QAAIABJREFUeF7snQeUVdX1xr/pvTB0ZoaiFCkiRQVBMLZoTCIo0WiMgi22GEuMCRoNJiqW/C0pikqiorHEiiVGjUoRrEhvivSZgaFM7+2/vjPe4c2b1++beW/mfWctlsK755bf3fee7+599j5RTU1NTVATAREQAREQAREQARGIGAJREoARc691oSIgAiIgAiIgAiJgCEgAyhBEQAREQAREQAREIMIISABG2A3X5YqACIiACIiACIiABKBsQAREQAREQAREQAQijIAEYITdcF2uCIiACIiACIiACEgAygZEQAREQAREQAREIMIISABG2A3X5YqACIiACIiACIiABKBsQAREQAREQAREQAQijIAEYITdcF2uCIiACIiACIiACEgAygZEQAREQAREQAREIMIISABG2A3X5YqACIiACIiACIiABKBsQAREQAREQAREQAQijIAEYITdcF2uCIiACIiACIiACEgAygZEQAREQAREQAREIMIISABG2A3X5YqACIiACIiACIiABKBsQAREQAREQAREQAQijIAEYITdcF2uCIiACIiACIiACEgAygZEQAREQAREQAREIMIISABG2A3X5YqACIiACIiACIiABKBsQAREQAREQAREQAQijIAEYITdcF2uCIiACIiACIiACEgAygZEQAREQAREQAREIMIISABG2A3X5YqACIiACIiACIiABKBsQAREQAREQAREQAQijIAEYITdcF2uCIiACIiACIiACEgAygZEQAREQAREQAREIMIISABG2A3X5YqACIiACIiACIiABKBsQAREQAREQAREQAQijIAEYITdcF2uCIiACIiACIiACEgAygZEQAREQAREQAREIMIISABG2A3X5YqACIiACIiACIiABKBsQAREQAREQAREQAQijIAEYITdcF2uCIiACIiACIiACEgAygZEQAREQAREQAREIMIISABG2A3X5YqACIiACIiACIiABKBsQAREQAREQAREQAQijIAEYITdcF2uCIiACIiACIiACEgAygZEQAREQAREQAREIMIISABG2A3X5YqACIiACIiACIiABKBsQAREQAREQAREQAQijIAEYITdcF2uCIiACIiACIiACEgAygZEQAREQAREQAREIMIISABG2A3X5YqACIiACIiACIiABKBsQAREQAREQAREQAQijIAEYITdcF2uCIiACIiACIiACEgAygZEQAREQAREQAREIMIISABG2A3X5YqACIiACIiACIiABKBsQAREQAREQAREQAQijIAEYITdcF2uCIiACIiACIiACEgAygZEQAREQAREQAREIMIISABG2A3X5YqACIiACIiACIiABKBsQAREQAREQAREQAQijIAEYITdcF2uCIiACIiACIiACEgAygZEQAREQAREQAREIMIISABG2A3X5YqACIiACIiACIiABKBsQAREQAREQAREQAQijIAEoI0b3tjYiPz8fKSlpSEqKsrGntRVBERABERABESgowg0NTWhrKwM/fr1Q3R0dEcdNqyOIwFo43bs3r0bubm5NvagriIgAiIgAiIgAqEisGvXLuTk5ITq8CE9rgSgDfwlJSXIzMwEDSg9Pd3GntRVBERABERABESgowiUlpYaB05xcTEyMjI66rBhdRwJQBu3gwZEw6EQlAC0AVJdRUAEREAERKADCWj8BiQAbRicDMgGPHUVAREQAREQgRAR0PgtAWjL9GRAtvCpswiIgAiIgAiEhIDGbwlAW4YnA7KFT51FQAREQAREICQENH5LANoyvEg2IKbQ19fXo6GhwRZDdRYBERABEQgdgZiYGMTGxkZcKbNIHr8ta9McQBvPXaQaUG1tLQoKClBZWWmDnrqKgAiIgAiEA4Hk5GT07dsX8fHx4XA6HXIOkTp+O8KVALRhapFoQCx+/c0334BfjT179jQvDBXBtmFE6ioCIiACISLASA4/6Pft22eiOUOGDImYosiROH47m5kEoI0HLxINqLq6Gtu2bcOAAQPAr0Y1ERABERCBzk2A0ZwdO3Zg0KBBSExM7NwX4+PZR+L4LQHoo3H4slkkGpAlACPpReGLLWgbERABEeisBCLxvR6J47cEYBCf0Eg0oEh8UQTRZLQrERABEQg7ApH4Xo/E8VsCMIiPXiQaUCS+KFyZzMCBA3H99debP760RYsW4cQTT0RRUZFZPlBNBMKdgLONc67va6+9hunTp7s89e3bt5sQ4sqVKzFmzJiALy9Y+wn4BCKwYyS+1yNx/JYADOLD3dkNqKymzCRwpMan+kyls74ovve975lB6aGHHvL5Wj1tyEnTKSkpPs+D5ETrgwcPonfv3kqaCcod0E7am4CzANyzZw+6deuGhISEoAnAWbNmmbVYX3/99ZZ9MhmBz1ePHj1MeRK19ifQWd/rdsh09vHbzrVbfZUEYoNiZzagDfs24MSnT0RFbQVumHgDbpp0EzISvS+I3VlfFL4IQGbEcfDRoGPjoXDRleI3kspLBJde6Pbmr5c7EM+dKwEYuisO/ZHr6uoQFxfX4SfSWd/rdkB15vHbznU79pUAtEGysxrQ1qKtOP6fx6OgvKDl6rOSsjD7+Nm45phrkBSX5JZKZ3xRcJB5+umnW10TM5k5YDEs+9///he33nor1qxZg3fffRf9+/fHjTfeiE8//RQVFRUYPnw45s6di1NOOaVlH67CY0888QTefvtts4/s7Gz83//9H84880zTxzkE/NRTT5nw8Ysvvmj+u2vXLhx//PF48sknTT0uNhba5nksWLDAlN257LLLQC9MSUlJK4+J44UdOHAAv/zlL7F06VLjcTz88MNxyy234Pzzz2/ZjKV87r//fvB8eVx6Ja+44grDgG337t246aab8N5776GmpsZc/9///ndMmDABrgZsnv+qVavMNbJRbI8aNcqIPp77yJEjsXjxYjzwwAPm+rZu3YqsrCz8+Mc/xn333YfU1EMe6GXLlpnz/eKLL4yn6dhjj8ULL7yAN998EzfccAPy8/NbeaBmzJhhPLE8jlozgcceewx//OMfzb2Njo5uwUJbpAePz8K3334bkI07hoA///xzYzcbN24095v2c/bZZ7eEgPkx9Ytf/AIffvihsVs+V1dffTWuu+46c05z5szBHXfc0eq2ffTRR+Cz5RxKpv385je/werVq43tzJw5E3feeWfLxxptbvTo0SaDdf78+cb2rrzySnMMd402RltjyJrCixGCBx98EOPGjWvpQu/kzTffjIULF5rnbvDgwbjnnnvwox/9yGzjzl7J2ZWA5jEYQrfOixGYRx99FO+88w7+97//mefu9ttv98jNOrl//vOf5h2zZcsWw4TPwt/+9jdccsklKCwsxFtvvdVyHXyX5OTk4O677za/O7fO+F63+7x31vHb7nU79pcAtEGzMxpQXmkepjw5BduKt2Fkz5G4ZcotuHPJndi4f6MhkZ2WjTnfm4NZY2YhNrpt+MXTi4JiyV2jgHEsL+BpWw5aSUmHRKirbTno+9r44v7BD35gBikOjGysYUiRRAHIgePPf/4zDjvsMDM/jwKI4m/SpEnmnDlg8kW7efNmM4ixuRKAfMFS0BxzzDH461//Cr6gWVqBL2dXApCD4wknnGDEJa/55z//OcaOHYt//etf5hh33XWXEU0c0CjCHn74YTz33HPmnB1DZo4c8vLy8Pzzzxuxmp6ebgQphRMHKgo4tt/+9rdG/HGwo+hkUe9NmzYZgVleXo6jjjrKCFgOFn369MFXX32F3NxcHHfccT4LwBUrVuCqq67CpZdeCnpWjzjiCBN+577JjgKcYuCkk07CI488Ys6LInLixIlmgKKwoCeWguC8884zIo/CmOd9zjnnmO33799vzpMCnkw6qvF6Kus6vgh6clyyT9MHKPzJ6j//+Q9OPvlkg4VzT3kvKaS///3vGyEViI1bApDPJEUa799tt91m7ieFHcW9NQeQoooijWKJ4dzly5cbYcOPgHPPPdfYGu2D71H+GxufFYp8RwFImx46dKixvWuvvdbY6uWXX45rrrmmRUhRAPK4/GD62c9+hk8++cRsz4+xU0891aVpUJjyWOPHjze/8xmnaGKd07S0NPBDafLkySgrKzPPCj+mNmzYYD7G+D7xZK+8Xl8FYK9evcw7gNfAfffr188jN54rRSOvlWKU58J3HJ9xfoyR89SpU80HgPUx+cYbbxguFOKOH1wWGAnA9I56fYTVcSQAbdyOziYA91XswwlPnWDE3uHdDsfSi5eib1pfNDQ24Jk1z+APi/6AnSU7DZFh3YfhzpPuxIzhM1oNOp5eFJ4KQp9xxhlGjFiNA7q7lUQoiixvkiXWONg7Ng7C/jRXIWBLlFFMTZs2zePu6MWioKF3jc2VAPz973+PP/3pT+Z3DpAcRDgIn3766S4F4MUXX2y+3jmwsFEIUaDyJc3GAZseAf5ho0eFIpUi0Z0AdHURP/zhD42ApMjlYEbxS08BBZ9ze/zxx83x6B3lYOzcfPUAckDigOypvfTSS4apdW85QO3cuRMff/yxy24UjDwvMmWjIP7LX/5iGHZkMXJOm0id6/u8WX/s1NO25bPLkRLv24cP7Zki5B//+IfZJe/rH/7wB/NxQ5Hhqvli45YA5P5mz55tRIZVD3TevHnmfnpKAqFo27t3L15++WVzCq7syTmUTM/iK6+8YjyN1n3ms8IPGdoZP574fPP54Eed1eg9pkClSPKlsT89d/zIomilB5ziiselAHVu3uzVVwFI0UaB6ak5c+OHD98fFNju7iW9pPResp111lnm49YS2s59JAAlAH15RrSNA4HOJABLqktw0oKT8FXBV8hJzzHib2DmwFb3s7q+GvO+nIe7lt6F/ZXNgmt83/GYe/JcnHLYKebl2xUFIAdFvlCtRvHG0BS9AfQQMHxSVVWFX//618bDx+ZKAP773/9u8U5xm4yMDOMJvOiii1wKQL7UHb2bHFwZxqHngQMbX9gMffFr3moMsfF3dwKQgxgHPIaW6TlhCJd/OADw/Bi2oyeQnhp6WZwbRdb69evNcV01XwUgVxSgt86x0ZtHryK9KHx2yJX2RE8QPwhGjBhh+DmHBa19UFjQu0qvKu8Xw2nkRQ9UR7bOIAB5r+lto9hiKJ0fVQxtWkIjUBu3BCC9yvQi0otmNf6d98RRAFIU0oPNe8ZniPNBuQ3tkM0XAUib57PkKF6sY3G/9MpTAFLAcqqC1SiCu3fvbjzxrhrDpAy38hrIic8OP0r5ccTngM8698djuGre7NVXAfjss8/iggsuaHUIT9x43py2wfN25/nmfaZIp3jl9oxOfPDBB5gyZYrLa5EAlADsyHdolzhWZxGADFed9uxp+Hjnx+iZ3BNLLl6CI3oc4fYelNaU4sFPHsSfP/kzymvLzXYnDTrJCMHR3UebcI+rQtDhGgLm+XvyADqXZuHLn6Ejesw454fh6J/85CdmH1YWsS8lMijguD0HOXdzADnHyGoUdRRq9G5aAnDJkiWtXtrW7+4EIAct/uFxjzzySCOs6GFgOJV91q5da0Le7gQgRe6XX37pVgAyPMt5hpwTZTUKWYpGxzmAzhnXHEQZBua8rJ/+9KfGu0hPH0OAFn+G4uh5cScAeTxuw3tx2mmnGTFIbxHD0x3Zwj0ETBYUWxQJnGtKTly5h3PerHCnXRunTXHOrCcBSBFKLxRDq5w+QI84555+9tlnJnzqqwCkzdMz5yjk2J+ecHqMef9dPd+ca8dnkAxcNUYlmG1MeyMfCmWeJz2OvD5+vPEd4E4AerNXeusZsqZYthpFKj9yHOcAOpfW8caNXnxO7/AkAPmMMpTMZ5LhcArKr7/+2u1jIgEoAdiR79AucazOIABr6msw7YVpePfbd5GRkIGPZn6EsX3H+sSfIeO7l96NR758BLUNtabP5aMvxy+H/BJDBw/tVEsGcd7TsGHDzEvdau5q81E4cY6S5Vmih4pf0BRyHSUAeY4MAXPiO0UZGz0UDBdTXLkTgEys4JwiK/RHbyHDv/zDPnzRU3wxdOoqBMz5jr/61a+MyHcVAmbYjZ48y4PD8+I8KWYuehKADOFxLh+9kVZiAsNXZGwJQIa0OP/KXQiYx+LcJ3o3eD+5LYW6mmsCtFeKBXp8KZ44d85qdm3cCgHTe27N12XyCQW+5QGk+KG3l54nq3FuKkP+lgCkl5JzUDk30Wq+hoB/97vfmRIyVgjY+aPDmwCkIGUo+cILLzSHZjib3kTaFwUgveAMIbsLAXuzV3Kn59WKGnC84DPNsKwnAegLN36A02voLgTM6+GHFj2nFIAMVzNk765JAEoA6j3qJ4FwF4D1jfU47+Xz8MrGV8AJ5O9f+D4m5U7y8yqBHcU7cMfiO/D06qeRm5yLeZPnYcjhQ5CTlYOEWNc1wfw+SDt34EDDQYdf15wETXFD75qr4sz0OHAQYsiJYW+KFIober86UgAyCYSDEcUcvWcUr88884wZlOg1cNXobaDYYuYsvSZMIuE1OyaO0OPB+XO8Foo3ekHowaM3jiE6igN6jzgxnZPIOaDTm0DvCAUX50XRq8K/M3zF/dAb40kAWh4bbkuRygnrHJAYprYEID0UPDbPg0KCmZwUm/SYcD4bG585nhPDx8z85SCn5prA+++/b1jTW80EI85RtZpdG+dHEUUIEyy4Xz4vTALhfExLANLGGGKl/XFb2i4/PPj/lgDklAAKR863Y7iWgoU24SoJhIKLc3CZjMWPF+ckEH8FIG2W82F5nrQrfmzR+81zsgq887mhYOVzxGgARTTfCZzX681ead98Tnj9fBb5HmGmLz/oPAlAX7jxQ43PyL333mueRwp9PlMUj1bj/adHnR+O9JTyGZYAPEQg3MfvjnivKQnEBuVwNqDGpkZcsvASI9riY+Lx1vlv4dTDXWfD+YqAtQMfXPogZvSdgR7ZPRAVG4X0hPQOnYDv67k6b7dtyzbcfM3N2LR+E6qrqvHRyo+QtzMPP5/2c6zYugLpGYe+AHfv3I3Z187GqhWr0C2rG37xq1/gnTfewfBRw/H7u5sH0e+N+R5mXjkTF195sfn7kO5D8MiCR3DqDw8xHjdoHG6961bM+NkMfPbxZ62O9cpzr+CuW+/CV9u+OvTCfvt9XH3R1fjmwDfm3yhy7r71brz24mtm4v5PL/opdu3YheiYaDz0hOuC1sVFxebcP1nyCRKTE02fgt0FKCstw6PPPmr2S6/gvAfn4d/P/BuFewrRs3dPnD/rfFx5w5Xm97xdeZh721wsW7TMDB6Dhw7GH+77A44af5T5/eF7HsYLT7+Amuoa/OSCn6C+rh6bN27Gv95ozl6+4MwLWrGyLvDJR5/E/L/ON4PtMccdgzN/ciZ+c/VvWvH/bNlneODOB7Bu9TrjYeYxH5r/UKv785urfoNF7y/Cx+s/dluUOFA76Ur9eO+mHjkVhXsL8cGKD9B/YHMGO1swbHzlFytx+02349uvv8XgYYNxza+vwS9n/RILFy3EiCNHGG/v7b++He+//b55R/zo7B8hLT0Niz9YjDcXN3v8Duw/gF9f8Wus+mKVmQ/77MJnkd0/GyeOPbFlP9yOdnHfH+7DxvUbkdktE2f99CzccOsNLWVgXNncVT+/CmkZabjv783zdp3b+jXrcdsNtxnb7ZfTDzf+/kbce/u9rZ5rPk/33H4PPvzvh2Z+4IBBA3DT7TfhxO83Z517slc+c7+/4fdY8sESc93Xz74eT857EqeecSp+9dtfmf6u3hu+cGPf5596Hk89+pR5J/A9ddqZp+H2e25vuUxOVSDHwUcMxvwX5ns07YbaBuzZvQfP7HwGB+oOuN2W7yTOiyYL3i8++44tq3sWOP/Xap99+pnbfTE8P+yIYS2/c4pCY0Ojy+3Jj3MurcYqAzz2hIkT8PyM55EYm+jx+lz9GM7jt98XE2AHCcAAwbFbuBoQH/zr/nsd/vr5XxETFYOXznkJZw0/y8aVHurKUMGmbzYhoUcCKprcl30JysG0k1YEKNzOOeEcnPLjU3DVzVdFLJ1rzrsGg4YMwk1/as6OVhMBEWhLgB+6Pxj3A9z2f7fhpDNO8oyoHtiftx9XLrsSOypcJ72EK2N/suMdryFcx++O5CwBaIN2uBrQbR/ehjuXNpcHWDB9AS48qnmOSzCaNVeEYaX66Howc1itfQjs2rkLiz5YhEnHTzLelH889g+88OwL+OiTjzD0iLZlKdrnLMJnr0UHiwyPqy69Ch9/+bHxTKp1TgK1dbWorKgEIxWuWkpySot3l/UEy8rL3F5oclJyy3xkeqhKy0rdbsv5ikmJzTVG6xvqzUe8u8btrPmNLJXFxCx3LTEhsaUcDj/UiksOJXc592GyCa+PjR/rRcVFbvebEJ9gErnMtmgy0yXctfi4eDO9hcen1/eB+x7Ae/95D+99/F6b1Y3iYuNMUo51Dpz/uK9gH678vyuxY29rAUjPm1W8m9tz3iT7clqGObeoQ2fUo3uPVp66JUuXuD3frG5Zpjar1RjCJmdXjVMDjhrdHIFg47zGuvo6TJ0yFRePuRhxMf6vnhKu47dbYO3wgwSgDajhaED3L7sfN/+vufbT38/4O64+5mobV9i2ayROFg4qQD92xpcyEyfWrVtnBgq+LFnixbEsjB+76/Sb8qODAyDnUlm1ETv9RXWxC2DY2So9xP9yTqn1dyZSMezHxmLVzER315iVy/l5bBReTPhx15gFzDmrbJwLxzmC7hrLB1nFkRnGZJKKu8btrPJQfO/xOXTXeHwrG53XzAxpd43Xxetjo2C15kO62p5zlZlNzEZhx6Ls7hrZcp6ilUTDZDDWfmQ9ROfGLGLH2oact8lSOEzyYkIXj2n94XuHRfG7WgvH8bujGUsA2iAebgb02JeP4cq3m+dxsWTL747/nY2rc91VAjDoSLVDEeg0BPgh4ijq6HGyvGQU51xizl1zFGp8j1BwcG6rqyLeFDOW54vbsqyJu0bvkLW6BcWmc9F4x34UPpbni9fBBCh3jdtxezZ6IVlPz13juVrilqKO1+ausXA2k0IsUccsaHeNbK1sfLLn/Dt3jXNmmUhjNSbTuGv0QlqJVdyG2dTcnlUGHFds6jSGGcCJhtv4HcAl2O7SpQQgXdOsM8UHivWWmHHorvAlyXHJLabo8+uSLxFmdrHuk+ND5IlwOBnQc2ufw89f/bkJE/xu8u8w95S5to3D1Q4kANsFq3YqAmFDgELDEmWsJ0jhY3nxKJocV+FxFHVMCmDJFNacpMBgFjf/a/2hmKF3SS38CETiez2cxu9QWUSXEYBc+YD1nCgCWdqCpQVYgZ4ufmv9VkfIrDXGGk0ss8FSCfz6YVo9M5jcldhwvknhYkBvbH4DZ794NhqaGnD10Vfjb2f8rd0ycyPxRRGqh1PHbR8CDKXRo0NRw5Cl1ejFoWhh42/ulirk747bUhR5KoJOjw9FEBuPyxIq7hpFkuWB8Wdbep0Y/nTXuM+W+WwNDS1z39jPOWTL0KcVUrVEneN+KQ4tgUcvkuWhIlf+oQBU61wEIvG9Hi7jdygtpcsIQBbd5FJHLBRrNRa/ZTFQ1jNzbvT0cVvHkAXrrNEjyLlXvrRwMKAPtn6AHz73Q9Q01ODC0RfiqelPIToq2pfTD2gbxyQQa0AJaEfqJALtRIAeKmsuGoWKJUgYRmQIjYLHVeP8KSuMxzAi51K5a5wfZQkfb/PZOHfRCrd5m8/Gj1XO3WLzNp+Nc+pYWJjNlVBzPHfH+Wz06rHuo7vG41sfzVY409GTRy9eR6693E5mot06EKBNWHMHFQKOHNPoEgKQX+D8yubi8ixwajVmLnGCrat1TZcvX26K49Lbx0KaDHNw9QeKRi6b40uzIwCZ7bRqT/NySIG23aW7ccGrF6CirgLTj5huyr3ERvv39c16Su4a58A4ThQmSw6srLxPDwHD5lbjvzkKQnpPHENFjsdw3pYvH3oOXDV/tuWgZC1Mz3152q/zthS2jt4g53Ox5iPx371ty3OwBkiKDQ6i7po/25KvtYoGbZ4eInfNn235wudcLDZv+3Xclsfn9u6aP9s6CjXLK+Vuv47b8l7wOXQMUTp69ujRt+yUApArnLBZXixHbxUFlTWXjELN09wsFtW15odRqHmab0XxZZ0DvX9cPcNd43NlzQ/jM8QCvu4akwms6Sq0dXdLlrE/t7OSKsjH4sD77ijuLM+eZQ+BvpvUr3MR4LPBMZDv+0i593bG7851d92fbZcQgPyqZ7YW08gds5VY0Z0V091lhb388stgdXkOIhx0zjzzTPDf3M1TsQYZCycNiHNgOFhYg4GvhhHMBeVPPexUvHn+m15X5eB58iF3LNTJa3UnUFjlnxX6rUbvCPfBFTHOP//8Fm8Jf6eQsbLV+HdWzHe3Xw4yjufAivruhARDco4ilPM13XlwOJhzxQyr0bvLgdFVo5ByLCzKAdFdGI9igXNKrcaB1lO4jfu1hBq9yZ7KR/B8LRFirYrhzobIwQpR0ubpeXLXyNcKO+7Zs8fjxHhO/LbEO+3D02R3rtBgiWF6ybhvd42eL0tQ8Vw9TWDnc2SJJCYTeBJUfNYtkeRpW3Klh8zy6lmClTYvL5avbypt15UJmHWtKyvNM8/nxMqQ7srX7Dh+850TyPjdVfh0KQFIrx6Xp7Ial9Li8kOOa2Bav3FuINel5NJZXFieX/tcCogLp1vrqDrfZC7f42qh+kAMqLKuEkf87ZBYCdSgJvefjPk/no+U+OZaUY6NQon1krj8EP+w0jpLiHB5LatRtLkTakygYaKM1VgOgKKXgmjGjBk4+eSTWwZSig3Lw8DtydOdR42DrzXHiNtSRLg7B2sQt86B2XXuPF/8cnV8gfGl5k5YUqA5Lo1EMcMPAVeN12uVg+Dv/Fp2Jyz5O7e1PIAUPp7mkvEcLLFIMeNpLhmvzfo6Z9aB80hXAAAgAElEQVSep7lkFD6WsKR9ehKsvBfWRw/vr6faaLzHlrDk8Xke7hrDnlY4idflqYYZPVSWCCUvT+KWoVfL08v7y3PgtfIPr4P/JSeLa6DPlvqJQKQQoPjjOyOSQvvyAAJdQgAGEgJmwggHfIaNrcbEEIoeeipcfQkF0wPYni8WJsK88cYbZq1bZ6HCtVYZyg3G4EiB5ykM2Z7XqH2LgAiIgAjYJ8CPpkgJ+zrSkgDsIgKQN5VJIOPHjzdZwFZjKG7atGkuk0DowaKngNnDVqO3jCFkhp88LZxtbR8OBsTQJc/7Zz/7Wct1MKv5rbfeMn+nd4eeTv6hx84qVmr/taE9iIAIiIAIiEDnJBAO43eoyXUJDyAhWmVgmMDBMPDjjz+OJ554wmS7ser67NmzjbBbsGCBYf7UU0/h8ssvx1/+8peWEPD1119vPGOffeZ+AetQf0Ew/Pjhhx+2hHWtavqcl2Zl7tH7x3+n6OPctUhy64f6gdLxRUAEREAEwp+ABGAX8gDS3Oj9YxkXzj/jfDXW+LOWzZo1a5ZJc1+0aFGLZbLsCwUjvWicA3HSSSfh3nvvbTXfy5MZd6QBMVv5zjvvBJfsccyupRdz4sSJRsiOHTs2/J86naEIiIAIiIAIhJhAR47fIb5Ut4fvMh7AUABuDwPivDqWZmHSBkO5nLPHxuzkc845x/w//80K6VLgWksbhYKBjikCIiACIiACnY1Ae4zfnY2BBKCNOxYMA6I3j6VNrExdhnetsiF//OMfzcL3bMyg/M9//mPm8VnFX22curqKgAiIgAiIQMQSCMb43dnhSQDauIN2DYjz9LgcnXNhWNYmYjh65syZJolFTQREQAREQAREIHgE7I7fwTuT0O1JAtAGe7sGxNp3rGnG8jJcv9jK1uWSdlpP08aNUVcREAEREAER8EDA7vjdFeBKANq4i8EwICZ1DBs2rNUSZjZOSV1FQAREQAREQAS8EAjG+N3ZIUsA2riDMiAb8NRVBERABERABEJEQON3FysD09F2JAPqaOI6ngiIgAiIgAjYJ6DxWwLQlhXJgGzhU2cREAEREAERCAkBjd8SgLYMTwZkC586i4AIiIAIiEBICGj8lgC0ZXgyIFv41FkEREAEREAEQkJA47cEoC3DkwHZwqfOIiACIiACIhASAhq/JQBtGZ4MyBY+dRYBERABERCBkBDQ+C0BaMvwZEC28KmzCIiACIiACISEgMZvCUBbhicDsoVPnUVABERABEQgJAQ0fksA2jI8GZAtfOosAiIgAiIgAiEhoPFbAtCW4cmAbOFTZxEQAREQAREICQGN3xKAtgxPBmQLnzqLgAiIgAiIQEgIaPyWALRleDIgW/jUWQREQAREQARCQkDjtwSgLcOTAdnCp84iIAIiIAIiEBICGr8lAG0ZngzIFj51FgEREAEREIGQEND4LQFoy/BkQLbwqbMIiIAIiIAIhISAxm8JQFuGJwOyhU+dRUAEREAERCAkBDR+SwDaMjwZkC186iwCIiACIiACISGg8VsC0JbhyYBs4VNnERABERABEQgJAY3fXUwAPvLII7j//vtRUFCAkSNH4qGHHsKUKVPcGldNTQ3++Mc/4tlnn8WePXuQk5ODW2+9FZdccolPBikD8gmTNhIBERABERCBsCKg8bsLCcAXX3wRF154ISgCJ0+ejMceewzz58/Hhg0b0L9/f5eGN23aNOzduxd33nknBg8ejMLCQtTX12PSpEk+GaoMyCdM2kgEREAEREAEwoqAxu8uJAAnTJiAcePG4dFHH20xsuHDh2P69OmYO3duG8P773//i/POOw9bt25FVlZWQIYpAwoImzqJgAiIgAiIQEgJaPzuIgKwtrYWycnJeOmll3DWWWe1GNV1112HVatWYfHixW0M7eqrr8bXX3+No48+Gs888wxSUlJw5pln4k9/+hOSkpJ8MkwZkE+YtJEIiIAIiIAIhBUBjd9dRADm5+cjOzsby5YtaxW+vfvuu/H0009j8+bNbQzv9NNPx6JFi3DKKafg9ttvx/79+0FReNJJJ+Gf//ynS0PlnEH+sRoNKDc3FyUlJUhPTw8r49bJiIAIiIAIiIAIuCYgAdjFBODy5ctx3HHHtdztu+66y3j3Nm3a1MYCvv/972Pp0qUm+SMjI8P8/uqrr+InP/kJKioqXHoB58yZgzvuuKPNviQA9YoRAREQAREQgc5DQAKwiwjAQELAM2fONB7DLVu2tFjsxo0bMWLECBMaHjJkSBtLlgew8zzcOlMREAEREAERcEdAArCLCEDeYCaBjB8/3mQBW41ijpm+rpJAHn/8cVx//fUm8zc1NdV0WbhwIc4++2yUl5f7NA9QBqSXiwiIgAiIgAh0PgIav7uQALTKwMybN8+EgSnwnnjiCaxfvx4DBgzA7NmzkZeXhwULFhhLpchjlvDEiRNNWJdzAC+77DKccMIJpp8vTQbkCyVtIwIiIAIiIALhRUDjdxcSgDQtev/uu+8+Uwh61KhRePDBBzF16lRjdbNmzcL27dtN4ofVODfw2muvNaHg7t2749xzzzU1AZUFHF4Pqs5GBERABERABIJJQAKwiwnAYBqHL/uSAflCSduIgAiIgAiIQHgR0PgtAWjLImVAtvCpswiIgAiIgAiEhIDGbwlAW4YnA7KFT51FQAREQAREICQENH5LANoyPBmQLXzqLAIiIAIiIAIhIaDxWwLQluHJgGzhU2cREAEREAERCAkBjd8SgLYMTwZkC586i4AIiIAIiEBICGj8lgC0ZXgyIFv41FkEREAEREAEQkJA47cEoC3DkwHZwqfOIiACIiACIhASAhq/JQBtGZ4MyBY+dRYBERABERCBkBDQ+C0BaMvwZEC28KmzCIiACIiACISEgMZvCUBbhicDsoVPnUVABERABEQgJAQ0fksA2jI8GZAtfOosAiIgAiIgAiEhoPFbAtCW4cmAbOFTZxEQAREQAREICQGN3xKAtgxPBmQLnzqLgAiIgAiIQEgIaPyWALRleDIgW/jUWQREQAREQARCQkDjtwSgLcOTAdnCp84iIAIiIAIiEBICGr8lAG0ZngzIFj51FgEREAEREIGQEND4LQFoy/BkQLbwqbMIiIAIiIAIhISAxm8JQFuGJwOyhU+dRUAEREAERCAkBDR+SwDaMjwZkC186iwCIiACIiACISGg8VsC0JbhyYBs4VNnERABERABEQgJAY3fEoC2DE8GZAufOouACIiACIhASAho/JYAtGV4MiBb+NRZBERABERABEJCQOO3BKAtw5MB2cKnziIgAiIgAiIQEgIav7uYAHzkkUdw//33o6CgACNHjsRDDz2EKVOmeDWuZcuW4YQTTsCoUaOwatUqr9tbG8iAfEalDUVABERABEQgbAho/O5CAvDFF1/EhRdeCIrAyZMn47HHHsP8+fOxYcMG9O/f363RlZSUYNy4cRg8eDD27t0rARg2j6dORAREQAREQATah4AEYBcSgBMmTDBC7tFHH22xluHDh2P69OmYO3euWws677zzMGTIEMTExOD111+XAGyfZ017FQEREAEREIGwISAB2EUEYG1tLZKTk/HSSy/hrLPOajGw6667zgi6xYsXuzS6J5980ngMP/nkE9x5550SgGHzaOpEREAEREAERKD9CEgAdhEBmJ+fj+zsbHAu36RJk1os5u6778bTTz+NzZs3t7Gib775BscffzyWLl2KoUOHYs6cOV4FYE1NDfjHajSg3NxcMIycnp7efpaqPYuACIiACIiACASNgARgFxOAy5cvx3HHHddiIHfddReeeeYZbNq0qZXRNDQ0YOLEibj00ktx5ZVXmt98EYDc5o477mhjgBKAQXsmtSMREAEREAERaHcCEoBdRAD6GwIuLi5Gt27dzLw/qzU2NqKpqcn823vvvYeTTjqpjQHKA9juz6QOIAIiIAIiIALtTkACsIsIQFoKk0DGjx9v5vRZbcSIEZg2bVqbJBCKPWYHOzb2+/DDD/Hyyy9j0KBBSElJ8WqAMiCviLSBCIiACIiACIQdAY3fXUgAWmVg5s2bZ8LAjz/+OJ544gmsX78eAwYMwOzZs5GXl4cFCxa4NERfQsDOHWVAYfdM64REQAREQAREwCsBjd9dSADybtOLd99995lC0Czq/OCDD2Lq1KnGEGbNmoXt27dj0aJFEoBeHw1tIAIiIAIiIAJdl4AEYBcTgB1tqjKgjiau44mACIiACIiAfQIavyUAbVmRDMgWPnUWAREQAREQgZAQ0PgtAWjL8GRAtvCpswiIgAiIgAiEhIDGbwlAW4YnA7KFT51FQAREQAREICQENH5LANoyPBmQLXzqLAIiIAIiIAIhIaDxWwLQluHJgGzhU2cREAEREAERCAkBjd8SgLYMTwZkC586i4AIiIAIiEBICGj8lgC0ZXgyIFv41FkEREAEREAEQkJA47cEoC3DkwHZwqfOIiACIiACIhASAhq/JQBtGZ4MyBY+dRYBERABERCBkBDQ+C0BaMvwZEC28KmzCIiACIiACISEgMZvCUBbhicDsoVPnUVABERABEQgJAQ0fksA2jI8GZAtfOosAiIgAiIgAiEhoPFbAtCW4cmAbOFTZxEQAREQAREICQGN3xKAtgxPBmQLnzqLgAiIgAiIQEgIaPyWALRleDIgW/jUWQREQAREQARCQkDjtwSgLcOTAdnCp84iIAIiIAIiEBICGr8lAG0ZngzIFj51FgEREAEREIGQEND4LQFoy/BkQLbwqbMIiIAIiIAIhISAxm8JQFuGJwOyhU+dRUAEREAERCAkBDR+SwDaMjwZkC186iwCIiACIiACISGg8VsC0JbhyYBs4VNnERABERABEQgJAY3fEoC2DE8GZAufOouACIiACIhASAho/JYAtGV4MiBb+NRZBERABERABEJCQON3FxOAjzzyCO6//34UFBRg5MiReOihhzBlyhSXxvXqq6/i0UcfxapVq1BTU2O2nzNnDk477TSfjVEG5DMqbSgCIiACIiACYUNA43cXEoAvvvgiLrzwQlAETp48GY899hjmz5+PDRs2oH///m2M7vrrr0e/fv1w4oknIjMzE08++ST+/Oc/47PPPsPYsWN9MlIZkE+YtJEIiIAIiIAIhBUBjd9dSABOmDAB48aNM149qw0fPhzTp0/H3LlzfTI8egF/+tOf4vbbb/dpexmQT5i0kQiIgAiIgAiEFQGN311EANbW1iI5ORkvvfQSzjrrrBYju+6660yId/HixV4Nr7GxEQMHDsTNN9+MX/7yl1635wYyIJ8waSMREAEREAERCCsCGr+7iADMz89HdnY2li1bhkmTJrUY2d13342nn34amzdv9mp4nDt4zz33YOPGjejVq5fL7TlXkH+sRgPKzc1FSUkJ0tPTvR5DG4iACIiACIiACISegARgFxOAy5cvx3HHHddiWXfddReeeeYZbNq0yaO1Pf/887jsssuwcOFCnHLKKW63ZZLIHXfc0eZ3CcDQP8w6AxEQAREQARHwlYAEYBcRgHZCwEweufjii034+Ic//KFH25EH0NdHS9uJgAiIgAiIQPgSkADsIgKQJsYkkPHjx5ssYKuNGDEC06ZNc5sEQs/fJZdcAv6XySL+NhmQv8S0vQiIgAiIgAiEnoDG7y4kAK0yMPPmzTNh4McffxxPPPEE1q9fjwEDBmD27NnIy8vDggULjOVR9F100UV4+OGHcfbZZ7dYY1JSEjIyMnyyThmQT5i0kQiIgAiIgAiEFQGN311IANKy6P277777TCHoUaNG4cEHH8TUqVON0c2aNQvbt2/HokWLzN+/973vucwOnjlzJp566imfDFUG5BMmbSQCIiACIiACYUVA43cXE4AdbV0yoI4mruOJgAiIgAiIgH0CGr8lAG1ZkQzIFj51FgEREAEREIGQEND4LQFoy/BkQLbwqbMIiIAIiIAIhISAxm8JQFuGJwOyhU+dRUAEREAERCAkBDR+SwDaMjwZkC186iwCIiACIiACISGg8VsC0JbhyYBs4VNnERABERABEQgJAY3fEoC2DE8GZAufOouACIiACIhASAho/JYAtGV4MiBb+NRZBERABERABEJCQOO3BKAtw5MB2cKnziIgAiIgAiIQEgIavyUAbRmeDMgWPnUWAREQAREQgZAQ0PgtAWjL8GRAtvCpswiIgAiIgAiEhIDGbwlAW4YnA7KFT51FQAREQAREICQENH5LANoyPBmQLXzqLAIiIAIiIAIhIaDxWwLQluHJgGzhU2cREAEREAERCAkBjd8SgLYMTwZkC586i4AIiIAIiEBICGj8lgC0ZXgyIFv41FkEREAEREAEQkJA47cEoC3DkwHZwqfOIiACIiACIhASAhq/JQBtGZ4MyBY+dRYBERABERCBkBDQ+C0BaMvwZEC28KmzCIiACIiACISEgMZvCUBbhicDsoVPnUVABERABEQgJAQ0fksA2jI8GZAtfOosAiIgAiIgAiEhoPFbAtCW4cmAbOFTZxEQAREQAREICQGN3xKAtgxPBmQLnzqLgAiIgAiIQEgIaPyWALRleDIgW/jUWQREQAREQARCQkDjdxcTgI888gjuv/9+FBQUYOTIkXjooYcwZcoUt8a1ePFi3HjjjVi/fj369euHm2++GVdeeaXPxigD8hmVNhQBERABERCBsCGg8bsLCcAXX3wRF154ISgCJ0+ejMceewzz58/Hhg0b0L9//zZGt23bNowaNQqXX345rrjiCixbtgxXX301nn/+ecyYMcMnI5UB+YRJG4mACIiACIhAWBHQ+N2FBOCECRMwbtw4PProoy1GNnz4cEyfPh1z585tY3i//e1v8cYbb2Djxo0tv9H7t3r1anzyySc+GaoMyCdM2kgEREAEREAEwoqAxu8uIgBra2uRnJyMl156CWeddVaLkV133XVYtWoVGOp1blOnTsXYsWPx8MMPt/z02muv4dxzz0VlZSXi4uK8GqsMyCsibSACIiACIiACYUdA43cXEYD5+fnIzs42YdxJkya1GNrdd9+Np59+Gps3b25jfEOHDsWsWbNwyy23tPy2fPlyEz7m/vr27dumT01NDfjHaiUlJSa8vGvXLqSnp4edgeuEREAEREAEREAE2hKgAMzNzUVxcTEyMjIiElFUU1NTU2e/cksAUsAdd9xxLZdz11134ZlnnsGmTZtcCsCLL74Ys2fPbvmNAvL44483SSR9+vRp02fOnDm44447Ojsunb8IiIAIiIAIiABgHDg5OTkRyaJLCMCOCgE7ewD55TBgwADs3LkzYr8g7Dw11heYPKiBURS/wLhZvcRP/OwRsNdb9hdafvR9lZWVmQog0dHR9k6mk/buEgKQ7JkEMn78eJMFbLURI0Zg2rRpbpNA3nzzTZMlbLWrrrrKzBlUEkjHWLPmYNjjLH7iZ4+Avd6yP/GzR8Beb9mfPX7s3WUEoFUGZt68eSYM/Pjjj+OJJ54wNf7opWOoNy8vDwsWLDDUrDIwLAHDUjAUfcwCVhkY+0bl6x70APtKyvV24id+9gjY6y37Ez97BOz1lv3Z49elBCAvht6/++67z8zhY42/Bx98EMz2ZWPCx/bt27Fo0aIWaswOvuGGG1oKQbM0jApB2zcqX/egB9hXUhKA9kiJn/i1BwF7+9T7T/zsEbDfu8t4AO2j8H8PnBPIGoP0LiYkJPi/gwjvIX72DED8xM8eAXu9ZX/iZ4+Avd6yP3v8upwH0D4O7UEEREAEREAEREAEuj4BeQC7/j3WFYqACIiACIiACIhAKwISgDIIERABERABERABEYgwAhKAEXbDdbkiIAIiIAIiIAIiIAEYoA0w4/j+++83GccjR47EQw89hClTpgS4t8jq5mpFld69e2PPnj2RBcLHq12yZImxtRUrVhh745rV06dPb+nNgqZcoYalj4qKikxNzL///e/GLtUAb/xYIYBLRjo2Mvz0008jHh+T3F599VWzmlJSUpJZavPee+/FsGHDZH8+WIcv/GR/7kE++uij4B9W8GDjO+3222/HD37wA/N3vft8MEIPm0gABsDPqjlIEci1gx977DHMnz/fFJXm2sBqnglQAL788sv43//+17JhTEwMevbsKXQuCLzzzjtmnetx48ZhxowZbQQgB2Que/jUU0+Ba1zfeeedRvRwDey0tLSIZ+qNHwfgvXv34sknn2xhFR8fj6ysrIhnd/rpp+O8887DMcccg/r6etx6661Yu3atedelpKQYPrI/92biCz/Zn3t+XKyBY8PgwYPNRvxQ48fwypUrjRiU7dl7RUkABsCP3gEOxvwysdrw4cONV4ZffGreBeDrr79uVl1R849AVFRUKwHIL2AuZXT99deDdSzZWB6BHlW+HFnoXO0QAWd+/IUDMJd1pE2qeSawb98+9OrVC6yhyhqrsj//LMaZn+zPP37cmh9mFIGXXHKJ3n3+42vVQwLQT4CBrDvs5yG6/Ob0APIBzsjIMPUTKajvvvtuHHbYYV3+2u1eoLOA2bp1Kw4//HB89dVXGDt2bMvuuQRiZmZmm9Cm3eN39v7uBCDFH71+ZHbCCScYjyqFjlprAlu2bMGQIUOMF5DF9mV//lmIMz9LAMr+vHNsaGjASy+9hJkzZxoPYGJiot593rF53EIC0E+A+fn5yM7ONiE5zoexGgUM3dMMu6l5JsCQXGVlpQlXMvTGkCXnGHHZvu7duwufBwLOAmb58uVmGgKXOaQn0Gq/+MUvsGPHDrz77rvi6UDAlQDklI7U1FSzZCSXiLzttttMuJNzLlXg/RA8evv4YcF5pkuXLjU/yP58f7xc8WNv2Z9nhvzY4PKu1dXV5jl97rnncMYZZ8j2fDc9t1tKAPoJ0RKAfPHRKK1Gj8EzzzxjhIyafwQqKirMl9zNN9+MG2+80b/OEba1OwFIu+zbt28LDa5vvWvXLvz3v/+NMEKeL9eVAHTuwUQbisEXXngBZ599tvh9R+Caa67B22+/jY8//hg5OTmtBKDsz7uZuOLnqpfsrzUVRt127txppmm88sorZr49pyDw7/z4le15tz13W0gA+slOIWA/gfm4+amnnmom+jrOq/Sxa0RtphCwvdvtiwDkERjmvOyyy1rmVdo7aufvfe2115o5kkwuGjRoUMsFKQTs2711x89db9mfe66nnHKKcRhwzrOmv/hmfxKA9ji16s05a+PHjwezgK02YsQIEx5REoj/oJm0wAeZYUum+Ku5J+AuCeSGG24wHlQ2fqRw/pqSQNpy9EUAHjhwwEzzYFmdiy66KKLNkWFLiheWHlq0aJERxo7NSgKR/bk2E2/8XPWS/Xl+5E4++WTk5uaarH1Oe5HtBf6KkgcwAHZWGZh58+aZMDAHiieeeMLMYWPoSM0zgZtuugk//vGPTcmcwsJCMweQLn3O9RC/tuzKy8vByeNsTPR44IEHcOKJJ5psODKk0OOHB1+IHKA5H5WDtcrANLP0xI8MmZTE8joMobPe2C233GJCThs3boz4MjpXX321mXO1cOHCVrX/mMDFuoBssj/37ztv/Gibsj/3/PgssuYfBV9ZWZmZlnHPPfeYqS2MGsn27KkNCcAA+dH7d99995nCvMyGe/DBB01ZBDXvBFhXjKGk/fv3m9p/EydOxJ/+9CfQi6rWlgDFHAWfc2M2HGv/WcVQWY/SsRA07VINRgy748cpByzfxKxCzimiCOS2tEcOOpHe6DF11fixwfI5bLI/91bijV9VVZXsz8NDdumll+KDDz4w4yw/OkaPHm1CvxR/sj37bycJQPsMtQcREAEREAEREAER6FQEJAA71e3SyYqACIiACIiACIiAfQISgPYZag8iIAIiIAIiIAIi0KkISAB2qtulkxUBERABERABERAB+wQkAO0z1B5EQAREQAREQAREoFMRkADsVLdLJysCIiACIiACIiAC9glIANpnqD2IgAiIgAiIgAiIQKciIAHYqW6XTlYEREAEREAEREAE7BOQALTPUHsQAREQAREQAREQgU5FQAKwU90unawIiIAIiIAIdBwBrtp0//33Y8WKFWZFDq4LzdVzPDUu7XnjjTea5VG5Xi/XKb/yyis77qR1JJ8ISAD6hEkbiYAIiIAIiEDkEXjnnXewbNkyjBs3zqyZ7U0Abtu2zSyPevnll+OKK64wfbkm8vPPP2/6q4UPAQnA8LkXOhMREAEREAERCFsCXNvYmwDkWr1vvPEGNm7c2HId9P6tXr0an3zySdheWySemASgjbve2NiI/Px8pKWlwd2i3zZ2r64iIAIiIAJdiEBTUxPKyspMWDQ6Orrdrqy6uhq1tbVu98/zcB6zEhISwD+emi8CcOrUqRg7diwefvjhll1RNJ577rmorKxEXFxcu123duwfAQlA/3i12nr37t3Izc21sQd1FQEREAERiDQCu3btQk5OTrtcNsXfoIF9sWdvsdv9p6amory8vNXvf/jDHzBnzhzbAnDo0KGYNWsWbrnllpZ9LV++HJMnTzYOk759+7bLdWun/hOQAPSfWUuPkpISZGZm4t8r/41dTQWt9nRY6lCve95Yuq3VNkNSB3vt47jBmpLtrbYfnna4X/2dN15dtKPVP43MGGhrf946rzqwq2WTIzMHeNs84N+/2r/b9B2T5b9YX7EvD2O7e++3Ym8exvfMdnuOK/bkm9/G93K/zVf5BRjfp5/bfXyVl+/59935GNfXdf+VuwowLrvti3fVjgKMzW3976u2FWDsgNb/tnprAcY4/Nuab5vtfcygQ9ut+aYARx126O9rN+dj9OBD57Nucx5GDzn097Ub8jB6mMPv6/Nw5PDW579+zW6MGtGW2fo1uzBqlGuW61fuxMgjvQ+uG1Zsx4ij+gdsV84dN3y5FSPGtZ8dB+1Eu+iONny2pdWVjRjfvu+vQDBWVlVi5k3nori4GBkZGYHswmuf0tJSs+9tX+SrO+8AACAASURBVP8F6WlJbbYvLavCoKG/AkVoenp6y+/B8gBSAF588cWYPXt2y745D/D44483SSR9+vTxeg3aoGMISADa4Gw9aG9teQspaSkte9pY+k2rvQ5OPcKno6wr+bbVdsPSvItIq8Oq4tZicmS6f2LS+QS/OthaXI7OPMynawh0oxX7D4nPMd0GBbobt/2+2LezWYB193+A/nzvLhzdw3u/z/fswrG93IvFzwuaheixvV1v80VeXvPvfV2Lly927cax/dwLmy927sYx2a5//3JHHo7JbSuYVmzLw9H9W//7iq15OHpg63/7aksexg869G8rv8nD+MMd/r45D+MGH/r7qo27MXbooXNZs6FZ7I85ovnfVq/97u/DHbZZuwtHjWx9/mtX7cToUa15rV25A6OPcs1w7ZfbceQY76Ju3edbzXmMCpJgW/ed+Bh1dPBtN+gPQxfZ4brlX7e6klHHtu87KhjYKqsqcM41PwKdB47iKxj7tvZhjUsHCp5Aenpym12Xllaie9/LAzoHhYCDeadCvy8JQBv3wJ0AdNylxKB/gMNRCFIAsnkTgRSARsB5EYHuBCD7UgR6EoBm/25EoL8CkOLPXJcXAUjxZ8SzTQFoiT9LAI5xEn/8d0cB6Er8cZtgCcBgiT+eEwWgxJ9/z7q/W3dGwed8jZEgAJkE8uabb2LDhg0tl3/VVVdh1apVSgLx1+jbeXsJQBuALQH4yOo/Y3if8T7tyVEQyjPoHll7CsFAvIHBEoG+eAHdCUAjED14Ad0JQH+9f0YUOngAXXn/jCD00wPoTQD66v3jsV15AOn9Y/PmAWwP75/En0+vP782chZ87NwZvHyeLrIzCkDOFdyypTm8zuSOBx54ACeeeCKysrLQv39/E+rNy8vDggULzDZWGRiWgGEpGGb+MgtYZWD8Mv8O2VgC0AZmSwC+t/V15KHZS2K1gSkjvO45EO+gnTAxT8gxVNwZwsTtJQTbWwS2pxfQXw+gvwKwPcK/zuKPtujsAXT2/hmh107h32B5/xT69fqa82uDruDlCxcBWJD/ONLTXcwBLK1C336/8DkEvGjRIiP4nNvMmTPx1FNPmYSP7du3g9tZjYWgb7jhhpZC0PQKqhC0X49Ch2wsAWgDs6MAdJwD+HVZ67kpnUEMEoMdQeg4Z7A95gu2hxBsLxFoNxTsLQzckQKwo8K/vnj/aKN2w7/y/tl44bVD167o5etqArAdbrt2GSYEJABt3Ah3AtBxlx0lBnlMO97BYCaRdJQYDEaySKhEoKdQsKdkEIaA2VyJwGCFgIMZ/nVO/uC5MwHEk/fPCL12TP4IpvdPoV//X6Bd3csnAei/TahHaAhIANrgbgnAJ9fNxdDeR/u0J38FYSBh4kgRg5ZXMBRC0Jc5gd48gRSBnjKC/c0GtisAmf3L5k0ABnPuH4+3xin7l+KPLZzDvwr9+vS6MxtFmpdPAtB329CWoSUgAWiDv6MATE5LbNlTTvIon/bqrxjkTgMRhF3dMxis8LC/3kBfRaC7+YB2vIB2PYAdWf4lnJI/5P3z6dVke6NI9vKFiwDcnud+DuDAbN/nANo2Bu0gbAlIANq4NZYA/GjbK0hNP1QHcGv5oTUQuXtfBKHEoI0b8V3XYHgEAxGBvpSH8SQC/fUCugsD0wPI5lwL0FUSiC8C0O78v3Cs/RcMAaiSL66fVYk+7++wjswClgD0fj8ifQsJQBsW4E4AOu5SYtAz4PYoOG3XIxhsERhoKNjbXEBfvYB2BKBz7T/eTSsEvHJzc8jYKgDtqvizN+8f+9up/dfRpV8U+m39PEv0+TeASAD6x0tbty8BCUAbfH0RgHbFIPv76x0MJEzM4wQrVBxoNnGwxaBdj6A/QtDbaiGeRKC3ULCruYDu6gG6mgcYTAEYzPl/znP/aIPukj/4m7vaf97q/hnb/nyr7VU/JP7azufr7HX5bLz+A+oqARgQNnVqJwISgDbAWgLwxY13wJoD2CfpKJ/3GAnewWCIQbtlZewIwWCLQH9Dwe68gOEuAAMJ/7ZX8kewSr9EauhXXj6fX+leN+xIAfj17vlIc7EUXFlpJYbmXOZzHUCvF6UNOi0BCUAbt84SgMt3vIjU9GTsqDi09A13KzHYuvA0mQQiCINVViZQIeirCLSTFBIsL2CgHkB/1//tqPAv6/7J+2fjJRVgV4m+AMF56SYB2D5ctdfACEgABsbN9HIWgM67ClQQBuIZ5LE7IlQcrDBxIGIwWCHiQIRgsERgIKFgf7yAwRKAzgkgK7/5bj3g75Z/owC05v7xXgYy/8+X4s8dWfi5sb4ejbXViIqOQXRCIrjwPVtnDv3ymhoqy4CmJsQkpSA6PsHjG89R+Cm8a2NwcNNVAjD4TLXHwAlIAAbOzqsAdNx1oGKQ+whEEPorBnkcu+sUD0sb6jNNu4Wng+EVbC8h6M0T6E0EusoKdrU6iKswsKtMYF/mADp7AF2t/xvo/D8WfmYL5tJv/iR/eMv8rS3ah/Jv1qFs82o0VlcB0dFI6tsfqcOOQvKAodiwYjs6W8Hn+spyVGxZj9JNK1FXUgSgCbHJqUgdehTShh6JuIyslmdVos/n15btDSUAbSPUDoJIQALQBkxvHkB3u3YWg9zO13BxKMQgz29w6hFeSQXqHQylGAwkY9gXbyBFYFNTEwbW9MKWNUXYsbEE9bUNSOuWgMFjuuFgzwpMGTSwDVN3oWA7XsD2FoD0/rGNHZpj/sv5f8HI/rUb/vVl7l/Ftk3Y99GbqCs5iJiUVMQkJKGpsQH1ZSVAVBTKErMRO2Iqjpw4zKv9h8sGdcUHUPjBa6jcvc14MmNTM4w3k6KQ3sCEHn1Q3G0MojP7mFOWp6/j7lxHCsC1u552OwfwyNyZmgPYcbc9bI8UsQJw7ty5ePXVV7Fp0yYkJSVh0qRJuPfeezFsmO8veksAvrn5FqSkJaJH0tiAbnS4ewcDySoOhRi0EyL2Vwh6E4GNDU14/uV12LqsBMl1CUhKjUNMTBRqqhpQX9eInjnJ6HFiAk47eohLEWjXC+hYC5ACkO2Y3OyWYznXAfTHAxiM+X8dFf715P2r3rMbe97+FxqqK5HQO6cl5GtB2rs1D02l+9H/xFPRY+oZbX4P6GFv506NtTUoePs5VO7cgsS+AxAdG9tyxMJdB9DU1Iim4j2ISsnEkZdejbj0bu18Rtq9IwEJQNlDOBGIWAF4+umn47zzzsMxxxyD+vp63HrrrVi7di02bNiAlJRDRZ093SxLAK7Y+S+UxH7batNgicFw8w52RjHoTxZxsITgig/24OOFeaiIq0ZyViyyUw4NtA0NjdizrQI1KXUYd15vnDzqcJ9EoD9hYG/FoO0KwGDO/3NV+oVA3M3/8yX864v3r/CD11Gy5jMk5R7mUtwV7j6IbmlRoKjKmXEp4rN6tfu7m17jmsJ8VGzfjPqyYkRFRyOhVw5SBg1DbEqa1+OXf7see/7zAhJ6ZSM6Lt5sT+FntV7Z3dDU2IiqvG3ocfzp6DZ2std9aoPgEZAADB5L7ck+gYgVgM7o9u3bh169emHx4sWYOnWqT2QdBSCzgB1bXuXadhGEnT1U3NGewUC9gv4IQWdvYHlxLV58YBMa6hqR1ScJeRUlxhb6JWe22ERTYxN2fl2K5GExyMxJRI+aFEQhCt37JGHQqExsrt1ntnX2BPoiAn1JBPFHADIBxNf5f67KvzjP/eN1eSv+HIzkD0/ev7rSIux+cR6iYmNdesEo/nr162bC+FW7vkX3409H1tG+vRd8enm42Igh2v1L30Hl9s3GKxkdG2/EGkPScRnd0G3cFKSPOsaIQndtzzsvgiKwrOGQWKToc241+/cgNi0DOTMub+UlDPTc1c8zgbUfbzIbVNdWYc6zN7Zr+NUalxQCllV6IyAB+B2hLVu2YMiQIcYLOGqUb2v5ehKAnsQgfwvEQxjucwftegd9TSJxnDPoT1mZQBJHfE0UcRSBaz7eh/89tx05g9MQHdOcSeosAukF/PqrgyjaW43EPtGIS4pBVkIyGuqbkJoZhxHH9sC+xAokFsSjuLAaMXFRyD4sHZW9a5DeJwET+uW2mJi7ZBDnMHCgIWBPAjCQ+X92wr/B8v5V5W1H3qv/aPaUOYRJCZXij40CkK2qYAfShh2F3qec7e19GvDvDTVV2PveK6jYugHx3fuYjF0rC5kikIkqTbU16D75NGSOOc54JSt3b0VDZQWiYmKQ0KM34nv0xcr/+xNQU4Wo1Cy4En7WCdZXlJmM59xzrkBsanrA562O7glYos/aYtRxQ0AP4Lm/miYBKMMJCwISgMyPa2rCtGnTUFRUhKVLl7q9MTU1NeAfq1EA5ubmgiFgZw+gp7sbCd5Bu2KQ/HwRhIGIwfYSgpYIXPvmQRR/2YCcIa1DdpYI7JuYge0bSrBnezkaGppw5OSeKE+sQXZKBugZLNxdgW3rS5CYEovoLKBfVhqaGoGq8nokpcYi+YhYXPDT0YiNa/YEuVob2NkL6JwIEiwPoN3yL/4Wf6YA9Lbyhy+rflTlb0feKxSA/RAdG9fqcbW8f9Y/VuXvQPqIseh10vR2e2mXrP0chR+9gUQKUjelWmoPFqIJQPrwcajcvgm1RftNeZeS/aVAXCKis7KRGVdhbCihu+dwdX15KRrrapD706tMdrCafQKuBJ/zXjtSALobl8pLKzG+/wXtKkLt09QeOoKABCCAa665Bm+//TY+/vhj5OQ0ZzK6anPmzMEdd9zR5id/BaDzDoIhCDvSO+hviZmuIgZ9DQv/4x8rsf3TMhw5ojnL0rFRBFYcqEPRhjrEJ0WjpqIBRxzTHRk9EpFXXoKeMan4etVB4/WLi49G91GJSMqIQ3ZquvlQYXh5264SDJ7cDRdeMLrFS+TsBbQjAF3VAHQXAvYkAH0t/zJ61CFvJlnZCf/6MvePx6ivKMXuF+aZ8ihxmd1bbpGz+GPSRNXubej5vR8h86jj2uWdzFp9+a/9EzUH9hoB6K41NtajZPWnRiAm9s5FcXkUomJijV10z4pHXVEh6NlDdAzSBnuOYlTv2YXEvv3R78cXegwpt8sFd5Gd+iL4JAC7yM3uopcR8QLw2muvxeuvv44lS5Zg0KBBHm9zsDyAng7iLAa5bTDCxeE0d9BfQejvvMFAysr44xX0FhZe8cFevPqvzehxeAKyk9vOv/ryiwKU7alFZloSGuobMWJCT+PZowAs2lWNsq31SMuKR1lRDfoOSEVcbrQRgFYrPViD/INluOo3R6N7n+a5p64EIP/dCgP74wH0VAPQUwFoX+b/dUT411vdP4vj/qX/QdEXS5DUn0kg0W1Cv9yOJVUoAnPO+UW7ZczWHtyH3S89bkKxDP26axSIZeu/RG1cGmJzRprNHMO8FIIV2zai9sBepB0xDnFpGS531VhXi5rCPPQ6+SykHzGmiw5twb8sZ8HHIzCs60+TB9AfWtq2vQlErADky5Li77XXXsOiRYvM/D9/mzUH8INvbkJqWgIyE4/2dxdetw+ld9C55iBPNifZs2ehvb2D7S0GgyEED+6txr8f2IyDjRVI7haLfkmHkj8492/1kn0or61GfQ2Q2z8Dh4/ONJ48isHPluahvroRPbJSUFVeh9jYaIye0gv55WUtIpC2u2ZNIY44qTsuOHd0iw354wX0FAK2IwC91f9zTv7gyfvqAQxW+NcCVnugEAVv/8uIvMQ+udhXUNIy74+MG8pLUVt6AN2PPRlZE070+qwGugETMna//ATiMrojJiHR5W54Pnu++BRNpfuQmp2L9GGu1xxvqKtF2foViE5KQnLu4Dbh3YaaaiP+mFXc+9SfuD1eoNfS1foF4uXzxEACsKtZSOe+nogVgFdffTWee+45LFy4sFXtv4yMDFMX0JfmmG1VGfd1my6dRRC2l3cwnMWgt+QRX4Wgu7Dwopd34csP9qJXdhIORlcY26AQZA3AVUv2oraqEfWxDeg7KgWDs3uY3yvL6rD+032oi6lHTFw04upiERUNjJnSG/kM7QEtInDPzgpE9QKuueGYTi0A/RF/vFBP8/98Df86PqjVBTtR+OFC7Nv8DTJ7ZiI6IQlgIeiqclMUOuPIY5E14SQTaq0t3o+qnd+CCRv8O8O1if36m6Xj7DTOx9vFjOSYmDZeRquES1NtFeKLtprEDYrVlIHu65VW7v72u3mEzSVsYhKTTVFra5WT5AGD0WvqD02BaLXWBIIt+Jz5dqQAtNaodz4HzgGcNOCnmgMo40fECkArw87ZBp588knMmjXLJ9PwlG6/t2pVuwvCUIaL29s72JFhYl/FoKd6gs5h4dqaBix6aRc2fMYQIlCdXIOomChkNCZj4+cHOB5j2PjuqMmobRaHyZmoLG0WgAnJsahqqkV1WQN6dEvGyIk9zTZ5ZaUtArBwVwUaujdhws/64di+zfNWnZNBnJeF8xQGdiwE7asH0J8EkDXfLQcXaPmXYHv/HB/OtUvWoLFwO3pE7UN9aZEpDZPcfzBSDx+JhD65aKyuxIFP/oeKrRvRUFGGpigW7AGi4uKMGMuaeIpZOs5OowgtWfcFknMOM7txrt1HkViycYVJ+mBGcnzGoXmLzsetLsxDUs5hpsZf+bcbjMePYeyErF7mmhL7DVTpFwdo7S36HO+PBKCdp0R9g00gYgVgMEB6q7fkeIzOIggDXZXE3yXq2tM76E+Y2J/5gpZX0FchSG8fs3kp+Aq2VaCwshxxidHok5aO3V+XYciYbib0a2UH94xNw7rl+0zGb0JyDAoLKzDiqJ7o3f/QvDBLBO78phTjTuiD5GNjWwSgJQKP7XcokckxGaQjBSATQDyt/ctzdVUA2lPx52B7/6znc91nW1rW+mWo1fHjkPX4THmWbRsRl9mjZVk19m2orkLtAdbTy0Tv7/8ESf0GBPxaYVmagv88R1mJksrm1Tsc5/dRABatWoa4jB7IGDnOo9eRGc7pw8ei14nT3J4Pz71y5zeo3psHzglkkemUAUNMWRxPdQYDvsAw6tiRgs/5siUAw8gQdCqR6wEMxr33RwC6Ol57i8L2mD/oS7i4K4hBT15Bb+Fh57CwKSZcXm9q/K0q2o38tZVY/2wJyoprkZIeh/RuCajLqEdSZgzqd0Uj/9syE/pNSolD5sh4DMg6lEhCAVhb2YCkqjhM+8UwFCQ1h4Y9eQHd1QN0nAcYbA+gNwHoTvzxWkYf5ZQV/OV2c43eBKCvyR+OzyLFH9uoo10ngB38cgkOLHvXePqslTUc+/PeVhfsMFm5/c662JZnbfVLC1G/eTky0mLNfMBohm6bGs26xHXlxWgoL0NsWhqSc93PV26sr0NN4W70/v45LjOBTSb5t+tx8PNFpragqVIZHYOmhjpExyciOfdw9Dj+B24TSILx3gzFPkIp+hyvVwIwFHdfx3RHQB5AG7ZhVwA6HzrcBWGgpWbaUxD6Eyp29Ax6qjHoa21Bbx5B57Bw6YEa/O/5ndixqRT5+0pRVliH6LpoxERHIT4xBjHdgfiUaBR/XWcKSI86rrk+IBtrBLIxtLxh4z5MPC4HP7hosNnOeXUQx2QQT2FgdwKQx3EMAzsWgnbMAnYMATMD2N8EEH/m/wWj9p/z8+ZN/DFhIu/lx1FfWYGE7r3dvik4z67mwB70+9HPkTzA/2Sydcub5w+POvYwU9y5dONXqNz1LRpraow3knP1UoceaQTogU/eR3y3ni6zhY0Y3bsL8d16IHv6JS4TPMq+WYt9i98yK4zwmjiX0WoNVRXmOhiGpoDszPUBw0XwORtNRwrAj7a9gtT0tlnl5aUVOHHQDM0BtDH2d5WuEoA27mSwBWBnF4Sh9g6GSgx6EoKWCKTXbucrjdi+oRR9ByYbwfdtwUGU5NeisSgKVRX1qK1uQGKvGOSOT0XcwXhUV9YjLTMexVFVprhvSnUC6uoacdjITGR+Lx5TBx/yWjmKQFdzAV2Vg3EWgLS/owc216FzJQCbGhqwaslnGFC+F3WFe7BnfykGHDkSKUcciU0lURg7/JDnLlAPoLP3j+fibf6fL4WfXT3mjqFfV79TjOW//rTLYtFtBvbd3yLr2JPQfeLJPr9RLOFniT+rI4VcXclBNFSWm3AsQ88xiUlgvUCWrinZ8CVik9MQl55lEkfYGKquPbgXsSkZ6HXSNOPJc24M++5+dT4YTnZXb5AeRCbGdJ90KrLGt+/Sdz6D8nHDcBV9jqcvAejjzdRmHUJAAtAG5vYWgB0pCO0mlHSEd9CfeYPBFoO+eAUdhSAH8eLCGiPsYmKjsWjlVqx5vQjDh/ZCbPyhtVzzKktQV9WIbkhpFoFVDTjiokzEJ0cjcXsyNq84gJrKeuyvrkBqz3iccuJhGDy6G1YX7zHmYa0VTAFo/u6QEGLNBXSeB8jtrGXhfAkD0wM4pnc6it5ZiLzPv0BmYiyiE5NQVFyBjDggOikZxd0H4qgLfo7o78qYOApAXxJAOPePraMEoDfvH8+lYvtm5L/xDJKyB7WaF+jqlVGVt80s0dZjyhle3yjuhJ/XjgAo0IpXfYKyTSuNSDQx3CYmpMSbRJRu46e6nYtY9vUa7H3/FRPOdvT8OR+35kAhYpOSkT3jsrAvE9MZRJ8EoC+WrW1CQUAC0Ab1jhaArk61vcLGwRaEofQO+ioGfUke8SYEVxzYhoKNNdjxZQXqd8ajtqbRzOcr3FmF6uhaZB+ZbNb17ZN4qDYgRSBbz5g0bFldjMOPzEDy0THI6p+AY/sNNPUAo6OjsL5qrwn5Htur2dP2ecHuNiLQl7mAjskgPgnADdvRf9ViVKxdiZLEDPTp17zCyd79peidlYbG8jLs37oNOSechB4/ONt4pZwFYHtl/wZS+sUX8cfr4xJw+a8/ifis3m6XZ7OeSYZsjdfsmO95fKM4hnttvHrAcK1ZC7iqwngJuX5wYu8cjwkc+5a+g5I1nxpB66kxKYTFpPtNm2k7u9nONbrr6yj6/C3E3B7n488+5QH0h5a2bW8CEoA2CIeDAHQ+/Y4ShP6uTuJvdrG/ZWZ89Q4GIgbdzRd0FoP0+n32XiGWv7UH+yrKkdojBnFJ0UhvSMXKj/Y1i7ycJKQOjTJeQEsE1tU2mizhHTuKUVlcj5SkBPQfloaqhFr0OzIVM6aNMskiRvTt2dUiAC0RGIgX0B8P4Kp33kf3Ze8ivm82Cstr0SereVUSCsA+3b/7/535yIyqRZ/zL8PXZQkeM4CDnf3rb/KHt9Cv9Uwx5Jr36nxwpQ5PS7RRhNWVFXsUTHa8fjZeUa267v1wIco2r/Yq6hjqry7cjX4/vgjJOZ7FYrDOzdt+OrPoc7w2CUBvd1q/dyQBCUAbtC0B+Nm3vzQrgTi3lISJNvYenK7hKAj9FYMk4U8iSUeLQUsI7lpTjfUvNiApNQbdejbbQ0FVsQnxbv20AslxiaipakCv3CSkDGFt3ijU1zaibDNQtLca8UkxKK+tMaViJk7JAZd8255XjF5Dk3HxFWORkhHfRgT66gV0TAahB5CNIpAeQLaj+2fDVSawWXXkLw+h+8ECJPQfZOb9uRSA+0qQUV6ItHHHIb//BL8EII+xasV2jBnT33g6HVuw5//56v2zzqFk7efYt+hNxGf1cp140dAAll1JOewI9P3hBS49cMHy+tl9Ixz47EMc/OIjJOe0nR/ouO/6ynI0VJYhe/rFSOjRdj1ru+fha/+uIvpCJQDf2/o6UtLaJoFUlFXg+4dNVxKIr4bYhbeTALRxcy0B+PXu+UhLb16T1WoHq1e43HOoRWFXEIShEoOesogbG5vwlz9/hT1f16LnoDj0TMhquf+NDU1YvigfddWNSE9KRF1tE0Yel2W8eqtX7cX+bTXo2SMZ0THRRvTF9YhC31HJzSuH1DYib2sZMsbG4ZJLx7fs09ET6EkEfrluPUYcLEb9/v3YdrAIQ4YOQ+LwEVhZXN5mHiAFoBGDA7NbkkAaKiux/u4/ome3dMRmdvMoALNQZVad2D/5PIwdcSghhHMAXYWAi4qqsfnrIixevB21tY3o1SsdQ4dkYNiQbujXNxnrVjTPC3SXARxI8oev3j8LtEm8WPI2Std9YcqymPIscXEmi7a+rNjMw0vs2x+9T/sJ4jObV3SxWrgIP+t8qvfsQt4bCxCXlulxzeGqgp2mEHbfM873OvfRxuvTZdeuKPokAINtJdpfsAhIANog6UkAutttOArDYItCO/MHA0km8VUQBuIZHJx6hMtb6SwG9+yswr/+vAXpWfEojWuuzcdmCcG8LRXYtqEU0ekNqCxqxLBR3dEzJxlrl+1HRV0N4pNjkIR4VJbWY+j4bqhKay7/QhFIUZhfUobJV/TFCUMHm3+nAGRzNR/QJITU1+OIdZuw84MPEVVaim4pKSiqrAIaG9GjXz/sGTwcY846y6x64WoeILOA2cb0ymgWgN27ITYj06MA7B5VY5IRDkz5mVcBmBCfiv99uAvFxTWoqapGnz7pqG9oRFl5HZKTYjHh6N5Ija3GUWPdF1f2VwD66/1zFIGl679A6foVqD1YCDQ2oSmqCbEp6Ug9fAQyx05us4RbuIk/XgtF6553/23qACb26Y/o2OZpBY6trrQIDVXl6H3KDHNtHdE6WyKHHSYdGQKWB9DOnYqMvhKANu5zIALQ1eHCTRSGsyD0lkzS0WLQEoJ5G2vx6RMNyB2a0hLG3FNd1HK70xsysOHzIlSW16O6vhbd+scjpVsM9q1rMKKRHsR9+yuR2iMWR0/oZ5I9rOSQvgkZ2LWlDP1/mIL+49NwdI9mUeRKBHI+IOdwrX76GfRevQ7R3TJRGBOL7IzmpJP8omL0qq3F/r2FqJ8wCWN+9jN8tT3fhIDZnMPA4/r3wZp77kKPplozB9BTCDiz+iCSBg7F3lHf9xgC/t+7m7FhYyXq6hvRp3cy9u0pQe8+zXUOGQ4uKak1QvDwnDicM2Ok2yc0Vpn7LwAAIABJREFUEAHoruCzL68B1vtjiRSWU4mKjUFCz35thB/3E47iz7q+urISFH7wmqkzGGNKyWSaVUW4vjG9mUwo6Xb0VHQbN6XdvX9d3dvnyqYkAH150rRNRxGQALRBOlgC0N0phIswbG9B6GtCib/ewWCKQW/JI+99vhHvP1qCHgNi0TuldSjQEoKVxQ04uDEK+/Kq0WdgEhpT61H4dQ1SUxLAMHFmzwSkDo4yiSNWgoglAht2x+CEs3NRP7J57WBXItAKBR9VVIktf3sETRkZ6Nu3H/JLSk2f7LT0lv/v3diAfXv3Yegvr0PCwIEtXkBX8wBXvfwaope+iz6jjsTeg+Wu5wDuOYCMqoPoNf0CfIuebgXgmpU78O23tdj8dTFyc1ONyCgsKG4RgNazULivChWllfj1r8YjKfFQsWLr90DEH/vaEYDeXhXhLPwcz51z/FhsunzzGtDjx3WCWWQ6se8ApB8xBimHDW838ReJos+RfUcKwLe2vOV2DuCPBv9IcwC9PdAR8LsEoI2b3N4C0NWphYMoDFdB6Mk72F5i0AoRV5TW4x93f4t9lcXI6NUsWHokdG91CykEq8oaUbQB6JGdZJaH27KxGOl9YpGZHY/B/XuY7OCC78rCdGtKQ1FhNcoO1iIvrwyDJqXh3AtGYXdKkfEQuhOB8Qv/g94bvsH+3r3M8ft9J/woANkoCPulp2PP6jWoP3Yixl40s5UANALTYR5gfXER1v/lIWTVVqA4pTv69Gj2JlpZwPQ4Fq5bj94jhqP3+Zdh3Tf73QrAJR99gxVflSElNQ6pKc0hSFcCMH93EfYfqMWF5w/HqOGH5lPaEYASf63fKFzphCFt3r+YpGST6OK4DrKNV2ObrpEu/CwgEoDBtCrtyy4BCUAbBEMhAMNRFAZTEAY6f9DfzGJHQZiTPMqlFXibM+joFaQQ/ODVPVj6ZiEGDEtBTGwUCr8LATsKwYIdlahJrsKJV2Whb002Xv7rVsQnRaE6rdqcQ+/EbiYMundHJb7eXIT6qkbENcWaEHFyTgwS02Iw9qi+SD8hFsndYluJQM4HbKytxfrb5qBXYjJie/RAfmmZEYCW8HP0AvasqsK++kaM/sMdJgxshJ9DNnCr1UCWfIa+Kxdj35atQHIqevbrg/0Hy5AV22jWqC1N7o6RMy9DfK8+HmsA/ufNjVi9pgz9+6e1iA1XAnBvfjGq66IxdVJfHD+xb5v7448HMNC5f76+GjqL58/X6wnmdhJ+rWlKAAbTurQvuwQkAG0QDBcB6O4SQuktdBaFmYlHB0TaWRAGGi721TvoixgcmNJ2cjzFYNmBBnzwzwpU7E5Ev4FJSEhqXqaLQpAh3sb9KZzkhtN/noO6ocX/z957gMdZ3fn+n3d614x6L5YtW5ItdxtjMCUBEkqAkE0Wkmz6buryv5t9/k+y/70JuxuSvcndDdm9NwVIlp4CZEmAJBAwmGKDe5Usy7a6NJJG0vRe/s8545HVLVk22HiOHz+WZ855531/552Zj76/Jp/b9aSH/u0K5UvNDMXSrtqR7iietpSEyJguRsidpKLSxuJVDjpcI3h6Y6xsKqboZj16i3oCBK4z5zH0v/6NwWiE0rJyCYBiZFRA8XMGAkUs4FAwRORzX2R9deWsbmDRDaTJrpfFoDu2vUaeJsWoJ0hBRSmWpnV0aApZvbZevtZsRaDnA4ChaBoAr9y0cAA8X+pfFv6mvq2z0DfzR10WAM/qayC76DxZIAuACzDshQ6A013auwWF7yYQzkcdnK8yOBkGdxw7yptPBXEej2FMWdHqFRIC/hKQsodZ/UETH7h6xZj69Wb7cbY/4mbwRJTqily0ehVvv+aUvX+tRiNBXxzFkqRsuZHKvLRLucvjZqQjwm13LSWxPCYfE+5gmRSSSFDz8NMMOPtJFhVSZrFNgcAMACpOJ8WLaui8+Y6xmoAZBVAec5wbWADg2tp0osi+QydZUWjhyHEnmKysbqzhYHM3q5aVTwFA8cD4MjDbtraxZ58Xq0WH+Qwu4FhCzS0fqGJFw0RX+oWi/mXhb2b4u9g6dCzga2BeS99JAPzvY3+cMQbw9roPZmMA57Vz783JWQBcwL5ejAB4oUDhuQDCs3EXn0sYHO8iHg+CiXiKrXta6G6O4h9NUWorprLORF2TjfZUuradGJkOI57hKE/86gh9zRGCPWoGu4NgSKLRqygptlJZZ2VUFUBJxClLqUU9D5r7QmjzzPzt/7eB/e5T7ttTEKh9bTtF23biKikU1abHIHC8ClhqtjBw6BDVd32Co+U1MxaFnuAGPgWB+1p7WbM4DYP7W3pYXVc+ZwAUSSBtbRHaTnioKJ85CaT16BAlxTY+dWcdJtPEciXzBcDzof5l4W/iJ0lG9cuC3+xfKFkAXMAXbnbpObdAFgAXYNL3CgBeCFB4PoDwTO7iucLgXBJIJsNgOJig7bCfV944gW84iUarsGZlFcvW2HDog7S07JFqnWIxsqxpC4pWw2utx3n1/hEG2iJUV+ViFTUFNX7UsRDVw0OYeluIeURLuRQajZUTmhKqv7qRq993OTsH0nUBhRK46/B+DI/8mvyEwpAjhzJrzkQV0O2hYGQUl1ZL9C/vZF19I5n+wJmagJls4Ew9wLU1ZWRUwOkAUCp9p1RA4QIWY1V9WhEcrwCKNnBWSw6/e/YEiSQUFRonlIER8z3eKF0do3z4Q4vZsLZoyu35bgNgFv5Ob0nW3Tu/L5AsAM7PXtnZ59cCWQBcgH0zAHjw5N9gPdUKznABtH9bwCXNunQ69/H56myyUCA8m9jB8UA4U8zgmVzEAgRdPQle/VUIX5cZtVrBYFIJ1sPT30WF7ygNZg95tjiKSsGbDKIqy0ezqZGG6z7Ebx/oZs/efvKq0t1E1EEPmt3PYxnpRau3kjBaCcRjqMIhdL4gpqZ81B+/gdWbr50AgXte2Yr+2RfIi8ZxmY2UFJfS7/Wh+HyoXCMkHXbqPvUZDhnNrC8rnwCAEiSnSQaZ4AY+pQJmFMDxACh+nikOMNMHuLllhJdf6cbrjRINhykuySEeT+LzRdHr1RTnqrjzY8tRT2oNJ449VwA8H8kfWfibCn9Z1W/un7hZAJy7rbIzz78FsgC4ABtnALCj935sNqM8kie864xHfC9B4jsFhe80EJ4tDI4MRvnlj3tpPeGiuEYtlT+7rgCNx4V1759JDQ4yqujJb7DT0FhCMhwhOjCM2zeC5qqVDBqu460/j2CoiaEkEpTve5m8YScj1lxSag1mjUHeX6O+EMl4ktXVSbwWNbFPfZg1jesmQODunTuobemgd+9elGAIh8HIiEpFxbr1nKipIVlSghJP366TIXAmABRzRSxgRgU8WwAUx3G5QrS2jfLatg7ZSlHA8uJFdpYttePud9E0QxeQ+QDguXb/CgBcvmHRGd/j7/UJWZfv2e1wFgDPzm7ZVefHAlkAXIBdpwPAMx3uUgDE8w2FCy07Mx91cC5u4vGq4NEXinj5GRc1y9JAMxwZlq7e4l0vY3MNkbA6CIUhSIolV1jQmRRydfnE3T5c/d34t2zhhTcKqS7Lxx7owfjWfxOxODAY0p0y/LF0uZiER4WuEEobjdg7Xbg2ryL+gS2szauaAIE7+7tYoxg4cKxVrkvZckk5HGwoKWdXdw8bSsvZ1dXD2rx89r+xnaUGPccHR0jm5kFROajU0yaDzCcOcLILuGn56T7BUi3c08GyhjI0agWNRiXP89Dujml7AAv4E2P5mnQ3FFEyJzLQIzt0pOJxVHo9xvJadLkFzLfv75neu1n1L22hLPyd6U6Z+fl3EgCfPPoiJqt5yskEfQH+Ytn12SSQs9/G98zKLAAuYCvPBgDP9HJnAsSLUT28kIFwITBYqG+S26k65aYM+BJ871sHiEVT5BarsGnThZgNx/eTs+NZErEwSiqFSlETjqmx15VgWGFDsaRVPVN3CP3iKp5JLKfzUIzL40ew9rYwkpPOgjVp0irz8GhAVJNh5YZigqYQDI1QYHbQ/fkPg9EwAQKT8TRQiRqBolOIaBUnewVDWv1LJNDs3odm/z7yAgFGA0FyjQZGIwmKly2lu3Y5a6/eQiYZZHwcoDiGSAY5UyLIZAAU68ZD4KF9nTStnAiFswFgBv4iQ/0M73iJUM9JkpGwzKwWQKg2WbAsqmdAt4gVV6w401tuzs9n1b8s/M35ZplhYhYAF2rB7PpzaYEsAC7AmucDAGc7nfcSHJ5PKDxbd/FcYDAcSnL0YJCtr51gsDclt6tucTmrNloxmBT+64d9FJfrCCgjafgb7KXs9T+jHXGSsNhJabTEkzESgSgWHdgr7VhXL8NtipH0h7D4NNi/8Ff86cUY0d8+ijXuRldULpJ58YRCRIMpzHo9tloVjnIthQYH/S4nyqiPpf/4DfarQvJ1M0rgWHmYaSBwfVEpBx5/gsJ9+xlJJCmuqqY/EKI0x0Z/nxOlvxdFpSGxaQuppo2gN6A65TKezg0skkDEEOVgZqsFmIkDzNzrZwOAAv4G/vQbIsNO9HnFqIymMQBM+L1ER4fwq3JZ8Zm/RmO2LuBdnl6aVf/SdhDqXzbm7+xvpywAnr3tsivPvQWyALgAm77TAHimU50NEC905XAhQBiLpWg7GsHZFyOZBLtDTf1yPWaLmrN1F08Hg8NDMZ56yEXr4SBqNVhzNFKJcw6PSPjILwVPr43KWgM6vQqVz43uzacw9HdBNEFcY5PzFZUQ3lLoLQnMSgi1yY6moRG9WU3Y1Y7lazdTUbuFA9+4n/4DbXg1DnSKXhaG1jgSOMq0VJcW4Ayli0nnxbQMDTqJf/UuVi1Zw66hrhkhkIQin1PiKtRtxyl9/gVcKlW6aLTHS4nVivNkOzleN/6BQfSiXRgKuZs2M1ixlKYPfoCDTt+UOEBxzPHlYMZnAgsFUIyVjems4PkAYMbFGzjRTHR4CGfPCJWrGgieaCE80I2xrAZFlVY5x4+BzkGsKQ+ODVeTf/n1Z3rrnPH5rPqXBcAz3iRzmJAFwDkYKTvlHbNAFgAXYOoLDQBnu5SLEQ4nQ+F0GccH94Z44Tkvne1R4vEUAm8EjBUWa9h8lZlrbrCi0aShR4yzUQcFDIaCKZ78WYrjh6BuWZEEvPFDKIOHDg/hHUlR26BQUV6CoXUvugPbCXhi6P2DhNTpeBwFFaJWoFanQqNPoIsEcdvLUYx2HAkX6ptWU7SpgERrL5q3e0hVVDEc9qDRK5TaCxA9hcUQWcISArsHKKheTO8nbwCthlWOmjEITJxyAWeUwIwrWJCo/qnn0BxuJllRmT6vRApVTzeq7m7ZsUQofoJ0FbcHS1UVvjgUr1xJ76qrBP2OqYFn4waeiwu4YVk+rm3P4287QjIcQqXT4x72YdZEiQz0YSipxFhZS9w7SnR4kGQ4KPzxaGwO/FEd9lwjKrWa8r/4mwWrgJciAIpewcHu48Tcw4jfrDTWHDr6Uqy4etUCPjUv7aXvJAA+0fzyjDGAdzW8LxsDeGnfiunP/JT4FTs7zsoCFxMAznSBFxMYTgbCln1N/OrhESKRFKXlWgzGNJTFYykGB+L4fQne/0Ebt300RyZkTB7zgcGdb/h57H4XuZUetLr0kUzaiS3KBATu3OZDbQrRuC5F7tYXCDjjMiawKNZFTGMiqaTPLx5Nxw4KkLTpQiRVKYgnScSThG3lFFfoUdsjJPpGcCypw7CofEJv4QwE5ikWBo4ehTveT9NNt7PHlS40fSYI3NV2lMpHnmQoEqGkvEKqf4XBEKN792IymwkoKqwGPf5QBGMkTMhgonDtegabmyndeBnda97H2iUVU7KB5+IGnosCePCtNgoHduFr2Ysuv1jG9QmwH+wdJTnSh2akA0VIqUL9U6nkz4pGI8E1GYsRiSbIqahEUakp+dAnZUzg2Y5Lzf0rvhJ8rQdw73uT6OigjKsU7x7xTeELQNXVV2FfdQUqYe/smJcFsgA4L3NlJ59nC2QBcAEGfi8A4GyXPxMcXgju5JHhBP/yzy0E/VBelf4dRqtJuxgzw+NOMDwU5/NfyWfl2nQCxUxjNhgUX4A//sEAJ1sjVC/Wjx3CGxsY+zkDg4d2+/F6ElSXJSh8+7/xBBVUFi1F3m60iQhhxUAkLNzAKbQGSMTAqgQwJQMk9UZ8RWW4dYXo0FPfqCF49AApdwBNUzUFSxsZiqTdvvn6PBKhMIPHjlG0eg0Fn/sYB6JDNNkXzQqBmaQQZciN8YGHSJltYDJBIoW6uRnV6Cgpq03UmpZDEf8GA1INLLriKpy9TvJSMYauvpW1mzdOAEAxf7IbeKZi0GJuRgWcLgbwwDMvYG7biq6gBLV+4t45Dx5C6+kjEQ6QjEbQF5WjNkyc4x/xYFAnURmMVNz5VWzLVp71O/1SU/88h3fievNFCc9aRz4qTboTSyqZZOBYO0QCVFx1Lfmbr5dzsmPuFsgC4NxtlZ15/i2QBcAF2Pi9DoAzmeZCAMNXXgrw6H+5WVavG8vCDcX7J5yyAMLjrWFWrzPxua/kjfXfPdOWT4ZBfXIN3/uHXqki5uZPVT3Gg2BgOB/PaBxtKoLl5adQa1UkTVY0kVFy/b0oyTiRlBGtUSsTOxKRCIaoD5M2Srx0EdHiGoKpCN7hJCW1ahobihh9fQ9BrxtNRQE2RzHuuB/CUVCrKGpazcDNq1AcVhpti9k70jEtBI4vDyMgUPH5Mf7kIVlbEJsd/AG0+w+Q0utBo8Wq1+MLR9Iq4JALS2ER3qrFFOdYGTx4kOT6q0ht2DJtTcDp+gLPFgcoAFAC4alMYAHc+/7zPmzRQYxl1VO2y3m4GZWzlVQ8JlUprc2ONid3yjy/awQtUao/9XfYV24607bP+PylBIDRURe9v3sIkilZSme64TzeBSEvS+/6LObqpWdt10txYRYAL8Vdv3CvOQuAC9ibSxUALwQwvO9/D9PWEqGm9pQ/dtJJZWBwdEQhGoG/+1aEsuLL5r3bAgZDAbjvHhUaNRQXlc56jPaeAcFP1K1I0fZvu8kNduPRFkrYs6j8GDx96BIhNKdKxyjxKEo8Tjgnl+SSReh1Nnn8oD9BlDCNV+qwq+wEmk8SWVGMQgqHOhfFkcuJXDOJ8lIay+ro0vdJwJ0LBMp4wP4ulrz4Jn1vvEnx8uX0d3ZjaTlKyGDAajRK+JMqoChb4/GQrF1CqqCEEruNgcNHKL9yC52Nm2cEQHENs2UDz+YGTgT97P/f3wG1huJFE8vDiOMKAFGOvy3tpIgYRbUafWG6N/H44R8aRqdOUPbhz5N/+XXz3vvxCy4VCBzZ+wbDb76AsXzRrL8whfo68KkKUDdey4rL6xZk20tp8TsJgI8c3orJapli3qDPz18tvzYbA3gp3XgzXGsWABdwE2QB8MzGO19q4b33DDE0EKe8Iu2emmn4fUlGRxN8+Zv9FBSdDnedTwu7ZDLFj/7XEEdbnVScagJh0EwPgieOhVm+yoS13MnOn3ezdHAHCUsx6AyCZ+g6EUIXC6AngJJMoA/4CKZ0xKrrKSpUEU4E05cSNcrWaIsvT6HVK8SO91N6xZU4r2+gdUeYoQMWPMNRRiN+tAaFVU1l6FeEKF6qY3nOkilKYCyedtWNLw+z57VXKfvdiwypVShJ0Bw8RMpklokfYy5gnw+TxYJvcT2lBfn0j3pRujtJNq2Hzens2kxpGFUsbd/ZsoHHZwKLudO5geN+L92P/AcjnijF1VPBznmiB1X7LpKxCIpaKwHQIIpWjxupRJyAaxhLQSH21ZdTctNdZ75ZZ5lxqQBgz29/QXR4AH3BxPjWyaaJ+dwIG/sq34eiS7vfs+VhznyLZQHwzDbKznjnLJAFwAXYOguAZ2e8cwGF/+ffRzhyOEzt4ukVwMyZDbsSiDIx//NfCsixpyFoLtnFk69s+zY/jzwwwqI6PVqtQiDmHJuSgcFIJEl3R5RPf7mAYCDJrx4cYElwK8budhJmK2pzIT0dUWKxJDq9gjbowuhzcUKzlILaPEps6Rp+AgLDwRQ6lZEN19jRaFU4jx0lXFpBm+Fm2o8GSJiDWHJV5Bny6BkewetKkG+1U3EdLNlsmhUCM5nByZiCduvraLe9JaFP3d0LiaSMCbSoVARG3aDRkKhdAvZ0MWolGifXO4Jr8/Ws+cANjC8MPTkbWMyfqwo43g0skjh6HvtPhrqcFNfVTrnJ4gEfg2+/jiYVIxkKyBqABqEACpk1lSQRDpGMRolqrDhKizBVLqbklk+c3c16atWlAoBdv/4JiYAPXW66iPlMIx70kwgFqPiLv5Eu+Ex3kMz8LAxOb7ksAC7obZhdfI4tkAXABRg0C4ALMN6kpdNB4WzJJq+/GuShB93ULdVOm+GbOXxrS5TLrzTymb92THuyc4VBkVH80x+5pNu5dmkaAsXIgGA0Cn3tDpavMvLZrxbiHk3wo3v7MWriFPXvhe7DqKIRfEE1Qa8aoy5JTNHRF7LRb6xi46JeLNrTiQyjrhiO8hiLmjSyo4jnaDd/8tfTr6ugflmRhMIh0WYOZCu5wfAobmccq8pGw10aiuv0ROLpeMVMYsh0mcHJKGj2HED79j5yjp0k2N2DzmohGk9INTBZVkaqoJhSe45U//J9HoYVDYk7PjWhMPSZVMCZkkFmKgcz8tZWhrf9Aa+2gOIS+4S9S8aiDLz1BnptirjPi9poSmcCpxEVlcGEvqAYQ3EFA4cOU3XdB8nffMOCb9ZLAQL7nn2UUF/nFEV1svFinmFZ0LLiY1+ckqSThcGZb7UsAC74bZg9wDm0QBYAF2DMDACebP8CVltaiVLr5h9ntoBTeE8vnQ0KRYbvD747zIgrQU2tZtp4JddQHJ8vxZfvzqVxxens3dmMNh4IJ7uJRaHpx34xwvHWiFTwcnLU0lMqzsUfHmVJA3z4r5LUlK+TL/HLX7h442Ufi5boMITdaJyd+Ae66elQCOsK8TsqGen20qA/QV2NUP4Ccl0yYiQaSbFyoxUlx0MqnsB5SMPrySswLdWiNynYdekA/ckQ6DweZdOVlVTenpDPzwSBkzuFrMspZN+LL2L45dOoe3pl/B06newFbCkowCPKsMQTMkEkcc0HSNY1SNfv2pqyWVVAoQCKkekMMtdkEJGM0PebB3Cd7Kagrha13oAizunU6N+7j9TASUyl5Vhq60mGgjJLVZSCEfXqROaqiCUcau+h8bNfkTUDFzouhXIw3uY9DGz9HcbSqlkzfEX7vZyVmyi48oOzmnUyDIrJl7I6mAXAhb4Ls+vPpQWyALgAa2YAsL/vfmy2tHoTjLw15yNmYXHOppITJwPhkYNqfvloBZ7RJCVlaqxWUQ9OIRxO4uxPu35vuc3KTbema8jNZ8ykDAolUBSffnt7kKGBmDxkcamWjZvNNK024lUOyscSCXB1NPCbh0dobxO19rQUFKXVypNtYfbt9hCPweKCEJvCr6C2WUloDLi9QVknsHaZg9plJpGFwUBrC3tGl9BjvpzFK6wMn1L+poPA9n4XsUiKu7+1jg5N71hSiDinTBxgRgnMQKAsDRONoXv+JbQ796ByDqFEY7L4ryRckUUjwLSskvgdd7L6+uvZ2953OvbvVHu4c6UCNiyy4T96gJHtLzF8aD+iaKLBnoO+sFxmporsXxGn5ut3Yisvx1RdN2V/RW/gsLMHv7kSzcr3s2L9VFfyfO6HzNz3OgQK127vMw8R845gKKqY9n0j2uwJ2C658U6M8wTrSx0I30kA/K8Dr86YBPKZlVdnk0DO5gPgPbbmkgfAH//4x/zgBz+gv7+fxsZG7rvvPq688so5bfN0ADinhfMAxSwkzmxRAYRtrSpeekFHW6uaSDhXfmGJHIbKKi1Xv8/MpiuMY2Vi5ro3k+dNpwqKxJBIOJ30YDAqY1+UohvJ7reCvPzKCXo6FZlB7B2xEA2nMJpV5NhV0n0r1vsjXtRKgtLufRQMdxHXGtHm2alYbEJb6Ke724DPqUNj1NNiriKsMtPQcDo4X4DgZAi0pHJp63Dxga/ZuWJ5I/vd7dNCYAYGM/GA6j9vR7f1NRKFRZQWFOHs7UMZHkUR8KeoSWnUKPEU1Xd9nGOli1DGQd9CVECRDSxGxhV88NU9JHb/CWtsRBZ/Fps5fLIDfdhNMhKSdf3Mi+qxr7uSnr4QiWM7sWrCqEWMpagFmEwhEhQEvFqWLKfg6g/RcihdHmj5upqzvQUmrHuvQ2Cw5ySDW39HzDuKzp4vbSveV4lwEKHMCpU1f9N15Cxff07sOR0Uyv3atOScHP9COkgWAC+k3cieyyUNgL/+9a/55Cc/iYDAzZs387Of/YwHH3yQ5uZmKivP7DJaCADO9dabi6J4qUOiqBvXfjJOW8dhWRcux55icV0Cm/XcuuNncw9LhS2W4uknRnn9lYDwnFJYpCGuGkLEBzp7IeSzUb/CyA232lnaYJBqYOuRMC0nj5La14qlexRDws/R4QIOD+QxEtGgMuuxlJXhdKkIBZNUrkhQt6R4DGonQ2AkmCTltnL5lzRsqm+QACjG5PIwE+IB3SEqH/4dA95RSqsW0evzooiWJVL2gzKrTXYKUZwDskh0/GOfku5hAYHrqsvYe7x3qho4z4xgAYAiecP59C9wHj5CcWOjzO7NjIGuQXJtWqKDfeiLSin9yBfSyQdb91BmcONvPSCTEkQiiL6wFOuyVZhrlsn2cWIcfvt4FgLn+qEjEpEGenEf3EGw64QsuK2IP1qdVPxsy9fL+n/zVdXn8fJTkkrGr72YwTALgPO5C7Jzz7cFLmkA3LhxI2vWrOEnP/nJmJ3r6+u57bbb+N73vndG278TAHimk8gC4vQWmuwuPtfdSzIwOD5O8M9/8PL0Lz2UlGmw5ailyiegTXhS4+ohYhFw9Tm45aMObvjQxMQG0W845g2z4xETW7dGMegViqpNxBxR+UU73GnjwE4/JpOKwto45XUqcnRFU9xQzzn3AAAgAElEQVTBLccHqCot5FP/7yKOhdtZaq07IwTWnxzhxM9+TqKiPF0C5hT8CQgszcBfUhIuqv5+lnzxKxwx5ExQAcUuyJjA2rI5tYcbHwuYUQFrVC4GnnsCr8ou4/2KinNOA2BfugNKYaGVcPdJ8q+9BfuaK9Jwt/MkDU2lsl+wSAZRG83TwomAwHOlAsrX3X4sDZUbTtUGOtOb9SJ7XvxiFRt1EXW7ZD1IjcWGvqAURbTfexfHTIqh3IsLXDXMAuC7eONkX3qKBS5ZAIxGo5hMJp588kluv/32McPcfffd7N+/n23btp3xdrkQAPBMJ3kmQLwU1MPzBYPjFUElvoHv//MAAX+SgkIN/b0xerqiiDqEQpUUxaFLSrUk1B5y8+Gb/7wSi3ViG623Dx/ksR+lyLHkk1d4ur6hJzok3c3N2/WEAgm0OhWL1sUpLyqW25+JCTSl8ug+EaTp9hS33byKw54T8vkMBGZUQPFYxgUs/lW/sRv1c6+QrKkkNab8KRNUQLFG1ArMGxpicMs1pOpXyWOPVwHn4goWySBiiKzgyRAY3/YUOeEhDGVVDPa7JwCgWDPQl34sMtiHNsdB+Z1flqAoAFB++a+pmvbtIGLWgp3HpQu5+8QQSzatkh1GxieVnOl9NN3zog5eqLeD1he2kvIOUlCcI/sWWxY3YiyrGWuhdjbHzqxJREIEO9uIjAzKwFIBYaaqOnSnyvIs5NjvxbWzweGFAIjvJAD+fN+2GWMAP7f6qmwM4HvxDTDPa7pkAbCvr4+ysjLefPNNLr/88jGzffe73+Xhhx+mtbV1iikjkQjib2YIAKyoqGB8Esg87f+uTp8NDt+rYDgeBs+VKihA8OAeFQ/9REd5eREth8MMOONoNApGkyLdtdFoWg0UibVGW5gvfTPJlivXTtj/Z341wkvPeShe4sWsm1iIV0Bgf2eS7hY9AV+CxQ1GihvCskSMUGp6h124epKsWlPK8r8MY7CoWGxZNi0EvtlxgtJwMUdG+zDmqDE1N1PwpzdxlRVSarLT6/dMUAHlSSZBSaRQdXUSu/FWEo0rpOK3rrKMPSencQFnYgRPuYLV8XS85Ex1AcU17L/3f+Kw6cdakE2GQAGAYuSZQZSCqfzk36bjBE+pgJMhMBEMMLz9RfwnW2RtO1kmMAVeTwSVo4hlt98hQe1sRiIUZOi15wmcOCILInu96azrVCxETp4VU9USCrfcLIHtbIawh6/1AKN7Xk8rcKSkGzaVSqIxWWV8Y+6Ga2V2dHbM3QJnAsTMkc6XkpgFwLnvVXbm+bfAJQ+A27dvZ9Om031C7733Xh599FGOHj06xfr33HMP//RP/zTl8YsVAGe7vWaCw/cSGJ5LGHz15QA/f7CDYEDB2WvCkauWADh+pEjhdSelKvjJr4W47ZMp7IZ0yRhRHPqbX+qWqqGIH4wmg+Tn5VBUqiMnVy1dmu7oIANdKdoP6WSZmPKGOBqtgjZhw2BSY18S5IqPGFhetpwWb5s8bgYChQq4rbWNE28F8Tfr6R5yk6MxE1CHuaw8ie3AU2gqC8BomKACllls9Hl9aQD0+1H8AZb9j69zIHa6A8h8EkLEOSmnoLCpJo9D+47LUjNNa+ro/cUPGeoZpHhJ2qUqAFCMya7gVNBLrsNI5V/djdogMqXTY7wSKBIWBl54ksCJZlnUWG2xjbmFE5EwQ63HUQxm6u/6DMby+UFgMh5n8OXf4mvZL+MNx5/DYPcwqXiElGeQkrXrKL7ujrE4xPl8nHuO7MH1xh9lKRaR+ZxRKwUYxv0eRB0+69JVFF59Cyrt7MXQ5/O62bnMGn842T7zBcUsAGbvsAvJApcsAJ6NC/i9pgDO90YcD4XDwwpHDmvw+RT0xqVUVmupq5u9KPN8X++dmr8QEBQFn0W3kT07Q/z6CS/Drjh6U0gqfWrVVPUnmUrS1R7jxtty+OK3B+QlKuFV/Ornwzz12LCMFxSuYaGXhcJh9DoDJeU66pabJOwJJdDVnyQ4bOOya3MYjgxgy1PYtL6e8hoDxwNtVJsb5HEFBAoAFOOVg0dpflLDsfZh2UFEZ9NKRSzoTREcirKidysVtmF0K6plIoVwBZeZc04nhIiewO2dlF22iRNXXYPiDaJyOlFF4jQuquJwXIOKtNt6rBzMJBUw0yKu3q7Q/KeXofMouSYNo+4Q5BZTYFYYbGlFVVJNUWE6/m86CHQePIJSVMXqL909JdYvA4GlyU6GX/+DrP83HSAJkBo41IwqJ5+VX/n6vNy1gY5jOJ9/HK2jYAL8jb9fB9qd0i2sabqOlbfOrwh1zOum579/Lus/6vOm78ghXMOiFE7h+27DtjTtjs+Od9YCc1UTM2clYDELgO/sHmVfbXYLXLIAKMwikkDWrl0rs4Azo6GhgVtvvfWiSQJ5p2/wQCDJM8/42fFWmJGRpISIeHwYkWxZW5vg1lsjLF2WuCgLYs8HBLu7Yrz5WpA9u0JS0ROxf/v3hiXAVVVrUWnTRZ0nQ2A0kpRdQupXGPjOv5fKpI7/+I/DtO63M9Afl5nEIoEkM3zBABG/nvIaPQ0rTSgqhePtgxTkFnD3P1el6wr6W+T0ctNy+e8x37EJEFimLOa+7+1nuCdGQ30xg1E3Bfpc+kNuigwO+gJurCdcFJ54gWW1JrzldlKqUzGIIiYwFkPdO0BBaSk9l21C3dmNuqUFlceDw2DAHYmRyisi0bgSlq6UiSTjITDTIk7CYVszvP0SOUoUT1wru4nk200Md/ZiUcWIe92E8mtQmW3TQqCo7ydiAAOLr0ZVvowVq6Zm6x968yixN36F3aKgz0/HSU43ktEIkaF+gjVXoSqsmXOCyMCfn8Z3dP8Z3ceh/i58SRuaNTeNgepcEkbcB9+S7mXhnlaUmRMuws5u9CUV5F/2fsS1KBot+ryis1Ic3+nPkUvx9QQwhqMh7nns785r/F0mNv2BPa/LPt6TR9Dv5wtrrzyv53Ap7u/FeM2XNABmysD89Kc/lW7g+++/nwceeIAjR45QVTV9QPn4Tb4YkkDO5U0ZCiV54AEv27eHKCpSk5+vHitHIsCwqzuOw6HiU5/ppKEhHROVGReT6zgDgjPFCB5tjvDIz904++PkF6ix29UyDu+3T/oYHk6Qk6OSCR9aw0QIFHMGnXHZRk5RpVh/mRm1RuHQERfLloNnKJejh8PkFqRdvmJEEgFZhzkeNrBmkwVHvoa25hAbbglyx0dOxxAKCBwPgGKtUAKFCti2M8KuJ1ToyyMUmfNxhkflsePJNFwkkyp5/pG97azVHyRPFwAVEgJzVVpGIiFSRSXELtuI9u3dqDo7wWYnZc2RsKeEoyijIyjRKBXXvp/2urWsXVw5tUPIKzvI2/UCoy435BVRXJCDc8BLcUFaKXV2OTH2pmE2VLgYxWihuCidLS2UwFQkhC3lw7KsiaIPfpQjB9P1/SZDoOhS0fvUz/HGTRKKCoptMgZQuH7FEPUC1SZR2w5Z5sSx4RqcmtOZvLNlCgs7dT16H6Jfsc6RP+vbS9QjTCUSVH78q7JdWiZrWCyaDQT7n3+cYE/7rO3YRCygaNkmYNhQWCpVW+Eu1trzsC5twla/Nl0XMTsuKAsIBfAvvnLzeYWvLABeUFt+QZ/MJQ2AYmeE+vf9739fFoJevnw5P/zhD9myZcucNu1SA8AX/xzkoYe81FRrpHI1eYgvx7a2GBUVWv7xHx0YDOk5k+MJBQx6PQlaW2MIqNTpFJYs0ZGXPzErdk6bcJ4mzaQGCsC7738NMzgYZ/ES4UY9Hef3+rYgrS0R2QXEZFKkWxxVGgJTKQv9PXGZCGIyi44lAlwMnDgexedJUlAapWpxivYWC9FoEpt9IgS6h6G6xkpOrhadXsXNX/GTX6pQbFwpjz+bCvij/70Xd5sRXVk6gSlfnychcLwK2B/0oHMb8ERG+es7cjhxcDcVGOhKRknU1pJYVI3+6T9QcLKD4fx8FNWptmyiPMypDOEilcJQZyfJq28k2bhqqiv42V/BoZ1Yc+0E/GFs+Q68mMBkpbgwh4EBDym/B5OnF60jD69wDesNOOwWkpEgHl8MVWktTZ/4FBpzWtk4tLtjCgQGO47R+9tfyNi+wWMdJN0D6AlDIu2TFvF0GptD9gqO+93kNF1GwdW3yOfOVC9QdMDofPQ+mfghiiTPNuI+D8lEjMq7vjrFVTweBsUxxgNhzzP/RdQ1MKN6KeGvp51QX4fsimJtXIfO5pCwKTp4iHqIok5f4bW3oTmVJHOe3ibZw87TAlkAnKfBstPPqwUueQBciHUvJQAUWaz/8i8j9Dvj1AiwmWFEIina22PcfbeddeumZigODr/FCy/oefstDa4hFYo6T6pPDoeadesN3HSzWSZQXChjshr48gt+Hn/Yy7IG7ZQOIyePR9mzKyzduF5vUnb9sFlVRBNh2REkHNKRm69Cq1MkHK+7zMSO10WRXQiHUxjMYUoqUvScNMusYbNFJXsOizE0FISknlUbLdz+iXypBnYGmscAMAOB06mA3/nmboK+JLWVRQyGR8cAUKwZD4HtAyOEfSm2fK0EvSWdWCDbxAnXbWc/ZU/+HpeoZ2NMJ16IsjClNht9blE4Or1jqv4+UgVFJG79xGlXcCKBetufUb34lGw5Jy5Yk4yRiMZQa7UkzA6oXkZxTSVOpxucnZRfeZWEvK59R2QGbEXDUix1y2kbSitdTSsrJtwi40Ew3NdF71MPkkomCPd1EvIFUXRGzPY0NIoMYtE7WNFqZc3Aous/Qu7GayccbzYQ7Hv2MYJdbRhLZvcShAd7pUu27I7Pz1o7bzIM5nv34W87POPxhds60JmuQSgUzpzGtai0p3tdi+sT7mHxeME1t57Xgs0Xyvv0YjmPLABeLDt1aZxnFgAXsM+XEgCeOBHjO/cOU1oyvfo33oxHmiPc+EEzn/jExCQIvz/Jz37mYffuMHl5agoL1STplXFzLpfCkCufxkYdf/MlO7l5Fx4E6rQb+e49LlniRcT5TR7BYJI3tgVlAWgREyhIR8YDKtDR5ZO1AO12q0waaViup7xKx+tb/TK5Q69XcA0lKF8UpqysgM6TEYaH0jGBAgFD0QjC43jPv9ezdEUawAQAipFRAaeDQOEGfvC77RzvGmTJomIJgGJMpwKe7BsmR7HQ8AUbm6sXsXOgm3X5Vex0dqN9ZQe6l14lWZVOEhnfIUT8nAFCImFUw8Ms/drfcSSmZU11KQee+CX23a/hbzuGWgUJ0RpFESqoCpIJlESMlM4Iixoprq/HeaQZpWY5az7/1/K4k9vFHdrXKR+fCQKXryil65H7cO/dgdoo3L1p8Av40i5gsyUNS6LVmZBrq7/wD9hO1TWcvKcZEBSPZ1zDvmMHZYax6E2s0k7/y5BQ44RCV3jtreSs2DDnTxkBgwnnceIH/4xiK5RqZWGZY2y9UCB9rfsRPXsF4BoKSqXaN3nE/V5Eokj57Z8dK6sz55PITjxvFnhHAXDXLDGA67MxgOdtky+iA2cBcAGbdSkBYHNzlO/96wiLa7VTyptMNuGxY1E2bTLwxS9O7Hbxm9/4+O1v/SxZoh1zD2fWxhI9IteAY8c0XH1tKZ//64lrF7BN52SpUAJFCNm9365BpaTILzjlAp109ONtUVqOREglUxhNKq55v5m+njj79oYxWUMEvCZy89Ws2WCS6t6brwakW1gkfghojCWDrL9SFIMuka5hAZWieN3QQJy6DT6+9ven4/6Ecrq77QhDzZV0toVIJFLgGGbzZcuoXmriZChdCqb9pXx+92Q7pUvV5OkLZlQB4z1adDVJvvj/rGe3q5tMn2ChAur++DLa7bspXr58rCzMGPSJH4QamGOjf8RD7vAgQ+//EErlEpShATS/fRw8o2g7jpFIKWjNJmKxBDqthlgsLvv2quIRkjoz1DZiN6rx5C1GfdkHWdlYflYQ6Gj9I65XnkWbVzghCzgDgcKNqk+GZJ/j8o9+gdz1V896n4wHwfoVJTiff4Jg93EMJdWoNBPvhYzyqC8qo+TmT6AxW+d1D4qEjt5nHiIy0I03IWIVT4dbOGyKrA+YBmgFy5JGtNbTgJh5IXFviFjIgi03Yl95uszVvE4kO/mcWyALgOfcpNkDLsACWQBcgPEuJQAUbt3v3DtCYYEai2X2VlBHjkS55UNm7vzL0198Hk+Cb98zQjyWoqRkengSWzEw1EsopPCNbwQpr567crKAbZzzUpdvF/d+q4ZEPEVR8fTXkEzBibYoRw4KVzCsW6+XyTE9XXHMtiiOvBSrVhdiPtUFpP14hMMHwjKZRBQpFhnCKzaGKC0uHTsvkTnc1R7lps95aVynkG9cLZXBl34/yosvdhLwgsPqEEzAsNeDWgtNTWXc8okiBnXtGIYX8fC/deBXvFSXpt3AYoxXATuHR1CG9Sy+w8jNVzeya6iLtXlVYyrgvieeoOjNPbhKi6btEzzmCo7GUQ30E7v1L0lV1qB54zXUb29D8Xqht0O6LIUUqtWk4W8MAqMR1FoNCWs+2BzUfvSTdOrTGb5ngkAxZ7waKNS3nfd+m9RAOwYiqHVaGYMnegsLMBKZxGG3B4xWMDsoqqul/M6vzMlVmgHBVMBNrmuPjMVTdIaxgs+JoA9Rg9BQVCFLtOgLJhb0nuvNJnrxDr70W5mlrM3JRWO1S+V14HAzSecx0JtR5Zaj5BRSVJY77WGDve3krt1C3mXvm+vLZuedZwtkAfA8Gzh7+HlZIAuA8zLXxMmXEgAKdUkogCeOx1m8eOYYQKFY9fTG+frXHTStOB2XtHNnmB/9h5u6JbMriOILWqiNH/+rPq65JnbBlZP5z/97kF07ilhaP3vxXVESpqxcQ2m5lr27QvT1xlm1xoDZMYRBn1a1xBDq367tQXy+BHaHCvdokhUbQojybwZNqQTmE8fC1DUY+Ov/UcRw6gh5hlX88ekRXn7WLbOCVbZRrLrT5U6c7iE8PRbqmsysuSvIssJGXnvOxe9/e5JCh4PcQh2uqFsCYH9oBEPQysnuUWrWm2i6PQ+1VjXWKi6jAirN7eifeJpkURFluflSBZzQJ1hcjCgW7RoCvZHYnZ9G0RiofPEZBg4dwuR2EfL60ERCJAShyrYcaRtICAxHpDtYYzQSL6xG+ehXWb2ucUK7ODF3sjtYPnbKJbyiqZyOTh9HDvSReukBgsE42lSEPLUXkyqCXpdOvlF0ottIoawR6OoakDGJ2ms+xYr1tfP6NDj02iGSAydI9rRgN6X94BpLDtb6VVhqG8+6C0jmJKIjQ3gO7cR/4ojMYhbvjVQsSsTVj3lRI7q8Aoa6R6acc8ZlHOw5KeEvd91V87qu7OTzZ4EsAJ4/22aPPH8LZAFw/jYbW3EpAaC46NdeC/HAAx5KStTkjKtVlzGIiH1rOSpqzen4xjccE1zF27aFuP9+Nw0Np6FwJtM3t0S48y9tXHvdITnlQiohs2P3Xh74v2VYpYKXkvGLIn4vx6GWsX5ieNzp4tBf+3oe9Y16XviDn1895qGhUUc44USrOQ2AYv7ocJyD+0JyjTjGxitMqAwjhLx5BANJFi8zcOfn8ikq0dIbPER0qIEf/2sfBqOK3AKtLA5t0RaNmXM06sKYyqfreIhbP1WMY90QFYZ63nppmD89307Ka8KfCmLTWonHkkQMYTZdXoltS4y1JYvZ4+pklaNmggq41lbKoX/9PgXDHoYK8lBSp1TgcfF/JQYDA80tVH34DtoWNcg2cQfv/Q52Vz++jk5MDgfBvl60qQRxkUU8DgJF05REKAwaDYU3fRjXutvk9UzuGTwTBB7c20Frm5fu/jjRUJQV/U+jUSWJ6XJwjwTQxgMU5+spL7eis+WMtVATxZRVBhPe+tPJEjP1FJ7pfj20oxWiIr4wBVoDKzYuWcCnytSlopxMbGRItoETNhva+rv0S03TD1h0IhEjFY+CfxjNqg+iKqiatezMOT3Z7MFmtUAWALM3yIVkgSwALmA3LjUAjMdTPP64D1EOxmxSKC7WSPgRysTwSJK+vnSG8Je+lENV1USV8K23wvzHf7pZtnT2biFpBTDGpz9j4/rrTGMlZC4UCGxt382/fdfI2zssEtaER1OrVeHIVVFZpZVQNjSQ4H03mPnYx22yUHNfb4wf3DuM0ahgzBmUd9xkCBSgt/ftIGoNVFRpiaTclBQVsf4KC6vXm2VZGDEEAL76+yRvPGdnSYNBui0FAIqRgUABgGL4emwUlevY8uUItTmN8rFdna3Ejxex73g3gieWlpcTrfZwed0y9o120mRfJAFQjFg8/ZoZFZC2HgxPPwu+AEWLanFGY2kV0O2hOAVDHe0kapYQu+lWVLp0oor22WdQ7XkL1ego2HIgEgXPCNpknHgihUarIS7iAONC7dWSzCmA6z7K6o99lAOHumeEwMkgeKzNze+fO0nAF8Rm1bIkuA9z/36iOelsYY8nRCiUID9XR1NTMX1eLU6fGv/AEHn19TS+/0rKS800724f+0SYLwiKheNjBcX/Z6speLYfPaJF3Oju1zGWVY+1iBt/LPEeCju70BeUUvbhz9K88/Q1jZ83l6LUZ3uO2XXTW+AdBcC335i5EPTGK85rLcLs/l8cFsgC4AL26VIDQGEqAYFbXwnx6itBunviwmuH6HFrz1GzerWeG280U1Y2NT5O1M+755+GZUZrUdHMMYBudwKPN8n/9w+5YxCZqSP4bkNgT0+cB37qprlliFDIJt21oiev/MINp9XARYt1fPQuG7feYUWvT6tk4vknHvHw8gsBqmq0qPQDEwBQPC8SRcS/f/X5XEordLjCh6koXiuLRgtl1TUYJxZNJ5b89CdHGO7Jpbz6tJo6nQqoCuXi98S5/n8kWVmd7hIyvj3cYc8JRI/g/e40IDTaFrN3pGMMAiergCIjePeON6nesY+BY8cgGk+reCL5xGwmsaiW2NXXoBjS2d+iG0ijd5jj//FDVMI1bDRLhY9EEiIh9PEo0VAErVZNLBKDkkpyCvLwrr0BZVEDq5aVT4BAccyDp6BwfFygsM+xtgjt7R6p8Imi0cZgPxVdL6C32kjo0+cTiSToGwW3vhR3zIBaSaGIZJCSKow2CxXlFt5/VRn5ecaxvsIS4tacuSj8dB8j5wsGRfay80+/IdwvIK9kQo3BZDxG1OVEpTdQdN0dmCoXT/sJN7n0TGZSFgoX8IUwh6VZAJyDkbJT3jELZAFwAaa+FAEwY65wOMnR1phsg6bRiHInGkpmSIzIrHnsMS/PPhdg2VKdLP48eYg4w+aWKJdfbuRrX82ZEJQvIPDdBECRdPHvPxil+Ygo2OxCpy3F70vh7I/h86WD2YSLzmhU87dfz2PV2ok1EEVs5BMPe9i5I0ScUYqKi0QTDULBlOwfLFzqt33MzuVbzPJYA6H9mNVr2b8zwM43/XR3RGXyicgc7ugaxWSwydZwEsCEwjWNG1gTycM9EuP6uxM4itVjnUHEfNEjOAOA4v8CAucCgKIkzFqthQNvvMGicIKOQIAlFZUctVhZt3I1u7t7WV9Wzu7OXtZXlLGntZ3Kl55j8MU/YdDpiOhNWIx6/MF0UWpFZM0E/RgtZkJmBylbLk1f/weOtKddmXOBwG2vtPHSy05MJjVVlacSIkSyx8Gt5Lv2obNYiBscuOMGdvbnSjjKVXnQEkPJr6C4sZ5wJMHAYIiiQiO331RDXu7p/cv0F76QYDDqduF67Y+EettJiVYxao0sCyMUYV1+MXmbrsNcXTevT7eZoFBe94bTnVLmddDs5AkWuJgBUDRN+MEPfiCbJjQ2NnLfffdx5ZVXTrvDr776Ktdcc82U51paWli2LN2fPDvefQtkAXABe3ApA+DZmG10NMGPf+zh0OGIzATOy02XshDKl9udTh6prdXy1a/Yp2QKv9sAeOhghPv+bZTySg0qrRONumxaE7S1Rlm5xsCX/tYxJatUFHfeuzvEK9uO4+wuJpFMSZVw9XojazeaWLT4tKLXNbqfl56slvAnQDGvQCvVQAHeb+8Yxj+qpWm9mUV1BgmB0wFgdNguW819+VvV9KROTOgPnAFAcREZFXAyAEoAGxcLuHvPTmpPDNC9Ywf5Ki2ucIhkfgGJphUkliwDgwEl3WwDJZ4GU6ECKsMutI8/hLr5IHqTSUKg2agn4AsgausoWj2pgmIcVhOjDZth+XpW15VzsPmUC/gMSuDx426efLINvS4NQEWFOemTSCYxOw+gnNiJNuLhQGwZ/SErJeYQOQU5MgnEkzCPQXRBoY2Obh9rmvK56fqpqp8AQXGvCs1blGY5W2VQnNq5UAdluZn+blmOJhH0yx7A4ppMFYvPaT/grFp4Np9206+5WAEw0zZVQODmzZv52c9+xoMPPkhzczOVlVP7cWcAsLW1FZvtdD3YgoIC1OIDLTsuCAtkAXAB25AFwPkbb2QkIWsB7tkbkckSikqRsWgWq8KK5Xo+8hHLtGVi3m0AfPxRLy++EKS+QUck3jcjAIpr8vmSfONbBRTPUO7GFdxDLLhOZviKtnCWUyVhxlvz4ccP8MYfcmWxaLNl4gdme88AB7frJNw1rTNTWqmfAoAjkSGG2ixc86F8rr+jgGO+Y1MAUILIODewAEAxpnMD17ePcOKxX+GIJBg26CjOK8Tp9ZAfiTHi9VCybgMnL7+cDUuWsqurZ6IK2N7LSquR5vvvR73zDZRgSLqCDWYzYZ0RW44Nr0pPasV6Vn3sLzlwLN3jdyYIlGBaXz7mDu7vC7J/j4eyMguuIe+YGTMgqMTCDDY3c6xPVKGOUV7loHJZJSoRwHlqDPS55U+BYBxbjolPfqyOXEdaBRTQFx3ql905AidaZCeRUY9oS1eH4ihlUbVVtmTLAJgoOTOfMRkGxdrzETs4n3Oay9wsGM7FShPnvJMA+OCON2eMAfz8ps3zigHcuHEja9as4Sc/+cnYBdXX16brMNoAACAASURBVHPbbbfxve99b4ohMgA4OjqK3X5h1XSd/669d1dkAXABe5sFwLM3Xn9/XLp7w6G0W3NpnZaKCs20tdguhBjA//OjUQ4ciFJenU64mEkBFK3werpj/P0386ldMn2pmJHwHsz6y2Y03uhInH/6n83oNQUUFE0tueMOD9C610LXiQjFFVo2brHhi7vGkkAEsLQcG8RoVfGlv1tBcYVhTgAoTmg6N/DuXdvRPfY7UvEEyeJCUmMZwGmVTwnFUPX2kFixkuiHbkFJpB+f4Apu75VZwXtffYOqnhP07dknlbeCgjyGcoqpv/oqWmJG+diaxWXsb+mZFgLFg+OTQ8T/33q7nRf/1Et+rhnHKWgbcHrk+gwE9vUHEEphJBKlqtyCPUdLUfEppXDcTgjbHTo4wBXr86iuMLO8qRz3ntcZ3f2aLMWiNltRqTXEQwEizm6S0SgRxYBisMhzz6+uwLpsFTmrLh/LNJ7vu+RcqIPzfc1zOT/rSp7ZmhcSAHZ3d09Q5/R6PeLv5BGNRjGZTDz55JPcfvvtY0/ffffd7N+/n23bts0IgNXV1YTDYRoaGvjHf/zHad3C5/Leyx5rfhbIAuD87DVhdhYAF2C8eSx9t9U/caoP3u/hzTdCVC92TYA/4REUCS79vXFcrvippBD42tdzue4Gi1T4Jo8zAeAbr/h58P6TNDSWTOk3LI7ljQ1AtJD9bwdw9kZpWGkkpzyAVVdAwJfAPRzHUaBl/UeCbNm4Ur78mRRAMWe6OEDxeOXzu+l5ZRtFjSvoC3opNdnp9XsoM+fQ6/NSZrHR1+ekIJGg79ZbWbduwwQVUBxDuILFEBC452Qva0oL2N/SLlKoUanSXzpra8vY19orf54vBD700H5ajrgpLDJQOg7sMiAYi6k5enQUh0PPyqZ8RsYrhZNAsLPLx00fqEIV8ZNoP0DiwFbySgvR2PPkLyiiU0fg+BEioy7ZSk5tNGFZ2iQ7fgy2dZAKB1CVLKHp059DbTDO406ffurFDoSZq5oNDMWcSyHO8EICwMl327e//W3uueeeKTdhX18fZWVlvPnmm1x++eVjz3/3u9/l4YcfRrh5Jw/x2GuvvcbatWuJRCI8+uij/PSnP0Uog1u2bFnweyJ7gHNjgSwALsCOWQBcgPHmuPRCUP/Eqb7+WogHf+amsnYYgz4d/ycyokVSSHdnTP5sMCiynZtIilm8VMeSOj2f/GwO5RWnVbwzwZ847h+e8fDkb7poXD59FwkBgCZtCZFIkn07/OQXafBHvRjUdux5WlZdZmX1Jivh3HbKTens3/kCoFgjS8IcO0jhA88ylBRxm5X0BjxTAFDMVRKg6ugkfsUVxK66ig2l5dO6guXcDAxWl7H3eC9ra8rY15YGv9kgUDw/XVygeLyyKJ///u1x2tpc5OXpKS897Xbq73PT2xtkYDDM2tVFFBenE23EEBnDmSEUwXg8SV9/kDtuW0RtuZ6ex/8vru4BFFGe5tSwBPsIO3vQWHNkS7aEz402txBL3YpTgBiWsXmBgiY09ekvzJniBVOJOKHeDgInW4iL3sQaLcbSKsw1y2QHkOnGewUIx1/bpaIaXkgAOFcFMAOA27dvZ9Om020F7733Xgl2R48endOn+S233CLfH7///e/nND876fxbIAuAC7BxFgAXYLw5LL1Q4E+cqteb5NvfbsbvK6S6RiubWBw5GOHE8ShWq0rCn8gUFsksTav0EvpEX+CaRTq+dHcueXnpOL65AOCfnvXy6yc6aVwxOwCODsc48HaAolIdEXwYNDkUFOvYcJWNNZtzcGnapgVAcR6ZcjAiBlCM8Ykg4v+ZOMC9u7eT99AfGM41U2IrkAAoxnQqoLP5KGUbN3J8HACKuZmEEJkVfMoVLFRAMdYtEALFMTIu4ZLcXP78QgfNzUNyfwSI51iNxGIiO1tDb4+HHJsWi1V7OlHk1H2YAUG3J0ZZeQ6fumspsfbDOJ/7pQQy5VS/34GOfhIdh0GlxuJIB7cLRVDEAFob1oz1/RVdPBS1hoq7vozaZJlQViYDhDHPCEOv/F727E0m4qi0etkNJRmLSbi0r71S9vFVVLO3XnwvAuFc4VDa8iLKUH4nAfDnb8wcA/i5K+YeA3g2LuDpPt4FMD722GOITODsuDAskAXABexDFgAXYLwzLL2Q4E+cqie8iz07Nfzm8TKZvGG1qdn1dkiWszEYkHUARTmY0jINq9caZMauUAWPHY3yF3fZ+ODN6b7IcwHAg3tD/Ojf21hUWzxWS3C8uYQC6B3I4+Auv2wlt3azFX2uB6O6QLp/xV+rXUPxCj/luRVSFdQsGqCpKq0GjgdA8fNsiSB7975F3i+eZ9huBL2OEqN9WhVQuIGdR5qJL19O7Kab5WvMpgJmXMEZABTz56ME7j/cJSvgrK5PF3rOQGDj4hI62r20HXPTdnxIQmBxiYnr37eYnTud8q9GnZDFu8UYyxgWbflCcQYGg9RW6mlclkOieTuW7j2Yqk7X0osM9OI/3ozW5iDgT5eyEXGDhlQIS10TOtHDTzyWSBDq66Dk5o9LZXAC0Ihs4nCA+J7nSbq6KVi6BLX+tKtYlBOKjbpIhsPkXnEDjtWb5/VGu1iTSuZ1kacmX2xu5YsRAIWpRRKIcOeKLODMEHF9t95667RJINPt5Uc+8hFGRkbYunXr2Wx1ds15sEAWABdg1IsRADP1+4SrUkBKdbWGwsKZCzMvwDxnvfRCgz9xIQIADfrL2LMzxHPP+Nn5VpDenjhWm0q25dIbFEpKNdQ36GVSS2aIhBBbjppvfjufkLJPPjxbAoh4Xrh2v/NPhxkdyKO6dmpQds/gAC27DbhHEiyuN7BirUVmAZs1RfR2hGk/FmJ4IIbOEqd2qUMW305ZAlx5VQ1X3OAg3tHFsb07yI/noDYZ6C/VsmzdtRwM9sgkEDHGMoG7jlL84PMkgyGGcgwzAqAouZLvHGLgmqtY86Hb2NXdMwaA4niTE0LEY/OFQI87wot/bqP7pB+LzoDbG6C00sz119RSXGqW7qXJCSLidTLFo+sWFfOHP7TT2jqKyaQhFo3ITi0C1DUaPZFwgqYVedxwXaUsTr3/8V+SOPwGqsLKsYSRcG8ngY5jE9yzfm8IQl5UZUtRbHljc0PdJyi87g5sK9ZPeS+M7tqGa9vz+BT7lG4ehSXp5JToyKBUGss/+jdorWefSXkpAeFkQ19ogHixAmCmDIyI4xNu4Pvvv58HHniAI0eOUFVVxTe/+U16e3t55JFH5BaIGoEiAUTUCxQKolD+/vVf/5Wnn36aD3/4w2f93ZBdeG4tkAXABdjzYgJA4Z58eWuQV18N0dsbl10rRAKDw6Fi3VoDN95omrVDxwLMNK+lFyr8iYsQACiGgOhvfWOQ1pYoeXkq9AaVhGhRykZAiBiitZso/hwKpaRKd8/3CjHkHTwj/GWM9fKbB3ju0QLZBUR0BhGgkhm7dg/QfkRLQYmGVRstWHM0EgCHO20cbw7JdnLinNzeEGsvK5AJIR39LrR9YdZp9lNmcuMNe7Aa7DKz15sMkV+7FNcNK1BqyyZkAovX3PfLX+J4eS8j5fmUWPLGFEDxXCYZpK+9nUKzle6P3sH6ZY0SAMWYrAJKGJzFFSyen04JdHYF6D0cZMQVJhiNotWpcJiN+P0xwtEIS5fbufOjK2TSTAYCxbFEuZjMECAYjSaJumMc2ueUpVySioZgNIndrmNRjYVbb64fUwe9h3cz+Pyv8WocKKq0Cz814kQ72oU2xyGc2/IxURYmFQ1jbVzLsDfdHFnWC3R1U3fXZ2VW8PiRjIbp+dVPifu9spPH+DHYOzr2X6EEWuMjFF57KzkrZ84an9cbbJoahGL9xVB2Zr7XOZf57zQgXqwAKGwp1L/vf//7shD08uXL+eEPfziW0PHpT3+ajo4OmeQhhpgnIFFAodFolCAoIPHGG2+cy7Zk57xDFsgC4AIMfbEAoIC/Rx/18dLLQSwWheKidA9fARfDw0mcA3GWLNbJHr6lpe+OGnghgp+4NYTyJ0YG/jK3y30/GOZEW5Sq6ollWtyjCTo7Yjj744jCz6J7h+h89vFP5bDlxk6W1m084x0nuoCIcWJ/Pc8/NcqgUPO0Chqtgt+f5MghLzk2g0z0sOem96tveJCjb+ulWzRTN3Bg0E9ljZ36VRa8ruMU795F0uWianMNSkmMXF2+XDvgGcTiDKHJtTP6sctZ3nTlmAIont/TdojSp17DebSVwrplqPS6025gn5uiUIyhwUGqP/IXWK65il29vWwoKZ+XCiheZyZ3cInOxi9+doBoJEF9XYGEPKcrXe+vONeKUAZ7ejys3JDHx+447eaerAZGh5wcee6P0NVKNBAirrOT0pqwVlVRe+VaOjyny/Y0La8gHvDR+9j/kTF5uvyitK3ae0mKGECNDos97dYXyRtaRz6WZSvHfgEQ8X3DTjfaq+5EMacVvRWr0gVzI4N99Pz6p2gdBWcsFeM8dETWG9Su+YBcu5Di07PdeJeySjirXbYfO+P7dT4xiBczAJ7RENkJF50FsgC4gC27WABw6ytBfvELL2WlGmzCZTlpiBZsLUejrF6l5+//3jFt6ZEFmGnWpRnwE5PezVZv/3975wHlVnXt/b+mSdP7eIqnedwN7jaYZhxDIJBCCcTEmPKAjyR8SV5IvuQl4QV4vJCEkpD3eI9mElrohJpAIGAwBhKPje1xL9N77yojaeZb+8hXc6VRudK90kiafdby8kj3nHPP/Z+rq5/2PnsfT4P0Bn9U98VnB/G3v46KxNBS6eywoWavRVj+UlNpbaBOJIW224GiUiNKywqw+V9ysHCJ6zZx8nOT+/dIUw1SE05FWkY8aP3/wb0mHNlvgsk0jsGxThzaBcxfmC8sf1LZf7ATLUeTkJOfAN1Jy1RP3yhyc9Kw4owMxO18A1ltLejWFSC/2ICCU23I0TsAsMvcj9ykHJiP1mOwKg+n/uhW7BloElHAVHb3NGKJOQWH/vAEctv6RYBFH+zITUxGb38fcgsKkXruOTi8cA7WFle4ACC192UFpONyVzC9liKDxd/HWlD9906MtFJgBKXa06EoxxF8IUFgUW4GerqMGB41YeNXZsNgiBdbyFGRIHCiuxV5hz/CWFcHErJz0WsiA54OMI8iS2cW+f3yvniJgLj9e5ucupaOnkDv9reRmJXniPqdmMDI0RqM9XSK/H8Yo46A/GUrBQRSEZG9zXXIXHkmCs535E3bv6vB2ed4XzvSDrwltmyLS/ScK1KqTNHEKXMWiLWE8i3ppOMMhKF6Minr158F0TlPJwNVwgqAH/kIAlmvPAhEmRJcKxoVYABUMWvRAIBk/fvlL/vQ3GzDnDlTkwpLl0+WJbIE/uTHOVi40PeXkgrJXJpGqtWPBukL/uj48WNjeOCeXuTlxYt1gKOj4/jHJwRpE8jOdriChRewx44FC5NQNq8PzfX5yC9IwHd/nI+cXFdL6/CQHdWfGfHZx6Nobu9Goi4HhuQ4nLoiGavPTEMVbflGQFNXg2fuzUZGNrmcJ3cI2b6tA9ZRgxMKTXYjLEN6ZOcmYOUSOwzbX0CCPhMmJIOAv3IdUJDuSG1CAJinz4V9xIiutmYs+NH3cCh7wgUAaUu4ccsY9vzjY1Q09KKpsQHFmXloysuEbdF8rFmyHDvbW7B2VqkAQCr+rIBUR4oKFn/XtQorIBUJAnu7Tfjf+3chUR+HqpP7/Lb3DE2BwHH7BGyjOlx8yRyMJ5pFHxIE2kdHUPP7+4H+LqBgNgoLJtfTdXYOOty1/Z3IK8hB0ZU3IKlg0i1bs7se4wd2YPzYbmSmJgoIJJev8cRh2EYGkZCWAUtWGeKyKUH2hADKDJ0RhtkVKPzyN0+6il0/MdbBftQ88GuR8kWX4oBZT0mp6X1T0wlkrVmPvHOmus7cgTBUMCiNnq2EwT355JBoHjPh9j/834B24Qj0rNL30uMMgIFKN+PqMwCqmPJoAMDjx8dw96/6UFyUgJQU3+kkaI/eK65Iw+WXOVxboSqRbPVTAn9Uh9znf3hkAJ9sN6KyKhFtLVYc2D+G/Hxpf2OAtr2jRNBL1wwjLW0C8XElOHbIgk3XZWPD+ZMa9/Xa8PTWPhyqMSOVUspk9iFFX4jR4XH09lgF6F16VQ7WnpWGxsEavPDALJEAurRyMkBk23sdmLAmO6HQaBuFsS8Jc5ekYkFyM5Kq/wrkzxFuaevYOCpOBwpP5raTAJCuq72mBpXXbEbGhnVTtoSj49XdTViVW46dnc1YnefYL3dnRzPWFpQ6AVDUk7mBBQzK8gLS612Nrc61gPRasgKKv2UQmGVJwavPHENCBu2/q0PxSesfQSAVuTWwq9WIiy6eh7M3TO4jTHWqzO3ofvN5GEor0NljFO0K8yf3J6XXHR0D0HU0AkvPwsrN33S59SfGx1Hz7g5MNB/FeHsdcrNSRJTvuNUidgJxLPvUCesgxYPoCisQf+rZ0KVmYekyR5Sye+l65yUM7d+F5LIqSNvQSXUkGLSbjLAO9KL4suuRPLvS78cxEoCQBjlT1xP6nSBaG2waxRU3X8gAqEQsrhNyBRgAVUgcDQC4d58F997Tj0WLEj1usya/fHIDn39eCq691vXLUYVELk0jHfxosP4sf/ILIqvdn54cxK6dJhw5NCYOkTVwzDIh/tHfS1cYkJrVhcQEh0uyqWEMRcWJ+H+/KBDzQda4R/+rB3t2mVA1Xw+rrhOGhGLnaciqRHBJkHHj9wpgKD2OY59V4aUnulFeRRHHDqj/dEcHRnr1yMpxWHn7h0aRoDNg1VlZyO49hqTdfwMK5sA0YhP7L1esm0B+qqsFkNq179+PiquuROb5k+sAyQVMFkAq/gCQ6khWQLIAijZeIoLpmBQQQn+7QyBZAdvrRnBk2wBKK9PR0Tcs+vMGgTU1HVi0PAfXbHbsfkKFEkdPvPsiska7YJjtANaOzsn9guUgSG7d/mEb4i6+HrqERCxbMhlA4uyv+gRgtwHkuo2Lx4I8Hcwt9SIPoC5Jj+TSKiTlF4q53b+n0eX+l8MguYjbX3+GHMbCFSwFD0kwSP2hrw3Fa9ai8Mub/eYCdDkRBXrsrHN/K2TrB+UnYivhFNld3mAA9K0PHw2vAgyAKvSOBgA8fHgMv/51H8rLE0Xgh69y4OAYLr00FVdeoa0FMBrAj3QJBP4kHWnNXvU/TPjl7T3CDUwa6/VxmF2WIHIC6pI6nfBHbbq7bIjTAbf/pkik4Tl22IwH7+tGQWEiUtPiMGrtcAFAakMQePyIBWecm4ZzNrUjxb4Uf3qkCwf3GFFSkQRbUh+G2rJw8PMRZObEw2KawIjJjPmLs1ExPwVJ7fXQ/+N1IKcU/f0TKJubgoy5FmcQiGQBnLCPo+PQQVTddD3S1q30aQGkcUlWQMkCKN7z4QYWMNjUIlLCUJGsgPS3uyuY3iNL4LZ/1OHjP7egrDATqelJaO91wJs7BM7KSkdzwxDmrcjAnIWOHzAr5p9cB/jLXwBmE5CVi8K8yR837iBIUbnjJiNm3/ADHGqYhERPIEj9y9cKUtCIr+IJBocO7BapYOzGUZFWhraUmxi3wzbYD7L+pc5ZgJ5Zp0GXMvl5lAJJAn1sTRcQ0jgZCidnK5wA+Idt3tcA/ssGXgMY6GcoFuszAKqY1WgAQAocuPPOPvQP2FEm25LM/bKpXlOTDT+4NVsEg6gtWkNfd7cNzU02EVBBgSxVcxNFkl+tSjDwJ52b3Kq/+EknLJYJ5OUnCLCj4A2TrV1Ukax/9HdHuxUGQxxu/3WhCLZ5+U/9+Pvbw1iwxCDgj4rcAiido6/HhgFTL/7trmUi8new34ZXn+nBkRoj+kcHkZqUiWMHjTCNjiM7LwG55VYsWOA4h85qgX7bs8CQHdbUHJSsmEBKVtwUALR29aJ3bBiL//3fkJCVoQoABex5cQPTMSkvoPj7ZFoYAX1ljjWA0npAgt+H/3s32mpHcOoSh9XUEwQOD4yhIDsNV9+4BGnpSdh72JGGhiCwbesDsPX3oj/OsQUcQSC5cG3Dg5iw2dA3YAKS0wCzEbkZesy+8Vaxe4dUpDyC9FprGBzvbkHxeDtGaw9h3GIRgSlJOfliV5H0RctcxiEPJKGxBAuDAsqmyULoDQjp/ZngOmYA1OqJzf1ooQADoAoVowEA6fLeemsEf3p2GHOrEgV8uBf6kj161Io5VYm47ec5AmCCLVqDX3u7De/9zYjdu8ximzVaZ0W7b9B2bOs3pOCMMw2qopbVgJ9coye29mPHRyYsXDQZQEMAKIc/qn/0oBnnfjENm65x7PP68APdOHzQgoo5SR6tf9I5LOZx1DZ144f/fopz7R+tQ2w4bsZH/zgO20ARBvtsqD1sFBbDtCIjigtmCdfx6JAdI7s+RlXXQRQtn42Eygkn/FH/ZAHMtqfA0tCKoXOXYOnma8VpncmgPbiA6bjSdYBUV3IDi799WAHpuLsruKl2EE8+VgMK9Fg0L28KBI4MjaGuoR9L1uRi8zcmd92QILCipQYD29+DoXIeLd5E19FjQG8n9BNWsfUa7ecbn5wMo8kGLF4D3cYrsWLxVIteoCBI1+LPMljzeQMGh6zoaR/AuMWMefNnoWpRCdIyJ3cG8fZZjBUgnElQyAAY7DcLtwuFAgyAKlSNFgAk1+TDjwyieqcFhUXxyM2Jc0LTyOi4sKxRNOu3vpUZdASw1uBH09LUaMWjDw+iod6KWUUJIukyWbQozUpHu12sn7v4K6m45NK0oCBQK/ijsR7cb8GDv3NEBWdmxXu0/lGOwMF+O275YR7mLXRE9T7+Pz3Yu9uEOfP0PgGQkkk3tnbjh7efgpIyVwtt4+ghFCY71r21NZnx1ttHcGKfHROj6SJlS3JqPDLKBnGGrRUJJw5gGCbklFSKfH7jY1Z0tzYizaZH+tqlaP/KcpxSsFgVAFJjb+sAJQCk/92tgPSeN1fwkZpevPL8YRiHbZg9Kx16QwI6eofFmsaC7FQsX1OA9LJ4xMXrsHKuw4pIRUBgTzt0f30eOTkZsPV0YayzFRYbbd+SLNbxpRgSYR8ZhH1kGJmnn4u+dV+HzpAi2ssTSUt9KgFBqit3EXuCwZ4eEz75rB11DYMwGW0YGqQAFR3SUhNQUZaKK7++GHr9ZKS3r0eVOwxS3Wi1EErX6cl1TMei2VLIAKjiC5ebaq4AA6AKSaMFAOkSKR/dyy+PoHqXGf39dmeaEtrHlpJAX355GhYsCCz9SyigT5oOSl9z72/6cPSIFfMXJLrshCHV6e21Y6BvHDd/JxOr13jPredpirWEP+qfYPTFZ4fw97+NIi8vDilZ3dAnOdagiYTb3Xb099px3kXpuPQbmU5g3fbuMJ5/sh8l8waE29iT+5f6OFbXicwc4Md3rBC7fEiF4I+KBID0d93IYaSMzEdPx5hIRdOb1IxV808Ru34MfrYPJ7Z9gJQOKyasVgxOmJFXPhcZ61Ygfd0K1Jjbpm4HF4AFkM4vrQOkvyU3sPj7ZDCI+FuBFZDqkSuYCq0H7Ok04t2/n0DL8RFk6g2Ij49DXPo4Zs9NxwXnzHVquueoo40EgmQR3ffs88C216AfHURCdh7ikhz3+uioGRizQD9hQ2J+IeL1emScdg7yLrrc644ikvZyEKT3vLmH6Zg7DBYW5OKNt+rQ2WVCbq4BaWmOIC27fRwDA2MYHh5DTqYOp63ORWKCY769RRR7ur+1tA5S/9PpMpZfX7SvJwwnAP7x/U+Qkjq5lEHS0Tg6gus38hpAT5+bmfYeA6CKGY8mAJQuk3L9HTgwhsm9gBOxwAtgeZJGDn10PFTJm/ftteD3v+tHWXmCR7e1NLZjx8awbKke3/tBlt8oZ2qjNfjJNaK1gG+9NowPtjVhZDAX8SfXKNJuINk58ThrQxou+EqGi4u9t8eG+/6jE6PWXpSVTkb/yvu1jk3g4JEuXLmlCud+yXVPWLn1T2pDADg7ZXJHjGPDx1CR6rDqUTnUdwSlA1mwmy04YWnHwgXrEJeUiL0D9VPgj+p7igKm9z25gMX7JwNB6G+lAEh1aS2gOF99q3MtoHgtyw9Ir3cdbRGpbNbMLUFiUjz2HHcA36qqScufOwRSLsD9P/0BdF0twuVr0Cc5MreMjyMuSQ9zchZQVI48PTBus6L42luQmONI7Oxtazn5HCm1Cor+Pm/Eh9s70d5uQn6+HkWFU/f5NZtt6Ow04vyNZVizepbPiGJ/jzCtrYN0vukGQrvZCGNTLeyjQ2g81gldeg502cXOaOlItRIyAPq7W/l4OBVgAFShdjQCYLCXG0prn6cxPf3kEN5/z4iFsp02PNUbHLBjaGgcP/9Frt9t7EIJf9LY+sy70d2pQ93hxSLgg9bgFRYlioTOFCDiqbzyxj68/bIO2Rn5yC9McHFnC9dvvQXF84dw8/eWIz3T1SUYDAAeHjqOuWkLxVAODNZiQfp88Xe4AVBAn4eIYHcIlFsBJf3ku4XQe/4g0HjsELqf+wMS8mahp6nVkbA5xYC45BQk5uaJYIvO7kHHBtntjajadA0yVq1zma5AQZAae7IKNjUN48UXjyE7W4+hwVGXc8wqcGwbR6Wzy4isDD22bF6AxMTJefeVXkbJ51tr62A4gXDcZsPg3k8xdHCXyJEoysSESNtjKCxF9tpzkVI2N2IjjxkAldyhXCdcCjAAqlA61gEw3NAnn4oHf9+PffvGRLSvr0KRty1NVvzkZzmYO8+zCzsc4CcHwFT96QHdVe2je7Fv+zy8//YQ+nttIo1MfDxgNo8jMTEOhfMGcNE3dZhXunJKv9EGgHQBlBSairsbWAChzApIr+VRweL1ySTR9HcgEDh/tB09rz4HQ8VcB1ydTCRNfxfmuua97KzZD6w+F7ql65y7iciFVwKCVN+bVfCTT9rw4YctqKx0O2/HoMv8ZmeloavLhE1XR5ewAQAAIABJREFUzkd5mffUTDMFCClFTs/2twUAErAnZuVAF+/4UWU3mzDW24n45FQUbLwEqXMWTfmsRIL7mAEwoEcjVw6xAgyAKgSORQCcTuiTT8Xjjw1ix8cmsY2ar0Jb2PV02/Gz23JQVu4KixL4UXtDgFAWzG1B1r9A4U/AiGkvsgyr0dttxf7PTag/bobVNoHcvAScsiIFyaXHMSs9OPij/uUuYLn1j46F0wIooM/LOkA6Js8LSK89uYLlEEgASGVVpcz168EdTHX2/PXviHvvz5i1eIlLUmUJBOUQaK4/jpwLL0FDuiP5tbSlnPs9oRQE3WGwp2MC1dUdKPMBdZ0dji3qOjst2LB+Fr50wQJFt6TWMEgnVRNMIg1aC5fxyPH96HjnRZEzMSF1KhCTXpbOFiSkZaHkipuQIEvl4028cEMhA6Ci25grhUkBBkAVQscKAIZrXV8gUn+ywyQigOfN953vr77OitmlCfi3n7mmrwmn1Y+ui+CPSqAAKMGfL21ajfuRl7xiShW11r9wAKCAvpP5AN0BULx2Cwah99ytgKtmF2GsuQljjfWobenA3NJi6CurkFhUgj21bYogkPIAHrzvNyLqt7CizEVLOQTajSOwDfRj1ub/I7aOE/B2qNknCNLBQGDw2WdrsG9vHwoLk1FY4H3XHavVjo4OE1YtT8esWZNpYfyllpEuzh0G6f1AAkmofijcxeK+CzAPIcFd+5vPwNhwFMkljnnxVCi/o6mtAbPO/zoylqwK5JHjrBvK6ONwAuAT73kPArnufA4CCermiLFGDIAqJjSaATBSLH3e5Keo5bvv6kVv7zgq5yR4DPCgdDCN9TZc+y8Z+MJGR9qOcIOfNH611j9vOhD8UQkFAMqtf3SOUKwBFJCnEACprrsV8PN/fo7yA5/DUl+LcbMZAyYLMD6B3Pxc6KvmI+vCi1HTZ1YEgb1v/Rkt7/1NBHsU5mdPkZzWAE60N6FoxQrM2nSDi6VQgkBq5M0iqBQEGxuG8NKLx2AfH0NS0uTaPncY7OoyIj09CVu2LHLW85daxtfjTK11cDqB0DrYh+bnHxL5GhNSfW9VaWprRNqcxSi8+CoVT3fXplpBIQOgZlPCHWmgAAOgChGjDQAjHfrcp4KSPz/xhyHQLiWlpYnQGxwJqimtSl/vODo7bTh9XTJuuCkDFjgscFTC4e6Vj1UN/FE/5P71BYDe4I/auKd/ofd8RQD7CgChtksyHGvkpCTQ9LeaKGBqHygAUhuyAlrb29D33J/QXVuH/DlViE917OTR3jcIGEeRYxpBUnklcjddjZpekzgmdwfTawoOkaKDyQrY8+Iz6Dx8BMgpAAzJKMx1BF2M0xqyjlYM6JKBDV8D8oucW8nJ50YLEKSUQS+9cBQN9UOYXZYugn46O93W/2Wmor3diPPOK8XatYVe749ggTAU1kEapBbuYurH3UI4r0yPlhcfEev+4il/o49i6WqDvqAEJZff4LOe2oPBQCEDoFrVub2WCsxIAGxoaMBdd92FDz74AB0dHSguLsbVV1+Nn//850g6mR9MiciRDoCR6NpVoqu8zt49Frz+6ggaG62wWSlPLqU2BrKy4rH2NAM2XnwUKQ4uCDv40TmDdf1S21C7f+kc7uv/6D0lEcBUb2nWHKGrlgBI/bmvA6T3pGhg+pusgKtnF6PvqT/CdPAA+rLzQUkSizMnLT/t/UMoTKPdS2qRfuZ6ZH/lUhEUQsUXBFp7e9D/tzeERbGvm6JIdchKNUCXlAT97Apkn38x9CVlLlvJebpnlYIgtfXkHu7oGMWbr9eBkkHn5hmQmurIA0g/bupqezFqtKG8PA2nrcvH6uWuLmtvn6FgYZD6i2TroBjfh3sxtv056BL10CWnoaBoMlraXQ8TJTafuwSFF20K9HGjur4/KGQAVC0xd6ChAjMSAN955x288MILuOqqqzB37lwcOHAAN910E7Zs2YL77rtPsbyRCICxAH3uE0D59Q4dHENTk9W5F3D5/EPIL3DAYLgtftL41MKfANkgrH/UbjrW/wl4627CqtxyIYGSPICijcwF7A6A4rVsHaAEgMtgR8/WRxCXmYX4tDQQ8MkBkOrRe/njju3cCm7+LhKyc3xCoIDDqhIRXDHW2gRzfS3q69uAhEQsWrsS+rI50FH49ckibSVHL2lPYa1BkFy8n+xoRWPDMEy0DR0ZuCeAzEw9Fi/Jwbp1RTh63LGftFR8JZuW19MSBqnfQNcOUhst1w+KgJi3nsXIiQNInj0HXa39LrpIQDhht8Hc1oRZF1yB9EVT180qfrhrWFEOhWaLCbc/cjMGBweRkeHblR3sEKTvpSff8b4G8NoLeQ1gsPrGUrsZCYCeJvDee+/FQw89hLq6OsXzGykAGIvQ520Swh3Z6+tmCNb1S32qtf5RH3L3L732lwDa3f1LbYLNAUhtQwmA8Z9Xo+CT7UiqmuvY+7d/SEyFuxUQE+PI6e9B7lVbkHLqclHHlyWQjssTRtNr96TR7nMugaA3CKT6gVgEqb5kFSSwKc7NRXvbiNhNxmBIQHl5OtLSXaPfJ2xW1Lz/GSaGesR+xrqUdJy64UzES+ZvP08tLYFQCxik4QbqLh45cQidf3kWCelZSEh3tQASEJKWE/1t0KVmY9ktP0T8ye38FD/Qw1DRaBzFFTedzwAYBq35FP4VYAA8qdFtt90Gsgzu2rXLq2oWiwX0TyoEgKWlpWhvexQZGf43b/c/HcpqzCTgkxSJJPCjMamFP+rDn/WP6gQb/EFt3XcA8bb+j+oGEwBC7VbnnbQGdjRjbUGpmC75TiD0OlALoBjPSy8jf+dn0Fc51iRS8WYF1LU0ouqa65G6co2zrnuOQOmAp4TRzmNuW8i5fxpDCYLSuTztPTx6/CAGPv0AlvYWUJQrGQoHh0xAejZ0VUuhW7QGy5d6j4x1vw4tYZD61gIIlcAg7drS98m76N+1Xezekkhb+iUmCfAbN43C0tuJxPQsFHzxctR1uiZMP2Wl4z6d7sIAON0zwOeXK8AACKC2thYrV67E/fffjxtvvNHrHXLHHXfgzjvvnHI8HAA4E6GPhI408KMxqXH9Uns11j9qr9T9S3WlLeCU5v+jNoEGgFCbnSoBkPqQrwPc8/qbKPh4m9MCKAEg/S+3Ak7YbOg8cgTzbvo2khdPbn8nruNE65T1gPS+FhBI/WhpEaT+PK0VHDm0Fz1/exXjYxYk5RciTu/Y85pcndb+XoyPDmOwaDHiVn4BujgH9Ch1E1NdNTAo2u9pdHkeBgODop9dDS79eANCSgY9tL8aQzU7YenpoL38BABSYIhh9hxkrznHY5oY96CS6QJCBkAGsEhSIKYA0BugyQWvrq7G6tWTUZdtbW1Yv369+Ld161afcxNOC+BMBT536KPX07XGz9PNoAX8Ub++rH903FfuPzquxv1L7ZUkgKZ6SgJA1AIgtXdfB7h7734kPfsU8rOzkZA9mbLF3Qpo7epEj8UO66ZrsXrhpLVQAkD63z0oRC0EUnsl1kCqF6hrWA6CE8ZhTLzzDLINOugLPa9BtI0Mwz7Qi4KvbUbqwlNddh8JFAbVAqEWkcWeYJDecwdCAmJTSx1oj2cKEErKLYB+1mxF+4EHmoNQyy/scALg03/1vgZwy0W8BlDLeY3WvmIKAHt6ekD/fJWKigoYDI5f0QR/GzZswGmnnYYnnngCcXFxAc2j1msAZzL0uYNfJEGf/KZQ4/qlfpRa/6hupLh/ycLS22bG+7tqUZ6Uh4SkOPRkDGPjsnniC1du/RMw2N6CtbMc7mABd36CQDwBoICsx7Yi//BBJFVUiChdKvK1gOMmE8Zam5F5wUU4XrFYHJdvFUevva0HpGNKLIFUb+XcyZ1G5PeCkiARqX4wIDi0+1PUPvs0MKtMQA6VwvypgQPmpjqkzl+CgsuumQJA3rajU/KgizbroJJr8lQnnEDIABjsLHG7UCgQUwAYiECtra0C/latWoVnnnkG8bLoP6X9qAXAmQ58ks6R6Ob1dA+ohb9AANAT/FH7YN2/1NZb+hc65i3/35yJEnzyRguqd7cj2aqH7mSk6rDOguWnFuGML89Gra7Huf6P+tIKAHcdOorKHR/CfOQw4rKykJCbJ6J027v7kG+1wD46jJRlK5F96RWIMxiwu651CgCqgUABiX7WBQpQPdwibhdfLmH5/aRkZxGq3/H84zDVH4dhtmP9WkenIxDGHQRtQ4OYGDOj5LrvISFzaoJrqY0cBum96XQV0/lD7S5W+hx3rxdKIGQADHZWuF0oFJiRACi5fcvKyvDUU0+5wF9hofekq+4TECgAugMf9RefdHoo5jXi+4wW6JOEVOv6pX6UWP+oXiBbv1H9QKJ/qb4v96/4Yj6Z/+/jE7VoftWO5mPDsGVZUVmQKyxMZBGs7+pHfH8i8kuSUXRxCjaeUuW85zQDwMZWwGjE/IYTMO3bA3tvL2VJQb/JgonsHFSuPxdpp50h4I8KASAVdysgvResJZDaKoFAqhcsCFJbT7uLtD3xIKw9nUiaVTzl8yyHwfy0BNgH+1F87XeRlFeg6LOvBgbpBDPFOig+LwFuW+drAhgAFd2eXClMCsxIACR37/XXX+9RYvpyU1qUACBb+VzVjDbwo9FrBX/UV7Br/6itEusf1fMX/Ut1/KV/oTp/eHwvuj6zo3ReGjrGhlGcnCUms3V0EMUpWSJpccvxYeirdPj291Y73Y9aASCdS9oazj4yAmtzE8bHxhBn0OPgRBJWz5uETukuCwcE2gb7YWmqd4xFr4e+tBIJmQ5tAnELS2P25h7ufOkJGI8fdu5J7O251FHXAsqSrvvydVixapHSx5ezXqzAoABThcEkAYska6AmoIQBUI3y3FZrBWYkAGoloicAZOCbqm40Qp90FVrAH/Ult/7Rj4y2ZiuOHzaD9jNOStKhokqPynl6dFgOeF37R/0oCf6gevLoX3rtyf0rgGWg3qP79+PjtTiw1QhdnA5ZeXq0GgenACC1Nw5Z0dA5gJt/uBIFpanC/UtFizWAcgB0v6t217didZnntXneXMHUh7fIYDrma00gHbcPD2H/iy9DV38UWbDC4Q8HEjIykbJ4GTLP+gLi09JdQFCpW5gauYPg8N5qdL/5PAyllS4Jqt21MDfWIu3UVcj76ibUHHDoT8VTShl/zz61MEj9zyTrIF1vIEAYTgB85k3vQSBXf4WDQPx9FmbCcQZAFbMsAWBd/U1Iz5hM3DpT3bpyKeXQR+9HalCHv+nXat0fnYesf329Nrz5Yj8O15gwMmwXgEVAqDfEIa98ABsv02HZopVThuXN+kcVle79K76sBms9Wv8EHPU1ON2/r79/GEdfNGH23DS0m4ec8Ef1JAsg/d0yPIDeOjMuu3oRlp8zy+P6P6q3tmgyilW+FZx0oe5RwNL7kgXQ0zx5g0BfVkBxnV7Sw9AxbxBI6+x6X3kW5hNHMRBnANIyUZiXBUpLYhvoh72/F8nzFiHvsm9OgUDqNxgQnDAbkf+P12Ht74G+pNxjhKt1oA/jo6Mo+Po1SJm70CmTPKVMsDBI7dQCYaTAoADTMFgH/QEhA6C/Jy4fD6cCDIAq1JYAsKE1vImgVQw55E2j2drnLo4W8Ed9Sta/gX4bnnyoG8cOmVFUkoiMzHjnl/roiB21DT0oKc7H1d+ehdJKvctwwu3+/fM7h1D7mhml89JdrH/uANg6MojxNh3O+kopVp9X5DcAhNoHCoDUZk3pVGufPysgtQt0PaA3COx942UM/3MHDOVzoEtMREePIyCjMMcRlTtuHcNYUz0yzjwXORde4jJ3ga4NlDfe995HwI63gZFB5M+pQFxqmmPPYIsF1t4uUB7ErLPPR9bZ53lNgRIrMCggLsx5B9U+MN2tg2aLEbc/eENYdgJhC6Da2Yv99gyAKuaYAdAhXixBn3Q7aOn6pT7J+vfac314761BVC0wIDHR4T6UypC1E7T8tKM2C/NPScFNPyxEXJyjDsGfgI3kZS5tPAV/UAW17l/K/dd2wIgjz5tRXJnicf2fNBCyAE60xWHjNypw6pkFQQMg9ReoFZAAkEqwrmBq6ylHIL0vtwRa+3rRufW/BfglZOU458AdAq19PaBJLLqRonEdawLlJVgQNDfV4+jrrwPtjcCYGZnpKcIlTMEhGavOQNrSVdAFkMLKU8LpQB+Dai2DdL6ZaB3c9fEBBsBAbzauHzIFGABVSDuTATAWoU+6FbSCP+pPsv4NDtjwu//oEIET+bMSp9x1BIApiUUwjtjR02XF//lhEeYscGwvGG73LwHggsRSPHfvYbQODiG9INHj+j8a24nWXmTrUnDF9xciuyA5rABI5w/WCkhtfUUGyyFwwWArel99HvqKKo+gJYHgrKw0WBprkX/51UhdOtWNT30GEyRC7WiZwFhHK45U7xe7XyAlDcvOWYe4xKn3UiCPNC1gkM6nFgjVwqAAyhBZB6lvJVvVKdHdaBzBFddvDIsF8E+veV8DuPkSXgOoZL5ivQ4DoIoZnmkAGMvQJ78NtHL9ygGw5nMjHnugC1Xz9YiPd7X+UT0JAOnv44dM+OqmXGy4yGFFCof7l84jpX+Rdv/47C9teOvlE1g4Lx/6ZMc2Y/L1f039/ehrtGD9F8px3lWVigJAqA9PLmDxflOLy3Zw0pz4WwdI9YKxAiqFQF1NNXL3fAxDpetOI/J7RoLArKEu5F58OdLXnOHzyRKsNVDeaTDJpX0NimHQszparh1kAFTxhctNNVeAAVCFpDMBAGcK9Em3QSjgj/re/Y9RPPE/XZi/2GHVkxc5/EkA+KXLcvDFS7I9wh/VCaX7l/pfnl0Ji8mOxx/eg9EjFPOQCGOKBXEJOhTqMzHYa0Fj2wCWLy/EhVfPQWpm0hTrn4A6t11AvAGgN/ij+r4AUGjrIyJYHPeSIFqaA3+WwL1v/AVx77+FwlNP9bnVWEf3ANDWhLnX3Ii0ZasUPVkiEQRp4JGybpDGotY6qJVlUIxFZSAJA6CijwVXCpMCDIAqhI5FAIyV6N1gplVL1y+dX5765dA+Ix75XRfKKpOQlOS65aAcAMnVd+KQGZdtycPZX8ycFvcvwZ9UPm1ugK5Gj8PVvahr7kdWUgp0E0B6rh7xlRP4+teWIDXD4Yb0lP+P3nePABbvFbvubRtqAKRzegoIkUOgt/WA1p4uHL7/HiBJj6Iyz3vyUj+2/j709A5i/JJrsXLlEsW3YLBuYU8n0NoqGGswKCBOI1dxMEDIAKj4Y8EVw6AAA6AKkWMFAGcy9EnTH0r4o3OYTeP47V3tGB6wo7h0MmUQwR8VWv9HZbDfJtYB3vLTEliyT4j3Ag3+oDaHh4675P6j97wlf6Zj7u5feq+6uwmrch3bkJlHbXi3uhZzkwuQmBSHwoo0HDR2BLQFnOizuWUK/In3vbh/6ZgSC6AAPC95AemYWitg76svoHXbNqC4DEX5U7dbo2hcyseXvvYs5H7164p3D5E/frSwBkr9hQIEtYRB6iuW1g0qhcFwAuBzf96BlNS0Kd9wxtERXHXZWSFdh6jia5WbhlEBBkAVYkczADL0TU681vBHPXva9u39vw7i1Wf7MLs8CalpjjV1cuufdWwc9cctWHt2Oq66KR9NxsMe4Y/aBZv7j9p6S/5Ma/+oSBZAOQDu7GwWx1bnOYBwZ0fzFPij930lgA4WAJVCoD8AFOOv8Jw8mo75cgXb+nvR++Kf0HnwEJCdi8LSEue2ePbBAZGSxVA5D3lf3+yMAFa6hZwnCKT3Askd6OsxNhNgUADY3iYXGZaeUhrQ011Ly6A3IGQADGhKuHKIFWAAVCFwtAEgQ5/nydZy3Z83+KP3xyzjeOHJPuz8eBip6fEw5PQjMQnQxxWit8uK/l4bFpySgs03FyAzO0GT4A86r9Lkz1Lwh6SSOwBK8OcNAOXwJ2BP4fo/UdeHBVALAKQ+/CWI9g+BfRj84G9o3bUbMI4iOz1FpH2JS0tHyoLFyNxwwZT0L8FAII1DS2ugNJ9yEKT3PO0/HOzjUKs1g3R+tZZBLWBQ9BECVzHlAbzjt9eG1PomfS+xBTDYu3nmtGMAVDHXkQ6ADHz+J1dr+PMFgBIE7vhgGDt3jKCuqQeJOkekb05eIlacnirW/WVkeoY/qhdM8Ae1U+P+FcDX2ey0/ikBQII/KvL1fwL0gnABKwVAqufLCihBoC8roD8IpOPW7k4c/KQasI5hfkUJ9OWVSMwr8HqzBQuBoQJBAVmHHBZdKlqCoNSnVhHFYqz7J8dKr5ct8b4O09MkqLUMUp9awaCwAG7ZwADo/9HMNcKgAAOgCpEjEQAZ+pRPaLhcv55GZDGP45/7D8AwPh+JSTqUVhqQnulwC1MJVe4/6ntJhiOdiXzrt0Ddv9R+bYHDxaZk/1/pukIFgNS/v2hgUafuZPJoH65goY2P7eKka/G3d7B83rWAQOpPK7ewfGzhhEEBnIsCAziXsaqEQQFz0+gqDicAPv/Sx0hJ8bAG0DiCTVecHVIIVf4U5prTqQADoAr1IwEAGfiCm8BQwB+NxNPaP08jbDXuR17yCo+DV7rzBzU+NnzMufMHvZYHf9DraHD/0jjVuoCVAqAEgf6sgJEGgTSeULiFvYGgALWFwYOat0/lTHcVMwAG97zmVqFRgAFQha7TAYDuwEfDN+hPV3EVM6/pdMMfKe4PAN0jf6lNMO5fyfUrAGKg3qP1T0BRT6PH4A86Fqj7VwBdAOv/pgMA6Zz+INBffkDpkxOIJZDakDVw5VzvwSj+PpGhBkE6fyjXC0rXNxNhkAHQ393Nx8OpAAOgCrXDBYBs5VMxSW5NQwV/dJpArH9U35cF0F/qF2ofqPWP2mjt/hWA2N7iN/pXQF6Q6/+orb9UMNI0K3EDC+ANwBVM9b3lCFQDgdQ2WBDUMnegv09XqF3EdH4tYVAArMau4kAjimkMntYNMgD6u9v4eDgVYABUoXaoAJCBT8Wk+GkaiqAPCf7o/yzDar+D92f9ow7UAiC5fql4Cv6g993X/8lTvwjgOZn/T0vrH/XrngBagKGfCOBQAKAEgf6sgEKrE451g5EGgTSmcFgD5Tc0w6BDjWCBkKKA7/zN1SFdfyd9L/EaQL+P4hlfgQFQxS2gFQCyW1fFJATQNJqtf3SZ/nL/UZ25aQuFIvK1fwIUotj9G0oApL6VQqA/AKS+gnEHU7tgLYFyCKS/QxEk4u0jFg4YpHNraR3U2jIYKAySBfDKzevDAoAvvLDdaxDIN75xTkjHEMBjmatOowIMgCrEDxYAGfhUiB5k01DDHw0rUqx/7gBI8EclXO5fOpfS9C9U158FUKn7l/pS6gKWbiN/O4TIbzclkcHTBYFyEAwnBNJ5w7FeUJqHaIdBBsAgH+DcLCQKMACqkDUQAGS3rgqhVTYNJfzR0JSu/aO6/ty/wQR/UL9abf1GfWnt/hWQ52P9Hx1fU+I94jRQAKT+/OUDlAOgqO8nLQzVUeoKnk4InE4QjGYYFGOXrRsMNNegdD/JU8x4chMzAKp8mHNzTRVgAFQhpy8AZOBTIayGTSMN/ujSAg3+oDb+3L+S65fqzlT3rxPq6lsVAyC1CdQKKCC50n8U73S4gyUNwr020NNHNlotg+4wSK+DAUJP+QYZADV8uHNXqhVgAFQhoRwAJ5IOuPTEqVlUCKth01AFfUhDDLf1j84baPSv5PoVVqy+BizNmiOG72/rN6qjdu9f6sOb9U8c0zAAJFgAjFUIpOuKBBAUQCXbeYRehyLHoDT/WrqJtYZBs9mI//jlN0O6/k76XnrxTx95XQMY6nWIGj7CuasQKsAAqEJc6YNWU3cz8vPWq+iJm4ZCgXDAH41b6do/qhsK6x/1qzT4g+p6AkDa95dKON2/kQaAAngVuIIFSCvYKUS6p8kSuKrKv8XQWf+oI+pYTWCI/PMUzpQxSj7H4YRBGk8kbUtHFsBQB2AwACq5C7kOKcAAqOI+kD5ox1q2Ij0jRUVP3FRrBdxdv4MDduzfY8Ln1UYMDozDYNBh6QoDlq1KQWFxYlCn19L6RwNQm/qF+nDf+YPeCzT4g9rI07/s7HDsxSpt/SaOu+X+EzAXRPJnaudr/R8dD2QNINUPNBBEmvxAXMHRBoE03kixBso/bNMJgzSOcG9LxwAY1KOWG4VIAQZAFcIyAKoQL4RN3eHv8AEzXnymH+0tViQZdEhOjoN1bAIjI+PIzonHl76WgfUb06DT6RSPiuCPSqisf9S3v50/qI7S4A8BLAG4fyXXr4C9juYp8CeAcJZjL2Bv8Cfe9xL8IY4pcP+GGwDpfIFYAam+kvWAVG+6LYFyCKS/wx0t7O/DFc0wSNemJMUMA6C/u4CPh1MBBkAVajMAqhAvRE3d4a++1oKtD/ZicNCOijlJiI+fhLyJiQl0tttgMo5j07U5OOOcVMWj0sr6RyekvX89Wf/omL/gD6qj1P0ruX6pTTi2ftMCAAO1/olrCzAIRD7pwVgBlUJgoEEhAho1dgdL1xqJ1kD5PMQqDIYTAF96epvXNYBXbNkQ0nWIih+kXHFaFWAAVCE/A6AK8ULUVL7ujwDvjw/1Yeeno5i/WO/VwtfcMIasnHj8v3+fheSUOL8jC8T6R535S/1Cddj961n2YABQDQQq3SJOPtpQpocJBwTSOSLNGhirMEhBIP95x6aQwpf0vcQA6PdRPuMrMACquAUYAFWIF4Km7kEfbS1W/PbuTqSmxSMrO97rGckd3FBrwQ235GHVaf7XcgZq/aMTBxL8QfW1dP+S65eKVsEf1Fekun+lSQ6nFZDOGWhQiLAaTmNgiPzDEOnWwOmEQTq3lkEkO6uPMgCG4NnPXQanAANgcLqJVgyAKsTTuKmnfH97dhnxyO97MH+Rd+ufNIxjh8z4yuWZuOiSTJ8jiwbrH13AdG/9RmOYjvV/WgEg9aN0LaB0zmiGQHHPHG7H31GzAAAWxUlEQVQRlxLJ1sBIgkEaS6BBJOQC3nTF2WwB1Pj5z90FpwADYHC6MQCq0E3rpt6SPVPE76P/1YsFi/V+T0kA+OXLMnHxpf4BUEngB52QXL9U1Fr/qI/pyv1H5/YV/UuRv6JOketOHv7gj9ooif4V9UqVp1DRAgCpj0DXAlKbQFzBVD+S1gRKukUbBErjDveaQTpvMPkGGQD9Poq5QhgVYABUITZbAFWIp1FTXzt9UADI73/djdz8eKSle3cB2+0TOHHEgmtuysEZ69O8jkxL6x+dJFzBH3Qurdy/ctcv9esp9Yt4fxqif+UTp8YF7ITIutagrIDUPpDIYFE/QtzB0Q6CNP5IhsGwAuAf3/ceBHL9xpBaITV6vHM3IVaAAVCFwAyAKsTTqKmvZM/j4xP4n/u7cfSQBVXzvVsBO9utIjr4x7+YhUwfawUDWftHl+cv+EPJvr/UT6DWP2qjNvcf9eEp/YsSACT4o7K22PP+vqFK/xIKAKQ+g3EFMwRq9AHXoBs5EIZyBxJpqL4sgwyAGkwod6GZAgyAKqRkAFQhngZNlez0sW+3CU880ovEJB2KSqYmfB4asKO9zYavXp7hc/1fMNY/ukS17l+CPyoVqYudioUq95+AnbxycR6luf8E6EWY+5fGpIUFUPQThBWQ2gWyHpDqB5ojULQJUYoY+Ucz0nYRUfvYmG4YnF+eFb41gGwBVHu7xHx7BkAVU8wAqEI8lU19uX7lXVMqmB3bRvHGywMYGhpHTm48DJQI2jqBvh6bsPyd/YU0XPqNLCQmek8ErbX1j8YYbOoXaqs09x/VnWnuX60BUICxwi3i5PderEAgXVO0rg309ZgJNwzSWP5ZfRR33/aNkLpfnWlgGABVfsvEfvMZD4AWiwWnnXYa9u3bhz179mD58uWKZ50BULFUmlZUCn/yk9adsODzfxqxd7cJY5YJxMcD8xYZsOb0FCxeanBJEO0+2Omy/tE4psP9q3brN2EVnEb3r5YAqNYKSO1DuR4wXJZAOQTS39ESKaz0wRMuGDSOjuCqy84KDwA+9h5SUqYmtzcaR3HFTeeHdAxKded606vAjAfA73//+zh+/DjefvttBsDpvRcVnT0Y+JN3TLt+mEzjwtqXlh6naPs3La1/NJZAgj+ovuT+JdcvFbn1j14vSJ8v3penfqHXkbj1G40rlNG/0lxr5QKWAJD+D9YKGEsQKAfBWINA6d4JZRAJA6CixzxXCpMCMxoACfpuvfVWvPLKK1iyZAkDYJhuOjWnUbLuT03/ntoGAoDTlfrFHQDl8CcgpqcRy7MrxeVVdzdhVa5jrR+VnZ3NzrV/4rWHvX+VBH+Ivqc5+jcUAChBYDAAKED8hCNVTqxYAuUQSH/HKgjStWkNgwyAWj+duT81CsxYAOzs7MSqVavw2muvIS8vD5WVlX4BkNzF9E8q5AIuLS3FsZatSM/wv4OEmonitkCkwx/Nka/IXzruyfpH7we68we1OTBY62L9o/cCjf4l+KMiD/6g11rn/hNw2NTi1/pH9YLd/k3+GdHSAuiEyiADQiQIVAqAVD+YHIGiXRgCQ+Q6x+LaQG/PWi1gkAGQv8kiSYEZCYAUGHDRRRfhzDPPxG233YaGhgZFAHjHHXfgzjvvnDJ/DIChv6XVun6DHWEg1j9/AEjwR8VT8Ae9PzvlFOcwPa39o4NK3L/RvvUbXWcwyZ+doFbvsLitLgs8gbSv+ySYfYLl/QUTFELtA8kRyBAY7Cc9sHbBwmBYAfCRd5CS7GENoGkUV9x8Ia8BDGzKY7J2TAGgN0CTz1x1dTU+/fRTvPDCC9i+fTvi4+MVAyBbAKfnMxBN8EcKqU39Qn0EEvxB9ad767dIyP0nB0Ct4c/Zt0oroAC6SuVgGi2WQHEPRtlWclo9zQKBQQZArVTnfrRQIKYAsKenB/TPV6moqMCmTZvw5ptvugQA2O12AYObN2/Gk08+qUhbjgJWJJOqStMFfzRoLa1/1F8o3b+S65fOE0jwh7CUyXL/0Wut3b/k+qXiL/iD6kSq+1cOgEKzINLCiLkJcD0gtQkmR6BoF2Z38EyGQOn+8AeDDICqvg64scYKxBQAKtWmqakJBG9SaWtrwwUXXICXX35ZpISZPdvzDgbu/TMAKlU8+HrTse5Pgj/6X8t9f6k/zv3n/V4g+BOgGMTev/JeQ7H+z6V/FVZAhsDgnwXR1tITDDIARtssxvZ4ZyQAuk+p0jWADIDh/TBMF/xJAKgU/qi+VsEf1Fcg7l9y/VIJNPiD2sijf2Mh9590d4YDAOlcwVoBJQgMhyuYzkWWwJVzlbudtfqUz1SXsCf9JBg0m4341U+uDOn6O2ciaF4DqNWtHLP9MAACitcAMgCG73Mw3a7fQKx//gAw0OAP6i9St35z3/aNxqpF6hfqRwv3L/UTagAU51BpBWQIDN+zJJLORBbAzZecGR4A/J+3vAeB3PLlkI4hkjTnsXhXgAFQxd3BLmAV4vloOp3wF6z1j9rNxOAPrQBQS/gT1jmNI4A93a5qITDY9YA0lkiPDJbrxZbASTUYAEPzncG9BqcAA2BwuolWDIAqxPPSNBLgT0vrH/UVyuAP6j9Q92+4cv+FO/gjXNY/6dZVmxZGsgIKoIvRyGAGQdcHHQOg9t8Z3GPwCjAABq8dA6AK7bw1nc51f6Gy/lG/WgV/UF+etn7TOvdfuHf+oOtSG/wRbgAU59PIFcwQGIKHSQR2yQAYgZMyg4fEAKhi8tkCqEI8D01nsvWP5Agk+IPqc+6/qTdRONb/yc+qhRWQLYHaPkciubdwAuCL//W61zWAV37vawGvAfzf//1f3HvvvWhvbxdbpz7wwAM4++yzvcr90Ucfia1WDx48iOLiYvz4xz/Gt771rUienhk3NgZAFVPOAKhCPLem0w1/wVj/qE0w0b+07RsV950/6D1vwR90zH3rN87953oTEfxRCcf6P3cIVBMRLPUV6E4h1C6acgTKNZup6wKjFQBp44QtW7aAIJB20HrkkUewdetWHDp0CGVlZVO+COrr63HKKafgpptuws0334xPPvkE3/nOd/Dcc8/h8ssv1+6Lg3tSpQADoAr5GABViCdrGgnwFwwABgN/dB73fX/pPSVbv0muX6rvbv2j95ZmzRGq7u5pxPLsSvF3dXeT+H9Vbrn4X576RbzuaPaZ+Fn00dqKUEX/apX7T1x3fWvY4U+6jbVwBVNfMwkCxX08w3YPiVYApPy4K1euxEMPPeR8ci9atAiXXHIJfvWrX035IvjJT36CN954A4cPO37sUiHr3759+/DZZ59p88XBvahWgAFQhYSDg4PIysrC54f/G2npySp6mrlN+y17xcWnJK2ZVhG6zPuRpV8R0BjaTAeRa1jmtU2z8QgKDJP7+0oVG0aPoSRlkUu7E8MnUJa60Pne0aFazEmb73x9eKge89Lmitc1gw3i/0XpVeL/ff2NWJJZIf7e29ss/j81ywF8n/e0YHlOqbOf3d2tWJEre93ZilX5kznidne0YVWBa864z9vasaqw2GW8n7e2TXlPXuHzljasLHJt40moPc3tWFlSFJDu3irvbWzHilJt+gp0QHvr20WTFeXqzr+vztHP8gD6qak92aYy8HPXHHe0XTYn8LaBauSt/v6jbeLQ0rn+7xetzjld/RiNo7jpm1/EwMAAMjMzQzIMyTDxxG+eQ0pyypRzGE1GXPeTq9Dc3IyMjAzncb1eD/rnXsbGxpCSkoKXXnoJl156qfPw97//fezduxfk6nUv55xzDlasWIHf//73zkOvvvoqrrzyShiNRiQmJobk2rnTwBRgAAxML5faLS0tKC2d/DJV0RU3ZQVYAVaAFZghChB8Kd1xKlBJzGYzKisr0dHR4bVpWloaRkZGXI7ffvvtuOOOO6a0oZ2ySkpKhBv3jDPOcB6/++67xbapR48endJm/vz5uO666/Czn/3MeezTTz8V7mPqr6ho+n5wBKpnLNdnAFQxu+Pj4+JmTk9Pd9lXWN4l/RojSHT/taXitGFrymMPm9TOE7Hm4deczhitukfruKNZczVjn5iYwPDwsAiKiIuLC9nNThBIljtvhcah0+lcDnuzAEoASAC3bt06Z5tf/vKXePrpp3HkyBGPAHj99dfjpz/9qfMYAeRZZ50lgkgKCwtDdu3csXIFGACVaxVUTckcT+5iubk9qM7C3IjHHmbBZbkl+X4Jr/bReq9H67gliCI3KN/r4b3XAz0bu4ADVSx66jMAhniu+AEdYoG9dB+tukfruPkLne/zQBXgez1QxaavPgWBrFq1SkQBS2Xx4sX42te+5jUI5M033xRRwlL59re/LdYMchDI9M2j+5kZAEM8F/yQC7HADIDTI7CHs/K9Hv6pYM3Dr3m0/9gJRjEpDczDDz8s3MCPPvooHnvsMZHjr7y8XLh6W1tb8dRTT4nupTQwlAKGUsEQ9FEUMKeBCUb90LVhAAydtqJni8UifiHRB8RThFWIT6+qex67KvmCasyaByWb6kbRqnu0jpufjapv2bB3QNa/e+65R6zhoxx/v/vd70DRvlQo4KOhoQEffvihc1wUHfyDH/zAmQiaUsNwIuiwT5vPEzIARtZ88GhYAVaAFWAFWAFWgBUIuQIMgCGXmE/ACrACrAArwAqwAqxAZCnAABhZ88GjYQVYAVaAFWAFWAFWIOQKMACGXGI+ASvACrACrAArwAqwApGlAANgZM0Hj4YVYAVYAVaAFWAFWIGQK8AAGHKJPZ+AovcotxJtjr1nzx4sX758mkai7LRf/epXRQ6nrq4uZGdn47zzzsNvfvMbkdE+kgtFpt1111344IMPxNZINN6rr74aP//5z5GUlBTJQxdjo2z7f/nLX4T2NF7aQzQSC0UI3nvvvSJCcMmSJXjggQdw9tlnR+JQXca0fft2Me7du3eLsdN+pbTBfaQXyizw5z//WezCkJycLLboos/jggULIn3oeOihh8Q/+mxSofvlF7/4Bb70pS9F/NjlA6Q5oK3OaE9cut+5sALRpgAD4DTNGD00jh8/jrfffjsqAJBC/in/E+3hSPmefvSjHwnlaHugSC7vvPMOKIfVVVddhblz5+LAgQMiL9WWLVtw3333RfLQxdhof86srCzQvtOPP/54RAKglCOMIJD2+nzkkUewdetWkQS2rKwsojWmzx9tUbVy5UpcfvnlUQOAF154ITZt2oQ1a9bAZrOJHzT79+8Xmqempka05pQgOD4+XnweqdB+sgTh9EOYYDAaSnV1Na688kqxu9OGDRsYAKNh0niMUxRgAJyGm4K+dG699Va88sor4oEXDRZAd5neeOMNYSkhS2ZiYuI0qBj8KenLhiwQdXV1wXcS5pZPPPEE/vVf/zUiAZAs2QRQpKlUFi1aJO4PspJES6G9UaPFAuiuaXd3NwoKCkC516TcbNGiO40zJydHQOANN9wQ8cMeGRkR9zv94PnP//xP4b1hC2DETxsP0IMCDIBhvi06OzvFljqvvfYa8vLyUFlZGXUA2NfXB9rWhyyBO3bsCLOC6k932223gSyDu3btUt9ZmHqIVAAMZp/QMEkW8GmiGQBPnDiBefPmCSsgJemNlmK32/HSSy/h2muvFc9B2l4s0guNlYCVvCLnnnsuA2CkTxiPz6sCDIBhvDkmJiZw0UUXCTcZQQitgYkmAKRM7g8++CCMRiNOP/10vPXWW8jNzQ2jgupPVVtbK36933///bjxxhvVdximHiIVANva2lBSUiLcqLQOTSp33323cO0dPXo0TAqpP020AiA9V2hP1v7+fnz88cfqhQhDDwSqtKTEbDYjLS0Nzz77rHg2Rnp5/vnnhdWPfjwaDAYGwEifMB6fTwUYADW4Qe644w7ceeedPnuiNSO0Xo7WS9HCc1oDM90AqHTcq1evFtfW09MDsv41NjaK683MzBQQSF+c4S6Bjp3GR7Cyfv168Y/WqE1XCWbskQ6AdG/TF7pUKHjl6aefFkEK0VKiFQBvueUWEShE1vjZs2dHhdxkOW5qahJLGmgpDH0eyX0dyRbA5uZm0LPw3XffxbJly4TObAGMituNB+lFAQZADW4NAiP656tUVFSIRdu0AFoOTOQCIRjcvHmzsJiEsygdN/3SdS8UlFBaWiqgVv7FH67xBzp2gj9arE3r1Qim4uLiwjXUKecJdOzUQaQCILuAp+02Eif+7ne/K5aT0I9K8iZEa6GsAlVVVSKAKFIL6XzppZeK57VU6PlNz3N6ntB6aPmxSL0OHhcrICnAABjGe4F+8Q4NDTnPSFBywQUX4OWXXxZgEi2/3ukC6NcwRXhu27ZN/AqO5EJrFQn+aO3lM888E5UP6UgFQJp3undJW1oULxWy5JBbkoNAQvPJILcvwR8FrXz44Ydi/V80l40bN4oflHSfR2oZHh4W3g95uf7667Fw4ULQ8phoWnsZqRrzuMKrAANgePV2Odt0u4CVXvrOnTtB/8466yyRA5CiZylvF+VNO3jwIPR6vdKuwl5PcvsSrD711FMu8FdYWBj28QR6QvrRQG53irqmKElpjRel0KC1U5FQpDQwDz/8sLAGP/roo3jsscfEvVFeXh4JQ/Q6BoropAAKKitWrMBvf/tb8WOBFvlHcgqb73znO2Ld3Ouvv+6S+4+WZVBewEgulDuPcv4R8BFU0bq6X//61yIw6/zzz4/koU8ZG7uAo2q6eLBuCjAATuMtES0ASAu2KW8hJa0eHR0VuQApDxkFslAAQCQXsijQr3RPhawokV6uu+46j0sDIs3ySta/e+65R/woIEsIRUhGQzoSsp4R8LkXivSMZGuUt3W3f/zjH0H3TCQXSvXy/vvvi3uFgHXp0qXCghZt8EcaMwBG8p3GY/OnAAOgP4X4OCvACrACrAArwAqwAjGmAANgjE0oXw4rwAqwAqwAK8AKsAL+FGAA9KcQH2cFWAFWgBVgBVgBViDGFGAAjLEJ5cthBVgBVoAVYAVYAVbAnwIMgP4U4uOsACvACrACrAArwArEmAIMgDE2oXw5rAArwAqwAqwAK8AK+FOAAdCfQnycFWAFWAFWgBVgBViBGFOAATDGJpQvhxVgBVgBVoAVYAVYAX8KMAD6U4iPswKsACvACrACrAArEGMKMADG2ITy5bACrAArwAqwAqwAK+BPAQZAfwrxcVaAFWAFWAFWgBVgBWJMAQbAGJtQvhxWgBVgBVgBVoAVYAX8KcAA6E8hPs4KsAKsACvACrACrECMKcAAGGMTypfDCrACrAArwAqwAqyAPwUYAP0pxMdZAVaAFWAFWAFWgBWIMQUYAGNsQvlyWAFWgBVgBVgBVoAV8KcAA6A/hfg4K8AKsAKsACvACrACMaYAA2CMTShfDivACrACrAArwAqwAv4UYAD0pxAfZwVYAVaAFWAFWAFWIMYUYACMsQnly2EFWAFWgBVgBVgBVsCfAgyA/hTi46wAK8AKsAKsACvACsSYAgyAMTahfDmsACvACrACrAArwAr4U4AB0J9CfJwVYAVYAVaAFWAFWIEYU4ABMMYmlC+HFWAFWAFWgBVgBVgBfwowAPpTiI+zAqwAK8AKsAKsACsQYwowAMbYhPLlsAKsACvACrACrAAr4E8BBkB/CvFxVoAVYAVYAVaAFWAFYkwBBsAYm1C+HFaAFWAFWAFWgBVgBfwpwADoTyE+zgqwAqwAK8AKsAKsQIwpwAAYYxPKl8MKsAKsACvACrACrIA/BRgA/SnEx1kBVoAVYAVYAVaAFYgxBRgAY2xC+XJYAVaAFWAFWAFWgBXwp8D/B6qJ6lgwN8dtAAAAAElFTkSuQmCC\" width=\"640\">"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"res = train_and_plot(30,net,lr=0.01)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Why Not Always Use Multi-Layered Model?\n",
"\n",
"We have seen that multi-layered model is more *powerful* and *expressive*, than one-layered one. You may be wondering why don't we always use many-layered model. The answer to this question is **overfitting**.\n",
"\n",
"We will deal with this term more in a later sections, but the idea is the following: **the more powerful the model is, the better it can approximate training data, and the more data it needs to properly generalize** for the new data it has not seen before.\n",
"\n",
"**A linear model:**\n",
"* We are likely to get high training loss - so-called **underfitting**, when the model does not have enough power to correctly separate all data. \n",
"* Valiadation loss and training loss are more or less the same. The model is likely to generalize well to test data.\n",
"\n",
"**Complex multi-layered model**\n",
"* Low training loss - the model can approximate training data well, because it has enough expressive power.\n",
"* Validation loss can be much higher than training loss and can start to increase during training - this is because the model \"memorizes\" training points, and loses the \"overall picture\"\n",
"\n",
"\n",
"\n",
"> On this picture, `x` stands for training data, `o` - validation data. Left - linear model (one-layer), it approximates the nature of the data pretty well. Right - overfitted model, the model perfectly well approximates training data, but stops making sense with any other data (validation error is very high)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Takeaways\n",
"\n",
"* Simple models (fewer layers, fewer neurons) with low number of parameters (\"low capacity\") are less likely to overfit\n",
"* More complex models (more layers, more neurons on each layer, high capacity) are likely to overfit. We need to monitor validation error to make sure it does not start to rise with further training\n",
"* More complex models need more data to train on.\n",
"* You can solve overfitting problem by either:\n",
" - simplifying your model\n",
" - increasing the amount of training data\n",
"* **Bias-variance trade-off** is a term that shows that you need to get the compromise\n",
" - between power of the model and amount of data,\n",
" - between overfittig and underfitting\n",
"* There is not single recipe on how many layers of parameters you need - the best way is to experiment"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Credits\n",
"\n",
"This notebook is a part of [AI for Beginners Curricula](http://github.com/microsoft/ai-for-beginners), and has been prepared by [Dmitry Soshnikov](http://soshnikov.com). It is inspired by Neural Network Workshop at Microsoft Research Cambridge. Some code and illustrative materials are taken from presentations by [Katja Hoffmann](https://www.microsoft.com/en-us/research/people/kahofman/), [Matthew Johnson](https://www.microsoft.com/en-us/research/people/matjoh/) and [Ryoto Tomioka](https://www.microsoft.com/en-us/research/people/ryoto/), and from [NeuroWorkshop](http://github.com/shwars/NeuroWorkshop) repository."
]
}
],
"metadata": {
"celltoolbar": "Slideshow",
"interpreter": {
"hash": "86193a1ab0ba47eac1c69c1756090baa3b420b3eea7d4aafab8b85f8b312f0c5"
},
"kernelspec": {
"display_name": "Python 3.9.5 64-bit ('base': conda)",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.5"
},
"livereveal": {
"start_slideshow_at": "selected"
}
},
"nbformat": 4,
"nbformat_minor": 1
}