6g-ai-security.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "oriental-defensive",
   "metadata": {},
   "source": [
    "# 6G and Artificial Intelligence With Security Problems\n",
    "## 6G solutions with Adversarial Machine Learning Attacks: Millimeter Wave Beam Prediction Use-Case\n",
    "\n",
    "### 1. Introduction to 6G\n",
    "6G is the next generation for the communication systems. In recent years, machine learning algorithms have been applied widely in various fields such as health, transportation, and the autonomous car. The predictive algorithms will be used in 6G problems. With the rapid developments of deep learning techniques, it is critical to take the security concern into account to apply the algorithms. While machine learning offers significant advantages for 6G, AI models' security is ignored. Since it has many applications in the real world, security is a vital part of the algorithms. This post has proposed a mitigation method for adversarial attacks against 6G machine learning models with adversarial learning. The main idea behind adversarial attacks against machine learning models is to produce faulty results by manipulating trained deep learning models for 6G applications. \n",
    "\n",
    "Up to 4G, time-frequency domain technology has been explored to increase overall system capacity. The recent developments in 5G and beyond technologies support emerging applications such as smart homes, vehicular networks, augmented reality (AR), virtual reality (VR) with unprecedented rates enabled by recent advances in massive multiple-input multiple-output (MIMO), mmWave communications, network slicing, small cells, and Internet of things (IoT). These complex structures of 5G and beyond technologies  can be captured by using **data-driven approach** machine learning (ML) algorithm. The strong learning, reasoning, intelligent recognition abilities of ML allow the network structure to train and adapt itself to support diverse demands of the systems without human intervention.\n",
    "\n",
    "On the other hand, 6G is vulnerable for the adversarial machine learning attacks. Figure shows an attack scenario. The attacker gets an legitimate input, $\\mathbf{x}$, creates a noise vector with an $\\epsilon$ budget $\\eta = \\epsilon \\cdot sign(\\nabla_x \\ell(\\mathbf{\\theta},\\mathbf{x},y))$, sums the input instance and the craftily designed noise to create adversarial input $\\mathbf{x}^{adv} = \\mathbf{x} + \\eta$.\n",
    "\n",
    "![Adversarial Machine Learning Attack in 6G](blacksea_paper/img/adv-ml.png)\n",
    "\n",
    "### 2. Adversarial Machine Learning\n",
    "Adversarial machine learning is an attack technique that attempts to fool neural network models by supplying craftily manipulated input with a small difference. The obvious intention is to produce a failure in a neural network.\n",
    "\n",
    "There are many attacking methods for deep learning models, and FGSM is the most straightforward and powerful attack type. FGSM works by utilizing the gradients of the neural network to create an adversarial example to evade the model. For an input instance $\\mathbf{x}$.\n",
    "\n",
    "\\begin{equation}\n",
    "\t\\mathbf{x}^{adv} = \\mathbf{x} + \\epsilon \\cdot sign(\\nabla_x \\ell(\\mathbf{\\theta},\\mathbf{x},y))\n",
    "\\end{equation}\n",
    "\n",
    "![FGSM attack](blacksea_paper/img/fgsm_detail.png)\n",
    "\n",
    "### 3. Solution: Adversarial Training\n",
    "\n",
    "Adversarial training is a widely recommended defense that implies generating adversarial instances using the gradient of the victim classifier, and then re-training the model with the adversarial instances and their respective labels. This technique has demonstrated to be efficient in defending models from adversarial attacks. \n",
    "\n",
    "Let us first think a common classification problem with a training instances $X \\in \\mathbb{R}^{m \\times n}$ of dimension $d$, a label space $Y$ We assume the classifier $h_\\theta$ has been trained to minimize a loss function $\\ell$as follows:\n",
    "\n",
    "\\begin{equation}\n",
    "\t\\label{eq:cost-func}\n",
    "\t\\underset{\\theta}{min}\\frac{1}{m} \\sum_{i=1}^{m} \\ell(h_\\theta(\\mathbf{x}_i,y_i))\n",
    "\\end{equation}\n",
    "\n",
    "The following figure shows the diagram of **RF beamforming codeword adversarial training**.\n",
    "\n",
    "![FGSM attack](blacksea_paper/img/deepmimo_adv_learning.png)\n",
    "\n",
    "### 4. Let’s coding\n",
    "\n",
    "We import the usual standard libraries to build a deep learning model to predict RF beamforming codeword."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "hearing-benefit",
   "metadata": {},
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "from scipy.io import loadmat, savemat\n",
    "import numpy as np\n",
    "from keras.models import Sequential\n",
    "from keras.layers import Dense\n",
    "from tqdm import tqdm\n",
    "from keras.callbacks import ModelCheckpoint\n",
    "import pickle \n",
    "from sklearn.metrics import mean_squared_error\n",
    "import matplotlib.pyplot as plt\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "direct-majority",
   "metadata": {},
   "source": [
    "In this work, we will use publicly avaliable [DeepMIMO dataset](https://www.deepmimo.net) to attack RF beam forming prediction model. We will use FGSM attack, and **Mean Squared Error** loss function for the input manipulation. Our FGSM attack implementation is here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "postal-olive",
   "metadata": {},
   "outputs": [],
   "source": [
    "loss_object = tf.keras.losses.MeanSquaredError()\n",
    "\n",
    "def fgsm(model, input_instance, label, n_BS,n_beams, epsilon =0.01):\n",
    "    tensor_input_instance = tf.convert_to_tensor(input_instance, dtype=tf.float32)\n",
    "    adv_x = input_instance\n",
    "    for idx in range(0, n_BS*n_beams-2, n_beams):\n",
    "        with tf.GradientTape() as tape:\n",
    "            tmp_label = label[:, idx:idx + n_beams]\n",
    "            tape.watch(tensor_input_instance)\n",
    "            prediction = model(tensor_input_instance)\n",
    "            loss = loss_object(tmp_label, prediction)\n",
    "            gradient = tape.gradient(loss, tensor_input_instance)\n",
    "            signed_grad = tf.sign(gradient)\n",
    "            adv_x = adv_x + eps * signed_grad\n",
    "    return adv_x"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "working-cruise",
   "metadata": {},
   "source": [
    "## The parameters in our scenario\n",
    "\n",
    "| Parameter      | Value |\n",
    "| ----------- | ----------- |\n",
    "| Num of BS      | 4       |\n",
    "| Train Test Split   | 0.8        |\n",
    "| Number of beams | 512 |\n",
    "| Activation Function | ReLU|\n",
    "| Loss| Mean Squared Error|\n",
    "| Optimizer| Rmsprop|"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "outstanding-appeal",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Reading input and output sets generated from MATLAB\n",
    "# with DeepMIMO generator http://www.deepmimo.net\n",
    "In_set_file=loadmat('DLCB_dataset/DLCB_input.mat')\n",
    "Out_set_file=loadmat('DLCB_dataset/DLCB_output.mat')\n",
    "\n",
    "In_set=In_set_file['DL_input']\n",
    "Out_set=Out_set_file['DL_output']\n",
    "\n",
    "# Parameter initialization\n",
    "num_user_tot=In_set.shape[0]\n",
    "\n",
    "DL_size_ratio = .8\n",
    "count=0\n",
    "num_tot_TX=4\n",
    "num_beams=512"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "quick-ceremony",
   "metadata": {},
   "source": [
    "# The train procedure"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "literary-packet",
   "metadata": {},
   "outputs": [],
   "source": [
    "def train(In_train, Out_train, In_test, Out_test,\n",
    "          nb_epoch, batch_size,dr,\n",
    "          num_hidden_layers, nodes_per_layer,\n",
    "          loss_fn,n_BS,n_beams):\n",
    "    \n",
    "    in_shp = list(In_train.shape[1:])\n",
    "\n",
    "    AP_models = []\n",
    "    for idx in range(0, n_BS*n_beams-2, n_beams):\n",
    "        idx_str = str(idx / n_beams + 1)\n",
    "        act_func = 'relu'\n",
    "        model = Sequential()\n",
    "        model.add(Dense(100, input_dim=in_shp[0], activation=act_func))\n",
    "        model.add(Dense(100, activation=act_func))\n",
    "        model.add(Dense(100, activation=act_func))\n",
    "        model.add(Dense(100, activation=act_func))\n",
    "        model.add(Dense(n_beams, activation=act_func))\n",
    "        model.compile(loss=loss_fn, optimizer='rmsprop', metrics=['mean_squared_error'])\n",
    "        \n",
    "        history = model.fit(In_train,\n",
    "                            Out_train[:, idx:idx + n_beams],\n",
    "                            batch_size=batch_size,\n",
    "                            epochs=nb_epoch,\n",
    "                            verbose=0,\n",
    "                            validation_data=(In_test, Out_test[:,idx:idx + n_beams]))\n",
    "        \n",
    "        filehandler = open('history.pkl', 'wb') \n",
    "        pickle.dump(history.history, filehandler)\n",
    "        filehandler.close()\n",
    "        \n",
    "        AP_models.append(model)\n",
    "    return AP_models"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "passive-possession",
   "metadata": {},
   "source": [
    "# Training process (Normal Behaviour)\n",
    "\n",
    "Here we will train the RF beamforming codeword prediction model with out any attacker. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "artistic-coral",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "mse: 0.00030647155448710575\n"
     ]
    }
   ],
   "source": [
    "count=count+1\n",
    "DL_size=int(num_user_tot*DL_size_ratio)\n",
    "\n",
    "np.random.seed(2016)\n",
    "n_examples = DL_size\n",
    "num_train  = int(DL_size * 0.8)\n",
    "num_test   = int(num_user_tot*.2)\n",
    "\n",
    "train_index = np.random.choice(range(0,num_user_tot), size=num_train, replace=False)\n",
    "rem_index = set(range(0,num_user_tot))-set(train_index)\n",
    "test_index= list(set(np.random.choice(list(rem_index), size=num_test, replace=False)))\n",
    "\n",
    "In_train = In_set[train_index]\n",
    "In_test =  In_set[test_index] \n",
    "\n",
    "Out_train = Out_set[train_index]\n",
    "Out_test = Out_set[test_index]\n",
    "\n",
    "#In_train, In_test, Out_train, Out_test =  train_test_split(In_set, Out_set, test_size=0.33)\n",
    "\n",
    "# Learning model parameters\n",
    "nb_epoch = 10    \n",
    "batch_size = 100  \n",
    "dr = 0.05                  # dropout rate  \n",
    "num_hidden_layers=4\n",
    "nodes_per_layer=In_train.shape[1]\n",
    "loss_fn='mean_squared_error'\n",
    "\n",
    "eps = 2.0 * 16.0 / 255.0\n",
    "\n",
    "\n",
    "# Model training\n",
    "AP_models = train(In_train, Out_train, In_test, Out_test,\n",
    "                                      nb_epoch, batch_size,dr,\n",
    "                                      num_hidden_layers, nodes_per_layer,\n",
    "                                      loss_fn,num_tot_TX,num_beams)\n",
    "# Model running/testing\n",
    "DL_Result={}\n",
    "mse_list = []\n",
    "for id in range(0,num_tot_TX,1): \n",
    "    beams_predicted=AP_models[id].predict( In_test, batch_size=10, verbose=0)\n",
    "\n",
    "    DL_Result['TX'+str(id+1)+'Pred_Beams']=beams_predicted\n",
    "    DL_Result['TX'+str(id+1)+'Opt_Beams']=Out_test[:,id*num_beams:(id+1)*num_beams]\n",
    "\n",
    "    mse = mean_squared_error(Out_test[:,id*num_beams:(id+1)*num_beams],beams_predicted)\n",
    "    mse_list.append(mse)\n",
    "print('mse:',np.mean(mse_list))\n",
    "\n",
    "DL_Result['user_index']=test_index"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "peaceful-behalf",
   "metadata": {},
   "source": [
    "# Attack the RF Beamforming Codeword Prediction Model\n",
    "\n",
    "The RF beamforming prediction model's performance is quite good which is 0.00039. Now, we will attack the model with FGSM."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "antique-coverage",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "mse: 0.0151878763143215\n"
     ]
    }
   ],
   "source": [
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "# Model running/testing\n",
    "DL_Result={}\n",
    "mse_list = []\n",
    "for id in range(0,num_tot_TX,1): \n",
    "    # !!!!! Attack generation !!!!\n",
    "    In_test_adv = fgsm(AP_models[id], In_test,Out_test,num_tot_TX,num_beams,eps)\n",
    "    beams_predicted=AP_models[id].predict( In_test_adv, batch_size=10, verbose=0)\n",
    "\n",
    "    DL_Result['TX'+str(id+1)+'Pred_Beams']=beams_predicted\n",
    "    DL_Result['TX'+str(id+1)+'Opt_Beams']=Out_test[:,id*num_beams:(id+1)*num_beams]\n",
    "\n",
    "    mse = mean_squared_error(Out_test[:,id*num_beams:(id+1)*num_beams],beams_predicted)\n",
    "    mse_list.append(mse)\n",
    "print('mse:',np.mean(mse_list))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dense-barbados",
   "metadata": {},
   "source": [
    "## Adversarial Training\n",
    "The attacked model's performance (i.e. $mse=0.01532$) is worst than the normal performance (i.e. $mse=0.00039$). Let's train the normal model with adversarial training."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "adolescent-chess",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Model training function\n",
    "def adv_train(In_train, Out_train, In_test, Out_test,\n",
    "          nb_epoch, batch_size,dr,\n",
    "          num_hidden_layers, nodes_per_layer,\n",
    "          loss_fn,n_BS,n_beams, eps):\n",
    "    \n",
    "    in_shp = list(In_train.shape[1:])\n",
    "\n",
    "    AP_models = []\n",
    "    mcp_save = ModelCheckpoint('model.hdf5', save_best_only=True, verbose=0, \n",
    "                                   monitor='val_mean_squared_error', mode='min')\n",
    "    \n",
    "    for idx in range(0, n_BS*n_beams-2, n_beams):\n",
    "        idx_str = str(idx / n_beams + 1)\n",
    "        act_func = 'relu'\n",
    "        model = Sequential()\n",
    "        model.add(Dense(100, input_dim=in_shp[0], activation=act_func))\n",
    "        model.add(Dense(100, activation=act_func))\n",
    "        model.add(Dense(100, activation=act_func))\n",
    "        model.add(Dense(100, activation=act_func))\n",
    "        model.add(Dense(n_beams, activation=act_func))\n",
    "        model.compile(loss=loss_fn, optimizer='rmsprop', metrics=['mean_squared_error'])\n",
    "        \n",
    "        history = model.fit(In_train,\n",
    "                            Out_train[:, idx:idx + n_beams],\n",
    "                            batch_size=batch_size,\n",
    "                            epochs=nb_epoch,\n",
    "                            verbose=0,\n",
    "                            validation_data=(In_test, Out_test[:,idx:idx + n_beams]))\n",
    "        \n",
    "        callbacks = [mcp_save]\n",
    "        for _ in range(10):\n",
    "            In_train_adv = fgsm(model, In_train,Out_train, n_BS, n_beams)\n",
    "            In_train_adv = np.concatenate((In_train, In_train_adv), axis=0)\n",
    "            \n",
    "            Out_train_adv = np.concatenate((Out_train, Out_train), axis=0)\n",
    "            \n",
    "            history = model.fit(In_train_adv,\n",
    "                                Out_train_adv[:, idx:idx + n_beams],\n",
    "                                batch_size=batch_size,\n",
    "                                epochs=nb_epoch*3,\n",
    "                                verbose=0,\n",
    "                                callbacks=callbacks,\n",
    "                                validation_data=(In_test, Out_test[:,idx:idx + n_beams]))\n",
    "            model.load_weights('model.hdf5')\n",
    "        \n",
    "        AP_models.append(model)\n",
    "    return AP_models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "conventional-mumbai",
   "metadata": {},
   "outputs": [],
   "source": [
    "AP_models = adv_train(In_train, Out_train, In_test, Out_test,\n",
    "                      nb_epoch, batch_size,dr,\n",
    "                      num_hidden_layers, nodes_per_layer,\n",
    "                      loss_fn,num_tot_TX,num_beams,eps)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "efficient-lafayette",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "mse: 0.0006801871943259285\n"
     ]
    }
   ],
   "source": [
    "# Model running/testing\n",
    "DL_Result={}\n",
    "mse_list = []\n",
    "for id in range(0,num_tot_TX,1): \n",
    "    # !!!!! Attack generation !!!!\n",
    "    In_test_adv = fgsm(AP_models[id], In_test,Out_test,num_tot_TX,num_beams,eps)\n",
    "    beams_predicted=AP_models[id].predict( In_test_adv, batch_size=10, verbose=0)\n",
    "\n",
    "    DL_Result['TX'+str(id+1)+'Pred_Beams']=beams_predicted\n",
    "    DL_Result['TX'+str(id+1)+'Opt_Beams']=Out_test[:,id*num_beams:(id+1)*num_beams]\n",
    "\n",
    "    mse = mean_squared_error(Out_test[:,id*num_beams:(id+1)*num_beams],beams_predicted)\n",
    "    mse_list.append(mse)\n",
    "print('mse:',np.mean(mse_list))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "banned-afghanistan",
   "metadata": {},
   "source": [
    "### All results\n",
    "Now the RF beamforming codeword prediction model's performance under attack is better than the undefended model performance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "respective-boring",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAD7CAYAAAB9nHO6AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAA41UlEQVR4nO3df3wUd73o/9e7CQRqEhtSEyBLb4JNoklsONvQNDb8qN7apvoFey73tBx7W6ynHLS16qVfLlz5Yk8PStTwOF5stegR24qn0OM9VdQi9vaQQDBE0kgqiSbBhEIokGvAQk7pQsLn+8fMhmWbn7vZDDvzfj4e+2Bn5jMz782H9352PjPzGTHGoJRSSo3WNU4HoJRSKr5ow6GUUmpMtOFQSik1JtpwKKWUGhNtOJRSSo2JNhxKKaXGJKYNh4jcJSKtInJYRFYPslxEZJO9/HUR8Ycs2yIi3SJyaJD1Pm9vt1lEvhHLz6CUUupKMWs4RCQBeBqoAAqApSJSEFasAsi1X8uB74Ysexa4a5Dt3g4sBm4yxhQCVeMevFJKqSElxnDbtwCHjTEdACKyDesLvyWkzGLgeWPdhbhfRK4TkRnGmBPGmD0ikj3Idj8LVBpjAgDGmO6RArn++utNdvZgm3KHCxcuMHnyZKfDUBHQuotvbq+/11577c/GmPeFz49lw5EFHAuZ7gJKR1EmCzgxzHbzgHki8lXgHeBxY8yB4QLJzs6moaFhtHHHnerqahYuXOh0GJ7wq1/BF74A/f3wd38Hq8M6YI2xlr/8Mlx7LTz7LPjtDtiHHoJf/AIyMuCQ3QFbXV1NdfVCvv99eJ+dnl/7Gtx9N/z2t7B8+eXtPvEE3HOPNX3XXXDiBPT1wbx58PTTkJAQ60+vwrk990TkjcHmx7LhkEHmhY9vMpoy4RKBNOBWYC7woojMNmFjp4jIcqzuL2bOnEl1dTUAs2fPJiUlhaamJgDS09MpLCxkz5491sYTEykvL6exsZGzZ88CUFJSwqlTpzh2zGrjcnNzSUpK4pCd/RkZGeTl5VFbWwtAUlISZWVlNDQ00NvbC0BpaSldXV0cP34cgPz8fBISEmhpsQ7Apk+fTk5ODnV1dQBMnTqV0tJS6uvrOX/+PABlZWV0dnZy8uRJAAoKCujv76e/v5/q6mqysrLw+XzU19cDkJycTElJCXV1dQQCAQDKy8tpa2uju9s6UCsqKiIQCNDe3g7ArFmzyMzMHGhoU1NT8fv91NbW0tfXB8D8+fNpbm6mp6cHgOLiYs6dO0dHRwdgNdTTpk2jsbERgLS0NIqLi6mpqcEYg4iwYMECmpqaOHPmDAB+v5/Tp09z5MiRq7ae+vth+fIPs2uXoatrPytW3Ex2dgf33XfTQD3t3z+N1tYifv7zNnbvfptPfepGamreob+/n+Lik9x6az9VVR/i/PlL1NfX09/fz5tvvsmXvjST2267XE99feUkJLSzcWM3CQmG66//EAsXXkdqai0JCYbKyhu48cYMDhxo4CtfKeQb3+hlzZocradxyKfW1laAUeVTUVERLS0trs2nIRljYvICyoBdIdNrgDVhZTYDS0OmW4EZIdPZwKGwdX4FLAyZ/hPwvuFiufnmm42bvfHGG06H4Am/+Y0xH/vY5emvfc16hVq+3Jh/+ZfL03l5xrz55uXpzk5jCgsvT7/xxhvmK18x5pvfHH7fHR3GZGQYc/HilfMvXDDmE58wZtu2sXwSNV7cnntAgxnkOzWWV1UdAHJFJEdEJgP3ATvCyuwAHrCvrroVeMsYM1w3FcBPgY8AiEgeMBn487hGHmeCv0xUbB0/DrNmXZ72+ax5Yy0TKlh3Tz0FN91kdWfZPxoBqK+HwkL40IfgmWcgMaSP4M47rW6vlBRYsiSKD6Yi5tXci1nDYYzpAx4FdgF/AF40xjSLyAoRWWEXexnoAA4D3wc+F1xfRF4A6oB8EekSkc/Yi7YAs+3LdLcBD9oto1IxNdj/MpGxlwn32c/Cn/4EBw/CjBmwcuXlZaWl0NwMBw7Ahg3wzjuXl+3aZZ3nCATg3/991B9DqajF8hwHxpiXsRqH0HnPhLw3wCNDrLt0iPkXgPvHMcy45+Yrxq4mPh8cC7mUo6sLZs4ce5lQ2dnZZGZenn74YfjEJ95d7oMfhPe8xzqpHtr1PGUKLFoEP/sZ3HHH2D6Pip5Xc0/vHHeBadOmOR2CJ8ydC+3t0NkJFy7Atm3Wl3aoRYvg+eetI4/9++G977WOIoYybdo0ToR0zr70EhQVWe87O62rpgDeeANaWyE7G3p7GVinr8+6gusDHxi3j6nGwKu5F9MjDjUxGhsbXX1J4NUiMdE6F3HnndbluA89ZJ1/eMY+hl6xwrqM9uWX4cYbrctxf/jDy+svXQrV1fDnP1tHJv/wD/D+9zfygx8s5OBBq0srOxs2b7bK19ZCZSVMmgTXXAPf+Q5cfz2cOmU1UIGAFcdHPmLtW008r+aeeOH0QElJidH7ONTVSOsuvrm9/kTkNWPMu67L1a4qF0hLS3M6BFcQkQl/bdu2bcL3qcaPV3NPjziUsnnlS9ULOa/Ghx5xuFhNTY3TIbiKmcBX9Te+MWH7UuPPq7mnDYcL6C/I+GWu0RSMZ17NPf1f6wJe6WJxI7l0yekQVBS8mnt6jkMpW/BLwK0ZEfyK80LOq/Gh5zhcLDjipYo/TX//906HoKLg1dzThsMFzoSOiqfiypncXKdDUFHwau5pw6GUUmpMtOFwAX/wEXMq7vi//W2nQ1BR8GruacPhAqdPn3Y6BBWh0/n5ToegouDV3NOGwwWCj4dU8efIxz7mdAgqCl7NPW04lFJKjYk2HC4we/Zsp0NQEZr9y186HYKKgldzTxsOF0hJSXE6BBWhlK4up0NQUfBq7sW04RCRu0SkVUQOi8jqQZaLiGyyl78uIv6QZVtEpNt+tvhg235cRIyIXB/LzxAPvHoTkhvoDYDxzau5F7OGQ0QSgKeBCqAAWCoiBWHFKoBc+7Uc+G7IsmeBu4bY9izgDuDo+EatlFJqJLE84rgFOGyM6TDGXAC2AYvDyiwGnjeW/cB1IjIDwBizBxjqWrd/Albh3mGFxiQ9Pd3pEFSE0ltanA5BRcGruRfLhiMLOBYy3WXPG2uZK4jIIuC4Mcabx4iDKCwsdDoEFaHCZ591OgQVBa/mXmIMtz3YeMPhRwijKXO5sMi1wJeBES9+F5HlWN1fzJw5k+rqasC6CiIlJWWgbzI9PZ3CwkL27NkDQGJiIuXl5TQ2NnL27FkASkpKOHXqFMeOWW1cbm4uSUlJHDpknX7JyMggLy+P2tpaAJKSkigrK6OhoYHe3l4ASktL6erq4vjx4wDk5+eTkJBAi/2Lc/r06eTk5FBXVwfA1KlTKS0tpb6+nvPnzwNQVlZGZ2cnJ0+eBKCgoID+/n5ee+01kpOTycrKwufzUV9fD0BycjIlJSXU1dURCAQAKC8vp62tje7ubgCKiooIBAK0t7cDMGvWLDIzMwmOJpyamorf76e2tpa+vj4A5s+fT3NzMz09PQAUFxdz7tw5Ojo6AMjOzmbatGk0NjYC1uM1i4uLqampwRiDiLBgwQKampoGxvrx+/2cPn164Lp4J+qpqqqKyspK2ufN4/htt1n1tH07CRcv0nL//VY9HThAzs6d1K1bZ9VTTw+lGzZQv2YN5+1fn2VPPklnRQUn58616mnrVvonTaL13nsByNq3D9/evdSvXk2vz8f0Awco2biRunXrCKSmWvW0di1tS5bQPWeOVU9bthBIS6P9nnusetq9m8zGRhpWrrTq6ehR/Js2Ubt+PX1Tplj1tGoVzcuW0VNg9RDnbt6Mz+cbyIV4rSeIbT61trZa9TSKfDLGkJmZ6dp8GkrMhlUXkTLgCWPMnfb0GgBjzIaQMpuBamPMC/Z0K7DQGHPCns4GfmGMKbKnPwS8Crxtb8IHvAncYow5OVQsbh9Wvbq6moULFzodRtxzYlj16qoqFj7++ITsS4dVH39uzz0nhlU/AOSKSI6ITAbuA3aEldkBPGBfXXUr8Faw0RiMMeb3xpgMY0y2MSYbq2vLP1yj4QWJibE8cFSxlPjOO06HoKLg1dyLWcNhjOkDHgV2AX8AXjTGNIvIChFZYRd7GegADgPfBz4XXF9EXgDqgHwR6RKRz8Qq1nhXXl7udAgqQuVr1zodgoqCV3NPnwDoAo2NjZ4dpXM8OdFV1fjYY/g3bZqQfWlX1fhze+7pEwBdLHgyS8Wfszfc4HQIKgpezT1tOJRSSo2JNhwuMNxlc+rqVrJxo9MhqCh4Nfe04XCBU6dOOR2CitApF/ePe4FXc08bDhcI3kil4s+x2293OgQVBa/mnjYcSimlxkQbDhfIzc11OgQVodyXXnI6BBUFr+aeNhwukJSU5HQIKkJJ9thCKj55Nfe04XCB4OBwKv4ceughp0NQUfBq7mnDoZRSaky04XCBjIwMp0NQEco4eNDpEFQUvJp72nC4QF5entMhqAjl/eQnToegouDV3NOGwwWCD7xR8ad2/XqnQ1BR8GruacOhlFJqTLThcAGvXhLoBkkeHV3VLbyae/o8DqVsTjyPYyLp8zjUWOnzOFxMG8X41bBypdMhqCh4Nfe04XCB3t5ep0NQEeqdMcPpEFQUvJp7MW04ROQuEWkVkcMisnqQ5SIim+zlr4uIP2TZFhHpFpFDYet8U0T+aJd/SUSui+VnUEopdaWYNRwikgA8DVQABcBSESkIK1YB5Nqv5cB3Q5Y9C9w1yKZfAYqMMTcBbcCa8Y08/pSWljodgopQaWWl0yGoKHg192J5xHELcNgY02GMuQBsAxaHlVkMPG8s+4HrRGQGgDFmD3A6fKPGmF8bY/rsyf2AL2afIE50dXU5HYKKUNe8eU6HoKLg1dyLZcORBYQ+5aTLnjfWMsN5CNgZUXQucvz4cadDUBE6ftttToegouDV3EuM4bZlkHnh1wGOpszgGxf5MtAH/HiI5cuxur+YOXMm1dXVAMyePZuUlBSampoASE9Pp7CwkD179gCQmJhIeXk5jY2NnLWvsS8pKeHUqVMDT/vKzc0lKSlpYGTMjIwM8vLyBu4iTUpKoqysjIaGhoGTZ6WlpXR1dQ38R8vPzychIYGWlhYApk+fTk5ODnV1dQBMnTqV0tJS6uvrOX/+PABlZWV0dnZy8uRJAAoKCujv76e3t5fq6mqysrLw+XzU19cDkJycTElJCXV1dQQCAQDKy8tpa2uju7sbgKKiIgKBAO3t7QDMmjWLzMzMgatFUlNT8fv91NbW0tdnHejNnz+f5uZmenp6ACguLubcuXN0dHQAkJ2dzbRp02hsbAQgLS2N4uJiampqMMYgIixYsICmpibO2MOK+/1+Tp8+zZEjRxyrp6qqKiorK2mfN2/gCz1/+3YSLl6k5f77rXo6cICcnTupW7fOqqeeHko3bKB+zRrOp6db9fTkk3RWVHBy7lyrnrZupX/SJFrvvReArH378O3dS/3q1fT6fDSsXEnJxo3UrVtHIDXVqqe1a2lbsoTuOXOsetqyhUBaGu333GPV0+7dZDY2DlyVlXr0KP5Nm6hdv56+KVOselq1iuZly+gpsHqIczdvxufzDeRCvNYTxDafWltbrXoaRT4ZY2hpaXFtPg0lZvdxiEgZ8IQx5k57eg2AMWZDSJnNQLUx5gV7uhVYaIw5YU9nA78wxhSFbftBYAXwUWPM2yPF4vb7OE6cOMEMvTonak7cx3Fi7lxmHDgwIfvS+zjGn9tzz4n7OA4AuSKSIyKTgfuAHWFldgAP2FdX3Qq8FWw0hiIidwH/A1g0mkbDCxISEpwOQUUo4eJFp0NQUfBq7sWs4bBPYD8K7AL+ALxojGkWkRUissIu9jLQARwGvg98Lri+iLwA1AH5ItIlIp+xFz0FpACviMhBEXkmVp8hXgQPz1X8CXaBqfjk1dyL5TkOjDEvYzUOofOeCXlvgEeGWHfpEPNvHM8YlVJKjY3eOe4C06dPdzoEFaHpE3R+Q8WGV3NPGw4XyMnJcToEFaGcnZ6/mjyueTX3tOFwgeAlhyr+BC/rVfHJq7mnDYdSSqkx0YbDBaZOnep0CCpCU+2bvlR88mru6YOclLLpg5yUupI+yMnFgkMiqPhTv8bzgzvHNa/mnjYcLhAce0fFn+D4Vio+eTX3tOFQSik1JnqOwwUCgQBJSUlOhxH3nDjHEUhNJckejTTW9BzH+HN77uk5Dhfr7Ox0OgQVoc6KCqdDUFHwau5pw+ECwecJqPgTfGaHik9ezT1tOJRSSo2JNhwuUGA/4U3Fn4KtW50OQUXBq7mnDYcL9Pf3Ox2CilD/pElOh6Ci4NXc04bDBYLPSFbxJ/gcchWfvJp72nAopZQaE204XCArK8vpEFSEsvbtczoEFQWv5l5MGw4RuUtEWkXksIisHmS5iMgme/nrIuIPWbZFRLpF5FDYOtNE5BURabf/TYvlZ4gHPp/P6RBUhHx79zodgoqCV3MvZg2HiCQATwMVQAGwVETCL0GoAHLt13LguyHLngXuGmTTq4FXjTG5wKv2tKd5daA1N6hf7fn/vnHNq7kXyyOOW4DDxpgOY8wFYBuwOKzMYuB5Y9kPXCciMwCMMXuA04NsdzHwnP3+OeCTsQheKaXU4GLZcGQBx0Kmu+x5Yy0TLtMYcwLA/jcjyjjjXnJystMhqAglnzjhdAgqCl7NvcQYblsGmRc+utpoykS2c5HlWN1fzJw5k+rqagBmz55NSkoKTU1NAKSnp1NYWMiePXsASExMpLy8nMbGRs7ag8+VlJRw6tQpjh2z2rjc3FySkpI4dMg6/ZKRkUFeXh61tbUAJCUlUVZWRkNDA729vQCUlpbS1dXF8ePHAcjPzychIYGWlhYApk+fTk5OzsAzjKdOnUppaSn19fUDQzeXlZXR2dk5MMxBQUEB/f399Pb2Ul1dTVZWFj6fb+DwOTk5mZKSEurq6ggEAgCUl5fT1tZGd3c3AEVFRQQCAdrb2wGYNWsWmZmZBAeFTE1Nxe/3U1tbS19fHwDz58+nubmZHvvpdcXFxZw7d46Ojg4AsrOzmTZtGo2NjQCkpaVRXFxMTU0NxhhEhAULFtDU1MSZM2cA8Pv9nD59miNHjjhWT1VVVVRWVtI+bx7Hb7vNqqft20m4eJGW+++36unAAXJ27hx4VvjUnh5KN2ygfs2agSHSy558ks6KioHhRAq2bqV/0qSBS2+z9u3Dt3fvQDdVw8qVlGzcSN26dQRSU616WruWtiVL6J4zx6qnLVsIpKXRfs89Vj3t3k1mYyMNK1da9XT0KP5Nm6hdv56+KVOselq1iuZly+ixb1LL3bwZn883kAvxWk8Q23wKXmI72nxqaWlxbT4NyRgTkxdQBuwKmV4DrAkrsxlYGjLdCswImc4GDoWtM1AGmAG0jhTLzTffbNzsN7/5jdMhuALWjxZjJvD1m3XrJmxfA59PjRu35x7QYAb5To1lV9UBIFdEckRkMnAfsCOszA7gAfvqqluBt4zdDTWMHcCD9vsHgZ+NZ9DxKPjrR8Wf4BGGik9ezb2YNRzGmD7gUWAX8AfgRWNMs4isEJEVdrGXgQ7gMPB94HPB9UXkBaAOyBeRLhH5jL2oErhDRNqBO+xppZRSE0Qf5OQCfX19JCbG8nSVNzjxIKe+KVNIfOedCdmXPshp/Lk99/RBTi7W1tbmdAgqQm1LljgdgoqCV3NPGw4XCF7RoeJP8KopFZ+8mnvacCillBoTbThcoKioyOkQVISKtmxxOgQVBa/mnjYcLuDVSwLdIJDm+TE645pXc08bDhcI3qWq4k/wTnAVn7yae9pwKKWUGhNtOFxg1qxZToegIjRr926nQ1BR8GruacPhApmZmU6HoCKUaQ9cp+KTV3NPGw4XcPNd8W4XHN1WxSev5p42HEoppcZEGw4XSNURVuNW6tGjToegouDV3NNBDpWyOTHI4UTSQQ7VWEU0yKGI3B/y/rawZY+OX3gqGsEnpan4U7t+vdMhqCh4NfdG6qr67yHvvx227KFxjkVFKPgIShV/go95VfHJq7k3UsMhQ7wfbFoppZQHjNRwmCHeDzatHDJ//nynQ1ARmr9qldMhqCh4NfdGajg+ICKvi8jvQ94Hp/MnID41Cs3NzU6HoCLUvGyZ0yGoKHg190Z65uEHo9m4iNwF/C8gAfhnY0xl2HKxl98NvA0sM8Y0DreuiMwBngGmAH3A54wxv40mznjX09PjdAgqQj0FBU6HoKLg1dwb9ojDGPNG6AvoBfzA9fb0kEQkAXgaqAAKgKUiEp4lFUCu/VoOfHcU634D+AdjzBxgnT2tlFJqgox0Oe4vRKTIfj8DOIR1NdWPROSLI2z7FuCwMabDGHMB2AYsDiuzGHjeWPYD19n7GW5dAwTvunkv8OYoPqerFRcXOx2CilDx5s1Oh6Ci4NXcG+kcR44x5pD9/tPAK8aY/wcoZeTLcbOAYyHTXfa80ZQZbt0vAt8UkWNAFbBmhDhc79y5c06HoCJ0zudzOgQVBa/m3kjnOC6GvP8o8H0AY8w5Ebk0wrqDXa4bfiXWUGWGW/ezwJeMMf9bRP4G+AHwn9+1c5HlWN1fzJw5k+rqagBmz55NSkoKTU1NAKSnp1NYWMiePXsASExMpLy8nMbGRs6ePQtASUkJp06d4tgxqy3Lzc0lKSmJQ4esNjUjI4O8vLyBm4GSkpIoKyujoaGB3t5eAEpLS+nq6uL48eMA5Ofnk5CQQEtLCwDTp08nJyeHuro6AKZOnUppaSn19fWcP38egLKyMjo7Ozl58iQABQUF9Pf38/rrr9PR0UFWVhY+n4/6+noAkpOTKSkpoa6ubuBJZeXl5bS1tdHd3Q1Yj74MBAIDD6SZNWsWmZmZA4O3paam4vf7qa2tHbhmff78+TQ3Nw/07xYXF3Pu3Dk6OjoAyM7OZtq0aTTaI7+mpaVRXFxMTU0NxhhEhAULFtDU1MSZM2cA8Pv9nD59miNHjjhWT1VVVVRWVtI+bx7Hb7Pud83fvp2Eixdpud+6F3b6gQPk7NxJ3bp1Vj319FC6YQP1a9ZwPj3dqqcnn6SzooKTc+da9bR1K/2TJtF6770AZO3bh2/vXupXr6bX56Pb76dk40bq1q0jYA9hUb52LW1LltA9Z45VT1u2EEhLG3jw06zdu8lsbBwYJDH16FH8mzZRu379wL0h81etonnZsoHzKLmbN+Pz+QZyIV7rCWKbT62trVY9jSKfjDH09va6Np+GZIwZ8gX8HPg8cA9wBrjOnj8VaB5h3TJgV8j0GmBNWJnNwNKQ6VZgxnDrAm9xeagUAc4OF4cxhptvvtm42e7du50OwRWwfpwYM4Gv3VVVE7avgc+nxo3bcw9oMIN8p47UVfUZoBBYBtxrjPmLPf9W4IcjrHsAyBWRHBGZDNwH7AgrswN4QCy3Am8ZY06MsO6bwAL7/UcAbz67MUR2drbTIagIZf/6106HoKLg1dwbtqvKGNMNrBhk/m5g2EeXGWP67PGsdmFdUrvFGNMsIivs5c8AL2NdinsY63LcTw+3rr3ph4H/JSKJwDvY3VFeNm3aNKdDUBGaZneLqPjk1dwbdnRcEQk/QriCMWbRuEcUA24fHbe6upqFCxc6HUbcc2J03OqqKhY+/viE7EtHxx1/bs+9oUbHHenkeBnW1U0vAPXo+FRKKeV5IzUc04E7gKXA3wK/BF4I6TZSV4G0tDSnQ1ARSmv3/Cm6uObV3Bv1g5xEJAmrAfkm8KQxJnyY9auW27uq1PjQBzkpdaWIHuRkr5gkIn8NbAUeATYB/zb+IapI1dTUOB2CilDNN3TEnHjm1dwbtqtKRJ4DioCdWONDHRquvHKG/oKMX+aaEX+7qauYV3NvpHMc/w34DyAPeCx4KI911GuMMd58UvtVJqReVJyRSyMNwKCuZl7NvVGf44hneo5DjYae41DqShGf41BXv+D4Myr+NP393zsdgoqCV3NPGw4XCA5spuLPmdxcp0NQUfBq7mnDoZRSaky04XABv9/vdAgqQv5vx83tUGoQXs09bThc4PTp006HoCJ0Oj/f6RBUFLyae9pwuEDwYS0q/hz52MecDkFFwau5pw2HUkqpMdGGwwVmz57tdAgqQrN/+UunQ1BR8GruacPhAikpKU6HoCKU0tXldAgqCl7NPW04XMCrNyG5gd4AGN+8mnvacCillBqTmDYcInKXiLSKyGERWT3IchGRTfby10XEP5p1ReTz9rJmEfH8uNTp6elOh6AilN7S4nQIKgpezb2RRseNmIgkAE9jPUGwCzggIjuMMaGZUgHk2q9S4LtA6XDrisjtwGLgJmNMQEQyYvUZ4kVhYaHTIagIFT77rNMhqCh4NfdiecRxC3DYGNNhjLkAbMP6wg+1GHjeWPYD14nIjBHW/SxQaYwJABhjumP4GeLCnj17nA5BRWiPPsgprnk192LZcGQBx0Kmu+x5oykz3Lp5wDwRqReRGhGZO65RK6WUGlbMuqq4PPx/qPAHAQxVZrh1E4E04FZgLvCiiMw2YQ8ZEJHlwHKAmTNnUl1dDVjXXaekpAxcDZGenk5hYeHAL4fExETKy8tpbGzk7NmzAJSUlHDq1CmOHbPastzcXJKSkjh0yHogYkZGBnl5edTW1gKQlJREWVkZDQ0N9Pb2AlBaWkpXVxfHjx8HID8/n4SEBFrsPu7p06eTk5NDXV0dAFOnTqW0tJT6+nrOnz8PQFlZGZ2dnZw8eRKAgoIC+vv76e3tpbq6mqysLHw+H/X19QAkJydTUlJCXV0dgUAAgPLyctra2ujutg7UioqKCAQCtLe3AzBr1iwyMzMJPr8kNTUVv99PbW0tfX19AMyfP5/m5mZ6enoAKC4u5ty5c3R0dACQnZ3NtGnTaGxsBCAtLY3i4mJqamowxiAiLFiwgKampoHRRf1+P6dPnx64E9eJeqqqqqKyspL2efM4ftttVj1t307CxYu03H+/VU8HDpCzcyd169ZZ9dTTQ+mGDdSvWcN5u7+77Mkn6ayo4ORc6zdNwdat9E+aROu99wKQtW8fvr17qV+9mt6sLBpWrqRk40bq1q0jkGo9G6187Vraliyhe84cq562bCGQlkb7PfdY9bR7N5mNjTSsXGnV09Gj+Ddtonb9evqmTLHqadUqmpcto6egwPp7bN6Mz+cbyIV4rSeIbT61trZa9TSKfAru0635NJSYPchJRMqAJ4wxd9rTawCMMRtCymwGqo0xL9jTrcBCIHuodUXkV1hdVdX2sj8Btxpj/u9QseiDnNRo6IOclLqSEw9yOgDkikiOiEwG7gN2hJXZATxgX111K/CWMebECOv+FPgIgIjkAZOBP8fwc1z1gr9EVPxpfOwxp0NQUfBq7sWsq8oY0ycijwK7gARgizGmWURW2MufAV4G7gYOA28Dnx5uXXvTW4AtInIIuAA8GN5N5TXBQ0sVf87ecIPTIagoeDX3YnmOA2PMy1iNQ+i8Z0LeG+CR0a5rz78A3D++kSqllBotvXPcBYY7iaWubiUbNzodgoqCV3NPGw4XOHXqlNMhqAid8ugT5NzCq7mnDYcLBC9rVPHn2O23Ox2CioJXc08bDqWUUmOiDYcL5ObmOh2CilDuSy85HYKKgldzTxsOF0hKSnI6BBWhJPtOXxWfvJp72nC4QHCoBhV/Dj30kNMhqCh4Nfe04VBKKTUm2nC4QEaG5x9JErcyDh50OgQVBa/mnjYcLpCXl+d0CCpCeT/5idMhqCh4Nfe04XCB4PDTKv7Url/vdAgqCl7NPW04lFJKjYk2HC7g1UsC3SDJo6OruoVXcy9mD3K6muiDnNRo6IOclLqSEw9yUhNEG8X4FXz0q4pPXs09bThcIPgcZhV/emfMcDoEFQWv5p42HEoppcZEGw4XKC0tdToEFaHSykqnQ1BR8GruxbThEJG7RKRVRA6LyOpBlouIbLKXvy4i/jGs+7iIGBG5PpafIR50dXU5HYKKUNe8eU6HoKLg1dyLWcMhIgnA00AFUAAsFZGCsGIVQK79Wg58dzTrisgs4A7gaKzijyfHjx93OgQVoeO33eZ0CCoKXs29WB5x3AIcNsZ0GGMuANuAxWFlFgPPG8t+4DoRmTGKdf8JWIV7r5xUSqmrViwbjiwg9LmKXfa80ZQZcl0RWQQcN8Y0jXfA8So/P9/pEFSE8rdvdzoEFQWv5l5iDLctg8wLP0IYqsyg80XkWuDLwMdG3LnIcqzuL2bOnEl1dTUAs2fPJiUlhaYmq91JT0+nsLCQPXv2AJCYmEh5eTmNjY2cte/qLSkp4dSpUwPPF87NzSUpKWlgLP6MjAzy8vIGxq1JSkqirKyMhoaGgcv1SktL6erqGji0zc/PJyEhgZaWFgCmT59OTk4OdXV1AEydOpXS0lLq6+s5f/48AGVlZXR2dnLy5EkACgoK6O/vp7m5mdbWVrKysvD5fNTX1wOQnJxMSUkJdXV1BAIBAMrLy2lra6O7uxuAoqIiAoEA7e3tAMyaNYvMzMyB69NTU1Px+/3U1tbS19cHwPz582lubqanpweA4uJizp07R0dHBwDZ2dlMmzaNxsZGANLS0iguLqampgZjDCLCggULaGpq4oz9ICO/38/p06c5cuSIY/VUVVVFZWUl7fPmDXQh5W/fTsLFi7Tcf79VTwcOkLNzJ3Xr1ln11NND6YYN1K9Zw/n0dKuennySzooKTs6da9XT1q30T5pE6733ApC1bx++vXupX72avmuv5fj8+ZRs3EjdunUEUlOtelq7lrYlS+ieM8eqpy1bCKSl0X7PPVY97d5NZmPjwH0gqUeP4t+0idr16+mbMsWqp1WraF62jJ4Cq5c3d/NmfD7fQC7Eaz1BbPOptbXVqqdR5FN+fj4tLS2uzaehxOzOcREpA54wxtxpT68BMMZsCCmzGag2xrxgT7cCC4HswdYFfgG8CrxtT/uAN4FbjDEnh4rF7XeOV1dXs3DhQqfDiHtO3DleXVXFwscfn5B96Z3j48/tuefEneMHgFwRyRGRycB9wI6wMjuAB+yrq24F3jLGnBhqXWPM740xGcaYbGNMNlYXln+4RkMppdT4illXlTGmT0QeBXYBCcAWY0yziKywlz8DvAzcDRzGOor49HDrxirWeDd9+nSnQ1ARmn7ggNMhqCh4Nfd0kEMXCAQCnh2lczw50VUVSE2dsBFytatq/Lk993SQQxcLngBU8Sd4kl3FJ6/mnjYcSimlxkQbDheYOnWq0yGoCE21L8FU8cmruafnOJSy6YOclLqSnuNwseANSir+1K9ZM3IhddXyau5pw+ECwTthVfwJ3m2u4pNXc08bDqWUUmOi5zhcwO3Xkk8UvY9DjZXbc0/PcbhYZ2en0yGoCHVWVDgdgoqCV3NPGw4XCI7uqeJPcARdFZ+8mnvacCillBoTbThcoKAg/Im8Kl4UbN3qdAgqCl7NPW04XKC/v9/pEFSE+idNcjoEFQWv5p42HC4QfGKZij/BpwKq+OTV3NOGQyml1Jhow+ECWVlZToegIpS1b5/TIagoeDX3tOFwAZ/P53QIKkK+vXudDkFFwau5pw2HC3h1oDU3qF+92ukQVBS8mnsxbThE5C4RaRWRwyLyrgwRyyZ7+esi4h9pXRH5poj80S7/kohcF8vPoJRS6koxazhEJAF4GqgACoClIhJ+0XMFkGu/lgPfHcW6rwBFxpibgDbA8+NSJycnOx2CilDyiRNOh6Ci4NXci+URxy3AYWNMhzHmArANWBxWZjHwvLHsB64TkRnDrWuM+bUxps9efz/gzU7GECUl7xqDTMWJko0bnQ5BRcGruRfLhiMLOBYy3WXPG02Z0awL8BCwM+pI41xdXZ3TIagI1a1b53QIKgpezb3EGG5bBpkXPp7zUGVGXFdEvgz0AT8edOciy7G6v5g5cybV1dUAzJ49m5SUFJqamgBIT0+nsLCQPXv2AJCYmEh5eTmNjY2ctYe7Likp4dSpUxw7ZrVlubm5JCUlcejQIQAyMjLIy8ujtrYWgKSkJMrKymhoaKC3txeA0tJSurq6OH78OAD5+fkkJCTQ0tICwPTp08nJyRn4jzh16lRKS0upr68feFhMWVkZnZ2dAwOrFRQU0N/fT09PD9XV1WRlZeHz+QZO2CUnJ1NSUkJdXR2BQACA8vJy2tra6O7uBqCoqIhAIEB7ezsAs2bNIjMzk+Aw9Kmpqfj9fmpra+nrsw705s+fT3NzMz3287KLi4s5d+4cHR0dAGRnZzNt2jQaGxsBSEtLo7i4mJqaGowxiAgLFiygqamJM2fOAOD3+zl9+jRHjhxxrJ6qqqqorKykfd48jt92m1VP27eTcPEiLfffb9XTgQPk7Nw58IU/taeH0g0bqF+zZuChTGVPPklnRcXAAIYFW7fSP2nSwM1+Wfv24du7l/rVq+n1+WhYuZKSjRupW7eOQGqqVU9r19K2ZAndc+ZY9bRlC4G0NNrvuceqp927yWxspGHlSquejh7Fv2kTtevX0zdlilVPq1bRvGwZPfawGLmbN+Pz+QZyIV7rCWKbT8Gb+kaTT8YYWlpaXJtPQzLGxOQFlAG7QqbXAGvCymwGloZMtwIzRloXeBCoA64dTSw333yzcbPdu3c7HYIrYP04MWYCX7urqiZsXwOfT40bt+ce0GAG+U6NZVfVASBXRHJEZDJwH7AjrMwO4AH76qpbgbeMMSeGW1dE7gL+B7DIGPN2DOOPG+Xl5U6HoCJUvnat0yGoKHg192LWcBjrBPajwC7gD8CLxphmEVkhIivsYi8DHcBh4PvA54Zb117nKSAFeEVEDorIM7H6DPGira3N6RBUhNqWLHE6BBUFr+ZeLM9xYIx5GatxCJ33TMh7Azwy2nXt+TeOc5hxr7u727PDO8e77jlzdGj1OObV3NM7x5VSSo2JNhwuUFRU5HQIKkJFW7Y4HYKKgldzTxsOFwheGqjiTyAtzekQXO1Xv4L8fLjxRqisfPdyY+Cxx6zlN90E9hWvw657+jTccQfk5sJf/3Uy9hWw/PjHMGfO5dc118DBg/D22/Dxj8MHPgCFheCG4cm04XCB4DXjKv4E78tQ46+/Hx55BHbuhJYWeOEF699QO3dCe7v1+t734LOfHXndykr46EetdT7wgTcHGpVPfcpqKA4ehB/9CLKzrQYE4PHH4Y9/hN/9Dvbts7Ybz7ThUEq50m9/ax0tzJ4NkyfDfffBz352ZZmf/QweeABE4NZb4S9/gRMnhl/3Zz+DBx+03t9550l++tN37/uFF2DpUuv9tdfC7bdb7ydPBr8furpi8YknjjYcLjBr1iynQ1ARmrV7t9MhuNbx4xCaGj6fNW80ZYZb99QpmDHDej9nTib2TeNX2L79csMR6i9/gZ//3DpiiWfacDhovPpft269PM5jaP/rHXcw0P8adPQoJCdDVVWMPpQak8zQSlXjyoQPcIR1ZDGaMqNZFyAzM/Nd8+rrraOM8PPmfX1WY/LYY9aRTDzThsMh49n/+vzzFwftf/3oR9/dIH3pS1BREfvPp0YnONaUGn8+HxwLGSq1qwtmzhxdmeHWzcy0urMAdu16nYyMK7e5bdvgRxvLl1s/6L74xYg/0lVDGw6HjGf/60c+0j1o/+uDD3JF/+tPf2qtU1g4AR9QKYfNnWv9gOrshAsXrC/0RYuuLLNoETz/vHWEsX8/vPe9VjfUcOsuWgTPPWe937VrOotDHhZx6RL8679a+Rxq7Vp46y341rdi9nEnlDYcDhnP/tcbbrhm0P7XGTMY6H/9j/+Ar38dvvKV8f8sKnKpR486HYJrJSbCU0/BnXfCBz8If/M31o+mZ56xXgB33239mLrxRnj4YfjOd4ZfF6zLaV95xTp6OHjw+isur92zx8rT0K6ori746let3gG/37rS6p//eUL+BDET0yFH1NDGs/81Ozt70BN0ob7yFaubyqMPLLtq+TdtcjoEV7v7busVasWKy+9F4OmnR78uQHo6vPpqcCr1imULF1pHLqF8vsFzOZ7pEYdDxrP/tbb2yKD9rydOMND/Wl8Pq1ZZ15Z/61vwta9Zv6iUs2rXr3c6BNcQkQl/ffWrX53wfV4NtOFwyHj2v/6f/3P9oP2vzz3HQP/r3r1w5Ij1+uIX4X/+T3j00Qn6sGpIwYcuqfg0xaP1p11VDgntQ+3vh4ceutz/Ctbh9N13w8svW/2v114LP/zh4Ovefns3hYVWH9Tq1VZ/7A9+ADfcYJ2oU8pbJrJfqHoC93d1HG0AiHFb59sgSkpKTPDRjW506dIlrrlGDx6jFewGmMiMuHTNNVxz6dKE7Cv4tePWnL/cjTNxn++aay5x6dJE5Z79/3MC609EXjPGvOsZsvptM86c6Gd9+OGHPdnP6gbNy5Y5HYKKwrJlzSMXciFtOFzAiw+ScYserbu4VlDQ43QIjtBzHLHyxATua+YE7m+i9qOUumrF9IhDRO4SkVYROSwi7xqFXiyb7OWvi4h/pHVFZJqIvCIi7fa/nn+gweauzU6HoCJUvFnrLp5t3lzsdAiOiFnDISIJwNNABVAALBWR8OPyCiDXfi0HvjuKdVcDrxpjcoFX7WlP8yX5Ri6krkrnfFp38cznO+d0CI6I5RHHLcBhY0yHMeYCsA1YHFZmMfC8sewHrhORGSOsuxiw71TgOeCTMfwMceHj7/u40yGoCHV8XOsunn384x1Oh+CIWJ7jyAJC7m+mCygdRZmsEdbNNMacADDGnBCRsLEprxJPTOC+qiZ4fy43kdeMVQG3T+D+vEFrMNZi2XAMVnvhFyAPVWY06w6/c5HlWN1fAL0i0jqW9ePJ448/fj3w54ncp16SOz607uKbB+rvPw02M5YNRxcQ+mg6H/DmKMtMHmbdUyIywz7amAEMOryfMeZ7wPciDz9+iEjDYDfpqKuf1l1882r9xfIcxwEgV0RyRGQycB+wI6zMDuAB++qqW4G37G6o4dbdAdhPnOBBIOwpFkoppWIpZkccxpg+EXkU2AUkAFuMMc0issJe/gzwMnA3cBh4G/j0cOvam64EXhSRzwBHgf8aq8+glFLq3TwxVpXbichyu2tOxRmtu/jm1frThkMppdSY6FhVSimlxkQbjnEkIl8WkWZ7+JSDIlJqz58kIpX2MCmHROS3IlJhL/uqiBwTkd5R7iNbRIyIfD5k3lMisiwmH2roOI6IyPUTuc/xNNa6EpFrReSXIvJHe73KMezrSyLyjoi8N2TeQhH5cMj0JwcZWWG0288WkUORrGuvP6r/e1eDCHPsVyLSZK/3jD0yxXD7yBaR8yLyOxH5g72tB4dbJ2TdF+zYvjQOn/VZEVkyhvJR/T8YCx3kcJyISBnwCcBvjAnYX6qT7cX/CMwAiuxlmcACe9nPgaeA9jHsrhv4gohstu+sH2usicaYvrGu5xZR1FWVMWa3faXfqyJSYYzZOYpdLsW6UvAe4Fl73kKgF/iNPf1J4BdASzSfzc2iqLe/McacFesGiJ9gXVCzbYTd/ckY81f2fmcD/yYi1xhjfjhMfNOBDxtjBr33wU30iGP8zAD+bIwJABhj/myMeVNErgUeBj4fsuyUMeZF+/3+4J3wY/B/scbpetevIBGZIyL77V89L4k9CKSIVIvI10SkBqvRqRaRfxKRPfavqrki8m/2L7b1Idv7qYi8Zv9aWx6+vzg15royxrxtjNltz7sANGLdXzQsEXk/kAysxWpAEJFsYAXwJftX8wJgEfBNe/r9IvKwiBywfyn/bzs2RCTTrtcm+/XhsP3Ntn8pz7W38yu7/vaKyAfsMjkiUmdv/x+j/mtOnEhz7Ky9fiJWQzOmE7vGmA7gvwOPAYjIe0Rki/33+52IBIdD+jWQYdfhvGH+/s+KNbjrb0SkI3hUIZanRKRFRH4JDIyKISI3i0iNva1dYt3DFpzfJCJ1wCNj/YNGzBijr3F4YX05HATagO8AC+z5NwG/G8X6vWHTi4AnBymXDRwCcoA/Yl2u/BSwzF7+esi+nwS+Zb+vBr4Tsp1q4Ov2+y9g3WA5A0jCujEz3V42zf53qr3f4PwjwPVO/90dqqvrgA5g9nB1ZS9bC/x/WD/SjgAZ9vwngMdDyj0LLAmZTg95vx7rSxFgO/BF+30C8N6Q/xP5wO+AOfbyV4Fc+30p8O/2+x3AA/b7R8L/712tr2jqDevS/jPAvwAJw9Vb8O85SJ2ft99/Dbg/ZH4b8J7w9Yb5+z8L/Kv9f6IAa1w+gL8GXrHrdSbwF2AJMAnryPR9drl7sW5RgCvz/ZvhccfqpV1V48QY0ysiNwPzsAav2S7WcPCNEW5vB+++YTJ0eaeI/Bb42+A8sfrQrzPG1NiznsP6Dxq0PWwzwe3/Hmg29pGPiHRg3bnfAzwmIvfY5WZhjWQc10+viaauRCQReAHYZKxfoiPV1X3APcaYSyLyb1jdJE+PIswi+8jvOqwvzF32/I8AD9j77Qfeso8q34d1M+x/Mdb9UsnAh4F/lctDVCTZ/94G/Bf7/Y+Ar48iHsdFU2/GmDtFZArwY6y/4Ssj5ViY0HE+PgYsEpHH7ekpwA3A+YHCw//9AX5qjLkEtNjdagDzgRfsen1TRP7dnp8PFAGv2NtKAE4Mku8/whpRPOa04RhHdoVXA9Ui8nusrqQXgRtEJMUYM95jMH8Nq892zyjL/0fYdMD+91LI++B0oogsBP4zUGaMeVtEqrGSJO5FUVffA9qNMd8aaR8ichNWQxtM+MlYRyqjaTieBT5pjGkS68KHhSOUfwtrYNDbgGasX7N/McbMGaJ8XF6HH02OGWPeEZEdWCNsvzLGXf8V8Af7vWA10FeMf2d3QQaN9PcPzbfQRmmwehGsH3ZlYfu7bojyMafnOMaJiOSLSG7IrDnAG8aYt4EfAJvEOqmKiMwQkfuj3acx5o9YJ1M/YU+/BZwRkXl2kf8G1Ayx+mi8FzhjNxofAG6NJt6rRaR1ZR8BvBf44ih3tRR4whiTbb9mAlki8p+Ac0BKSNnw6RSsX5WTgE+FzH8V+KwdT4KIpNrzL2CdYH9ARP7WWP36nSLyX+2yIiLBpw7twzoSImzbV7VI6k1EkkPOByRijVTxxzHuNxtrGNxv27N2AZ8X+9eAiPxV+Doj/P2Hsge4z67XGVwedrcVeJ9YFwcEryArNMb8BeuIs9wuN2F1qQ3H+EkGnrNPbL2O1Xf5hL1sLdYJ7RaxLpf7qT2NiHxDRLqAa0WkS0SesOcvEpEnR7Hfr3LlSdoHsU6yvo6VWKPZxlB+hXXk8TrWVSv7o9jW1WTMdSUiPuDLdtlGsU6A/h0MW1f3AS+FzXvJnv9z4B57O/OwrvL5f8U62fp+rPMi9Vi/jEO/6L4A3G7/2n4NKAwuMMb8B9aPiC+JdcL2U8BnRKQJ6yhkccg2HhGRA1gNYbyIJMfeA+ywyzdhXZH4DIyYY++36+IPWEc03zaXr6j6R6zzDq/b+xrqAoOh/v5DeQnr6srfYz3UrgYGLsZYAnzd3tZBrG4wsIZpelqsk+PnwzcYK3rnuFJKqTHRIw6llFJjog2HUkqpMdGGQyml1Jhow6GUUmpMtOFQSik1JtpwKKWUGhNtOJRSSo2JNhxKKaXG5P8HWYdS13uX2X0AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "vals = [0.00039551136731949614,0.01532598592375935,0.0007175213740754419]\n",
    "df = pd.DataFrame({'Scenario':['SC1: Normal','SC2: Attacked','SC3: Defended'],'vals':vals})\n",
    "df.plot.bar(x='Scenario',y='vals', rot=0,color=['green', 'red','blue'], \n",
    "            edgecolor='black',legend='',linewidth=2)\n",
    "plt.xlabel('')\n",
    "ax = plt.gca()\n",
    "for i, v in enumerate(vals):\n",
    "    ax.text(-0.17+i*0.97, 0.0002 + v, str(np.round(vals[i],5)), color='blue')\n",
    "plt.ylabel('MSE')\n",
    "plt.grid(True, which=\"both\", ls=\"--\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "necessary-victorian",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}