Binary Search to gradient descent.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Q: How to find SQRT?\n",
    "A: BInary Search"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "625"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Find sqrt of 50\n",
    "l = 0\n",
    "r = 25\n",
    "guess = 25\n",
    "guess*guess"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "guess = (l + r)/2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "156.25"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "guess*guess"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "r = guess\n",
    "guess = (l + r)/2\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "39.0625"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "guess*guess"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "87.890625"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "l = guess\n",
    "guess = (l + r)/2\n",
    "guess*guess"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "61.03515625"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r = guess\n",
    "guess = (l + r)/2\n",
    "guess*guess"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "49.4384765625"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r = guess\n",
    "guess = (l + r)/2\n",
    "guess*guess"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "55.084228515625"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "l = guess\n",
    "guess = (l + r)/2\n",
    "guess*guess"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "52.22320556640625"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r = guess\n",
    "guess = (l + r)/2\n",
    "guess*guess"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "50.82130432128906"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r = guess\n",
    "guess = (l + r)/2\n",
    "guess*guess"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "50.127506256103516"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r = guess\n",
    "guess = (l + r)/2\n",
    "guess*guess"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "49.782395362854004"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r = guess\n",
    "guess = (l + r)/2\n",
    "guess*guess"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "49.95480179786682"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "l = guess\n",
    "guess = (l + r)/2\n",
    "guess*guess"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "50.041116774082184"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "l = guess\n",
    "guess = (l + r)/2\n",
    "guess*guess"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "49.997949972748756"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r = guess\n",
    "guess = (l + r)/2\n",
    "guess*guess"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "50.019531045109034"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "l = guess\n",
    "guess = (l + r)/2\n",
    "guess*guess"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "50.008739926852286"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r = guess\n",
    "guess = (l + r)/2\n",
    "guess*guess"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Q: x*x*x = 67, find x.\n",
    "#A: Binary Search"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "37595.375"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "l = 0\n",
    "r = 67\n",
    "x = (l + r)/2\n",
    "x * x * x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "74.08800000000001"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = 4.2\n",
    "x*x*x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "68.92099999999999"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "l, r = 4, 4.2\n",
    "x = (l + r)/2\n",
    "x*x*x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4.00625"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "67.06071885974123"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "l= x\n",
    "x = (l + r)/2\n",
    "x*x*x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "a = [0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1]\n",
    "# find the sum of this array in log(n) complexity.\n",
    "# Hint1: Solve using binary search\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Hint2: \n",
    "\n",
    "```\n",
    "mid = (left + right)//2\n",
    "\n",
    "if a[mid] == 1:\n",
    "    if a[mid-1] == 0:\n",
    "        we found position.\n",
    "    else:\n",
    "        We have to search the left and ignore the right\n",
    "        # right = mid\n",
    "else:\n",
    "    we have to search on the right\n",
    "    ignore the left\n",
    "    # left = mid\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "a = [0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1]\n",
    "left = 0\n",
    "right = len(a) - 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "left: 7, right: 15, mid: 11, a[mid]: 1, a[mid-1]: 0\n",
      "we found position.\n"
     ]
    }
   ],
   "source": [
    "mid = (left + right)//2\n",
    "# print(mid, a[mid])\n",
    "print(f\"left: {left}, right: {right}, mid: {mid}, a[mid]: {a[mid]}, a[mid-1]: {a[mid-1]}\")\n",
    "\n",
    "if a[mid] == 1:\n",
    "    if a[mid-1] == 0:\n",
    "        print(\"we found position.\")\n",
    "    else:\n",
    "        print(\"We have to search the left and ignore the right\")\n",
    "        right = mid\n",
    "else:\n",
    "    print(\"We have to search on the right ignore the left\")\n",
    "    left = mid"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Q: arr = [100, 89, 67, 45, 23, 10, -10, 12, 16, 19, 100, 200, 230]\n",
    "# Find min of this convex values?\n",
    "# Hint: Solve using binary search"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [],
   "source": [
    "# arr = [100, -10, 12, 16, 19, 100, 200, 230, 300, 305, 560]\n",
    "# Find min of this convex values?\n",
    "# Hint: Solve using binary search"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Q: Find the value of x for which value of y is minimum: y = x^2 - 5x\n",
    "\n",
    "# x = 0, y = 0\n",
    "# x = 1, y = -4\n",
    "# x = 2, y = -6\n",
    "# x = 3, y = -6"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [],
   "source": [
    "l = -100\n",
    "r = 100\n",
    "def f(x):\n",
    "    return x*x - 5*x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "l: -100, r:100, mid:0.0, vmid: 0.0\n",
      "Going right\n",
      "l: 0.0, r:100, mid:50.0, vmid: 2250.0\n",
      "Going left\n",
      "l: 0.0, r:50.0, mid:25.0, vmid: 500.0\n",
      "Going left\n",
      "l: 0.0, r:25.0, mid:12.5, vmid: 93.75\n",
      "Going left\n",
      "l: 0.0, r:12.5, mid:6.25, vmid: 7.8125\n",
      "Going left\n",
      "l: 0.0, r:6.25, mid:3.125, vmid: -5.859375\n",
      "Going left\n",
      "l: 0.0, r:3.125, mid:1.5625, vmid: -5.37109375\n",
      "Going right\n",
      "l: 1.5625, r:3.125, mid:2.34375, vmid: -6.2255859375\n",
      "Going right\n",
      "l: 2.34375, r:3.125, mid:2.734375, vmid: -6.195068359375\n",
      "Going left\n",
      "l: 2.34375, r:2.734375, mid:2.5390625, vmid: -6.24847412109375\n",
      "Going left\n",
      "l: 2.34375, r:2.5390625, mid:2.44140625, vmid: -6.2465667724609375\n",
      "Going right\n",
      "l: 2.44140625, r:2.5390625, mid:2.490234375, vmid: -6.249904632568359\n",
      "Going right\n",
      "l: 2.490234375, r:2.5390625, mid:2.5146484375, vmid: -6.249785423278809\n",
      "Going left\n",
      "l: 2.490234375, r:2.5146484375, mid:2.50244140625, vmid: -6.2499940395355225\n",
      "Going left\n",
      "l: 2.490234375, r:2.50244140625, mid:2.496337890625, vmid: -6.2499865889549255\n",
      "Going right\n",
      "l: 2.496337890625, r:2.50244140625, mid:2.4993896484375, vmid: -6.24999962747097\n",
      "Going right\n",
      "l: 2.4993896484375, r:2.50244140625, mid:2.50091552734375, vmid: -6.249999161809683\n",
      "Going left\n",
      "l: 2.4993896484375, r:2.50091552734375, mid:2.500152587890625, vmid: -6.249999976716936\n",
      "Going left\n",
      "l: 2.4993896484375, r:2.500152587890625, mid:2.4997711181640625, vmid: -6.249999947613105\n",
      "Going right\n",
      "l: 2.4997711181640625, r:2.500152587890625, mid:2.4999618530273438, vmid: -6.2499999985448085\n",
      "Going right\n",
      "l: 2.4999618530273438, r:2.500152587890625, mid:2.5000572204589844, vmid: -6.249999996725819\n",
      "Going left\n",
      "l: 2.4999618530273438, r:2.5000572204589844, mid:2.500009536743164, vmid: -6.2499999999090505\n",
      "Going left\n",
      "l: 2.4999618530273438, r:2.500009536743164, mid:2.499985694885254, vmid: -6.249999999795364\n",
      "Going right\n",
      "l: 2.499985694885254, r:2.500009536743164, mid:2.499997615814209, vmid: -6.249999999994316\n",
      "Going right\n",
      "l: 2.499997615814209, r:2.500009536743164, mid:2.5000035762786865, vmid: -6.24999999998721\n",
      "found: 2.5000035762786865\n"
     ]
    }
   ],
   "source": [
    "while True:\n",
    "    vl = f(l)\n",
    "    vr = f(r)\n",
    "    mid = (l + r)/2\n",
    "    vmid = f(mid)\n",
    "    print(f\"l: {l}, r:{r}, mid:{mid}, vmid: {vmid}\")\n",
    "    if abs(vl - vr) < 1e-10:\n",
    "        print(\"found:\", mid)\n",
    "        break\n",
    "    else:\n",
    "        righofmidv = f(mid + .000001)\n",
    "        if vmid > righofmidv:\n",
    "            l = mid\n",
    "            print(\"Going right\")\n",
    "        else:\n",
    "            r = mid\n",
    "            print(\"Going left\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Q: Write you own strategy to minimize x*x - 10x + 10"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "500.0"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "l = 0\n",
    "r = 50\n",
    "x = (l + r)/2\n",
    "x**2 - 5*x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [],
   "source": [
    "def f(x):\n",
    "    return x**2 - 5*x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = 5\n",
    "x1 = x + 1e-10\n",
    "y = f(x)\n",
    "y1 = f(x1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "4.0 -4.0\n",
      "5.0000000001 4.999982650133461e-10\n"
     ]
    }
   ],
   "source": [
    "if y1 > y:\n",
    "    x -= 0.5\n",
    "else:\n",
    "    x += 0.5\n",
    "y = f(x)\n",
    "print(x, y)\n",
    "y1 = f(x1)\n",
    "print(x1, y1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [],
   "source": [
    "coordinates = [(1, 2), (5, 9), (10,  21)]\n",
    "# Q: Find a straight line (slope and intercept) which fits through points the best\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [],
   "source": [
    "from matplotlib import pyplot as plt\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = [x for x, y in coordinates]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1, 5, 10]"
      ]
     },
     "execution_count": 85,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {},
   "outputs": [],
   "source": [
    "y = [y for x, y in coordinates]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[2, 9, 21]"
      ]
     },
     "execution_count": 87,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0x7f973bffd898>"
      ]
     },
     "execution_count": 88,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAARGklEQVR4nO3df6zddX3H8edrbc0qmhXGXaUFrVlIFzSDmhvE6Yw/KTIibDEbZHO4kVQNbrCYGtkSWTRZTDrdD9kkHTIwY2imBclES8NM0MRft4C0Al0ZQ+ltpVex4PQmUnzvj/ut3l7Opbf3nNtz+fT5SE7O9/v5fL7f7/t+k/u63/s533NOqgpJUrt+adgFSJIWlkEvSY0z6CWpcQa9JDXOoJekxi0ddgG9nHzyybVmzZphlyFJzxnbt2//flWN9OpblEG/Zs0axsbGhl2GJD1nJPnObH1O3UhS4wx6SWqcQS9JjTPoJalxBr0kNW5R3nUjSceTW+8ZZ9PWXew9MMmqFcvZuH4tF61bPbD9G/SSNES33jPOVVt2MPnU0wCMH5jkqi07AAYW9k7dSNIQbdq66+chf8jkU0+zaeuugR3DoJekIdp7YPKo2ufDoJekIVq1YvlRtc+HQS9JQ7Rx/VqWL1tyWNvyZUvYuH7twI7hi7GSNESHXnD1rhtJathF61YPNNhncupGkhpn0EtS444Y9ElOS/KlJPcn+XaSK7r2k5JsS7K7ez5xlu0v7cbsTnLpoH8ASdKzm8sV/UHgvVV1BnAOcHmSM4D3A3dW1enAnd36YZKcBFwNvBI4G7h6tj8IkqSFccSgr6p9VXV3t/wj4AFgNXAhcGM37Ebgoh6brwe2VdXjVfVDYBtw3gDqliTN0VHN0SdZA6wDvg6srKp9Xdf3gJU9NlkNPDptfU/XJkk6RuYc9EleAHwWuLKqnpzeV1UFVD+FJNmQZCzJ2MTERD+7kiRNM6egT7KMqZC/qaq2dM2PJTml6z8F2N9j03HgtGnrp3Ztz1BVm6tqtKpGR0Z6fpG5JGke5nLXTYBPAA9U1Uendd0GHLqL5lLgcz023wqcm+TE7kXYc7s2SdIxMpcr+lcDbwfekOTe7nE+8GHgzUl2A2/q1kkymuQ6gKp6HPgQ8M3u8cGuTZJ0jGRqen1xGR0drbGxsWGXIUnPGUm2V9Vorz7fGStJjTPoJalxBr0kNc6gl6TGGfSS1DiDXpIaZ9BLUuMMeklqnEEvSY0z6CWpcQa9JDXOoJekxhn0ktQ4g16SGmfQS1LjDHpJapxBL0mNW3qkAUmuBy4A9lfVy7u2TwNruyErgANVdVaPbR8BfgQ8DRyc7dtPJEkL54hBD9wAXAN88lBDVf3BoeUkHwGeeJbtX19V359vgZKk/hwx6KvqriRrevUlCfD7wBsGXJckaUD6naP/beCxqto9S38BdyTZnmRDn8eSJM3DXKZuns0lwM3P0v+aqhpP8mvAtiQPVtVdvQZ2fwg2ALz4xS/usyxJ0iHzvqJPshT4PeDTs42pqvHueT9wC3D2s4zdXFWjVTU6MjIy37IkSTP0M3XzJuDBqtrTqzPJCUleeGgZOBfY2cfxJEnzcMSgT3Iz8FVgbZI9SS7rui5mxrRNklVJbu9WVwJfSfIt4BvA56vqi4MrXZI0F3O56+aSWdrf0aNtL3B+t/wwcGaf9UmS+uQ7YyWpcQa9JDXOoJekxhn0ktQ4g16SGmfQS1LjDHpJapxBL0mNM+glqXEGvSQ1zqCXpMYZ9JLUOINekhpn0EtS4wx6SWqcQS9JjTPoJalxBr0kNW4u3xl7fZL9SXZOa/vrJONJ7u0e58+y7XlJdiV5KMn7B1m4JGlu5nJFfwNwXo/2v6uqs7rH7TM7kywB/gl4C3AGcEmSM/opVpJ09I4Y9FV1F/D4PPZ9NvBQVT1cVT8FPgVcOI/9SJL60M8c/XuS3NdN7ZzYo3818Oi09T1dW09JNiQZSzI2MTHRR1mSpOnmG/QfB34dOAvYB3yk30KqanNVjVbV6MjISL+7kyR15hX0VfVYVT1dVT8D/oWpaZqZxoHTpq2f2rVJko6heQV9klOmrf4usLPHsG8Cpyd5aZLnARcDt83neJKk+Vt6pAFJbgZeB5ycZA9wNfC6JGcBBTwCvLMbuwq4rqrOr6qDSd4DbAWWANdX1bcX4oeQJM0uVTXsGp5hdHS0xsbGhl2GJD1nJNleVaO9+nxnrCQ1zqCXpMYZ9JLUOINekhpn0EtS4wx6SWqcQS9JjTPoJalxBr0kNc6gl6TGGfSS1DiDXpIaZ9BLUuMMeklqnEEvSY0z6CWpcQa9JDXuiEGf5Pok+5PsnNa2KcmDSe5LckuSFbNs+0iSHUnuTeJXRknSEMzliv4G4LwZbduAl1fVbwL/DVz1LNu/vqrOmu0rriRJC+uIQV9VdwGPz2i7o6oOdqtfA05dgNokSQMwiDn6PwW+MEtfAXck2Z5kw7PtJMmGJGNJxiYmJgZQliQJ+gz6JH8FHARummXIa6rqFcBbgMuTvHa2fVXV5qoararRkZGRfsqSJE0z76BP8g7gAuAPq6p6jamq8e55P3ALcPZ8jydJmp95BX2S84D3AW+tqp/MMuaEJC88tAycC+zsNVaStHDmcnvlzcBXgbVJ9iS5DLgGeCGwrbt18tpu7Kokt3ebrgS+kuRbwDeAz1fVFxfkp5AkzWrpkQZU1SU9mj8xy9i9wPnd8sPAmX1VJ0nqm++MlaTGGfSS1DiDXpIaZ9BLUuMMeklqnEEvSY0z6CWpcQa9JDXOoJekxhn0ktQ4g16SGmfQS1LjDHpJapxBL0mNM+glqXEGvSQ1zqCXpMbNKeiTXJ9kf5Kd09pOSrItye7u+cRZtr20G7M7yaWDKlySNDdzvaK/AThvRtv7gTur6nTgzm79MElOAq4GXgmcDVw92x8ESdLCmFPQV9VdwOMzmi8EbuyWbwQu6rHpemBbVT1eVT8EtvHMPxiSpAXUzxz9yqra1y1/D1jZY8xq4NFp63u6tmdIsiHJWJKxiYmJPsqSJE03kBdjq6qA6nMfm6tqtKpGR0ZGBlGWJIn+gv6xJKcAdM/7e4wZB06btn5q1yZJOkb6CfrbgEN30VwKfK7HmK3AuUlO7F6EPbdrkyQdI3O9vfJm4KvA2iR7klwGfBh4c5LdwJu6dZKMJrkOoKoeBz4EfLN7fLBrkyQdI5maXl9cRkdHa2xsbNhlSNJzRpLtVTXaq893xkpS45YOuwBpod16zzibtu5i74FJVq1Yzsb1a7loXc+7fKUmGfRq2q33jHPVlh1MPvU0AOMHJrlqyw4Aw17HDadu1LRNW3f9POQPmXzqaTZt3TWkiqRjz6BX0/YemDyqdqlFBr2atmrF8qNql1pk0KtpG9evZfmyJYe1LV+2hI3r1w6pIunY88VYNe3QC67edaPjmUGv5l20brXBruOaUzeS1DiDXpIaZ9BLUuMMeklqnEEvSY0z6CWpcQa9JDXOoJekxs076JOsTXLvtMeTSa6cMeZ1SZ6YNuYDfVcsSToq835nbFXtAs4CSLIEGAdu6TH0y1V1wXyPI0nqz6Cmbt4I/E9VfWdA+5MkDciggv5i4OZZ+l6V5FtJvpDkZbPtIMmGJGNJxiYmJgZUliSp76BP8jzgrcB/9Oi+G3hJVZ0JfAy4dbb9VNXmqhqtqtGRkZF+y5IkdQZxRf8W4O6qemxmR1U9WVX/1y3fDixLcvIAjilJmqNBBP0lzDJtk+RFSdItn90d7wcDOKYkaY76+jz6JCcAbwbeOa3tXQBVdS3wNuDdSQ4Ck8DFVVX9HFOSdHT6Cvqq+jHwqzParp22fA1wTT/HkCT1x3fGSlLjDHpJapxBL0mNM+glqXEGvSQ1zqCXpMYZ9JLUOINekhpn0EtS4wx6SWqcQS9JjTPoJalxBr0kNc6gl6TGGfSS1DiDXpIaZ9BLUuP6DvokjyTZkeTeJGM9+pPkH5M8lOS+JK/o95iSpLnr66sEp3l9VX1/lr63AKd3j1cCH++eJUnHwLGYurkQ+GRN+RqwIskpx+C4kiQGE/QF3JFke5INPfpXA49OW9/TtR0myYYkY0nGJiYmBlCWJAkGE/SvqapXMDVFc3mS185nJ1W1uapGq2p0ZGRkAGVJkmAAQV9V493zfuAW4OwZQ8aB06atn9q1SZKOgb6CPskJSV54aBk4F9g5Y9htwB93d9+cAzxRVfv6Oa4kae76vetmJXBLkkP7+veq+mKSdwFU1bXA7cD5wEPAT4A/6fOYkqSj0FfQV9XDwJk92q+dtlzA5f0cR5I0f74zVpIaZ9BLUuMMeklqnEEvSY0z6CWpcQa9JDXOoJekxhn0ktQ4g16SGmfQS1LjDHpJapxBL0mNM+glqXEGvSQ1zqCXpMYZ9JLUOINekho376BPclqSLyW5P8m3k1zRY8zrkjyR5N7u8YH+ypUkHa1+vkrwIPDeqrq7+4Lw7Um2VdX9M8Z9uaou6OM4kqQ+zPuKvqr2VdXd3fKPgAeA1YMqTJI0GAOZo0+yBlgHfL1H96uSfCvJF5K87Fn2sSHJWJKxiYmJQZQlSWIAQZ/kBcBngSur6skZ3XcDL6mqM4GPAbfOtp+q2lxVo1U1OjIy0m9ZkqROP3P0JFnGVMjfVFVbZvZPD/6quj3JPyc5uaq+389xe7n1nnE2bd3F3gOTrFqxnI3r13LROmeSJGneQZ8kwCeAB6rqo7OMeRHwWFVVkrOZ+g/iB/M95mxuvWecq7bsYPKppwEYPzDJVVt2ABj2ko57/VzRvxp4O7Ajyb1d218CLwaoqmuBtwHvTnIQmAQurqrq45g9bdq66+chf8jkU0+zaesug17ScW/eQV9VXwFyhDHXANfM9xhztffA5FG1S9LxpIl3xq5asfyo2iXpeNJE0G9cv5bly5Yc1rZ82RI2rl87pIokafHo666bxeLQPLx33UjSMzUR9DAV9ga7JD1TE1M3kqTZGfSS1DiDXpIaZ9BLUuMMeklqXBbgEwn6lmQC+M6w6+jTycDAP7ztOcpzcTjPx+E8H7/Qz7l4SVX1/OjfRRn0LUgyVlWjw65jMfBcHM7zcTjPxy8s1Llw6kaSGmfQS1LjDPqFs3nYBSwinovDeT4O5/n4hQU5F87RS1LjvKKXpMYZ9JLUOIN+gJKcluRLSe5P8u0kVwy7psUgyZIk9yT5z2HXMkxJViT5TJIHkzyQ5FXDrmmYkvxF93uyM8nNSX552DUdS0muT7I/yc5pbScl2ZZkd/d84iCOZdAP1kHgvVV1BnAOcHmSM4Zc02JwBfDAsItYBP4B+GJV/QZwJsfxOUmyGvhzYLSqXg4sAS4eblXH3A3AeTPa3g/cWVWnA3d2630z6AeoqvZV1d3d8o+Y+kU+rj8kP8mpwO8A1w27lmFK8ivAa4FPAFTVT6vqwFCLGr6lwPIkS4HnA3uHXM8xVVV3AY/PaL4QuLFbvhG4aBDHMugXSJI1wDrg60MuZdj+Hngf8LMh1zFsLwUmgH/tprGuS3LCsIsalqoaB/4W+C6wD3iiqu4YblWLwsqq2tctfw9YOYidGvQLIMkLgM8CV1bVk8OuZ1iSXADsr6rtw65lEVgKvAL4eFWtA37MgP4tfy7q5p4vZOoP4CrghCR/NNyqFpeauvd9IPe/G/QDlmQZUyF/U1VtGXY9Q/Zq4K1JHgE+Bbwhyb8Nt6Sh2QPsqapD/+F9hqngP169CfjfqpqoqqeALcBvDbmmxeCxJKcAdM/7B7FTg36AkoSpOdgHquqjw65n2Krqqqo6tarWMPVC239V1XF51VZV3wMeTbK2a3ojcP8QSxq27wLnJHl+93vzRo7jF6enuQ24tFu+FPjcIHZq0A/Wq4G3M3Xlem/3OH/YRWnR+DPgpiT3AWcBfzPccoan+8/mM8DdwA6msui4+iiEJDcDXwXWJtmT5DLgw8Cbk+xm6r+eDw/kWH4EgiS1zSt6SWqcQS9JjTPoJalxBr0kNc6gl6TGGfSS1DiDXpIa9/+JWb5Qkc5lgQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(x, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "coordinates = [(1, 2), (5, 9), (10,  21)]\n",
    "\n",
    "\n",
    "y = mx + c\n",
    "\n",
    "for x = 1, ypred = m + c\n",
    "\n",
    "for y = 2, how much is error: m+c - 2\n",
    "\n",
    "....\n",
    "Overall Error: `(m+c - 2)**2 + (5m+c - 9)**2 + (10m+c - 21)**2`\n",
    "\n",
    "\n",
    "**Question: Find m and c such that Error is minimum**\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {},
   "outputs": [],
   "source": [
    "def error(m, c):\n",
    "    return (m+c - 2)**2 + (5*m+c - 9)**2 + (10*m+c - 21)**2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "109"
      ]
     },
     "execution_count": 91,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "error(1, 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "57.36"
      ]
     },
     "execution_count": 116,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "error(1.6, 6)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "ename": "SyntaxError",
     "evalue": "invalid syntax (<ipython-input-1-6f7c446b62bf>, line 6)",
     "output_type": "error",
     "traceback": [
      "\u001b[0;36m  File \u001b[0;32m\"<ipython-input-1-6f7c446b62bf>\"\u001b[0;36m, line \u001b[0;32m6\u001b[0m\n\u001b[0;31m    for x = 1, ypred = m + c\u001b[0m\n\u001b[0m          ^\u001b[0m\n\u001b[0;31mSyntaxError\u001b[0m\u001b[0;31m:\u001b[0m invalid syntax\n"
     ]
    }
   ],
   "source": [
    "coordinates = [(1, 2), (5, 9), (10,  21)]\n",
    "\n",
    "\n",
    "y = m1*x*(mx + c) + c1\n",
    "\n",
    "for x = 1, ypred = m + c\n",
    "\n",
    "for y = 2, how much is error: m+c - 2\n",
    "\n",
    "....\n",
    "Overall Error: `(m+c - 2)**2 + (5m+c - 9)**2 + (10m+c - 21)**2`\n",
    "\n",
    "\n",
    "**Question: Find m and c such that Error is minimum**\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {},
   "outputs": [],
   "source": [
    "def heaviside(z):\n",
    "    if z < 0:\n",
    "        return 0\n",
    "    else:\n",
    "        return 1\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 118,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "heaviside(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Assignment: Study the Hebbian principle\n",
    "\n",
    "Hebb’s Rule\n",
    "In order to understand how we train a perceptron we need to understand Hebb’s rule.\n",
    "In his book The Organization of Behavior, published in 1949, Donald Hebb suggested that \n",
    "\n",
    "\"When a biological neuron often triggers another neuron, the connection between these two neurons grows stronger. \"\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}