3_tensor_derivative.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 第3天：张量的自动求导机制"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensorflow的版本:2.1.0\r\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "tf.print('tensorflow的版本:{}'.format(tf.__version__))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1、自动求导机制\n",
    "所谓的自动求导机制，就是对于属性为变量的张量，tensorflow会自动的将该变量加入到它的求导记录器tf.GradientTape()中，实现自动求导。对于属性为常量的张量而言，需要将该常量手工加入，涉及的函数就是**watch**，具体参见下面给出的示例。 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 1.1、一元函数求导\n",
    "\n",
    "一元函数，也就是函数中变量的个数为1。引入一个例子，$y = 3x^{3} - 4x^{2} - 2x + 2$，计算函数$y$在$x=3$处的一阶导数$y1$，二阶导数$y2$的值。\n",
    "因为$$\\frac{\\mathrm{dy}}{\\mathrm{d}x} = 9x^{2} - 8x-2$$\n",
    "$$\\frac{\\mathrm{d}^{2}y}{\\mathrm{d}x^{2}} = 18x-8$$\n",
    "\n",
    "所以$y1=9\\times 3^{2} - 8\\times 3-2=55，y2=18\\times 3 - 8=28$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "函数y在x=3处的一阶导数:y1=55.0\n"
     ]
    }
   ],
   "source": [
    "# 注意类型必须均为浮点型，并且一致\n",
    "# 常量\n",
    "a = tf.constant(3.)\n",
    "b = tf.constant(4.)\n",
    "c = tf.constant(2.)\n",
    "d = tf.constant(2,dtype=tf.float32)\n",
    "# 变量\n",
    "x = tf.Variable(3, dtype=tf.float32, name='x')\n",
    "\n",
    "with tf.GradientTape() as g: # 自动求导机制\n",
    "    y = a * tf.pow(x, 3) - b * tf.square(x) - c * x + d  # 函数\n",
    "y1 = g.gradient(y, x) # 一阶导数\n",
    "print('函数y在x=3处的一阶导数:y1=', y1.numpy(), sep='')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "函数y在x=3处的关于a的一阶导数:y1_a=27.0\n",
      "函数y在x=3处的关于b的一阶导数:y1_a=-9.0\n"
     ]
    }
   ],
   "source": [
    "# 注意类型必须均为浮点性，并且一致\n",
    "# 常量\n",
    "a = tf.constant(3.)\n",
    "b = tf.constant(4.)\n",
    "c = tf.constant(2.)\n",
    "d = tf.constant(2,dtype=tf.float32)\n",
    "# 变量\n",
    "x = tf.Variable(3, dtype=tf.float32, name='x')\n",
    "\n",
    "with tf.GradientTape() as g: # 自动求导机制\n",
    "    g.watch([a, b]) # 手动的将常量的a，b加入到求导的记录器中\n",
    "    y = a * tf.pow(x, 3) - b * tf.square(x) - c * x + d  # 函数\n",
    "y1_x, y1_a, y1_b = g.gradient(y, [x, a, b]) # 一阶导数\n",
    "\n",
    "print('函数y在x=3处的关于a的一阶导数:y1_a=', y1_a.numpy(), sep='')\n",
    "print('函数y在x=3处的关于b的一阶导数:y1_a=', y1_b.numpy(), sep='')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在**tf.GradientTape()**下，只会保留计算一次的导数，因此如果需要多次调用**.gradient**，参数persistent需要设置为**True**，该值默认为**False**。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "函数y在x=2处的一阶导数:y1=18.0\n",
      "函数z1在x=2处的一阶导数:z1=43.0\n"
     ]
    }
   ],
   "source": [
    "# 注意类型必须均为浮点型，并且一致\n",
    "x = tf.Variable(2, dtype=tf.float32, name='x')\n",
    "\n",
    "with tf.GradientTape(persistent=True) as g:\n",
    "    y = 3 * tf.pow(x, 3) - 4 * tf.square(x) - 2 * x + 2\n",
    "    z = 4 * tf.pow(x, 3) - 2 * tf.square(x) + 3 * x - 1\n",
    "y1 = g.gradient(y, x) \n",
    "z1 = g.gradient(z, x) \n",
    "print('函数y在x=2处的一阶导数:y1=', y1.numpy(), sep='')\n",
    "print('函数z1在x=2处的一阶导数:z1=', z1.numpy(), sep='')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "计算函数的二阶导数，是利用嵌套来完成的，高阶亦如此。示例如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "函数y在x=2处的二阶导数:y2=28.0\n"
     ]
    }
   ],
   "source": [
    "# 注意类型必须均为浮点型，并且一致\n",
    "x = tf.Variable(2, dtype=tf.float32, name='x')\n",
    "\n",
    "with tf.GradientTape() as g2:\n",
    "    with tf.GradientTape() as g1:\n",
    "        y = 3 * tf.pow(x, 3) - 4 * tf.square(x) - 2 * x + 2\n",
    "    y1 = g1.gradient(y, x) \n",
    "y2 = g2.gradient(y1, x) \n",
    "print('函数y在x=2处的二阶导数:y2=', y2.numpy(), sep='')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 1.2 多元函数求导\n",
    "多元就是函数中包含多个变量，下面引入一个例子：$$y = 6x_1^{2} - x_2^{2} + 4x_1x_2-5x_2$$\n",
    "则函数$y$关于$x_1,x_2$的一阶导数分别为：\n",
    "$$\\frac{\\mathrm{dy}}{\\mathrm{d}x_1} = 12x_1 + 4x_2$$\n",
    "$$\\frac{\\mathrm{dy}}{\\mathrm{d}x_2} = -2x_2+4x_1-5$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "函数y在x_1=3, x_2=1处的关于x_1的一阶导数:dx1=40.0,关于x_2的一阶导数:dx2=5.0\n"
     ]
    }
   ],
   "source": [
    "# 注意类型必须均为浮点型，并且一致\n",
    "# 变量\n",
    "x1 = tf.Variable(3, dtype=tf.float32, name='x_1')\n",
    "x2 = tf.Variable(1, dtype=tf.float32, name='x_2')\n",
    "\n",
    "with tf.GradientTape() as g: # 自动求导机制\n",
    "    y = 6 * tf.pow(x1, 2) - tf.pow(x2, 2) + 4 * x1 * x2 - 5 * x2 \n",
    "dx1, dx2 = g.gradient(y, [x1, x2]) # 一阶导数\n",
    "print('函数y在x_1=3, x_2=1处的关于x_1的一阶导数:dx1=', dx1.numpy(), ',关于x_2的一阶导数:dx2=', dx2.numpy(),sep='')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 1.3 向量求导\n",
    "向量求导就是把向量作为变量，就是下面引入一个例子：$$Y=AX$$\n",
    "其中$X$是一个2行一列的向量，$$A = \\begin{bmatrix}\n",
    "-3 & 6 \\\\ \n",
    "4 &  6 \\\\\n",
    "5 &  7\n",
    "\\end{bmatrix}$$\n",
    "则函数$Y$关于$X$的一阶导数为：\n",
    "$$\\frac{\\partial Y}{\\partial X} = \\begin{bmatrix}\n",
    "-3 & 4 &5 \\\\ \n",
    "6 &  6 &7\\\\\n",
    "\\end{bmatrix}$$\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "函数y关于X的一阶导数:dX=[[[[-3.]\n",
      "   [ 6.]]]\n",
      "\n",
      "\n",
      " [[[ 4.]\n",
      "   [ 6.]]]\n",
      "\n",
      "\n",
      " [[[ 5.]\n",
      "   [ 7.]]]]\n",
      "函数y关于X的梯度:gX=[[ 6.]\n",
      " [19.]]\n"
     ]
    }
   ],
   "source": [
    "# 注意类型必须均为浮点型，并且一致\n",
    "# 变量\n",
    "X = tf.Variable([[1], [1]], dtype=tf.float32, name='X')\n",
    "A = tf.constant([[-3, 6],[4, 6], [5, 7]], dtype=tf.float32)\n",
    "with tf.GradientTape(persistent=True) as g: # \n",
    "    Y = tf.matmul(A, X)\n",
    "dX = g.jacobian(Y, X) # 一阶导数,注意此处不是梯度\n",
    "print('函数y关于X的一阶导数:dX=', dX.numpy(),sep='')\n",
    "\n",
    "gX = g.gradient(Y, X) # 一阶导数,注意此处不是梯度\n",
    "print('函数y关于X的梯度:gX=', gX.numpy(),sep='') # 梯度的维度一定和变量的维度是一模一样的，这样才能利用梯度下降法更改参数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 1.4 矩阵求导\n",
    "矩阵求导就是把矩阵作为变量，这种情况在机器学习中经常出现。下面引入一个例子：$$Y= X^{T}AX$$\n",
    "其中$X$是一个3行2列的矩阵,$$A = \\begin{bmatrix}\n",
    "-3 & 6 & 4\\\\ \n",
    "4 &  6  & 3\\\\\n",
    "5 &  7 & 9\n",
    "\\end{bmatrix}$$\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "函数y关于X的一阶导数:dX=[[[[13.  0.]\n",
      "   [32.  0.]\n",
      "   [37.  0.]]\n",
      "\n",
      "  [[ 7.  6.]\n",
      "   [13. 19.]\n",
      "   [21. 16.]]]\n",
      "\n",
      "\n",
      " [[[ 6.  7.]\n",
      "   [19. 13.]\n",
      "   [16. 21.]]\n",
      "\n",
      "  [[ 0. 13.]\n",
      "   [ 0. 32.]\n",
      "   [ 0. 37.]]]]\n",
      "函数y关于X的梯度:gX=[[26. 26.]\n",
      " [64. 64.]\n",
      " [74. 74.]]\n"
     ]
    }
   ],
   "source": [
    "# 注意类型必须均为浮点型，并且一致\n",
    "# 变量\n",
    "X = tf.Variable(tf.ones((3, 2)), dtype=tf.float32, name='X')\n",
    "A = tf.constant([[-3, 6, 4],[4, 6, 3], [5, 7, 9]], dtype=tf.float32)\n",
    "with tf.GradientTape(persistent=True) as g: # \n",
    "    Y = tf.matmul(tf.matmul(tf.transpose(X), A), X)\n",
    "dX = g.jacobian(Y, X) # 一阶导数,注意此处不是梯度\n",
    "print('函数y关于X的一阶导数:dX=', dX.numpy(),sep='')\n",
    "\n",
    "gX = g.gradient(Y, X) # 一阶导数,注意此处不是梯度\n",
    "print('函数y关于X的梯度:gX=', gX.numpy(),sep='') # 梯度的维度一定和变量的维度是一模一样的，这样才能利用梯度下降法更改参数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2、结合优化器计算函数最小值\n",
    "\n",
    "下面展示计算最小值的几种实现方式\n",
    "\n",
    " + **optimizer.apply_gradients**\n",
    " + **optimizer.apply_gradients+函数形式**\n",
    " + **autograph**\n",
    " + **autograph+函数形式**\n",
    " \n",
    "下面给出计算函数最小值的示例：\n",
    "\n",
    "首先引入一个示例：$y = 2x^{2} - 6x + 5$，计算函数的最小值。\n",
    "因为$y = 2x^{2} - 6x + 5 = 2(x-1.5)^{2}+0.5$，所以当$x=1.5$时，$y$取得最小值$ymin = 0.5$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "最小值ymin=0.5 此时x=1.5\n"
     ]
    }
   ],
   "source": [
    "# optimizer.apply_gradients\n",
    "x = tf.Variable(2., name=\"x\")\n",
    "# 优化器，有很多优化器可供选择\n",
    "# 优化器可以获得最小值的原理，就是变量x沿着梯度的相反的方向以learning_rate大小的步长变化，\n",
    "# 也就是x=x-learning_rate*dy_dx,这样持续一定的步数，可以取得比较满意的最小值。\n",
    "optimizer = tf.keras.optimizers.Adam(learning_rate=0.02)\n",
    "for _ in range(1280):  # 持续1280步\n",
    "    with tf.GradientTape() as tape:\n",
    "        y = 2 * tf.square(x) - 6 * x + 5  # 需要获得最小值的函数\n",
    "    dy_dx = tape.gradient(y, x) # 计算一阶导数\n",
    "    optimizer.apply_gradients(grads_and_vars=[(dy_dx, x)])  # 应用一阶导数\n",
    "print('最小值ymin={}'.format(y.numpy()), '此时x={}'.format(x.numpy()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "最小值ymin=-11.5 此时x1=1.5 此时x2=-1.999998927116394\n"
     ]
    }
   ],
   "source": [
    "# optimizer.apply_gradients+函数, 机器学习中比较常用的形式\n",
    "x1 = tf.Variable(1.)\n",
    "x2 = tf.Variable(1.)\n",
    "\n",
    "#注意func()无参数\n",
    "def func():    \n",
    "    y = 2 * tf.square(x1) - 6 * x1 + 5 + tf.square(x2) + 4 * x2 - 8\n",
    "    return y\n",
    "# 优化器\n",
    "optimizer = tf.keras.optimizers.Adam(learning_rate=0.02)   \n",
    "for _ in range(1280):\n",
    "    optimizer.minimize(func, [x1, x2]) # 最小值   \n",
    "y = func()  # 此时x1，x2已经是使得y取得最小值的数值\n",
    "print('最小值ymin={}'.format(y.numpy()), '此时x1={}'.format(x1.numpy()), '此时x2={}'.format(x2.numpy()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "最小值ymin=-11.5 此时x1=1.5 此时x2=-2.000000476837158\n"
     ]
    }
   ],
   "source": [
    "# autograph 动态计算图\n",
    "x1 = tf.Variable(2.)\n",
    "x2 = tf.Variable(2.)\n",
    "optimizer = tf.keras.optimizers.Adam(learning_rate=0.08)\n",
    "\n",
    "@tf.function\n",
    "def targetfunc():\n",
    "    for _ in tf.range(1280): # 使用tf.range(1280)\n",
    "        with tf.GradientTape() as g:\n",
    "            y = 2 * tf.square(x1) - 6 * x1 + 5 + tf.square(x2) + 4 * x2 - 8\n",
    "        dy_dx = g.gradient(y, [x1, x2])\n",
    "        optimizer.apply_gradients(grads_and_vars=zip(dy_dx, [x1, x2]))\n",
    "        \n",
    "    y =  2 * tf.square(x1) - 6 * x1 + 5 + tf.square(x2) + 4 * x2 - 8 # 根据得到的x值计算y的最小值\n",
    "    return y\n",
    "y = targetfunc()\n",
    "print('最小值ymin={}'.format(y.numpy()), '此时x1={}'.format(x1.numpy()), '此时x2={}'.format(x2.numpy()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "最小值ymin=0.5 此时x=1.5\n"
     ]
    }
   ],
   "source": [
    "# autograph+函数形式\n",
    "x = tf.Variable(0.2)\n",
    "optimizer = tf.keras.optimizers.SGD(learning_rate=0.32)   \n",
    "\n",
    "@tf.function\n",
    "def func():   \n",
    "    return 2 * tf.square(x) - 6 * x + 5\n",
    "\n",
    "@tf.function\n",
    "def train(epoch):  \n",
    "    for _ in tf.range(epoch):  \n",
    "        optimizer.minimize(func, [x])# 最小值\n",
    "    return func()\n",
    "\n",
    "y = train(1280)\n",
    "print('最小值ymin={}'.format(y), '此时x={}'.format(x.numpy()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3、线性回归示例\n",
    "\n",
    "对于线性回归模型$$Y=WX+B$$\n",
    "也就是计算使得最小二乘误差$$cost =\\sum_{j=1}^{m} \\sum_{i=1}^{n}(y_{ji}-\\tilde{y}_{ji})^{2}$$\n",
    "得到最小值的$W,B$。\n",
    "\n",
    "其中$m$是一条数据中因变量的个数，$n$是数据条数，$y_{ji}$是$i$条数据的因变量$j$的值，$\\tilde{y}_{ji}$是对应的模型输出值。\n",
    "假设自变量矩阵的维度为$[d, n]$，也就是一条数据有$d$个自变量；因变量的维度为$[m, n]$。也就是一条数据有$m$个因变量，则$W$的维度就是$[m, d]$，$B$的维度就是$[m, 1]$。当d等于1时，$W$就可看作一个标量；当m等于1时，$B$也可以看成一个标量。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.1 简单线性回归\n",
    "\n",
    "简单线性回归就是只有一个自变量x，一个因变量y的模型，模型可以写成：\n",
    "$$y=ax+b$$\n",
    "下面给出一组数据：\n",
    "![](31.jpg)\n",
    "计算出$a, b$的值。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "误差：5824.0\n",
      "误差：5.684341886080802e-14\n",
      "误差：5.684341886080802e-14\n",
      "误差：0.0\n",
      "误差：0.0\n",
      "最终最小二乘误差cost=0.0 此时模型参数a=2.0, b=1.0000001192092896\n",
      "模型输出值：[ 7. 11. 17. 19.  3. 31. 35. 53.]\n"
     ]
    }
   ],
   "source": [
    "# 3.1 代码实现\n",
    "x = tf.constant([3, 5, 8, 9, 1, 15, 17, 26], dtype=tf.float32)\n",
    "y = tf.constant([7, 11, 17, 19, 3, 31, 35, 53], dtype=tf.float32)\n",
    "a = tf.Variable(0.)\n",
    "b = tf.Variable(0.)\n",
    "# 优化器\n",
    "optimizer = tf.keras.optimizers.Adam(learning_rate=0.6)\n",
    "for _ in range(4001):  # 迭代的次数\n",
    "    with tf.GradientTape() as t:\n",
    "        y_modelout = a * x + b # 模型的输出值\n",
    "        cost = tf.reduce_sum(tf.square(y - y_modelout))  # 定义最小二乘法误差\n",
    "        if _ % 1000 == 0: # 打印误差\n",
    "            print('误差：{}'.format(cost))\n",
    "    d_cost = t.gradient(cost, [a, b]) # 计算一阶导数\n",
    "    optimizer.apply_gradients(grads_and_vars=zip(d_cost, [a, b]))  # 应用一阶导数\n",
    "    \n",
    "print('最终最小二乘误差cost={}'.format(cost.numpy()), '此时模型参数a={},'.format(a.numpy()), 'b={}'.format(b.numpy())) \n",
    "print('模型输出值：{}'.format(y_modelout.numpy()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2  多重线性回归\n",
    "\n",
    "又称多因素线性回归，也就是有多个自变量，一个因变量y的模型，模型可以写成： \n",
    "$$y=AX+b$$\n",
    "![](32.jpg)\n",
    "计算出$A, b$的值。\n",
    "\n",
    "上面的数据中有8条数据，每条数据有3个自变量，所有自变量构成的数据矩阵就是3行8列的，因变量矩阵是1行8列的。所以参数矩阵A就是1行3列的，参数矩阵b就是一个标量。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "误差：1625.72119140625\n",
      "误差：2.1829843521118164\n",
      "误差：1.9039669036865234\n",
      "误差：1.6688652038574219\n",
      "误差：1.6241304874420166\n",
      "最小二乘误差cost=1.6241304874420166 此时A=[[ 0.79583377  0.80701506 -0.28584385]], b=0.5898457169532776\n",
      "模型输出值：[[ 3.969173   6.8273287  9.410822  11.237818   2.1309967 16.903612\n",
      "  18.977118  28.54341  ]]\n"
     ]
    }
   ],
   "source": [
    "# 3.2 代码实现\n",
    "x = tf.constant([[3, 6, 8, 10, 2, 16, 17, 26],[3, 5, 8, 9, 1, 14, 17, 26],[5, 9, 14, 16, 3, 27, 31, 48]], dtype=tf.float32)\n",
    "y = tf.constant([4, 7, 9, 11, 2, 17, 20, 28], dtype=tf.float32)\n",
    "A = tf.Variable(tf.zeros((1, 3)), dtype=tf.float32, name='A') # 1行3列\n",
    "b = tf.Variable(0., name='b')\n",
    "\n",
    "# 模型输出函数\n",
    "def modelout():\n",
    "    y_modelout = tf.matmul(A, x) + b\n",
    "    return y_modelout\n",
    "# 计算误差的函数    \n",
    "def costfunc():    \n",
    "    cost = tf.reduce_sum(tf.square(y - modelout())) \n",
    "    return cost\n",
    "# 优化器\n",
    "optimizer = tf.keras.optimizers.Adam(learning_rate=0.01)   \n",
    "for _ in range(8001):\n",
    "    optimizer.minimize(costfunc, [A, b]) # 最小值   \n",
    "    if _ % 2000 == 0:\n",
    "        print('误差：{}'.format(costfunc()))  # 误差的变化\n",
    "        \n",
    "cost, y_modelout = costfunc(), modelout()   \n",
    "print('最小二乘误差cost={}'.format(cost.numpy()), '此时A={},'.format(A.numpy()), 'b={}'.format(b.numpy())) \n",
    "print('模型输出值：{}'.format(y_modelout.numpy()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.3 多重多元线性回归\n",
    "\n",
    "多元指的是有多个因变量的模型。多重多元线性回归的模型，可以写成：\n",
    "$$Y=AX+B$$\n",
    "![](33.jpg)\n",
    "计算出$A, B$的值。\n",
    "\n",
    "上面的数据中有8条数据，每条数据有3个自变量，所有自变量构成的数据矩阵就是3行8列的；有2个因变量，因变量矩阵是2行8列的。所以参数矩阵A就是2行3列的，参数矩阵b就是2行一列的。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "最终的最小二乘误差cost=3.6218783855438232 此时A=[[ 0.5145932   0.4480298   0.05619257]\n",
      " [-0.23077865  0.02671658 -0.8352975 ]], B=[[0.88975173]\n",
      " [1.2307721 ]]\n",
      "模型输出值：[[  4.0585837   6.723193    9.377432   10.967032    2.5355456  16.912859\n",
      "   18.996311   28.615192 ]\n",
      " [ -3.5579019  -7.5379944 -12.09589   -14.201325   -1.709961  -24.640688\n",
      "  -28.132507  -44.16912  ]]\n"
     ]
    }
   ],
   "source": [
    "# 3.3 代码实现\n",
    "X = tf.constant([[3, 6, 8, 10, 2, 16, 17, 26],[3, 5, 8, 9, 1, 14, 17, 26],[5, 9, 14, 16, 3, 27, 31, 48]], dtype=tf.float32)\n",
    "Y = tf.constant([[4, 7, 9, 11, 2, 17, 20, 28],[-3, -7, -13, -14, -2, -25, -28, -44]], dtype=tf.float32)\n",
    "A = tf.Variable(tf.zeros((2, 3)), dtype=tf.float32, name='A')  # 2行3列\n",
    "B = tf.Variable(tf.ones((2, 1)), dtype=tf.float32, name='B')  # 2行一列\n",
    "# 优化器\n",
    "optimizer = tf.keras.optimizers.SGD(learning_rate=0.000022)   \n",
    "\n",
    "@tf.function  \n",
    "def modelout():\n",
    "    y_modelout = tf.add(tf.matmul(A, X), B)\n",
    "    return y_modelout\n",
    "\n",
    "@tf.function    \n",
    "def costfunc():    \n",
    "    cost = tf.reduce_sum(tf.square(Y - modelout())) \n",
    "    return cost\n",
    "\n",
    "@tf.function\n",
    "def train(epoch):  \n",
    "    for _ in tf.range(epoch):  \n",
    "        optimizer.minimize(costfunc, [A, B]) # 最小值\n",
    "    return costfunc()\n",
    "\n",
    "cost = train(20000)\n",
    "y_modelout =  modelout()   \n",
    "print('最终的最小二乘误差cost={}'.format(cost.numpy()), '此时A={},'.format(A.numpy()), 'B={}'.format(B.numpy())) \n",
    "print('模型输出值：{}'.format(y_modelout.numpy()))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "TensorFlow2",
   "language": "python",
   "name": "tf2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}