吴恩达 第三周第四周编程作业

作业"/>

吴恩达 第三周第四周编程作业

前言

第三周目的是建成浅层神经网络，给的用例是点集数据分类
第四周目的是建成深层神经网络，给的用例同第一周的识别猫
主要参考
《深度学习——Andrew Ng》第一课第三周编程作业
【中文】【吴恩达课后编程作业】Course 1 - 神经网络和深度学习 - 第三周作业
《深度学习——Andrew Ng》第一课第四周编程作业
【中文】【吴恩达课后编程作业】Course 1 - 神经网络和深度学习 - 第四周作业(1&2)

由于第三周代码是第四周代码的基础，比较简单，于是详写第四周，略写第三周
写的内容包括文件结构图、函数列表、 参数列表、代码

文章目录

• 前言
• 第三周代码
• • 文件结构
  • 代码
  • • network . py 神经网络
    • runwork . py 跑数据
    • planar_utils . py 提供部分功能
    • testCases . py 提供测试示例
• 第四周编程作业
• • 文件结构
  • 代码
  • • work_space . py 跑数据
    • network_model . py 神经网络模型
    • forward_model . py 前向传播模型
    • backward_model . py 反向传播模型
    • dnn_utils . py 激活函数相关
    • lr_utils . py 数据集
  • 规律总结

第三周代码

文件结构

代码

network . py 神经网络

from datasets.testCases import *
from datasets.planar_utils import sigmoid# GRADED FUNCTION: layer_sizes
def layer_sizes(X, Y):"""Arguments:X -- input dataset of shape (input size, number of examples)Y -- labels of shape (output size, number of examples)Returns:n_x -- the size of the input layern_h -- the size of the hidden layern_y -- the size of the output layer"""n_x = X.shape[0]n_h = 4n_y = Y.shape[0]return n_x, n_h, n_y# GRADED FUNCTION: initialize_parameters
def initialize_parameters(n_x, n_h, n_y):"""Argument:n_x -- size of the input layern_h -- size of the hidden layern_y -- size of the output layerReturns:params -- python dictionary containing your parameters:W1 -- weight matrix of shape (n_h, n_x)b1 -- bias vector of shape (n_h, 1)W2 -- weight matrix of shape (n_y, n_h)b2 -- bias vector of shape (n_y, 1)"""np.random.seed(2)  # 设置种子，令random随机数相同W1 = np.random.randn(n_h, n_x) * 0.01  # W的行确定该层神经元节点个数，列匹配上一层神经元节点个数b1 = np.zeros((n_h, 1))  # zeros(shape, dtype=float, order='C')，shape = (n_h, 1)，需要加括号，否则1会填充给dtypeW2 = np.random.randn(n_y, n_h) * 0.01b2 = np.zeros((n_y, 1))assert (W1.shape == (n_h, n_x))assert (b1.shape == (n_h, 1))assert (W2.shape == (n_y, n_h))assert (b2.shape == (n_y, 1))parameters = {"W1": W1,"b1": b1,"W2": W2,"b2": b2}return parameters# GRADED FUNCTION: forward_propagation
def forward_propagation(X, parameters):"""Argument:X -- input data of size (n_x, m)parameters -- python dictionary containing your parameters (output of initialization function)Returns:A2 -- The sigmoid output of the second activationcache -- a dictionary containing "Z1", "A1", "Z2" and "A2""""W1 = parameters["W1"]b1 = parameters["b1"]W2 = parameters["W2"]b2 = parameters["b2"]Z1 = np.dot(W1, X) + b1A1 = np.tanh(Z1)Z2 = np.dot(W2, A1) + b2A2 = sigmoid(Z2)assert(A2.shape == (1, X.shape[1]))cache = {"Z1": Z1,"A1": A1,"Z2": Z2,"A2": A2}return A2, cache# GRADED FUNCTION: compute_cost
def compute_cost(A2, Y, parameters):"""Computes the cross-entropy cost given in equation (13)Arguments:A2 -- The sigmoid output of the second activation, of shape (1, number of examples)Y -- "true" labels vector of shape (1, number of examples)parameters -- python dictionary containing your parameters W1, b1, W2 and b2Returns:cost -- cross-entropy cost given equation (13)"""m = Y.shape[1]  # number of exampleloss = - (np.dot(np.log(A2), Y.T) + np.dot(np.log(1 - A2), (1 - Y).T))cost = loss / mcost = float(np.squeeze(cost))     # makes sure cost is the dimension we expect.assert(isinstance(cost, float))return cost# GRADED FUNCTION: backward_propagation
def backward_propagation(parameters, cache, X, Y):"""Implement the backward propagation using the instructions above.Arguments:parameters -- python dictionary containing our parameterscache -- a dictionary containing "Z1", "A1", "Z2" and "A2".X -- input data of shape (2, number of examples)Y -- "true" labels vector of shape (1, number of examples)Returns:grads -- python dictionary containing your gradients with respect to different parameters"""m = X.shape[1]W1 = parameters["W1"]W2 = parameters["W2"]A1 = cache["A1"]A2 = cache["A2"]dZ2 = A2 - YdW2 = np.dot(dZ2, A1.T) / mdb2 = np.sum(dZ2, axis=1, keepdims=True) / mdZ1 = np.multiply(np.dot(W2.T, dZ2), 1 - np.power(A1, 2))dW1 = np.dot(dZ1, X.T) / mdb1 = np.sum(dZ1, axis=1, keepdims=True) / mgrads = {"dW1": dW1,"db1": db1,"dW2": dW2,"db2": db2}return grads# GRADED FUNCTION: update_parameters
def update_parameters(parameters, grads, learning_rate=1.5):"""Updates parameters using the gradient descent update rule given aboveArguments:parameters -- python dictionary containing your parametersgrads -- python dictionary containing your gradientsReturns:parameters -- python dictionary containing your updated parameters"""W1 = parameters["W1"]b1 = parameters["b1"]W2 = parameters["W2"]b2 = parameters["b2"]dW1 = grads["dW1"]db1 = grads["db1"]dW2 = grads["dW2"]db2 = grads["db2"]W1 -= learning_rate * dW1b1 -= learning_rate * db1W2 -= learning_rate * dW2b2 -= learning_rate * db2parameters = {"W1": W1,"b1": b1,"W2": W2,"b2": b2}return parameters# GRADED FUNCTION: nn_model
def nn_model(X, Y, n_h, num_iterations=10000, print_cost=False):"""Arguments:X -- dataset of shape (2, number of examples)Y -- labels of shape (1, number of examples)n_h -- size of the hidden layernum_iterations -- Number of iterations in gradient descent loopprint_cost -- if True, print the cost every 1000 iterationsReturns:parameters -- parameters learnt by the model. They can then be used to predict."""np.random.seed(3)n_x = layer_sizes(X, Y)[0]n_y = layer_sizes(X, Y)[2]parameters = initialize_parameters(n_x, n_h, n_y)W1 = parameters["W1"]b1 = parameters["b1"]W2 = parameters["W2"]b2 = parameters["b2"]# Loop (gradient descent)for i in range(num_iterations):A2, cache = forward_propagation(X, parameters)   # Forward propagation.cost = compute_cost(A2, Y, parameters)  # Cost function.grads = backward_propagation(parameters, cache, X, Y)  # Backpropagation.parameters = update_parameters(parameters, grads)  # Gradient descent parameter update.# Print the cost every 1000 iterationsif print_cost and i % 1000 == 0:print("Cost after iteration %i: %f" % (i, cost))return parameters# GRADED FUNCTION: predict
def predict(parameters, X):"""Using the learned parameters, predicts a class for each example in XArguments:parameters -- python dictionary containing your parametersX -- input data of size (n_x, m)Returnspredictions -- vector of predictions of our model (red: 0 / blue: 1)"""A2, cache = forward_propagation(X, parameters)predictions = np.round(A2)  # np.round()对浮点数四舍五入，取0-1预测值return predictions

runwork . py 跑数据

import numpy as np
import matplotlib.pyplot as plt
from network import nn_model, predict
from datasets.planar_utils import plot_decision_boundary, load_planar_datasetX, Y = load_planar_dataset()
"""或者测试不同的数据集拿到选中测试集的X, Ynoisy_circles, noisy_moons, blobs, gaussian_quantiles, no_structure = load_extra_datasets()datasets = {"noisy_circles": noisy_circles,"noisy_moons": noisy_moons,"blobs": blobs,"gaussian_quantiles": gaussian_quantiles}选择测试集dataset = "blobs"X, Y = datasets[dataset]X, Y = X.T, Y.reshape(1, Y.shape[0])# make blobs binaryif dataset == "blobs":Y = Y % 2# Visualize the dataplt.scatter(X[0, :], X[1, :], c=np.squeeze(Y), s=40, cmap=plt.cm.Spectral)plt.title("Decision Boundary for hidden layer size " + str(4))"""
parameters = nn_model(X, Y, n_h=4, num_iterations=10000, print_cost=True)#绘制边界
plot_decision_boundary(lambda x: predict(parameters, x.T), X, Y)
plt.title("Decision Boundary for hidden layer size " + str(4))predictions = predict(parameters, X)
print('准确率: %d' % float((np.dot(Y, predictions.T) + np.dot(1 - Y, 1 - predictions.T)) / float(Y.size) * 100) + '%')

planar_utils . py 提供部分功能

import matplotlib.pyplot as plt
import numpy as np
import sklearn
import sklearn.datasets
import sklearn.linear_modeldef plot_decision_boundary(model, X, y):# Set min and max values and give it some paddingx_min, x_max = X[0, :].min() - 1, X[0, :].max() + 1y_min, y_max = X[1, :].min() - 1, X[1, :].max() + 1h = 0.01# Generate a grid of points with distance h between themxx, yy = np.meshgrid(np.arange(x_min, x_max, h), np.arange(y_min, y_max, h))# Predict the function value for the whole gridZ = model(np.c_[xx.ravel(), yy.ravel()])Z = Z.reshape(xx.shape)# Plot the contour and training examplesplt.contourf(xx, yy, Z, cmap=plt.cm.Spectral)plt.ylabel('x2')plt.xlabel('x1')plt.scatter(X[0, :], X[1, :], c=np.squeeze(y), cmap=plt.cm.Spectral)def sigmoid(x):s = 1/(1+np.exp(-x))return sdef load_planar_dataset():np.random.seed(1)m = 400  # number of examplesN = int(m/2)  # number of points per classD = 2  # dimensionalityX = np.zeros((m, D))  # data matrix where each row is a single exampleY = np.zeros((m, 1), dtype='uint8')  # labels vector (0 for red, 1 for blue)a = 4  # maximum ray of the flowerfor j in range(2):ix = range(N*j, N*(j+1))t = np.linspace(j*3.12,(j+1)*3.12, N) + np.random.randn(N)*0.2  # thetar = a*np.sin(4*t) + np.random.randn(N)*0.2  # radiusX[ix] = np.c_[r*np.sin(t), r*np.cos(t)]Y[ix] = jX = X.TY = Y.Treturn X, Ydef load_extra_datasets():  N = 200noisy_circles = sklearn.datasets.make_circles(n_samples=N, factor=.5, noise=.3)noisy_moons = sklearn.datasets.make_moons(n_samples=N, noise=.2)blobs = sklearn.datasets.make_blobs(n_samples=N, random_state=5, n_features=2, centers=6)gaussian_quantiles = sklearn.datasets.make_gaussian_quantiles(mean=None, cov=0.5, n_samples=N, n_features=2,n_classes=2, shuffle=True, random_state=None)no_structure = np.random.rand(N, 2), np.random.rand(N, 2)return noisy_circles, noisy_moons, blobs, gaussian_quantiles, no_structure

testCases . py 提供测试示例

import numpy as npdef layer_sizes_test_case():np.random.seed(1)X_assess = np.random.randn(5, 3)Y_assess = np.random.randn(2, 3)return X_assess, Y_assessdef initialize_parameters_test_case():n_x, n_h, n_y = 2, 4, 1return n_x, n_h, n_ydef forward_propagation_test_case():np.random.seed(1)X_assess = np.random.randn(2, 3)parameters = {'W1': np.array([[-0.00416758, -0.00056267],[-0.02136196,  0.01640271],[-0.01793436, -0.00841747],[ 0.00502881, -0.01245288]]),'W2': np.array([[-0.01057952, -0.00909008,  0.00551454,  0.02292208]]),'b1': np.array([[ 0.],[ 0.],[ 0.],[ 0.]]),'b2': np.array([[ 0.]])}return X_assess, parametersdef compute_cost_test_case():np.random.seed(1)Y_assess = np.random.randn(1, 3)parameters = {'W1': np.array([[-0.00416758, -0.00056267],[-0.02136196,  0.01640271],[-0.01793436, -0.00841747],[ 0.00502881, -0.01245288]]),'W2': np.array([[-0.01057952, -0.00909008,  0.00551454,  0.02292208]]),'b1': np.array([[ 0.],[ 0.],[ 0.],[ 0.]]),'b2': np.array([[ 0.]])}a2 = (np.array([[ 0.5002307 ,  0.49985831,  0.50023963]]))return a2, Y_assess, parametersdef backward_propagation_test_case():np.random.seed(1)X_assess = np.random.randn(2, 3)Y_assess = np.random.randn(1, 3)parameters = {'W1': np.array([[-0.00416758, -0.00056267],[-0.02136196,  0.01640271],[-0.01793436, -0.00841747],[ 0.00502881, -0.01245288]]),'W2': np.array([[-0.01057952, -0.00909008,  0.00551454,  0.02292208]]),'b1': np.array([[ 0.],[ 0.],[ 0.],[ 0.]]),'b2': np.array([[ 0.]])}cache = {'A1': np.array([[-0.00616578,  0.0020626 ,  0.00349619],[-0.05225116,  0.02725659, -0.02646251],[-0.02009721,  0.0036869 ,  0.02883756],[ 0.02152675, -0.01385234,  0.02599885]]),'A2': np.array([[ 0.5002307 ,  0.49985831,  0.50023963]]),'Z1': np.array([[-0.00616586,  0.0020626 ,  0.0034962 ],[-0.05229879,  0.02726335, -0.02646869],[-0.02009991,  0.00368692,  0.02884556],[ 0.02153007, -0.01385322,  0.02600471]]),'Z2': np.array([[ 0.00092281, -0.00056678,  0.00095853]])}return parameters, cache, X_assess, Y_assessdef update_parameters_test_case():parameters = {'W1': np.array([[-0.00615039,  0.0169021 ],[-0.02311792,  0.03137121],[-0.0169217 , -0.01752545],[ 0.00935436, -0.05018221]]),'W2': np.array([[-0.0104319 , -0.04019007,  0.01607211,  0.04440255]]),'b1': np.array([[ -8.97523455e-07],[  8.15562092e-06],[  6.04810633e-07],[ -2.54560700e-06]]),'b2': np.array([[  9.14954378e-05]])}grads = {'dW1': np.array([[ 0.00023322, -0.00205423],[ 0.00082222, -0.00700776],[-0.00031831,  0.0028636 ],[-0.00092857,  0.00809933]]),'dW2': np.array([[ -1.75740039e-05,   3.70231337e-03,  -1.25683095e-03,-2.55715317e-03]]),'db1': np.array([[  1.05570087e-07],[ -3.81814487e-06],[ -1.90155145e-07],[  5.46467802e-07]]),'db2': np.array([[ -1.08923140e-05]])}return parameters, gradsdef nn_model_test_case():np.random.seed(1)X_assess = np.random.randn(2, 3)Y_assess = np.random.randn(1, 3)return X_assess, Y_assessdef predict_test_case():np.random.seed(1)X_assess = np.random.randn(2, 3)parameters = {'W1': np.array([[-0.00615039,  0.0169021 ],[-0.02311792,  0.03137121],[-0.0169217 , -0.01752545],[ 0.00935436, -0.05018221]]),'W2': np.array([[-0.0104319 , -0.04019007,  0.01607211,  0.04440255]]),'b1': np.array([[ -8.97523455e-07],[  8.15562092e-06],[  6.04810633e-07],[ -2.54560700e-06]]),'b2': np.array([[  9.14954378e-05]])}return parameters, X_assess

第四周编程作业

文件结构

代码

work_space . py 跑数据

import numpy as np
from network_model import L_layer_model, L_model_forward
from lr_utils import load_datasetdef predict(X, y, parameters):m = X.shape[1]L = len(parameters) // 2p = np.zeros((1, m))probas, caches = L_model_forward(X, parameters)for i in range(0, probas.shape[1]):if probas[0,i] > 0.5:p[0,i] = 1else:p[0,i] = 0print("准确度为: "+ str(float(np.sum((p == y))/m)))return ptrain_set_x_orig, train_set_y, test_set_x_orig, test_set_y, classes = load_dataset()train_x_flatten = train_set_x_orig.reshape(train_set_x_orig.shape[0], -1).T
test_x_flatten = test_set_x_orig.reshape(test_set_x_orig.shape[0], -1).Ttrain_x = train_x_flatten / 255
train_y = train_set_y
test_x = test_x_flatten / 255
test_y = test_set_ylayers_dims = [12288, 20, 7, 5, 1]
parameters = L_layer_model(train_x, train_y, layers_dims, num_iterations=2500, print_cost=True)predictions_train = predict(train_x, train_y, parameters)  # 训练集
predictions_test = predict(test_x, test_y, parameters)  # 测试集

network_model . py 神经网络模型

import numpy as np
import matplotlib.pyplot as plt
from forward_model import L_model_forward
from backward_model import L_model_backwarddef initialize_parameters_deep(layers_dims):np.random.seed(3)parameters = {}  # W[L]与b[L]的字典L = len(layers_dims)  # 神经网络层数for l in range(1, L):parameters['W' + str(l)] = np.random.randn(layers_dims[l], layers_dims[l - 1]) / np.sqrt(layers_dims[l - 1])  # 使用 “/ np.sqrt(layer_dims[l-1])”避免梯度爆炸和消失parameters['b' + str(l)] = np.zeros((layers_dims[l], 1))assert (parameters['W' + str(l)].shape == (layers_dims[l], layers_dims[l - 1]))assert (parameters['b' + str(l)].shape == (layers_dims[l], 1))return parametersdef compute_cost(AL, Y):m = Y.shape[1]cost = -np.sum(np.multiply(np.log(AL),Y) + np.multiply(np.log(1 - AL), 1 - Y)) / mcost = np.squeeze(cost)assert(cost.shape == ())return costdef update_parameters(parameters, grads, learning_rate):L = len(parameters) // 2for l in range(L):parameters["W" + str(l + 1)] = parameters["W" + str(l + 1)] - learning_rate * grads["dW" + str(l + 1)]parameters["b" + str(l + 1)] = parameters["b" + str(l + 1)] - learning_rate * grads["db" + str(l + 1)]return parametersdef L_layer_model(X, Y, layers_dims, learning_rate=0.0075, num_iterations=3000, print_cost=False):np.random.seed(1)costs = []acc_dev = []parameters = initialize_parameters_deep(layers_dims)for i in range(0, num_iterations):AL, caches = L_model_forward(X, parameters)cost = compute_cost(AL, Y)grads = L_model_backward(AL, Y, caches)parameters = update_parameters(parameters, grads, learning_rate)if i % 100 == 0:# 记录成本costs.append(cost)# 是否打印成本值if print_cost:print("第", i, "次迭代，成本值为：", np.squeeze(cost))plt.plot(np.squeeze(costs))plt.plot(np.squeeze(acc_dev))plt.legend(["costs", "acc_dev"], loc=0)plt.ylabel('cost')plt.xlabel('iterations (per tens)')plt.title("Learning rate =" + str(learning_rate))plt.show()return parameters

forward_model . py 前向传播模型

import numpy as np
from dnn_utils import sigmoid, reludef linear_forward(A_prev, W, b):Z = np.dot(W, A_prev) + bassert (Z.shape == (W.shape[0], A_prev.shape[1]))cache = (A_prev, W, b)return Z, cachedef linear_activation_forward(A_prev,  W, b, activation):# linear_cache = (A_prev, W, b)# activation_cache = ZZ, linear_cache = linear_forward(A_prev, W, b)if activation == "sigmoid":A, activation_cache = sigmoid(Z)elif activation == "relu":A, activation_cache = relu(Z)assert (A.shape == (W.shape[0], A_prev.shape[1]))cache = (linear_cache, activation_cache)return A, cachedef L_model_forward(X, parameters):caches = []A = X  # A[0] == XL = len(parameters) // 2for l in range(1, L):A_prev = AA, cache = linear_activation_forward(A_prev,  parameters['W' + str(l)], parameters['b'+ str(l)], "relu")caches.append(cache)AL, cache = linear_activation_forward(A, parameters['W' + str(L)], parameters['b' + str(L)], "sigmoid")caches.append(cache)assert(AL.shape == (1, X.shape[1]))return AL, caches

backward_model . py 反向传播模型

import numpy as np
from dnn_utils import sigmoid_backward, relu_backwarddef linear_backward(dZ, linear_cache):A_prev, W, b = linear_cachem = A_prev.shape[1]dW = np.dot(dZ, A_prev.T) / mdb = np.sum(dZ, axis=1, keepdims=True) / mdA_prev = np.dot(W.T, dZ)assert (dA_prev.shape == A_prev.shape)assert (dW.shape == W.shape)assert (db.shape == b.shape)return dA_prev, dW, dbdef linear_activation_backward(dA, cache, activation):linear_cache, activation_cache = cacheif activation == "sigmoid":dZ = sigmoid_backward(dA, activation_cache)elif activation == "relu":dZ = relu_backward(dA, activation_cache)dA_prev, dW, db = linear_backward(dZ, linear_cache)return dA_prev, dW, dbdef L_model_backward(AL, Y, caches):grads = {}L = len(caches)m = AL.shape[1]Y = Y.reshape(AL.shape)dAL = - (np.divide(Y, AL) - np.divide(1 - Y, 1 - AL))current_cache = caches[L - 1]  # caches从0开始计数grads["dA" + str(L)], grads["dW" + str(L)], grads["db" + str(L)] = linear_activation_backward(dAL, current_cache, "sigmoid")for l in reversed(range(L - 1)):  # reversed()反转遍历current_cache = caches[l]grads["dA" + str(l + 1)], grads["dW" + str(l + 1)], grads["db" + str(l + 1)] = linear_activation_backward(grads["dA" + str(l + 2)], current_cache, "relu")return grads

dnn_utils . py 激活函数相关

import numpy as npdef sigmoid(Z):activation_cache = ZA = 1/(1+np.exp(-Z))return A, activation_cachedef sigmoid_backward(dA, activation_cache):Z = activation_caches = 1/(1+np.exp(-Z))dZ = dA * s * (1-s)assert (dZ.shape == Z.shape)return dZdef relu(Z):activation_cache = ZA = np.maximum(0, Z)assert(A.shape == Z.shape)return A, activation_cachedef relu_backward(dA, activation_cache):Z = activation_cachedZ = np.array(dA, copy=True)dZ[Z <= 0] = 0assert (dZ.shape == Z.shape)return dZ

lr_utils . py 数据集

import numpy as np
import h5pydef load_dataset():train_dataset = h5py.File('datasets/train_catvnoncat.h5', "r")train_set_x_orig = np.array(train_dataset["train_set_x"][:]) # your train set featurestrain_set_y_orig = np.array(train_dataset["train_set_y"][:]) # your train set labelstest_dataset = h5py.File('datasets/test_catvnoncat.h5', "r")test_set_x_orig = np.array(test_dataset["test_set_x"][:]) # your test set featurestest_set_y_orig = np.array(test_dataset["test_set_y"][:]) # your test set labelsclasses = np.array(test_dataset["list_classes"][:]) # the list of classestrain_set_y_orig = train_set_y_orig.reshape((1, train_set_y_orig.shape[0]))test_set_y_orig = test_set_y_orig.reshape((1, test_set_y_orig.shape[0]))return train_set_x_orig, train_set_y_orig, test_set_x_orig, test_set_y_orig, classes

规律总结

• 两个字典parameters和grads，分别记录每层正向传播和反向传播记录下来的W,b参数值和对应成本梯度，用于梯度下降

• 正向传播相关的返回值常常是 A/Z, cache

  • cache = (linear_cache, activation_cache) 均用于反向传播的计算
    • linear_cache = (A_prev, W, b) 上一层激活值，本层参数W, b
    • activation_cache = Z 本层激活函数的输入值
  • 而每层的激活值不在字典中，因为要在下一层中通过循环开头的A_prev = A来作为A_prev存储在下一层中

• 反向传播相关的返回值常常是dA_prev, dW, db或dZ