Course Content

Introduction to TensorFlow

1. Tensors

Welcome to TensorFlow Introduction to Tensors Tensor Properties Applications of Tensors Batches Creating Tensors Data Types Basic Operations: Arithmetic Basic Operations: Linear Algebra Challenge: Creating a Neural Network Layer Transformations Reduction Operations Quiz: TensorFlow Basics

2. Basics of TensorFlow

Gradient Tape Graph Execution Neural Network Implementation Quiz: Conclusion Summary

Neural Network Implementation

Basic Neural Network Overview

You've now reached a stage where you're equipped with the essential knowledge of TensorFlow to create neural networks on your own. While most real-world neural networks are complex and typically built using high-level libraries like Keras, we'll construct a basic one using fundamental TensorFlow tools. This approach gives us hands-on experience with low-level tensor manipulation, helping us understand the underlying processes.

In earlier courses like Introduction to Neural Networks, you might recall the time and effort it took to build even a simple neural network, treating each neuron individually.

TensorFlow simplifies this process significantly. By leveraging tensors, we can encapsulate complex calculations, reducing the need for intricate coding. Our primary task is to set up a sequential pipeline of tensor operations.

Here's a brief refresher on the steps to get a neural network training process up and running:

Data Preparation and Model Creation

The initial phase of training a neural network involves preparing the data, encompassing both the inputs and outputs that the network will learn from. Additionally, we will establish the model's hyperparameters - these are the parameters that remain constant throughout the training process. We also initialize the weights, typically drawn from a normal distribution, and the biases, which are often set to zero.

Forward Propagation

In forward propagation, each layer of the network typically follows these steps:

Multiply the layer's input by its weights;
Add a bias to the result;
Apply an activation function to this sum.

Then, we can calculate the loss.

Backward Propagation

The next step is backward propagation, where we adjust the weights and biases based on their influence on the loss. This influence is represented by the gradient, which TensorFlow's Gradient Tape calculates automatically. We update the weights and biases by subtracting the gradient, scaled by the learning rate.

Training Loop

To effectively train the neural network, we repeat the training steps multiple times while tracking the model's performance. Ideally, we should see the loss decrease over epochs.

Task

Swipe to show code editor

Create a neural network designed to predict XOR operation outcomes. The network should consist of 2 input neurons, a hidden layer with 2 neurons, and 1 output neuron.

Start by setting up the initial weights and biases. The weights should be initialized using a normal distribution, and biases should all be initialized to zero. Use the hyperparameters input_size, hidden_size, and output_size to define the appropriate shapes for these tensors.
Utilize a function decorator to transform the train_step() function into a TensorFlow graph.
Carry out forward propagation through both the hidden and output layers of the network. Use sigmoid activation function.
Determine the gradients to understand how each weight and bias impacts the loss. Ensure the gradients are computed in the correct order, corresponding to the output variable names.
Modify the weights and biases based on their respective gradients. Incorporate the learning_rate in this adjustment process to control the extent of each update.

Conclusion

Since the XOR function is a relatively straightforward task, we don't need advanced techniques like hyperparameter tuning, dataset splitting, or building complex data pipelines at this stage. This exercise is just a step towards building more sophisticated neural networks for real-world applications.

Mastering these basics is crucial before diving into advanced neural network construction techniques in upcoming courses, where we'll use the Keras library and explore methods to enhance model quality with TensorFlow's rich features.

Switch to desktop for real-world practiceContinue from where you are using one of the options below

Everything was clear?

Thanks for your feedback!

Section 2. Chapter 3

Neural Network Implementation

Basic Neural Network Overview

In earlier courses like Introduction to Neural Networks, you might recall the time and effort it took to build even a simple neural network, treating each neuron individually.

Here's a brief refresher on the steps to get a neural network training process up and running:

Data Preparation and Model Creation

Forward Propagation

In forward propagation, each layer of the network typically follows these steps:

Multiply the layer's input by its weights;
Add a bias to the result;
Apply an activation function to this sum.

Then, we can calculate the loss.

Backward Propagation

Training Loop

To effectively train the neural network, we repeat the training steps multiple times while tracking the model's performance. Ideally, we should see the loss decrease over epochs.

Task

Swipe to show code editor

Create a neural network designed to predict XOR operation outcomes. The network should consist of 2 input neurons, a hidden layer with 2 neurons, and 1 output neuron.

Start by setting up the initial weights and biases. The weights should be initialized using a normal distribution, and biases should all be initialized to zero. Use the hyperparameters input_size, hidden_size, and output_size to define the appropriate shapes for these tensors.
Utilize a function decorator to transform the train_step() function into a TensorFlow graph.
Carry out forward propagation through both the hidden and output layers of the network. Use sigmoid activation function.
Determine the gradients to understand how each weight and bias impacts the loss. Ensure the gradients are computed in the correct order, corresponding to the output variable names.
Modify the weights and biases based on their respective gradients. Incorporate the learning_rate in this adjustment process to control the extent of each update.

Conclusion

Switch to desktop for real-world practiceContinue from where you are using one of the options below

Everything was clear?

Thanks for your feedback!

Section 2. Chapter 3

Switch to desktop for real-world practiceContinue from where you are using one of the options below