Course Content

Introduction to TensorFlow

1. Tensors

Welcome to TensorFlow Introduction to Tensors Tensor Properties Applications of Tensors Batches Creating Tensors Data Types Basic Operations: Arithmetic Basic Operations: Linear Algebra Challenge: Creating a Neural Network Layer Transformations Reduction Operations Quiz: TensorFlow Basics

2. Basics of TensorFlow

Gradient Tape Graph Execution Neural Network Implementation Quiz: Conclusion Summary

Basic Operations: Arithmetic

Arithmetic Operations

TensorFlow provides numerous arithmetic operations for tensor manipulations. These operations and many others in TensorFlow support broadcasting, making it easier to perform element-wise operations on tensors of different shapes.

Addition

For tensors addition we can use tf.add(), .assign_add() methods and a plus + sign. Also we can use broadcasting with the plus sign + or with the tf.add() method.

Broadcasting makes it possible to carry out element-wise operations on tensors of different, but compatible, shapes by virtually expanding the smaller tensor to match the shape of the larger tensor.


              12345678910111213141516171819202122232425
            
import tensorflow as tf

# Create two tensors
a = tf.Variable([1, 2, 3])
b = tf.constant([4, 5, 6])

# Perform element-wise addition with TF method
c1 = tf.add(a, b)  

# Same as `c1` calculation, but shorter
c2 = a + b  

# Using broadcasting; 
# Same as `[1, 2, 3] + [3, 3, 3]`
c3 = a + 3  

# The most efficient one; 
# Changes the object inplace without creating a new one;
# Result is the same as for `c1` and `c2`.
a.assign_add(b)

print('TF method:\t', c1)
print('Plus sign:\t', c2)
print('Broadcasting:\t', c3)
print('Inplace change:\t', a)

Note

For the inplace method, the fundamental element must be a mutable Variable type rather than a constant.

Subtraction

We have analogues of all methods for subtraction as for addition:

tf.add() changes into tf.subtract();
Plus sign + changes into minus sign -;
.assign_add() changes into .assign_sub().


              123456789101112131415161718192021
            
import tensorflow as tf

# Create two tensors
a = tf.Variable([4, 5, 6])
b = tf.constant([1, 2, 3])

# Perform element-wise substraction
c1 = tf.subtract(a, b)  
c2 = a - b 

# Using broadcasting; 
# Same as `[4, 5, 6] - [3, 3, 3]`
c3 = a - 3  

# Inplace substraction
a.assign_sub(b)

print('TF method:\t', c1)
print('Minus sign:\t', c2)
print('Broadcasting:\t', c3)
print('Inplace change:\t', a)

Multiplication (Element-wise)

For multiplication, there isn't an inplace method since matrix multiplication inherently results in a new object. However, other operations have their counterparts:

tf.add() corresponds to tf.multiply();
The plus sign + corresponds to the asterisk sign *.


              1234567891011121314151617
            
import tensorflow as tf

# Create two tensors
a = tf.constant([1, 2, 3])
b = tf.constant([4, 5, 6])

# Perform element-wise multiplication
c1 = tf.multiply(a, b)  
c2 = a * b 

# Using broadcasting; 
# Same as `[1, 2, 3] * [3, 3, 3]`
c3 = a * 3  

print('TF method:\t', c1)
print('Asterisk sign:\t', c2)
print('Broadcasting:\t', c3)

Division

Similar to multiplication, but with tf.divide() and / sign.


              1234567891011121314151617
            
import tensorflow as tf

# Create two tensors
a = tf.constant([6, 8, 10])
b = tf.constant([2, 4, 5])

# Perform element-wise division
c1 = tf.divide(a, b)  
c2 = a / b 

# Using broadcasting; 
# Same as `[6, 8, 10] / [2, 2, 2]`
c3 = a / 2  

print('TF method:\t', c1)
print('Asterisk sign:\t', c2)
print('Broadcasting:\t', c3)

Broadcasting

Broadcasting is the term used to describe how tensors with different shapes are automatically and implicitly treated during arithmetic operations so that they appear as if they have the same shape. It allows operations to be performed on tensors of different sizes without explicitly resizing them first.

Task

Swipe to show code editor

Given a set of matrices, perform the following operations:

Inplace addition of a 2x2 matrix.
Subtraction using tf.subtract() method for a 2x3 matrix.
Broadcasting multiplication of a 3x2 matrix with another 1x2 matrix.
Broadcasting division between two matrices, one of size 2x3 and the other 2x1.

Note

Note the broadcasting behavior in the multiplication and division operations. In multiplication, it's like multiplying [[1, 2], [3, 4], [5, 6]] with the [[2, 4], [2, 4], [2, 4]]. In division, it's like dividing [[2, 4, 6], [4, 8, 12]] by [[2, 2, 2], [4, 4, 4]].

In the first case, broadcasting expands matrix along the 0th axis (first parameter of shape), while in the second case, matrix is expanded along the 1st axis (second parameter of shape). It depends on the shape of the matrices.

Switch to desktop for real-world practiceContinue from where you are using one of the options below

Everything was clear?

Thanks for your feedback!

Section 1. Chapter 8

Basic Operations: Arithmetic

Arithmetic Operations

Addition

For tensors addition we can use tf.add(), .assign_add() methods and a plus + sign. Also we can use broadcasting with the plus sign + or with the tf.add() method.


              12345678910111213141516171819202122232425
            
import tensorflow as tf

# Create two tensors
a = tf.Variable([1, 2, 3])
b = tf.constant([4, 5, 6])

# Perform element-wise addition with TF method
c1 = tf.add(a, b)  

# Same as `c1` calculation, but shorter
c2 = a + b  

# Using broadcasting; 
# Same as `[1, 2, 3] + [3, 3, 3]`
c3 = a + 3  

# The most efficient one; 
# Changes the object inplace without creating a new one;
# Result is the same as for `c1` and `c2`.
a.assign_add(b)

print('TF method:\t', c1)
print('Plus sign:\t', c2)
print('Broadcasting:\t', c3)
print('Inplace change:\t', a)

Note

For the inplace method, the fundamental element must be a mutable Variable type rather than a constant.

Subtraction

We have analogues of all methods for subtraction as for addition:

tf.add() changes into tf.subtract();
Plus sign + changes into minus sign -;
.assign_add() changes into .assign_sub().


              123456789101112131415161718192021
            
import tensorflow as tf

# Create two tensors
a = tf.Variable([4, 5, 6])
b = tf.constant([1, 2, 3])

# Perform element-wise substraction
c1 = tf.subtract(a, b)  
c2 = a - b 

# Using broadcasting; 
# Same as `[4, 5, 6] - [3, 3, 3]`
c3 = a - 3  

# Inplace substraction
a.assign_sub(b)

print('TF method:\t', c1)
print('Minus sign:\t', c2)
print('Broadcasting:\t', c3)
print('Inplace change:\t', a)

Multiplication (Element-wise)

For multiplication, there isn't an inplace method since matrix multiplication inherently results in a new object. However, other operations have their counterparts:

tf.add() corresponds to tf.multiply();
The plus sign + corresponds to the asterisk sign *.


              1234567891011121314151617
            
import tensorflow as tf

# Create two tensors
a = tf.constant([1, 2, 3])
b = tf.constant([4, 5, 6])

# Perform element-wise multiplication
c1 = tf.multiply(a, b)  
c2 = a * b 

# Using broadcasting; 
# Same as `[1, 2, 3] * [3, 3, 3]`
c3 = a * 3  

print('TF method:\t', c1)
print('Asterisk sign:\t', c2)
print('Broadcasting:\t', c3)

Division

Similar to multiplication, but with tf.divide() and / sign.


              1234567891011121314151617
            
import tensorflow as tf

# Create two tensors
a = tf.constant([6, 8, 10])
b = tf.constant([2, 4, 5])

# Perform element-wise division
c1 = tf.divide(a, b)  
c2 = a / b 

# Using broadcasting; 
# Same as `[6, 8, 10] / [2, 2, 2]`
c3 = a / 2  

print('TF method:\t', c1)
print('Asterisk sign:\t', c2)
print('Broadcasting:\t', c3)

Broadcasting

Task

Swipe to show code editor

Given a set of matrices, perform the following operations:

Inplace addition of a 2x2 matrix.
Subtraction using tf.subtract() method for a 2x3 matrix.
Broadcasting multiplication of a 3x2 matrix with another 1x2 matrix.
Broadcasting division between two matrices, one of size 2x3 and the other 2x1.

Note

Note the broadcasting behavior in the multiplication and division operations. In multiplication, it's like multiplying [[1, 2], [3, 4], [5, 6]] with the [[2, 4], [2, 4], [2, 4]]. In division, it's like dividing [[2, 4, 6], [4, 8, 12]] by [[2, 2, 2], [4, 4, 4]].

In the first case, broadcasting expands matrix along the 0th axis (first parameter of shape), while in the second case, matrix is expanded along the 1st axis (second parameter of shape). It depends on the shape of the matrices.

Switch to desktop for real-world practiceContinue from where you are using one of the options below

Everything was clear?

Thanks for your feedback!

Section 1. Chapter 8

Switch to desktop for real-world practiceContinue from where you are using one of the options below