Vectors

Import

First we shall import the NumPy package.

import numpy as np

Python Lists

Consider the following vectors:

\[\begin{split} \mathbf{x} = \begin{bmatrix} 1\\ 2\\ 3 \end{bmatrix}, \mathbf{y} = \begin{bmatrix} 4\\ 5\\ 6 \end{bmatrix} \end{split}\]

We can add these two vectors:

\[\begin{split} \mathbf{z} = \mathbf{x} + \mathbf{y}= \begin{bmatrix} 5\\ 7\\ 9 \end{bmatrix} \end{split}\]

Using Python lists:

x = [1, 2, 3]
y = [4, 5, 6]
z = [ ]
for x_i, y_i in zip(x, y):
    z.append(x_i + y_i)
z

[5, 7, 9]

Vectors as NumPy arrays

Consider the following vector:

\[\begin{split} \mathbf{x} = \begin{bmatrix} 1\\ 2\\ 3 \end{bmatrix} \end{split}\]

In NumPy:

x = np.array([1, 2, 3])
x

array([1, 2, 3])

Vector addition

Addition is one of the elementary operations that can be performed on vectors. Given two vectors

\[\begin{split} \mathbf{x} = \begin{bmatrix} 1\\ 2\\ 3 \end{bmatrix}, \mathbf{y} = \begin{bmatrix} 4\\ 5\\ 6 \end{bmatrix} \end{split}\]

we have:

\[\begin{split} \mathbf{z} = \mathbf{x} + \mathbf{y}= \begin{bmatrix} 5\\ 7\\ 9 \end{bmatrix} \end{split}\]

x = np.array([1, 2, 3])
y = np.array([4, 5, 6])
z = x + y
z

array([5, 7, 9])

Element-wise multiplication

Element-wise multiplication of two vectors is called the Hadamard product. The operator corresponding to it is \(\odot\). For example, given two vectors:

\[\begin{split} \mathbf{x} = \begin{bmatrix} 1\\ 2\\ 3 \end{bmatrix}, \mathbf{y} = \begin{bmatrix} 4\\ 5\\ 6 \end{bmatrix} \end{split}\]

we have:

\[\begin{split} \mathbf{z} = \mathbf{x} \odot \mathbf{y}= \begin{bmatrix} 4\\ 10\\ 18 \end{bmatrix} \end{split}\]

In NumPy:

x = np.array([1, 2, 3])
y = np.array([4, 5, 6])
z = x * y
z

array([ 4, 10, 18])

Scaling vectors

If \(\mathbf{x}\) is a vector, scaling it by a constant \(k\) is equivalent to element-wise multiplication by \(k\). For example, given

\[\begin{split} \mathbf{x} = \begin{bmatrix} 1\\ 2\\ 3 \end{bmatrix} \end{split}\]

we have:

\[\begin{split} \mathbf{y} = 3 \mathbf{x} = \begin{bmatrix} 3\\ 6\\ 9 \end{bmatrix} \end{split}\]

In NumPy:

x = np.array([1, 2, 3])
y = 3 * x
y

array([3, 6, 9])

Element-wise functions of vectors

Scaling a vector \(\mathbf{x}\) by a constant \(k\) can be seen as the outcome of the function \(f(x) = kx\) applied element-wise:

\[\begin{split} \begin{bmatrix} f(x_1)\\ \vdots\\ f(x_d) \end{bmatrix} = \begin{bmatrix} kx_1\\ \vdots\\ k x_d \end{bmatrix} \end{split}\]

NumPy extends this feature for any arbitrary function.

Example-1

For example, consider the function \(f(x) = x^2\). This can be applied element-wise:

\[\begin{split} \begin{bmatrix} x_1^2\\ \vdots\\ x_d^2 \end{bmatrix} \end{split}\]

Let us do this for:

\[\begin{split} \begin{bmatrix} 1\\ 2\\ 3\\ 4 \end{bmatrix} \end{split}\]

In NumPy:

x = np.array([1, 2, 3, 4])
x ** 2

array([ 1,  4,  9, 16])

Example-2.1

As another example, consider \(f(x) = \log(x)\). This can also be applied element-wise:

\[\begin{split} \begin{bmatrix} \log(x_1)\\ \vdots\\ \log(x_d) \end{bmatrix} \end{split}\]

Let us do this for \(\begin{bmatrix}1 & 10 & 100 & 1000 & 10000 & 100000\end{bmatrix}^T\). Use base \(10\).

In NumPy:

x = np.array([1, 10, 100, 1000, 10_000, 100_000])
np.log10(x)

array([0., 1., 2., 3., 4., 5.])

Example 2.2

We can also use log to the base \(e\), the natural logarithm. For this, let us take a specific vector:

\[\begin{split} \mathbf{x} = \begin{bmatrix} 1\\ e\\ e^2\\ e^3 \end{bmatrix} \end{split}\]

and apply the function \(f(x) = \ln(x)\) element-wise.

In NumPy:

x = np.array([1, np.e, np.e ** 2, np.e ** 3])
np.log(x)

array([0., 1., 2., 3.])

Example 3

Just as we can scale a vector, we can also add a constant to each component. This is equivalent to applying the element-wise function \(f(x) = x + c\). Let us take the case of \(\begin{bmatrix}1 & 2 & 3\end{bmatrix}^T\) and \(c = 5\).

In NumPy:

x = np.array([1, 2, 3])
c = 5
x + c

array([6, 7, 8])

Example 4

Now for a slightly more involved example with \(f(x) = 5x - 2\) applied element-wise on the following vector:

\[\begin{split} \mathbf{x} = \begin{bmatrix} -1\\ 0\\ 1\\ 2 \end{bmatrix} \end{split}\]

In NumPy:

x = np.array([-1, 0, 1, 2])
5 * x - 2

array([-7, -2,  3,  8])

Dot Product

The dot product between two vectors \(\mathbf{x}\) and \(\mathbf{y}\) is given as follows:

\[ z = \mathbf{x}^T \mathbf{y} = \mathbf{x} \cdot \mathbf{y} = \sum \limits_{j = 1}^{m} x_j y_j \]

Let us use the following vectors:

\[\begin{split} \mathbf{x} = \begin{bmatrix} 1\\ 2\\ 3\\ 4\\ \end{bmatrix}, \mathbf{y} = \begin{bmatrix} -4\\ -5\\ -6\\ -7\\ \end{bmatrix} \end{split}\]

In NumPy:

x = np.array([1, 2, 3, 4])
y = np.array([-4, -5, -6, -7])
np.dot(x, y)

-60

Alternatively, we can do the following.

x @ y

-60

Since this resembles the mathematical expression quite accurately, we shall stick to this.

Vector of zeros or ones

On many occassions, we might want to create a NumPy array all of whose elements are zeros or ones or some other constant. Let us create the following vectors:

\[\begin{split} \mathbf{0} = \begin{bmatrix} 0\\ 0\\ 0\\ 0\\ 0 \end{bmatrix}, \mathbf{1} = \begin{bmatrix} 1\\ 1\\ 1\\ 1\\ 1 \end{bmatrix} \end{split}\]

In NumPy:

zeros = np.zeros(5)
zeros

array([0., 0., 0., 0., 0.])

ones = np.ones(5)
ones

array([1., 1., 1., 1., 1.])

What if we wanted to create a vector all of whose elements are equal to \(5\)?

fives = np.ones(5) * 5
fives

array([5., 5., 5., 5., 5.])

Range

Sometimes we might want to create a sequence of integers such as:

\[ [1 \quad 2 \quad 3 \quad 4 \quad 5] \]

np.arange(1, 6)

array([1, 2, 3, 4, 5])

Norm of a vector

For a vector \(\mathbf{x}\):

\[\begin{split} \mathbf{x} = \begin{bmatrix} 1\\ 2\\ 3 \end{bmatrix} \end{split}\]

The \(L_2\) norm is defind as:

\[ ||\mathbf{x}||_2 = \sqrt{1^2 + 2^2 + 3^2} = \sqrt{14} \]

The \(L_1\) norm is defind as:

\[ ||\mathbf{x}||_1 = |1| + |2| + |3| = 6 \]

In NumPy:

# L2-norm
x = np.array([1, 2, 3])
norm = np.linalg.norm(x)
norm

3.7416573867739413

By default, the norm computed is the L2 norm. If we want to compute the L1 norm, we have to pass an additional argument to the function.

# L1-norm
x = np.array([1, 2, 3])
norm = np.linalg.norm(x, ord = 1)
norm

6.0

Shape and dimension of a vector

Vectors are “one dimensional” arrays. So all vectors in NumPy have array-dimension equal to one. The term “array-dimension” here is defined for a NumPy array. It is different from the dimension used in the vector-space sense. The vector-space dimension can be obtaind by looking at the shape of the NumPy array. Let us explore these two ideas for:

\[\begin{split} \mathbf{x} = \begin{bmatrix} 1\\ 2\\ 3\\ 4 \end{bmatrix} \end{split}\]

# Shape
x = np.array([1, 2, 3, 4])
x.shape

(4,)

# N-dim
x = np.array([1, 2, 3, 4])
x.ndim

1