Matrix Manipulation

Matrices are multi-dimensional data containers, which have very important properties enabling a branch of mathematics called Linear Algebra. Matrices can have multiple dimensions. However, the most commonly used type is a two dimensional matrix (e.g. $X_{m,n}$) or $m \times n$. The first dimension of a matrix (i.e. m) is the number of rows and the second dimension (i.e. n) is the number of columns of that matrix. A matrix with one of dimensions being one is called a vector. Matrices may contain one or multiple data types, including mixed data types (e.g. combination of strings and integers). A matrix with multiple type data is called an Array while the homogeneous data container is a matrix. This module only focuses on the homogeneous matrices containing only numerical entries, given their mathematical properties.

Matrix dimensionality

The very first step to be able to work with matrices is being able to handle them, including being able to select specific rows, columns, and/or entries in a matrix. As mentioned before each entry in a matrix has its own coordinates (i.e. the row and column number). For example, the $a_{2,1}$ represents the entry on the second row and the first column.

\[ A_{m,n} = \begin{pmatrix} a_{1,1} & a_{1,2} & \cdots & a_{1,n} \\ a_{2,1} & a_{2,2} & \cdots & a_{2,n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m,1} & a_{m,2} & \cdots & a_{m,n} \end{pmatrix} \]

Entry selection

In julia to select an entry of a matrix you can use "[m,n]", where m is the row number and n is the column number. It should be noted that these numbers are also referred to as indices. In the below example we first generate a matrix of 15 by 20 filled by random numbers, which will be used for our examples.

using DataSci4Chem

# Generating the data

m = 15          # number of rows
n = 21          # number of columns

X = randn(m,n)

15×21 Matrix{Float64}:
 -0.261242    0.640841  -0.0383476  …  -0.461675    0.735311  -0.533559
 -0.370764    0.948384  -0.0399103      0.595118   -1.32276   -0.335234
  2.41513     1.44591   -1.12539        0.922323    1.45203   -0.44027
  0.0528401   0.737434  -2.34295       -1.06559     0.692782   0.357873
  1.79908    -0.378802  -1.31851       -1.35407     1.44918    0.254611
  0.719962   -0.481266   0.509049   …   0.371926    0.148728   0.360237
  1.1654     -1.2174     0.459346      -0.150894    0.186405  -0.100618
  2.04242    -1.06084    0.210052       2.4811     -1.55696    1.27181
  0.680357    0.521498  -1.3151        -1.32405    -1.59695   -0.933656
 -0.0506918  -1.10537   -0.458522      -0.587213   -0.674997   0.31733
  0.243483    0.452193  -0.0650256  …  -0.0315809   0.354381  -1.44279
 -0.438872   -0.249622  -0.135071      -0.0795303  -0.555185  -1.21156
  0.686815    0.862209   0.716504      -0.754809    1.13362   -0.189733
 -1.47735    -0.698712  -0.995605       1.07751    -0.809657   0.774989
 -0.940401    0.199382   0.742045       0.170181    1.30623    0.858447

As you can see, we have generated our matrix of random floats and have stored it in a variable called X. Now as our first example we will attempt at selecting the matrix entry at the row 5 and column 20. To do that we use the provided coordinates with "[]". Here we first select the specified entry and store it in the variable "x1" and then printed.

using DataSci4Chem

x1 = X[5,20]

println(x1)

1.4491812501055228

We can also select more than one entry at a time. For example if we want to select the entries between 3rd and 10th rows on the 2nd column we take advantage of ranges (i.e. 3:10). Similar to the single entry selection we put the relevant range in the spot for rows and select the column #2.

using DataSci4Chem

x2 = X[3:10,2]

8-element Vector{Float64}:
  1.4459146414524355
  0.7374341578962736
 -0.3788019327364567
 -0.4812662138872235
 -1.2173955349571228
 -1.060836670530379
  0.5214980705069386
 -1.10537161806265

As you can see, the output of this operation is a matrix of 8 by 1, thus a vector of 8 elements. We can perform the range selection in both row and column level at the same time. The example below shows how to select rows 5:10 and columns 1:3. This should result into a 6 by 3 matrix.

using DataSci4Chem

X1 = X[5:10,1:3]

6×3 Matrix{Float64}:
  1.79908    -0.378802  -1.31851
  0.719962   -0.481266   0.509049
  1.1654     -1.2174     0.459346
  2.04242    -1.06084    0.210052
  0.680357    0.521498  -1.3151
 -0.0506918  -1.10537   -0.458522

There are also cases where you would like to select a column or an entire row in a matrix. In this case also we can use ranges expressed by ":". Since this is the full range from 1:end you do not need to specify the details. For example for selecting the second column of the X we can use the expression X[:,2], which implies that we are selecting all the rows in the second column.

using DataSci4Chem

X2 = X[:,2]

15-element Vector{Float64}:
  0.6408413585075055
  0.9483841787231454
  1.4459146414524355
  0.7374341578962736
 -0.3788019327364567
 -0.4812662138872235
 -1.2173955349571228
 -1.060836670530379
  0.5214980705069386
 -1.10537161806265
  0.4521930882119125
 -0.24962225175561514
  0.8622089120220822
 -0.6987119992293666
  0.19938192995797466

You can also select multiple columns or rows of a matrix. In this example we are selecting rows 3:5 of the matrix X.

using DataSci4Chem

X3 = X[3:5,:]

3×21 Matrix{Float64}:
 2.41513     1.44591   -1.12539   0.972592  …   0.922323  1.45203   -0.44027
 0.0528401   0.737434  -2.34295   1.18838      -1.06559   0.692782   0.357873
 1.79908    -0.378802  -1.31851  -1.15971      -1.35407   1.44918    0.254611

So far we have selected rows and/or columns that are next to each other, thus accessible with ranges. We can also access entries in a matrix that are not next to one another. In the below example we are selecting the columns 1, 3, and 21 of the matrix X. For this we need to generate a vector of the column indices (i.e. [1,3,21]), which is used for our operation.

using DataSci4Chem

X4 = X[:,[1,3,21]]

15×3 Matrix{Float64}:
 -0.261242   -0.0383476  -0.533559
 -0.370764   -0.0399103  -0.335234
  2.41513    -1.12539    -0.44027
  0.0528401  -2.34295     0.357873
  1.79908    -1.31851     0.254611
  0.719962    0.509049    0.360237
  1.1654      0.459346   -0.100618
  2.04242     0.210052    1.27181
  0.680357   -1.3151     -0.933656
 -0.0506918  -0.458522    0.31733
  0.243483   -0.0650256  -1.44279
 -0.438872   -0.135071   -1.21156
  0.686815    0.716504   -0.189733
 -1.47735    -0.995605    0.774989
 -0.940401    0.742045    0.858447

Now that we know how to select entries or chunks of a matrix, we can start working on matrix operations.

Matrix operations

Element wise

Most matrix operations are divided into two categories: element wise and the matrix wise. For the element vise operations the two matrices must have exactly the same size as the desired operation is performed element by element. Consequently, the resulting matrix will have the same size as the starting matrices.

Summation

An example of such operations is summing or subtracting two matrices. Let's generate two matrices 4 by 5 populated with random numbers called A and B.

using DataSci4Chem

A = rand(4,5)

B = rand(4,5)

4×5 Matrix{Float64}:
 0.813133  0.943504   0.180033  0.0980552  0.656813
 0.775892  0.836673   0.326705  0.263429   0.597806
 0.933627  0.938316   0.566102  0.929852   0.329847
 0.258826  0.0611276  0.594636  0.409667   0.515435

Now we can try to sum up these two matrices and store the results in the matrix C.

using DataSci4Chem

C = A + B

4×5 Matrix{Float64}:
 1.8116    1.83831  0.425873  0.328398  1.40672
 1.66338   1.72445  0.526975  0.898413  1.3956
 1.61821   1.49791  1.54727   1.91011   0.669231
 0.609491  0.65962  1.32372   1.03523   1.39107

The matrix D is the result of subtraction of two matrices A and B.

using DataSci4Chem

D = A - B

4×5 Matrix{Float64}:
  0.185336   -0.0486965   0.0658061  0.132288   0.0930914
  0.111601    0.0511014  -0.126436   0.371555   0.199988
 -0.249048   -0.378719    0.415066   0.0504098  0.00953592
  0.0918379   0.537365    0.134444   0.215893   0.360205

Transpose

Another very useful element wise matrix operation is the "transpose" operation. When a matrix is transposed its rows and columns are switched. This means that if you have started with $X2_{3,2}$, you will end up with matrix $X2^{T}_{2,3}$. In the below example we have a 3 by 2 matrix X2 and we will try to transpose it to X2_t.

using DataSci4Chem

X2 = [1 2 ; 4 3 ;  5 4]

3×2 Matrix{Int64}:
 1  2
 4  3
 5  4

using DataSci4Chem

X2_t = DataSci4Chem.transpose([1 2 ; 4 3 ;  5 4])

2×3 transpose(::Matrix{Int64}) with eltype Int64:
 1  4  5
 2  3  4

Scalar multiplication

Another element wise matrix operation is the multiplication of a scalar (i.e. a number) to a matrix. Here the scalar is multiplied into each element individually, thus element wise. For example if we multiply the scalar 2 to the matrix X2 we will get the following.

using DataSci4Chem

2 * X2

3×2 Matrix{Int64}:
  2  4
  8  6
 10  8

Element wise multiplication

The last element wise matrix operation that is discussed here is the element-wise matrix multiplication or Hadamard product. This operation is denoted as "$\odot$" and is performed using ".*" in julia language.

using DataSci4Chem

X2 .* (2 * ones(3,2))

3×2 Matrix{Float64}:
  2.0  4.0
  8.0  6.0
 10.0  8.0

In the above example with the function ones(3,2) we are generating a 3 by 2 matrix of ones, which is multiplied by the scalar 2, resulting in a 3 by 2 matrix of twos. In the next step the two matrices are multiplied element wise. As you can intuitively imagine and combination of these three components will result in the same output.

using DataSci4Chem

2 * X2 .* ones(3,2)

3×2 Matrix{Float64}:
  2.0  4.0
  8.0  6.0
 10.0  8.0

or

using DataSci4Chem

2 * (X2 .* ones(3,2))

3×2 Matrix{Float64}:
  2.0  4.0
  8.0  6.0
 10.0  8.0

Matrix wise

Matrix multiplication

One of the most important matrix wise operations is the matrix multiplication. Let's say we have a matrix $A_{m,p}$ and a second matrix $B_{p,n}$ for these matrices to be multipliable they must have the same inner dimensions.

\[ A_{m,p} \times B_{p,n} = C_{m,n} \]

As you can see the result of this multiplication is a matrix $C_{m,n}$. Each entry in the product matrix is the result of first element-wise multiplication of the first matrix rows to the second matrix columns followed by summation of those products. This process can be described with the below formula.

\[ i = 1:m, \\ j = 1:n, \\ c_{i,j} = \sum_{k=1}^{p} a_{i,k}b_{k,j} \]

Let's see this in practice:

\[ A = \begin{pmatrix} 1 & 6 \\ 9 & 3 \end{pmatrix} \\ and\\ B = \begin{pmatrix} 0 & -1 \\ -1 & 2 \end{pmatrix} \]

Based on the above formula the first entry of the $c_{1,1}$ will be calculated as:

\[ c_{1,1} = a_{1,1}b_{1,1} + a_{1,2}b_{2,1} = 1 \times 0 + 6 \times -1 = -6 \]

while the $c_{1,2}$ is calculated a following.

\[ c_{1,2} = a_{1,1}b_{1,2} + a_{1,2}b_{2,2} = 1 \times -1 + 6 \times 2 = 11 \]

Following this process we can calculate the product os A and B.

\[ \begin{pmatrix} 1 & 6 \\ 9 & 3 \end{pmatrix} \times \begin{pmatrix} 0 & -1 \\ -1 & 2 \end{pmatrix} = \begin{pmatrix} -6 & 11 \\ -3 & -3 \end{pmatrix} \]

Matrix powers

Matrix power is an extension of the matrix multiplication where $A^{n} = \prod_{i=1}^{n} A$, which denotes n multiplications of the matrix A. However, it should be noted that this is only possible for square matrices as non-squared matrices cannot be multiplied by themselves.

Inverse matrix

Calculation of the inverse matrix (i.e. $A^{-1}$) is one of the most fundamental operations that you will do with matrices. It has applications in regression, optimization, matrix decomposition and many other areas of scientific computing. For a matrix to be "invertible" (i.e. nonsingular), it has to satisfy two criteria:

1. it must be square
2. $A^{-1} \times A = I$.

In fact for calculating the inverse of a matrix you can use the second criteria for invertible matrices. Let's look an example for this.

\[ \begin{pmatrix} a & b \\ c & d \end{pmatrix} \times \begin{pmatrix} 1 & -1 \\ 1 & 2 \end{pmatrix} = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} \]

If we solve this problem we will end up with the below matrix.

\[ \begin{pmatrix} a & b \\ c & d \end{pmatrix} \times \begin{pmatrix} 1 & -1 \\ 1 & 2 \end{pmatrix} = \begin{pmatrix} a+b & 2b-a \\ c+d & 2d-c \end{pmatrix} = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} \]

Now we have a set of four equations and four variables that can be rewritten as a system of equations.

\[ a + b = 1\\ 2b - a = 0 \\ c + d = 1 \\ 2d + c = 0 \]

Solving these equations ultimately will result in the inverse matrix.

\[ \begin{pmatrix} 1 & 2 \\ 1 & -1 \end{pmatrix}^{-1} = \begin{pmatrix} 0.333 & -0.333 \\ 0.333 & -0.666 \end{pmatrix} \]

You can calculate the inverse of invertible matrices using the functions inv() or pinv(-) (i.e. pseudo inverse).

using DataSci4Chem

A = [1 2; 1 -1]
inv(A)

2×2 Matrix{Float64}:
 0.333333   0.666667
 0.333333  -0.333333

Matrix decomposition

Matrix decomposition is another important operation where a matrix X is decomposed/factorized into product matrices. There are several matrix decomposition methods, depending on the applications. For example for solving systems of equations LU decomposition is employed. One of the highly relevant matrix decomposition approaches is singular value decomposition (SVD). The SVD decomposes a matrix X into the product is three matrices $U_{m \times n}$, $D_{n \times n}$, and $V_{n \times n}^{T}$. The matrix $U_{m \times n}$ is the left singular matrix and it represents a rotation in the matrix space. The $D_{n \times n}$ is diagonal matrix and contains the singular values. Finally, $V_{n \times n}^{T}$ is called the right singular matrix and is associated with rotation. For more details of SVD please look at SVD course material.