> For the complete documentation index, see [llms.txt](https://amanalok.gitbook.io/linear-algebra/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://amanalok.gitbook.io/linear-algebra/basic-operations.md).

# Basic Operations

## Vector Equality

Let $$x \isin \mathbb{R}^{n}$$ ​and $$y \isin \mathbb{R}^{n}$$ ​with:

$$
x = \begin{pmatrix}
x\_0 \\
x\_1 \\
... \\
x\_{n-1}
\end{pmatrix},
y = \begin{pmatrix}
y\_0 \\
y\_1 \\
... \\
y\_{n-1}
\end{pmatrix}
$$

Then $$x = y$$, if and only if:

$$
x\_0 = y\_0; \\
x\_1 = y\_1; \\
... \\
x\_{n-1} = y\_{n-1};
$$

That is, the corresponding elements of $$x$$​ and $$y$$​ are equal.

## Vector Addition

Vector addition $$x+y$$ (sum of vectors) is defined by:

$$x+y = \begin{pmatrix} x\_0 \ x\_1 \ ... \ x\_{n-1} \ \end{pmatrix} +  \begin{pmatrix} y\_0 \ y\_1 \ ... \ y\_{n-1} \ \end{pmatrix} =  \begin{pmatrix} x\_0 + y\_0 \ x\_1 + y\_1 \ ... \ x\_{n-1} + y\_{n-1}  \end{pmatrix}$$

In other words, the vectors are added element-wise, yielding a new vector of the same size.

### Geometric Interpretation

In this interpretation, let $$x \isin \mathbb{R}^{n}$$ and $$y \isin \mathbb{R}^{n}$$. These 2 vectors actually lie in a plane and if we view them in that plane, we can observe the result of doing $$x + y$$. If lay the vectors "heel to toe" , e.g. if we lay vector $$y$$ 's heel (where $$y$$ starts) to toe of $$x$$ (where $$x$$ ends), $$x + y$$ can be given by going from heel of $$x$$ to toe of $$y$$. Vice-versa is also true.

Another geometric interpretation is, if we lay $$y$$​'s heel to toe of $$x$$ or we lay $$x$$'s heel to toe of $$y$$, this would create a parallelogram. The result of $$x + y$$​is given by the diagonal of the parallelogram. This is known as the parallelogram method.

## Vector Scaling

Scaling a vector is same as stretching the vector. Let $$x \isin \mathbb{R}^{n}$$​ and $$\alpha \isin \mathbb{R}$$​. Then, vector scaling is given by:

$$
\alpha x = \alpha \begin{pmatrix}
x\_0 \\
x\_1 \\
... \\
x\_{n-1}
\end{pmatrix} =
\begin{pmatrix}
\alpha x\_0 \\
\alpha x\_1 \\
... \\
\alpha x\_{n-1} \\
\end{pmatrix}
$$

## Vector Subtraction

Let $$x \isin \mathbb{R}^n$$ and ​$$y \isin \mathbb{R}^n$$​. Then, vector subtraction can be given by:

$$
x - y = x + (-y)
================

\begin{pmatrix}
x\_0 \\
x\_1 \\
... \\
x\_{n-1} \\
\end{pmatrix} -
\begin{pmatrix}
y\_0 \\
y\_1 \\
... \\
y\_{n-1} \\
\end{pmatrix} =\
\begin{pmatrix}
x\_0 - y\_0 \\
x\_1 - y\_1 \\
... \\
x\_{n-1} - y\_{n-1} \\
\end{pmatrix}
$$

### Geometric Interpretation

Taking the negative of $$y$$​ means it points in the opposite direction. Now, we have two vectors $$x$$​ and $$-y$$ that are heel to toe. $$x - y$$​ can be given by going from heel of $$x$$​ to toe of $$-y$$​.

Same as for Vector Addition, the parallelogram method can be applied here as well. $$x - y$$​ is the other diagonal of the parallelogram.

## Scaled Vector Addition

One of the most commonly encountered operations when implementing more complex linear algebra operations is the scaled vector addition, which (given $$x \isin \mathbb{R}^n$$​ and $$y \isin \mathbb{R}^n$$​)computes to:

$$
y := \alpha x + y =
\alpha
\begin{pmatrix}
x\_0 \\
x\_1 \\
... \\
x\_{n-1} \\
\end{pmatrix} +
\begin{pmatrix}
y\_0 \\
y\_1 \\
... \\
y\_{n-1} \\
\end{pmatrix} =
\begin{pmatrix}
\alpha x\_0 + y\_0 \\
\alpha x\_1 + y\_1 \\
... \\
\alpha x\_{n-1} + y\_{n-1} \\
\end{pmatrix}
$$

It is often referred to as the `AXPY` operation, which stands for **a**lpha times **x** plus **y**. We emphasize that it is typically used in situations where the output vector overwrites the input vector y.

### Counting Flops & Memops

Consider the following scaled vector addition:

$$
y := \alpha x + y
\\
\= 2 \begin{pmatrix}
1 \\
-2 \\
3
\end{pmatrix} +
\begin{pmatrix}
-1 \\
2 \\
1
\end{pmatrix}
$$

Now, imagine all the data for this scaled vector addition starts by being stored in the main memory of a computer. Further, in order to compute, data must be brought into registers and there are only a few registers available.

Now, we know that the scaling factor 2 in the above example is going to be used many times. So, we store it in a register and keep it there during the entire computation. This costs us 1 memory operation (memops).

Further, for each pair of one element from $$x$$ and one element from $$y$$​, we need to read these elements and write the result back to $$y$$. That means 2 memops for reading, 1 more for writing.

Finally, for each pair of elements from $$x$$ and $$y$$, we perform a multiplication and then an addition. That would be 2 floating point operations (flops).

Now, if the size of the vectors is n, then we would have to do $$3n+1$$memops and $$2n$$​ flops.

## Vector Function

A vector (valued) function is a mathematical function of one or more scalars and/or vectors whose output is a vector.

For example,

$$
f(\alpha, \beta) = \begin{pmatrix}
\alpha + \beta \\
\alpha - \beta
\end{pmatrix}
$$

Another example can be,

$$
f(\alpha, \begin{pmatrix}
\chi\_0 \\
\chi\_1 \\
\chi\_2
\end{pmatrix}) =
\begin{pmatrix}
\chi\_0 + \alpha \\
\chi\_1 + \alpha \\
\chi\_2 + \alpha \\
\end{pmatrix}
$$

The `AXPY` and `DOT` operations are also examples of vector functions.

### Vector Functions That Map a Vector to a Vector

We will now focus on vector functions as functions that map a vector to another vector:

$$
f: \mathbb{R}^n  \rightarrow \mathbb{R}^m
$$

For example,

$$
f\begin{pmatrix}
\alpha \\
\beta
\end{pmatrix} =
\begin{pmatrix}
\alpha + \beta \\
\alpha - \beta
\end{pmatrix}
$$

Another example can be,

$$
f\begin{pmatrix}
\alpha \\
\begin{pmatrix}
\chi\_0 \\
\chi\_1 \\
\chi\_2
\end{pmatrix}
\end{pmatrix} =
g \begin{pmatrix}
\alpha \\
\chi\_0 \\
\chi\_1 \\
\chi\_2
\end{pmatrix} =
\begin{pmatrix}
\chi\_0 + \alpha \\
\chi\_1 + \alpha \\
\chi\_2 + \alpha
\end{pmatrix}
$$

Do note the following:

* $$f:\mathbb{R}^n \rightarrow  \mathbb{R}^m, \lambda \isin \mathbb{R} \hspace{1.5mm} and \hspace{1.5mm} x \isin  \mathbb{R}^n$$, then $$f(\lambda x) = \lambda f(x)$$happens sometimes.
* $$f:\mathbb{R}^n \rightarrow \mathbb{R}^m \hspace{1.5mm} and \hspace{1.5mm} x,y \isin  \mathbb{R}^n$$, then $$f(x+y) = f(x) + f(y)$$ happens sometimes.

## Matrix Product

For given matrices A and B, their product is given by:

$$
C = A \times B
\\
C\_{ij} = \sum\_k A\_{ik}B\_{kj}
$$

## ​Hadamard Product

Hadamard product is element-wise multiplication. Following equations provide details of how the this product operation is calculated for given matrices A and B:

$$
C = A \odot B
\\
C\_{ij} = A\_{ij}B\_{ij}
$$

## ​Dot (Inner) Product

The dot (inner) product between two vectors *a* and *b* is given by:

$$
\vec{a}.\vec{b} = \alpha = a^Tb = b^Ta
\\
\alpha := \sum\_ia\_ib\_i
$$

Alternatively,

$$
\alpha : = (((0 + a\_0b\_0) + a\_1b\_1) + ...) + a\_{n-1}b\_{n-1}
$$

### Cost

The cost of a dot product with vectors of size $$n$$ is $$2n$$ memops and $$2n$$​ flops.

## ​Transpose

The transpose of a matrix A is given by:

$$
B = A^T
\\
B\_{ji} = A\_{ij}
$$

## ​Inverse

The following equation provides the deifnition of inverse of a matrix:

$$
I = A^{-1}A = AA^{-1}
$$

Not all square matrices have an inverse​.

For such cases, and non-square matrices, "pseudoinverse" is defined.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://amanalok.gitbook.io/linear-algebra/basic-operations.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.