Add documentation for required Tensor operations for mvp implementation

Signed-off-by: Julius Koskela <julius.koskela@unikie.com>

docs/tensor-operations.md

@@ -0,0 +1,138 @@
+Great, you're using a Rust library for tensor operations. Let's go through the key tensor operations with mathematical definitions and Rust code examples, tailored to your library's capabilities. The examples will assume the presence of necessary functions in your tensor library.
+
+### 1. Addition
+**Mathematical Definition**: \( C = A + B \) where \( C_{ijk...} = A_{ijk...} + B_{ijk...} \) for all indices \( i, j, k, ... \).
+**Rust Code Example**:
+```rust
+let tensor1 = Tensor::<i32, 2>::from([1, 2, 3, 4]); // 2x2 tensor
+let tensor2 = Tensor::<i32, 2>::from([5, 6, 7, 8]); // 2x2 tensor
+let sum = tensor1.add(&tensor2); // Element-wise addition
+```
+
+### 2. Subtraction
+**Mathematical Definition**: \( C = A - B \) where \( C_{ijk...} = A_{ijk...} - B_{ijk...} \).
+**Rust Code Example**:
+```rust
+let tensor1 = Tensor::<i32, 2>::from([4, 3, 2, 1]); // 2x2 tensor
+let tensor2 = Tensor::<i32, 2>::from([1, 2, 3, 4]); // 2x2 tensor
+let difference = tensor1.sub(&tensor2); // Element-wise subtraction
+```
+
+### 3. Element-wise Multiplication
+**Mathematical Definition**: \( C = A \odot B \) where \( C_{ijk...} = A_{ijk...} \times B_{ijk...} \).
+**Rust Code Example**:
+```rust
+let tensor1 = Tensor::<i32, 2>::from([1, 2, 3, 4]); // 2x2 tensor
+let tensor2 = Tensor::<i32, 2>::from([5, 6, 7, 8]); // 2x2 tensor
+let product = tensor1.mul(&tensor2); // Element-wise multiplication
+```
+
+### 4. Tensor Contraction (Matrix Product)
+**Mathematical Definition**: For matrices \( A \) and \( B \), \( C = AB \) where \( C_{ij} = \sum_k A_{ik} B_{kj} \).
+**Rust Code Example**: Your provided example already demonstrates tensor contraction (though it's more of a tensor product given the dimensions).
+
+### 5. Element-wise Division
+**Mathematical Definition**: \( C = A \div B \) where \( C_{ijk...} = A_{ijk...} \div B_{ijk...} \).
+**Rust Code Example**:
+```rust
+let tensor1 = Tensor::<i32, 2>::from([10, 20, 30, 40]); // 2x2 tensor
+let tensor2 = Tensor::<i32, 2>::from([2, 2, 5, 5]);    // 2x2 tensor
+let quotient = tensor1.div(&tensor2); // Element-wise division
+```
+
+### 6. Reduction Operations (e.g., Sum)
+**Mathematical Definition**: \( \text{sum}(A) \) where sum over all elements of A.
+**Rust Code Example**:
+```rust
+let tensor = Tensor::<i32, 2>::from([1, 2, 3, 4]); // 2x2 tensor
+let total_sum = tensor.sum(); // Sum of all elements
+```
+
+### 7. Broadcasting
+**Description**: Adjusts tensors with different shapes to make them compatible for element-wise operations.
+**Rust Code Example**: Depends on whether your library supports broadcasting. If it does, it will handle shape adjustments automatically during operations like addition or multiplication.
+
+### 8. Reshape
+**Mathematical Definition**: Changing the shape of a tensor without altering its data.
+**Rust Code Example**:
+```rust
+let tensor = Tensor::<i32, 1>::from([1, 2, 3, 4, 5, 6]); // 6-element vector
+let reshaped_tensor = tensor.reshape([2, 3]); // Reshape to 2x3 tensor
+```
+
+### 9. Transpose
+**Mathematical Definition**: \( B = A^T \) where \( B_{ij} = A_{ji} \).
+**Rust Code Example**:
+```rust
+let tensor = Tensor::<i32, 2>::from([1, 2, 3, 4]); // 2x2 tensor
+let transposed_tensor = tensor.transpose(); // Transpose the tensor
+```
+
+### 10. Concatenation
+**Description**: Joining tensors along a specified dimension.
+**Rust Code Example**:
+```rust
+let tensor1 = Tensor::<i32, 1>::from([1, 2, 3]); // 3-element vector
+let tensor2 = Tensor::<i32, 1>::from([4, 5, 6]); // 3-element vector
+let concatenated_tensor = tensor1.concat(&tensor2, 0); // Concatenate along dimension 0
+```
+
+### 11. Slicing and Indexing
+**Description**: Extracting parts of tensors based on indices.
+**Rust Code Example**:
+```rust
+let tensor = Tensor::<i32, 2>::from([1, 2, 3, 4, 5, 6]); // 2x3 tensor
+let slice = tensor.slice(s![1, ..]); // Slice to get the second row
+```
+
+### 12. Element-wise Functions (e.g., Sigmoid)
+**Mathematical Definition**: Applying a function to each element of a tensor, like \( \sigma(x) = \frac{1}{1 + e^{-x}} \) for sigmoid.
+**Rust Code Example**:
+```rust
+let tensor = Tensor::<f32, 2>::from([-1.0, 0.0, 1.0, 2.0]); // 2x2 tensor
+let sigmoid_tensor = tensor.map(|x| 1.0 / (1.0 + (-x).exp())); // Apply sigmoid element-wise
+```
+
+### 13. Gradient Computation/Automatic Differentiation
+**Description**: Calculating the derivatives of tensors, crucial for training machine learning models.
+**Rust Code Example**: Depends on if your tensor library supports automatic differentiation. This is typically more complex and may involve constructing computational graphs.
+
+### 14. Normalization Operations (e.g., Batch Normalization)
+**Description**: Standardizing the inputs of a model across the batch dimension.
+**Rust Code Example**: This is specific to deep learning libraries and may not be directly supported in a general-purpose tensor library.
+
+### 15. Convolution Operations
+**Description**: Essential for image processing and CNNs.
+**Rust Code Example**: If your library supports it, convolutions typically involve using a specialized function that takes the input tensor and a kernel tensor.
+
+### 16. Pooling Operations (e.g., Max Pooling)
+**Description**: Reducing the spatial dimensions of
+
+ a tensor, commonly used in CNNs.
+**Rust Code Example**: Again, this depends on your library's support for such operations.
+
+### 17. Tensor Slicing and Joining
+**Description**: Operations to slice a tensor into sub-tensors or join multiple tensors into a larger tensor.
+**Rust Code Example**: Similar to the slicing and concatenation examples provided above.
+
+### 18. Dimension Permutation
+**Description**: Rearranging the dimensions of a tensor.
+**Rust Code Example**:
+```rust
+let tensor = Tensor::<i32, 3>::from([...]); // 3D tensor
+let permuted_tensor = tensor.permute_dims([2, 0, 1]); // Permute dimensions
+```
+
+### 19. Expand and Squeeze Operations
+**Description**: Increasing or decreasing the dimensions of a tensor (adding/removing singleton dimensions).
+**Rust Code Example**: Depends on the specific functions provided by your library.
+
+### 20. Data Type Conversions
+**Description**: Converting tensors from one data type to another.
+**Rust Code Example**:
+```rust
+let tensor = Tensor::<i32, 2>::from([1, 2, 3, 4]); // 2x2 tensor
+let converted_tensor = tensor.to_type::<f32>(); // Convert to f32 tensor
+```
+
+These examples provide a general guide. The actual implementation details may vary depending on the specific features and capabilities of the Rust tensor library you're using.