current position：Home>Vectorization of deep learning formula
Vectorization of deep learning formula
20220127 05:34:50 【LolitaAnn】
This is my participation 8 The fourth of the yuegengwen challenge 18 God , Check out the activity details ：8 Yuegengwen challenge
Σ( ° △ °)︴ Let's start with a few nonsense . As a biology student （ At that time, all majors chose computers , As a result, I was forced to learn biology ）, But I had to study by myself . I use C++ As an introductory language , When I first started learning data structures , Because it's selfstudy , A lot of pain , I make complaints about it every day. ： Yes STL Why write it yourself ！！！ I learned later , In order to practice , Therefore, when you often brush questions, you can write some functions that even the library function already has to realize . At that time, all roads lead to Rome in my eyes , You can get the same result anyway . Although I learned the time complexity of the algorithm, I didn't put it into practice , After all, there is not so much test data in question brushing .
Later, an elder told me , You can have a look at the source code of the library function in your spare time . For example, the ranking of others may be a combination of several sorts , They are all algorithms optimized by various bosses .
Get down to business
This is also true for some formulas in our indepth learning , If you can simplify , It's great to use the library of linear algebra built in the language .
Benefits of using builtin algorithms ：
 Faster
 Implemented in less code
 It's less error prone than what you write yourself
 Better coordination with hardware system
Take a simple chestnut ：
$h_{\theta}(x)=\sum_{j=0}^{n} \theta_{j} x_{j}$
You can think of it as $\theta^{T} x$ And calculate , This translates into two vectors $\theta=\left[\begin{array}{l}\theta_{0} \\ \theta_{1} \\ \theta_{2} \\ ... \end{array}\right] \quad x=\left[\begin{array}{l}x_{0} \\ x_{1} \\ x_{2} \\ ...\end{array}\right]$ The product of .
octave：
Unvectorized implementation
Is to declare two vectors , Traverse .
% No vectorization code is as follows ：
prediction =0.0;
for j = 1:n+l
prediction = prediction + theta(j) * x(j)
end
Copy code
Vectorized implementation
% After vectorization, the code is as follows ：
prediction= theta' * x; % Single quotation mark in English ' Represents the transpose of a matrix or vector . Forget to go back and see octave grammar Copy code
C++：
Unvectorized implementation
double prediction = 0.0;
for(int j = 0; j<=n; j++)
{
prediction += theta[j]*x[j];
}
Copy code
Vectorized implementation
double prediction = theta.transpose()*x;
Copy code
Another chestnut ：
This formula is familiar , Is the formula for gradient descent . In this formula x It's a data matrix ,y Is a column vector .
For multiple linear regression ：
First simplify the gradient descent formula to make it easier to write code ：
among δ It's a column vector ：$\delta=\left[\begin{array}{l} \delta_{0} \\ \delta_{1} \\ \delta_{2} \\... \end{array}\right]$
And for x Come on ,$x^{(i)}=\left[\begin{array}{c} x_{0}^{(i)} \\ x_{1}^{(i)} \\ x_{2}^{(i)} \\... \end{array}\right]$
So it must be $\delta_{n \times 1} = Count ×(...)× x_{1×n}^{(i)}$ namely $x_j^{(i)}$ To transpose .
If this logic is not vectorized, it may need to write many cycles to complete . Just a little garbage like me can't write it for a while . I'm handy c++ I wrote it , No guarantee, right , Just take a general look , I didn't run this code either .
for(int j=0;j<n;j++)
{
// First calculate h(x) Come on
for(int i=0;i<len;i++)
{
hx[i] += theta[i][j]*x[i][j];
}
// Calculation [h(x)j]*x And sum up
for(int i=0;i<len;i++)
{
sum += (h[i]  y[i])*x[i][j];
}
// The rest of the formula
theta[j] = theta[j]  alpha * (1/m) * sum;
}
Copy code
But if you vectorize it, you can write it as ：
% Suppose it's binary now
hx = X * theta;
theta(1) = theta(1)  alpha * (1/m) * sum((hxy)*X(:,1:1))
theta(2) = theta(2)  alpha * (1/m) * sum((hxy)*X(:,2:2))
% Be careful octave and C And so on , Subscript from 1 Start
Copy code
% If it is n yuan
hx = X * theta;
for i = 1:n
theta(i) = theta(i)  alpha * (1/m) * sum((hxy)*X(:,i:i))
% Be careful octave and C And so on , Subscript from 1 Start
endfor
Copy code
After vectorization, everything is easy to deal with , But without vectorization , Then you may have to nest a lot for Circulated . So we should make good use of vectorization to reduce the workload .
copyright notice
author[LolitaAnn],Please bring the original link to reprint, thank you.
https://en.cdmana.com/2022/01/202201270534167776.html
The sidebar is recommended
 Spring IOC container loading process
 [thinking] the difference between singleton mode and static method  objectoriented programming
 Hadoop environment setup (MySQL environment configuration)
 10 minutes, using node JS creates a realtime early warning system for bad weather!
 Git tool
 Force deduction algorithm  92 Reverse linked list II
 What is the sub problem of dynamic programming?
 C / C + +: static keyword summary
 Idea does not have the artifacts option when configuring Tomcat
 I don't know how to start this
guess what you like

Matlab simulation of transportation optimization algorithm based on PSO

MySQL slow log optimization

[Vue] as the window is stretched (larger, smaller, wider and higher), the text will not be displayed

Popular Linux distributions for embedded computing

The 72 year old uncle became popular. Wu Jing and Guo fan made his story into a film, which made countless dreamers blush

How to save computer research

The final examination subjects of a class are mathematical programming, and the scores are sorted and output from high to low

Two pronged approach, Tsinghua Professor Pro code JDK and hotspot source code notes, onetime learning to understand

C + + recursive knapsack problem

The use of GIT and GitHub and the latest git tutorial are easy to understand  Video notes of crazy God speaking
Random recommended
 Ignition database test
 Context didn't understand why he got a high salary?, Nginxfair principle
 Bootstrap switch switch control user's guide, springcloud actual combat video
 A list that contains only strings. What other search methods can be used except sequential search
 [matlab path planning] multi ant colony algorithm grid map path planning [including GUI source code 650]
 [matlab path planning] improved genetic algorithm grid map path planning [including source code phase 525]
 Iinternet network path management system
 Appium settings app is not running after 5000ms
 Reactnative foundation  07 (background image, status bar, statusbar)
 Reactnative foundation  04 (custom rpx)
 Linux checks where the software is installed and what files are installed
 SQL statement fuzzy query and time interval filtering
 69. Sqrt (x) (c + + problem solving version with vs runnable source program)
 Java project: OA management system (java + SSM + bootstrap + MySQL + JSP)
 Titanic passenger survival prediction
 Configuration and use of private image warehouse of microservice architect docker
 Relearn JavaScript events
 How does Java dynamically obtain what type of data is passed? It is used to judge whether the data is the same, dynamic data type
 How does the database cow optimize SQL?
 [data structure] chain structure of binary tree (pre order traversal) (middle order traversal) (post order traversal) (sequence traversal)
 Detailed explanation of red and black trees
 redhat7. 9 install database 19C
 Blue Bridge Cup notes: (the given elements are not repeated) complete arrangement (arrangement cannot be repeated, arrangement can be repeated)
 Detailed explanation of k8s management tool kubectl
 Android system view memory command