Theorem. If the elementary matrix E is obtained by performing a row operation on the identity matrix I_m and if A is an m by n matrix, then the product EA is the matrix obtained from A by applying the same row operation.

Proof. Indeed, let us represent A as a row of columns A=[c₁,c₂,...,c_n]. Let us also represent E as a column of rows r₁,...,r_m. In order to multiply E by A we need to multiply each r_a by each c_b. Since there are three types of row operations, we need to consider three cases.

Case 1. Suppose that E is obtained from I_m by adding the j-th row multiplied by x to the i-th row. Then each row r_a except r_i has 1 on the (a,a)-place and zeroes everywhere else. The row r_i has 1 on the (i,i)-place and x on the (i,j)-place and zeroes everywhere else. If a is not equal to i then the element EA(a,b), the product of r_a and c_b, will be equal to the a-th entry of c_b, that is A(a,b). Therefore EA(a,b)=A(a,b) whenever a is not equal to i. Thus all rows of EA except for the i-th row coincide with the corresponding rows of A. Let us look at the i-th row. The entries of this row are products of r_i and c_b (b=1,...,n). Since r_i has 1 on the i-th place and x on the j-th places and zero everywhere else, the product of r_i and c_b is the sum of the i-th entry and the j-th entry of c_b multiplied by x. Thus EA(i,b)=A(i,b)+xA(j,b) for every b=1,2,...,n. Therefore the i-th row of EA is obtained by adding x times the j-th row of A to the i-th row of A. This is exactly what we needed to prove (EA is obtained from A by applying the same operation used in order to get E from I).

Case 2. E is obtained from I_n by multiplying a row by a non-zero number.

Case 3. E is obtained from I_n by swapping two rows.

These two cases are left as exercises.