Lemma. Every elementary matrix is invertible and the inverse is again an elementary matrix. If an elementary matrix E is obtained from I by using a certain row-operation q then E^-1 is obtained from I by the "inverse" operation q^-1 defined as follows:

If q is the adding operation (add x times row j to row I) then q^-1 is also an adding operation (add -x times row j to row I).

If q is the multiplication of a row by x then q^-1 is the multiplication of the same row by x^-1.

If q is a swapping operation then q^-1=q.

Proof. Indeed, it is easy to see that for every elementary operation q if we apply q and then q^-1 then we return to the original matrix. In particular, if we apply q and then q^-1 to I, we return to I. But by the theorem about elementary matrices, application of a row operation to a matrix is equivalent to multiplying this matrix by the corresponding elementary matrix. Thus let E be the elementary matrix corresponding to the operation q, let F be the elementary matrix corresponding to the operation q^-1. Then

Thus E is invertible and F is the inverse of E. The lemma is proved.