Notation: I'm writing X→Y to mean Hom(X,Y) because I suspect the arrow notation may be a little more familiar to some readers.
Well, there are third-rank tensors, which can be kind of like "3D matrices." Matrices in the ordinary sense (i.e. a nxmarray) are special, though, in that they're able to function as linear maps between vector spaces. A nxmxl array would not map a vector (a single column) to another vector, but to a matrix. (Or possibly to a fourth-rank tensor, i.e. a "4D matrix." There are both inner and outer products.)
Matrix multiplication and inversion are pretty much composition and inversion of functions, since matrices can act as functions. You can have inversion and composition of tensors, too, if you use them as maps. (The first only exists, of course, if the map is one-to-one.) It just isn't as pretty as 2D matrices, since they map columns to columns.
As
schrodingasdawg just said, the generalization from matrices to tensors of higher order is easier to understand if you regard them both as multilinear functions.
A matrix m can be regarded as a multilinear function m:V
1xV
2→F that takes two vectors and returns a scalar from the underlying field F (typically the real or complex numbers). It can also be regarded (via lambda abstraction
WP) as a map of type V
1→(V
2→F) that takes one vector and returns an element of the vector space V
2→F, which is the dual of V
2. (Each V
i is finite-dimensional here, so V→F is isomorphic to V, which is why it's okay to think of m as a mapping from V
1 to V
2.)
Similarly, a tensor T of order 3 can be regarded as a multilinear function T:V
1xV
2xV
3→F that takes three vectors and returns a scalar. From T we can obtain (via lambda abstraction
WP) a map of type V
1xV
2→(V
3→F) that takes two vectors and returns an element of the vector space V
3→F, which is the dual of V
3.
There are lots of possibilities here. By changing the pattern of lambda abstraction, we can obtain maps of the following types:
V
1xV
2→(V
3→F)
V
2xV
1→(V
3→F)
V
1xV
3→(V
2→F)
V
3xV
1→(V
2→F)
V
2xV
3→(V
1→F)
V
3xV
2→(V
1→F)
V
1→(V
2→(V
3→F))
V
2→(V
1→(V
3→F))
V
1→(V
3→(V
2→F))
V
3→(V
1→(V
2→F))
V
2→(V
3→(V
1→F))
V
3→(V
2→(V
1→F))
V
1→(V
2xV
3→F)
V
2→(V
1xV
3→F)
V
1→(V
3xV
2→F)
V
3→(V
1xV
2→F)
V
2→(V
3xV
1→F)
V
3→(V
2xV
1→F)
If the vector spaces are finite-dimensional, then every one of those types is isomorphic to the original type of T: V
1xV
2xV
3→F. Thus a single tensor T can be used in lots of different ways, which is a source of power as well as confusion.
