C++ FAQ Celebrating Twenty-One Years of the C++ FAQ!!!
(Click here for a personal note from Marshall Cline.)
Section 13:
[13.11] Why shouldn't my Matrix class's interface look like an array-of-array?

Here's what this FAQ is really all about: Some people build a Matrix class that has an operator[] that returns a reference to an Array object (or perhaps to a raw array, shudder), and that Array object has an operator[] that returns an element of the Matrix (e.g., a reference to a double). Thus they access elements of the matrix using syntax like m[i][j] rather than syntax like m(i,j).

The array-of-array solution obviously works, but it is less flexible than the operator() approach. Specifically, there are easy performance tuning tricks that can be done with the operator() approach that are more difficult in the [][] approach, and therefore the [][] approach is more likely to lead to bad performance, at least in some cases.

For example, the easiest way to implement the [][] approach is to use a physical layout of the matrix as a dense matrix that is stored in row-major form (or is it column-major; I can't ever remember). In contrast, the operator() approach totally hides the physical layout of the matrix, and that can lead to better performance in some cases.

Put it this way: the operator() approach is never worse than, and sometimes better than, the [][] approach.

  • The operator() approach is never worse because it is easy to implement the dense, row-major physical layout using the operator() approach, so when that configuration happens to be the optimal layout from a performance standpoint, the operator() approach is just as easy as the [][] approach (perhaps the operator() approach is a tiny bit easier, but I won't quibble over minor nits).
  • The operator() approach is sometimes better because whenever the optimal layout for a given application happens to be something other than dense, row-major, the implementation is often significantly easier using the operator() approach compared to the [][] approach.

As an example of when a physical layout makes a significant difference, a recent project happened to access the matrix elements in columns (that is, the algorithm accesses all the elements in one column, then the elements in another, etc.), and if the physical layout is row-major, the accesses can "stride the cache". For example, if the rows happen to be almost as big as the processor's cache size, the machine can end up with a "cache miss" for almost every element access. In this particular project, we got a 20% improvement in performance by changing the mapping from the logical layout (row,column) to the physical layout (column,row).

Of course there are many examples of this sort of thing from numerical methods, and sparse matrices are a whole other dimension on this issue. Since it is, in general, easier to implement a sparse matrix or swap row/column ordering using the operator() approach, the operator() approach loses nothing and may gain something — it has no down-side and a potential up-side.

Use the operator() approach.