C++, Numerical and Parallel Computing

This website is outdated.
Please go to http://pierre.pudlo.perso.luminy.univ-amu.fr/....

Some advices with C++

C++ is great for speed optimization. But:

1. Do not reinvent the wheel. Classical problems have already been solved. Use a widely tested implementation, that is to say a well known library. For instance, the Standard Template Library and Boost Libraries for general problems, and Armadillo for efficient linear algebra. Usually, our own implementations of classical algorithms are much more clumsy (and thus, inefficient or buggy) than what can be found in those libraries.

2. Use STL containers and avoid C-style arrays. C-style arrays are hellish: they are the major cause of memory leak and cannot be passed by value to a function. If you believe that std::vector will slow down our implementation, read the other points below. If you are doing linear algebra, use std::vector<double> or std::valarray<double> in the STL and arma::vec in Armadillo.
Never, ever use a double** (or float**) as a matrix (except, of course, if you are an expert and know exactly a good reason to use a double**): often this induces poor memory management and huge bugs...

3. Try to maintain readability of the source codes.

4. Develop unit tests (using, for instance, the boost unit test framework) while programming new classes, methodologies, procedures. This is a good way to fasten the debugging stages. This is also a first step that facilitates speed optimization.

5. If you do really care on speed:
a)Turn on aggressive optimization flags of your compiler
b)Begin with the simplest code that do the job, and then optimize (ideally with the help of a profiler to seek the parts that slow down the codes, or use unit tests to profile your code)


Why do I use C++?

1. Because any loop (for, while,...) will slow down any interpreted language. Interpreted languages such as R or python (with numpy) or scilab are convenient to quickly write a script that does not bug. But performance problems arise when one wishes to use a loop.

2. Because I hate pointers. It seems like computer scientists invented pointers to drive me crazy. My main criticism about pointers is that the compiler do not see many errors. When I program in C, I always hang time to find what part of the code causes the damn segfault. With C++, you can avoid pointers, or hide them into object’s definition.

3. Because C++ is object oriented. I know one can do some object oriented code with C. For numerical computation, the GNU Scientific Library is certainly a wonderful example of that. However, this library makes heavy use of pointers and I hate pointers.

4. Because of the Standard Template Library and the Boost Libraries.



• Stroustrup, B. (1997) The C++ Programming Language. Addison-Wesley (3rd ed.)
• Brokken, F.B. (2010) C++ Annotations. Available at http://www.icce.rug.nl/documents/cplusplus/.
• Meyers, S. (2005) Effective C++: 55 Specific Ways To Improve Your Programs And Designs. Addison-Wesley (3rd ed.)
• Meyers, S. (1995) More Effective C++: 35 New Ways to Improve Your Programs And Designs. Addison-Wesley
• Meyers, S. (2001) Effective STL: 50 Specific Ways to Improve Your Use of the Standard Template Library. Addison-Wesley


Parallel programming

I think it’s the easiest way to do parallel programming. One simply needs to add some directives to the C++ code to parallelize loops.

Concurrent Programming by hand
There exists various implementations. For instance, see the Boost Libraries at http://www.boost.org/doc/libs/?view=category_Concurrent

This is the way to use the power of your Graphical Processing Unit (GPU)