In a recent project, we needed to measure the increase in memory consumption for an application process. How to obtain “the right values” for this depends on the actual scenario and, apparently, is not straight forward in all cases.
Let me first describe the scenario a little more: We want to obtain measurements for both fully serial and MPI parallel applications. These applications are run in (1) an unchanged (vanilla), (2) an instrumented version (version 1) and (3) a version, which uses LD_PRELOAD to sneak-in another library that overloads MPI functions to do additional work (version 2).
More precisely what we want:
- A way to obtain reliable measurements for the different configurations, as we are interested in the additional amount of memory we need in version 1 and version 2, when compared to vanilla.
- The max memory consumption at runtime, not regarding potential /swap memory.
- We are only interested running on a Linux operating system
Eventually, we used the rusage feature. The returned struct offers different fields related to memory. We found that for our use case, the correct value was to use the maximum resident set size (max RSS). This proved to be reliable and reasonable compared to manual calculations of the memory we assumed we require. An example code is given below.
#include <sys/resource.h>
#include <stdio.h>
#include "mpi.h"
/* Needs to be called at the end of the process */
int MPI_Finalize() {
int rank;
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
// We assume only MPI root should output memory consumption
if (rank == 0) {
struct rusage r;
getrusage(RUSAGE_SELF, &r);
printf("MAX RSS: %ld\n", r.ru_maxrss);
}
// ... Call to PMPI for actual MPI_Finalize
}