Memory Allocation in UNIX:
UNIX uses C dynamic memory allocation libraries for memory allocation. Namely malloc, realloc, calloc and free are used. Functions of these liberaries are given below.
Function | Description |
malloc | allocates the specified number of bytes |
realloc | Increases or decreases the size of the specified block of memory. Reallocates it if needed |
calloc | allocates the specified number of bytes and initializes them to zero |
free | releases the specified block of memory back to the system |
Best memory allocators other than above TCMALLOC and JEMALLOC.
Gperftools tcmalloc():
gperftools is Google Performance Tools which also provides malloc() called tcmalloc. Works particularly well with threads and Standard Template Library (STL). gperftools is thread-friendly heapchecker, heap –profiler and cp-profiler. TCMALLOC is available in repository for more details please visit https://github.com/gperftools
JEMALLOC :
jemalloc is a general purpose malloc(3) implementation that emphasises fragmentation avoidance and scalable concurrency support jemalloc first came into use as the FreeBSD libc allocator in 2005, and since then it has found its way into numerous applications that rely on its predictable behaviour. In 2010 jemalloc development efforts broadened to include developer support features such as heap profiling and extensive monitoring/tuning hooks. Modern jemalloc releases continue to be integrated back into FreeBSD, and therefore versatility remains critical. Ongoing development efforts trend toward making jemalloc among the best allocators for a broad range of demanding applications, and eliminating/mitigating weaknesses that have practical repercussions for real world applications.
How to install tcmalloc:
yum list gperftools-libs yum -y install gperftools-libs
Now you will see few new libraries:
bash-4.2$ rpm -ql gperftools-libs /usr/lib64/libprofiler.so.0 /usr/lib64/libprofiler.so.0.4.14 /usr/lib64/libtcmalloc.so.4 /usr/lib64/libtcmalloc.so.4.4.5 /usr/lib64/libtcmalloc_and_profiler.so.4 /usr/lib64/libtcmalloc_and_profiler.so.4.4.5 /usr/lib64/libtcmalloc_debug.so.4 /usr/lib64/libtcmalloc_debug.so.4.4.5 /usr/lib64/libtcmalloc_minimal.so.4 /usr/lib64/libtcmalloc_minimal.so.4.4.5 /usr/lib64/libtcmalloc_minimal_debug.so.4 /usr/lib64/libtcmalloc_minimal_debug.so.4.4.5
How to install jemalloc:
yum list jemalloc yum install jemalloc.x86_64
Now you will see few new libraries:
bash-4.2$ rpm -ql jemalloc /usr/bin/jemalloc.sh /usr/lib64/libjemalloc.so.1 /usr/share/doc/jemalloc-3.6.0 /usr/share/doc/jemalloc-3.6.0/COPYING /usr/share/doc/jemalloc-3.6.0/README /usr/share/doc/jemalloc-3.6.0/VERSION /usr/share/doc/jemalloc-3.6.0/jemalloc.html
How and why to use tcmalloc or jemalloc with MySQL :
I will refer to used case. One of MySQL 5.6 instance I have been managing was struggling with memory leaks. MySQL have been using swap with all optimum recommended settings. As one of last resort I decided to use different memory allocation method and results were amazing. First take a look at swap usage with tcmalloc and with default malloc .
With default malloc()
with TCMALLOC()
Other benefits includes reduced sort activity , smooth i/o activity , less table level locks , low temp table usage , positive memory usage on MyISAM tables for meta data dictionary as show below. I also noticed the boost in performance and transactions as well because MySQL instance started generating more binary logs.
how to configure tcmalloc or jemalloc with MySQL:
there are two ways you can make MySQL use tcmalloc or jemalloc once libraries are installed.
-
configure library in my.cnf file under [mysqld_safe] as shown below.
[mysqld_safe] #malloc settings malloc-lib=/usr/lib64/libtcmalloc.so.4.4.5
or
[mysqld_safe] #malloc settings malloc-lib=/usr/lib64/libjemalloc.so.1
-
second method is to start MySQL directly using desired library on command line as show below
LD_PRELOAD=/usr/lib64/libtcmalloc.so.4.2.6 mysqld --defaults-file=/etc/my.cnf --daemonize
or
LD_PRELOAD=/usr/lib64/libjemalloc.so.1 mysqld --defaults-file=/etc/my.cnf --daemonize
I hope this will help you all, Don’t forget to like, share and comment
Thanks and Regards
Raja M Naveed