MySQL and memory Allocators malloc, tcmalloc and jemalloc

Memory Allocation in UNIX:

UNIX uses C dynamic memory allocation libraries for memory allocation. Namely malloc, realloc, calloc and free   are used.  Functions of these liberaries are given below.

 

Function Description
malloc allocates the specified number of bytes
realloc Increases or decreases the size of the specified block of memory. Reallocates it if needed
calloc allocates the specified number of bytes and initializes them to zero
free releases the specified block of memory back to the system

 

Best memory allocators other than above TCMALLOC and JEMALLOC.

Gperftools tcmalloc():

 

gperftools is Google Performance Tools which also provides malloc() called tcmalloc. Works particularly well with threads and Standard Template Library (STL). gperftools is thread-friendly heapchecker, heap –profiler and cp-profiler. TCMALLOC is available in repository for more details please visit https://github.com/gperftools

 

JEMALLOC : 

jemalloc is a general purpose malloc(3) implementation that emphasises fragmentation avoidance and scalable concurrency support   jemalloc first came into use as the FreeBSD libc allocator in 2005, and since then it has found its way into numerous applications that rely on its predictable behaviour.  In 2010 jemalloc development efforts broadened to include developer support features such as heap profiling and extensive monitoring/tuning hooks.  Modern jemalloc releases continue to be integrated back into FreeBSD, and therefore versatility remains critical.  Ongoing development efforts trend toward making jemalloc among the best allocators for a broad range of demanding applications, and eliminating/mitigating weaknesses that have practical repercussions for real world applications.

How to install tcmalloc:

yum  list gperftools-libs 

yum -y install gperftools-libs

Now you will see few new libraries:

bash-4.2$ rpm -ql gperftools-libs

/usr/lib64/libprofiler.so.0

/usr/lib64/libprofiler.so.0.4.14

/usr/lib64/libtcmalloc.so.4

/usr/lib64/libtcmalloc.so.4.4.5

/usr/lib64/libtcmalloc_and_profiler.so.4

/usr/lib64/libtcmalloc_and_profiler.so.4.4.5

/usr/lib64/libtcmalloc_debug.so.4

/usr/lib64/libtcmalloc_debug.so.4.4.5

/usr/lib64/libtcmalloc_minimal.so.4

/usr/lib64/libtcmalloc_minimal.so.4.4.5

/usr/lib64/libtcmalloc_minimal_debug.so.4

/usr/lib64/libtcmalloc_minimal_debug.so.4.4.5

 

How to install jemalloc:

yum list jemalloc

yum install jemalloc.x86_64

 

Now you will see few new libraries:

 

bash-4.2$ rpm -ql jemalloc

/usr/bin/jemalloc.sh

/usr/lib64/libjemalloc.so.1

/usr/share/doc/jemalloc-3.6.0

/usr/share/doc/jemalloc-3.6.0/COPYING

/usr/share/doc/jemalloc-3.6.0/README

/usr/share/doc/jemalloc-3.6.0/VERSION

/usr/share/doc/jemalloc-3.6.0/jemalloc.html

How and why to use tcmalloc or jemalloc with MySQL :

I will refer to used case. One of MySQL 5.6 instance I have been managing was struggling with memory leaks. MySQL have been using swap with all optimum recommended settings. As one of last resort I decided to use different memory allocation method and results were amazing.  First take a look at swap usage with tcmalloc and with default malloc .

With default malloc()

with TCMALLOC()

Other benefits includes reduced sort activity , smooth i/o activity , less table level locks , low temp table usage , positive memory usage on MyISAM tables for meta data dictionary as show below.  I also noticed the boost in performance and transactions as well because MySQL instance started generating more binary logs.

how to configure tcmalloc or jemalloc with MySQL:

there are two ways you can  make MySQL use tcmalloc or jemalloc once libraries are installed.

  •  configure library in my.cnf file under [mysqld_safe] as shown below.
[mysqld_safe]
#malloc settings
malloc-lib=/usr/lib64/libtcmalloc.so.4.4.5

or

[mysqld_safe]
#malloc settings
malloc-lib=/usr/lib64/libjemalloc.so.1
  • second method is to start MySQL directly using desired library on command line as show below
LD_PRELOAD=/usr/lib64/libtcmalloc.so.4.2.6 mysqld --defaults-file=/etc/my.cnf --daemonize

or

LD_PRELOAD=/usr/lib64/libjemalloc.so.1 mysqld --defaults-file=/etc/my.cnf --daemonize

 

I hope this will help you all, Don’t forget to like, share and comment

Thanks and Regards

Raja M Naveed

2 thoughts on “MySQL and memory Allocators malloc, tcmalloc and jemalloc”

  1. Hey,
    Interesting post. Though one questions that pops up in my head is. The memory allocators behaves just about the same with 8 or less CPU cores. Though Jemmalloc would probably give better memory allocation with more cores. However in your experiment above I assume you are using a single/two core and also missing side by side comparison. Perhaps you may like to produce a comparison chart with various CPU cores as an illustration that allocators have an impact with more or less CPU’s.

    https://www.percona.com/blog/2013/03/08/mysql-performance-impact-of-memory-allocators-part-2/

Leave a Reply

Your email address will not be published. Required fields are marked *