Saturday, December 5, 2015

Building MongoDB 3.0.x for ARM (armv7l) platforms

MongoDB does not officially support the 32-bit ARM platform. See the MongoDB Jira discussions for details.

Also note that for some Linux distributions of ARM, you can install MongoDB directly with their package managers, e.g. on Arch Linux: "pacman -Sy mongodb".  (But the latest build on Arch Linux ARM seems to be having issues.)

Updates (2016-01-18): Please note that the latest Arch Linux ARM package of MongoDB 3.2 seems to be working fine again.

Here are steps to compile MongoDB on 32-bit ARM.  I did it on my UDOO board with Arch Linux ARM.  But the procedure should be similar for other armv7l boards (e.g. Beaglebone Black).

While there are tutorials on how to compile MongoDB on ARM machines, most of them are based on fork such as this or this.  But these forks are pretty old (version 1.x or 2.x only).  Since I wanted to setup an ARM replica of my x86 primary running MongoDB 3.0.7, I decided to try to compile directly from the MongoDB git source.


Preparation

I will be compiling the source directly on the ARM board instead of cross-compiling.

Depending on your hardware and available RAM, you may need to add a swap file in order to build MongoDB successfully.  For reference, with the UDOO setup of 1GB of RAM, 1GB of swap space and swappiness equals to 1, the compilation process could hit 500MB of swap usage.

Make sure we have the latest software and compilers etc.  On the Arch Linux ARM, that means running:


sudo pacman -Syu
sudo pacman -Sy base-devel scons git-core openssl boost boost-libs


If you are using other Linux distributions, you may need other packages instead.  e.g. for Debian / Ubuntu with apt-get, you probably need "build-essential" instead of "base-devel":


sudo apt-get update
sudo apt-get upgrade
sudo apt-get install build-essential libssl-dev git scons libboost-filesystem-dev libboost-program-options-dev libboost-system-dev libboost-thread-dev



Get the MongoDB source

Get the MongoDB source from github and checkout the appropriate version.  I will be compiling version 3.0.7:


mkdir ~/test

cd ~/test

git clone git://github.com/mongodb/mongo.git
cd mongo
git checkout r3.0.7


Configure the V8 build for ARM

MongoDB depends on the V8 Javascript engine.  There are two versions of V8 in the source tree:  "mongo/src/third_party/v8" and "mongo/src/third_party/v8-3.25".  While the 3.25 version of V8 supports both ARM and ARM64, the SConscript is missing definitions for the 32-bit ARM.

(Note that 3.25 is not the default version.  There are problems with some of the memory tests.  And it seems that MongoDB will move to use SpiderMonkey soon)

Edit the file mongo/src/third_party/v8-3.25/SConscript to add definitions for "armv7l" accordingly (or you may get the patched file here):

@@ -77,6 +77,9 @@ LIBRARY_FLAGS = { 
    'arch:arm64': { 
      'CPPDEFINES':   ['V8_TARGET_ARCH_ARM64'], 
    }, 
+    'arch:arm': { 
+      'CPPDEFINES':   ['V8_TARGET_ARCH_ARM'], 
+    }, 
  }, 
  'msvc': { 
    'all': { 
@@ -308,6 +311,27 @@ SOURCES = { 
    arm64/stub-cache-arm64.cc 
    arm64/utils-arm64.cc 
    """), 
+  'arch:arm': Split(""" 
+    arm/assembler-arm.cc 
+    arm/builtins-arm.cc 
+    arm/code-stubs-arm.cc 
+    arm/codegen-arm.cc
+    arm/constants-arm.cc 
+    arm/cpu-arm.cc 
+    arm/debug-arm.cc 
+    arm/deoptimizer-arm.cc 
+    arm/disasm-arm.cc 
+    arm/frames-arm.cc 
+    arm/full-codegen-arm.cc 
+    arm/ic-arm.cc 
+    arm/lithium-arm.cc 
+    arm/lithium-codegen-arm.cc 
+    arm/lithium-gap-resolver-arm.cc 
+    arm/macro-assembler-arm.cc 
+    arm/regexp-macro-assembler-arm.cc 
+    arm/simulator-arm.cc 
+    arm/stub-cache-arm.cc 
+    """), 
  'os:freebsd': ['platform-freebsd.cc', 'platform-posix.cc'], 
  'os:openbsd': ['platform-openbsd.cc', 'platform-posix.cc'],
   'os:linux':   ['platform-linux.cc', 'platform-posix.cc'], 
@@ -362,6 +386,8 @@ def get_options(): 
        arch_string = 'arch:x64' 
    elif processor == 'aarch64': 
        arch_string = 'arch:arm64' 
+    elif processor == 'armv7l': 
+        arch_string = 'arch:arm' 
    else: 
        assert False, "Unsupported architecture: " + processor

Also, depending on your environment, you may want to patch the code otherwise the compiled mongod and mongo will cause segmentation fault.


Compile and install

Next we can compile and install MongoDB:


cd ~/test/mongo/

scons -j 2 --ssl --wiredtiger=off --js-engine=v8-3.25 --c++11=off --disable-warnings-as-errors CXXFLAGS="-std=gnu++11" all

scons --ssl --wiredtiger=off --js-engine=v8-3.25 --c++11=off --disable-warnings-as-errors CXXFLAGS="-std=gnu++11" --prefix=/home/cho/apps/mongo install

Although the i.MX6Q CPU on my UDOO has 4 cores, I used "-j 2" instead of "-j 4" due to stability reason.  Adjust according to your CPU power / memory usage.

The WiredTiger storage engine does not support 32-bit platforms.  So we are disabling it here with "--wiredtiger=off".

The "--js-engine" parameter is to select the non-default V8 engine to use.

Standard c++11 is disabled with "--c++11=off" and the "-std=gnu++11" flag is used instead.  The Google Performance Tools (gperftools) has problem with the "-std=c++11" flag when compiled for ARM.  This could be a problem if you tried to compile latest MongoDB source as "--c++11=on" has been made as mandatory.  You probably still can disable it by editing SConstruct if necessary. Or use the system allocator ("--allocator=system") instead of tcmalloc.

The "--prefix" parameter specifies where to install MongoDB.  You probably want to specify another location.

If you only want to build the database, no need to specify "all".  Or specify "core" to build mongo, mongod, and mongos.

Depends on the toolchain you are using, during compilation, the assembler will give warnings about SWP/SWPB instructions being deprecated:

Warning: swp{b} use is deprecated for ARMv6 and ARMv7

The generated code should be fine as Linux kernel can emulate these instructions (you may run "dmesg | grep -i swp" to check).  Not sure how to avoid it (tried specifying various gcc parameters such as -mcpu and -march but seems not effective) and whether it will affect the performance or not. Anyone? (See updates below)

And you can run "cat /proc/cpu/swp_emulation" and see the emulated count.


Build the tools (optional)

You may also want to build the tools. Install Go first:

sudo pacman -Sy go


Then get the source and build:

cd ~/test
git clone https://github.com/mongodb/mongo-tools
cd mongo-tools
git checkout r3.0.7
./build.sh


(Update 2015-12-08): It turns out that the boost library included in MongoDB is pretty old and it contains inline assembly with SWP instruction. You can apply the patch to the file src/third_party/boost/boost/smart_ptr/detail/spinlock_gcc_arm.hpp

Or, use the system boost library by specifying the "--user-system-boost" parameter when building.


5 comments:

Anonymous said...

Hi Clarence, what is the implication of an old boost library with SWP, since the latest Linux ARM kernel has a patch for SWP operations? What is patched and why is it better?

- Andy

Clarence said...

Hi Andy. I didn't notice any performance issue with SWP being emulated. However, some ARM Linux distributions may not has the SWP emulation (CONFIG_SWP_EMULATE) enabled.

You can refer to the link of the boost patch to see SWP being replace with LDREX. You can also refer to the ticket here.

Or, you can choose to link MongoDB with other versions of boost.

CB Terry said...

There is a certain amount of dedication that one must have, to pull off hacks like the above. Cheers, I just got mongo 3.0.7 to compile on my Nexus...

Anonymous said...

Hello, i want to compile MongoDB for a Raspberry V3 in rasbian jessie.
I made git checkout r3.2.0
After i try to edit mongo/src/third_party/v8-3.25/SConscript , but i don't find file :(
Have you a idea to pass this problem ?
Thank you very much, and you have made a great blog !!!

Clarence said...

I haven't tried to build version 3.2, but I believe MongoDB ditched V8 and moved to SpiderMonkey. So the steps for building 3.2 will be diff...