Avoiding pitfalls running Mongo 3.2 in Docker on OSX

Quick post after smashing my head off the desk for 5 hours trying to understand YTF I couldn’t get Mongo 3.2 running in a Docker container on my Mac.  Hopefully this post spares others the elastoplasts 🙂

logo-gray-1024x512-993867aa92

Pitfall #1 – Mongo cannot write data to the host OSX machine

There are three factors that cause this issue:

  1. A lot of mongo/docker docs tell you to mount a local volume
  2. By default docker-machine uses the virtualbox driver to create a local VM
  3. Mongo uses mmap to turbo-charge access to files on disk

These facts led me (by way of this issue) to the fact that there’s been an ancient known issue on VirtualBox that basically says docker ain’t going to be able to read/write files with mmap.

So it’s pointless trying to tell mongo, via docker, to mount a local data volume, as the above bug means mongo isn’t going to be able to access it.  (careful reading of the Docker Hub mongo docs may allow you to divine this, but it’s not at all obvious)

Solution – Create a data volume in the container rather than the host.

Like always once you understand the problem, the solution is simple.  We can tell docker to create a data volume in the container rather than the host, as follows:

In my (still) early days with docker I’d come across these cli commands which contained absolutely no description of WTF all the switches are doing, so a quick breakdown:

  • –name my-local-mongo – This tells docker to give the container the name “my-local-mongo” rather than generating a name like tuftybeaver
  • -v mongo-data:/data/db – This is the key switch, here we tell docker to create a new data volume called mongo-data and mount it as /data/db which is the default location mongo wants to write data to
  • -p 27017:27017 – Here we tell docker to bind the container port 27017 to port 27017 on the host vm, allowing us to access docker locally.
  • -d – This switch tells docker to run the process detached so we’re not running interactively.

If all has gone well docker should be running your mongo container successfully.  A quick docker ps will show your running container.

docker logs -f my-local-mongo – will allow you to follow the log files

Pitfall #2 – I can’t connect to mongo from localhost

This is the classic docker school boy error, that everyone makes.  The first time you do this you’ll be patting yourself on the back as you type localhost:27017 into your connection string and then wonder why the hell docker isn’t found.

This is because everything is actually running on your local docker virtual machine.  To find the IP of the VM enter docker-machine ls

You should see the URL listed something like this tcp://192.168.99.100:2376 You need to snarf this IP for your connection string.  The easiest way to sanity check that’s it all working is bash it into your browser, eg for this example:

http://192.168.99.100:27017/

The mongo database should helpfully respond with:

It looks like you are trying to access MongoDB over HTTP on the native driver port.

Demonstrating that your mongo db is running and responding to idiots trying to speak to it over HTTP 🙂

Pitfall #3 – No collections are displayed in RoboMongo

This was a real head scratcher, I was pretty sure everything was working, and yet RoboMongo wasn’t showing a damn thing.  This was a classic case of me Sherlock Holmes style spending 2 hours eliminating the impossible, to find whatever remains, however improbable, must be the truth

So in a turn up for the books, there was no bug in my code!!  It turns out by default Mongo 3.2 uses the WiredTiger storage engine, and RoboMongo can’t parse data stored in this format.

So instead if you’re using Mongo 3.2 you should use MongoChef to connect to your db.  MongoChef will show you all your lovely collections, living within your containerised Mongo.

Hope the above helps, thanks 🙂

 

Advertisements

9 thoughts on “Avoiding pitfalls running Mongo 3.2 in Docker on OSX

  1. Re: Problem #2, I’ve found that changing the Secondary Network adapter mode in VirtualBox gives you docker containers with their own DHCP address from your network (using bridged adapter in my case). This has been useful for sending data from hosts to monitoring dashboards or other reporting tools directly to the docker containers. The part I’ve struggled with, is that I believe the boot2docker iso image is by default configured to use host-only network in VB, so you have to change this by default every time you spin up the VB VM.

  2. Hey Iain. What was the symptom you were seeing when launching the container pointing the volume to the shared folder? I am trying to do that with VMware AppCatalyst to see if the behaviour is different but when I launch the container with….

    > docker run –name mongo -v /mnt/hgfs/dbpath:/data/db -p 27017:27017 -d mongo

    (where /mnt/hgfs/dbpath is a shared folder on my Mac visible in the Docker host running on AppCatalyst)

    …. it just exits.

    If I try to launch busybox with …

    > docker run –name busybox -v /mnt/hgfs/dbpath:/data -i -t busybox

    … it just runs fine and from within the container I can see in /data the Docker host folder /mnt/hgfs/dbpath.

    I am trying to understand if I am seeing your same problem (i.e. AppCatalyst has a similar limitation) or if it’s something else.

    Thanks.

    Massimo.

    P.S. love your writing style. 🙂

    1. Hi Massimo, sounds like a similar problem to the one I had. Virtualbox only has permission to write to directories living under /Users. Even if you chmod -777 /mnt/hgfs/dbpath it still won’t make any difference.

      Try:

      > docker run -name busybox -v /Users/massimo/data:/data -it busybox

      Where you created a data dir under your /Users dir. Hope it helps and glad you like the blog 🙂

      1. Hi Iain.

        That’s what I have already done but it’s a multiple hope.

        I have a folder on my laptop that is:

        /Users/mreferre/mongo

        I have then started a Linux VM on my laptop and shared this folder with this VM (via the AppCatalyst shared folders mechanisms) and it appears inside the VM as :

        /mnt/hgfs/dbpath

        If I run busybox with

        > docker run –name busybox -v /mnt/hgfs/dbpath:/data -it busybox

        It works just fine. From within busybox I can browse inside /data and if I create a file it shows up in “/mnt/hgfs/dbpath” in my Docker container VM (and hence it shows up in “/Users/mreferre/mongo” on my laptop since “/mnt/hgfs/dbpath” is a mount of “/Users/mreferre/mongo”).

        However, if I try to run

        > docker run –name mongo -v /mnt/hgfs/dbpath:/data/db -p 27017:27017 -d mongo

        it just exits. If I try to run it interactively ( -it ) it says:

        > chown: changing ownership of `/data/db’: Operation not permitted

        Google is full of similar issues in vm environments trying to share folders with the host. E.G. : https://github.com/boot2docker/boot2docker/issues/587

  3. Thanks for the post – it’s helpful!
    FWIW – I’ve pretty much given up on VirtualBox (on OSX, at least) and now use only VMware Fusion: too many networking issues which, let’s be honest, I can’t see Oracle fixing anytime soon.

    A quick note, though: if you use `docker-machine ip ` it will just give you the IP address, which if you’re bash-ing away can be useful, doing something like:

    curl -d foobar http://$(docker-machine ip myhost):${PORT}/foo/bar

  4. FWIW on OSX 10.11.5 + Docker native beta (1.11.1-beta12) you can create the data volume on the Mac. Bizarrely it gets the UID and GID of the running user + any sticky bits. (compared to native Docker on linux where the data volume and files get UID and GID 999)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s