Skip to Content
Technical Articles

Understanding containers (part 06): communication between containers

In my recent post Quickly load Covid-19 data with hana_ml and see with DBeaver I used the Jupyter notebook to connect to my SAP HANA database instance in the public cloud via its IP address. But what if I want to connect from a Jupyter notebook running in one container to the OrientDB server running in another container?

Some additional setup

If you’ve been following previous parts of this Understand Containers series, then you should have these two containers on your computer myjupyter01 and myorientdb01.

To get something meaningful we need some data in the OrientDB database and a Python client in the Jupyter environment to connect to that database.

pyorient driver

We need an OrientDB driver for Python installed in my Jupyter container from https://github.com/orientechnologies/pyorient.

docker exec myjupyter01 pip install git+https://github.com/orientechnologies/pyorient

[At the time of writing this post the proper version of the driver is not available at PyPI, therefore we need to install it from the source with
pip install git+https://github.com/orientechnologies/pyorient]

Ok, the driver has been installed in the Jupyter container.

OpenBeer database

We need some data, and — because it is Friday afternoon here already — what can be the better data to look at, if not at OpenBeer dataset?

If you haven’t played with it yet by following some of the tutorials or just by importing it as one of the sample databases in OrientDB Studio, then I prepared this one-liner to import it into the DB server running on orientdb01 container. (It was on macOS, and might not work the same way on Windows host)

BEERDIR=/orientdb/databases/OpenBeer sh -c 'docker exec myorientdb01 sh -c "mkdir -p $BEERDIR && cd $BEERDIR && wget -O- http://orientdb.com/public-databases/OpenBeer.zip | jar xv && ls $BEERDIR | wc -l"'

This command is certainly overengineered 🙂 as the same can be done in a less confusing way. But I did want to play with it a bit, and that’s the result. Some notes:

  • The database folder has to be in /orientdb/databases/,
  • I wanted to chain a few commands to be executed in the container in one line, so the only way to do it is via docker exec myorientdb01 sh -c "CMD1 && CMD2",
  • I wanted to pass the environment variable $BEERDIR to the commands, and the only way I found is by wrapping the docker command into another sh -c 'CMD0',
  • The downloadable sample database is a zip file, and — because there is no unzip available in the container — we do the trick using available jar x to uncompress a file coming from wget via stdin stream.

Once executed successfully…

…you should see this database in the OrientDB Studio.

admin / admin are default user and password for this sample db.

Once logged you should be able to run a query like below. This one is to find from the graph what the categories of beers brewed in the United States are (at the time the dataset was created ;-)) ordered by a number of beers linked to each category.

select CatName, count(*) as CatCount 
from 
  (select in(HasBrewery).out(HasCategory).cat_name as CatName 
   from Brewery 
   where country='United States' 
   unwind CatName) 
 where CatName is not null 
 group by CatName 
 order by CatCount desc

That was just the preparation for what we really want to practice today: two containers (or rather applications inside of them) talking to each other.

Containers talking to each other…

Basically, we want to get a result of the same query above, but in Jupyter (running in one container) connecting to the OrientDB (running in another container).

…using IP

Docker has a few networks preconfigured as can be seen using docker network ls.

Each container by default is assigned to the Docker network bridge and gets an IP address in that network. We can find these IP addresses by scrolling through the output of the docker network inspect bridge. Or we can use the --format option to output only what we need using a Go template.

I dunno about you, but it is the first time for me using Go templates. It took me a while to get this syntax working in the command below. I’ll be grateful if you can share some improvements in comments!

docker network inspect bridge --format '{{range .Containers}}{{.Name}} {{.IPv4Address}}; {{end}}'

So, in my environment, the IP addresses are:

  • 172.17.0.2 for myorientdb01,
  • 172.17.0.3 for myjupyter01.

Let’s open a new notebook OrientDB_OpenBeersDB in the Jupyter and try!

We imported pyorient and then used it to open OpenBeer db with the driver’s client. This client connected to server on 172.0.0.2 port 2424. This port is used by OrientDB for clients that support Network Binary Protocol, like pyorient.

Please note that we use the port from inside the container, not the one to which it might be mapped on the host running Docker!

Next, we just run a query and output its result.

import pyorient
#
client = pyorient.OrientDB("172.17.0.2", 2424)
odbclusters=client.db_open("OpenBeer", "admin", "admin")
#
categories = client.query(
    "select CatName, count(*) as CatCount from (select in(HasBrewery).out(HasCategory).cat_name as CatName from Brewery where country='United States' unwind CatName) where CatName is not null group by CatName order by CatCount desc"
)
#
for category in categories:
    print(category.oRecordData)

…using a hostname

It is not a good practice to communicate using IP addresses, because it might change. To solve this and to switch to hostnames we should use Docker’s user-defined networks.

Let’s create our own mynet01 network…

docker network create mynet01
docker network ls

…and connect our two containers to this network.

docker network connect mynet01 myorientdb01
docker network connect mynet01 myjupyter01
docker network inspect mynet01 --format '{{range .Containers}}{{.Name}} {{.IPv4Address}}; {{end}}'

Note that in the mynet01 our containers have IP addresses in the range 172.18.0.0/16, while in the default bridge network they have IP addresses are in the range 172.17.0.0./16.

What is cool, is that in the user-defined network container names are as well hostnames! Using example 3 from Understanding containers (part 03): one-shot containers we can use ping to check how it is working.

docker exec myorientdb01 ping myjupyter01
docker run -t --rm --net container:myorientdb01 nicolaka/netshoot ping myjupyter01

Let’s try from the Jupyter notebook now changing the IP address with the hostname.

client = pyorient.OrientDB("myorientdb01", 2424)

Everything works fine.

…using an alias

But using the container name as a hostname might still not be the best solution, in case we got another container with another name running our OrientDB server and serving data.

In that case, it is better to connect a container to a user-defined network using an alias. So, let’s disconnect and reconnect our container myorientdb01 to mynet01 network using --alias

docker network disconnect mynet01 myorientdb01
docker network connect --alias myorientdb mynet01 myorientdb01

…and update the code in the Jupyter notebook.

client = pyorient.OrientDB("myorientdb", 2424)

👍


We used quite a few new concepts today: not just Docker’s networks, but executing multiple commands in the container using sh -c "...&&..." and using Go templates to format the command output too.

Just like you, I am thirsty for more. But it is Friday evening, so let me quench my other thirst with … another beer. Repeating after Ray Bradbury “Beer’s intellectual…”

Cheers 🍻
-Vitaliy (aka @Sygyzmundovych)

1 Comment
You must be Logged on to comment or reply to a post.
  • Another excellent tutorial/experiment/application/learning about networking containers!

    After successfully completing it, I put a few notes, comments and questions together below.

    1. Why is there no proper OrientDB driver for Python at PyPI? That thread dates to Jan 2018. Is there a future for OrientDB 3.x or is it just an open source project for HANA graph, document, etc. tech? It seems strange it’s not in PyPI.
    2. That OpenBeer command to import it to the container is over-engineered, but it worked on Mac OS 10.15.4, although ultimately with a broken pipe (in zsh). It did seem to work, though and I can query via OrientDB Studio and Jupiter. (last output was “Cannot write to ‘-’ (Broken pipe). 221″)
    3. Go templates – completely new to me. No clue when to use it, but an interesting use to format output from “docker network inspect bridge” command. It seems that Golang is available for use since Docker is written in Go?
    4. Are you talking to me? – great reference, especially for a docker networking tutorial!
    5. My network IPs were similar, just opposite. I think the difference reflects the order of container startup. I started Jupiter first, then OrientDB so my IPs were opposite.
    6. The “docker network connect” was new to me, and the meat of this tutorial. Very easy to connect & disconnect containers to a common network for them to talk to each other!
    7. Cool use of the one-shot container to ping a container from the other using netshoot. And that container names are host names! Had to study it to understand why it’s 2 commands and couldn’t ping without netshoot. Of course the OrientDB container doesn’t include ping…
    8. Aliasing the container names seems good when have multiple containers of same type in Docker. I’m interested in how (or if) this plays out with K8S and pods in a public cloud environment like GKE. I’m thinking the aliasing is useful in local Docker networks, but not generally for broader container orchestration in Kubernetes.

    Thanks for this series of container blogs, Vitaliy! Your scenarios are very useful and insightful for those of us learning to apply these new technologies. Sincere thanks for your creative application of this tech to help us learn the fundamentals! Beer is always a good catalyst…

    Doug