Skip to Content
Technical Articles
Author's profile photo Witalij Rudnicki

Understanding containers (part 03): one-shot containers

Thank for all the feedback — here on the Community, as well as on LinkedIn and Twitter — after the part 02 of this series. Please keep it coming!

Quick recap before we dig into today’s short post:

  • a container is a running instance of an image (to simplify for now);
  • we can execute different commands from already running container…

…like some client commands.

But what if I want to execute a client without running the whole container with a server, like we’ve done so far executing OrientDB’s console client from within a container running OrientDB server.

Example 1: an MQTT subscriber

Some of you might remember that I’ve done some work with IoT in the past, and even today using IoT as a use case for SAP Data Hub CodeJams. On of the technologies we are using there is a lightweight protocol MQTT.

There are quite a few of public servers with MQTT brokers out there. One of them — http://test.mosquitto.org/bbc/ — has a constant stream of BBC closed captions published in three different MQTT topics.

For our exercise we want to receive a stream from a topic bbc/subtitles/bbc_two_england, but without installing any client software directly on our computer. Instead we are going to use eclipse-mosquitto Docker repository.

The latest image of it contains mosquitto_sub — a simple MQTT client that will subscribe to topics and print the messages that it receives.

So, let’s run the following command…

docker run -it --rm eclipse-mosquitto mosquitto_sub -h test.mosquitto.org -t bbc/subtitles/bbc_two_england/#

…and understand the results.

We requested Docker to run a new container using a (latest by default) image of eclipse-mosquitto and to execute the mosquitto_sub client to connect to the MQTT broker test.mosquitto.org and to output messages received on the subtopics (therefore # multi-level wildcard at the end) of bbc/subtitles/bbc_two_england topic.

The output within the red arrow is coming from the Docker, while the output within the yellow is what was coming from the stream of closed captions at the time of the live BBC2 broadcast (it was the history of “fairy? lights”).

I had to focus on writing this post instead of following the story of lights, so I pressed Ctrl+C. The command execution stopped, and Docker not only stopped the container, but as well removed it, because of --rm option in the run command.

Example 2: Drop-in replacement of local commands

I did not need to install MQTT client software on my laptop, check for any of its dependencies or think how to remove it when I not need it anymore!

What’s more I can create aliases (I am using MacOS with Docker Desktop right now) to create drop-in replacement of these commands on my laptop.

Now in one terminal window let me run mosquitto_sub as if it would be a client available in my OS…

alias mosquitto_sub='docker run -it --rm eclipse-mosquitto mosquitto_sub'
mosquitto_sub -h test.mosquitto.org -t sygyzmundovych

…while in another terminal session let me run mosquitto_pub connecting to the same MQTT broker and publishing messages to the same topic sygyzmundovych from my keyboard (option -l).

alias mosquitto_pub='docker run -it --rm eclipse-mosquitto mosquitto_pub'
mosquitto_pub -h test.mosquitto.org -t sygyzmundovych -l

Same messages are received and displayed in the first window as typed and published in the second.

If we check running containers we should see two of them based on eclipse-mosquitto running.

Try to stop — using Ctrl+C for mosquitto_sub and Ctrl+D (which is EOF i.e. end-of-input) for mosquitto_pub — and to run again these mosquitto pub and sub commands to see results changing in docker ps output.

Example 3: Connecting to a container’s network

I assume that you have the container myorientdb01 (from previous posts) running, e.g. by restarting it with docker start myorientdb01, if it was stopped in the meantime.

Let’s check its hostname with docker exec myorientdb01 hostname. The hostname of my currently running OrientDB server’s container is 99fe8d28d3c3.

Usually containers are very lightweight, as well in a terms of containing only minimal set of software required to run the container. Even many usual commands, like ifconfig, you take for granted are missing. This reduces the size of the image, but as well makes troubleshooting harder should things get out of control.

Instead of installing this and other network utilities inside the running OrientDB container we will use another method: a one-shot container with required network tools connecting to running OrientDB container’s network.

There is an image described as “network trouble-shooting swiss-army container” in the public Docker Hub registry: https://hub.docker.com/r/nicolaka/netshoot. That image has a set of powerful tools that can be used to troubleshoot Docker networking issues.

Actually that container is really useful, comparing to that real 85 tools Giant Swiss Army Knife 🙂

Let’s run ifconfig using that image…

docker run -t --rm nicolaka/netshoot ifconfig

…which will pull the image from the Docker registry, create one-shot container (i.e. automatically removed after execution of ifconfig command because of --rm option).

The IP address of that one-shot container was 172.17.0.3, but the finished container got deleted. Obviously the image itself remained locally, and will not be pulled during next run of a container.

Now, let’s find the IP address of the container running OrientDB server…

docker run -t --rm --net container:myorientdb01 nicolaka/netshoot ifconfig

…by adding --net container:<containerid> to the run command…

…and the address is 172.17.0.2.

We are not going to spend more time on Docker networks right now, as it requires a separate post or two. But you can still play with other tools included in the netshoot image.

Example 4: Useless, but…

This last example is over-engineering, as we have OrientDB console available in the container with the running DB server. We used it already in the previous post.

But “Repetition is the mother of all learning!“.

This time let’s delete the previously created mydb database using one-shot container with an OrientDB console connecting to the database running on 172.17.0.2. As you’ve seen above this is the IP address of a myorientdb01 container tht I have on my laptop right now, but might be different in yours.

docker run -t --rm orientdb bin/console.sh "drop database remote:172.17.0.2/mydb root root"

And the database is gone from that OrientDB server. Make sure you use your proper password of db user root. In my case the password was root too, and now I see it might be confusing, and is clearly not a good practice 😉


We will keep digging into Docker and containers in next posts. I will tag these posts UnderstandContainers for easy search.

Stay tuned!

-Vitaliy (aka @Sygyzmundovych)

Assigned Tags

      5 Comments
      You must be Logged on to comment or reply to a post.
      Author's profile photo Douglas Maltby
      Douglas Maltby

      Another fine blog on containers, Vitaliy!

      1. Great example use of MQTT with the BBC subtitle topics. I'd love to find more useful topics, but it seems there's not a topic directory (or topic of topics?) in MQTT, yet. The only way I see to find MQTT topics is to subscribe to all topics using wildcard "#" and scan through them manually. I know this isn't an MQTT series, but do you have other good example MQTT topics?
      2. Nice use of alias. The -rm option is a new favorite docker option, especially for these one-shot containers.
      3. Netshoot is a great find and looks to be a very useful image for troubleshooting the networking with containers and kubernetes. Will explore more of the tools in netshoot.
      4. Probably not useless and illustrates one-shot container well. Did it, but with a different database, "anotherdb", to keep exploring OrientDB using "mydb".

      I'm really enjoying this series. Thanks!

      Doug

      Author's profile photo Witalij Rudnicki
      Witalij Rudnicki
      Blog Post Author

      Hi Doug. It took me quite some time to find an MQTT server with a topic with a constant feed. I thought as well that it should be so obvious to find some sample stream with e.g. temperature or air quality data... but not. If you found something interesting publicly available, then please share it too.

      Cheers!

      Author's profile photo Douglas Maltby
      Douglas Maltby

      Hi Vitaliy. I've been searching, and the best I've found is the real-time public "high frequency positioning" MQTT feed at mqtt.hsl.fi of transit vehicles from the Finnish Transport Agency. Buses, trains and tram vehicles report their route, speed, long/lat, odometer, start/stop, etc. It's all very well documented here:  https://digitransit.fi/en/developers/apis/4-realtime-api/vehicle-positions/

      The feed can be overwhelming with such a large topic tree and payload, but parsing it down to a specific train (mode), operator (0090) and train (01021) running at the moment I write this was fun! Here's my train example, using a one-shot mosquitto container client to keep this on topic. Replace "01021" with + to see all active trains running for operator 0090.

      docker run -it --rm eclipse-mosquitto mosquitto_sub -h mqtt.hsl.fi -t /hfp/v2/journey/ongoing/vp/train/0090/01021/+/1/#

      This HFP data might make for a good future IoT, geospatial CodeJam pipeline app in Data Hub. I haven't figured out how to consume from mqtt.hsl.fi using the SDH delivered MQTT operator using tcp, though. The broker mqtt.hsl.fi seems to publish in mqtt(s) or ws(s), not tcp. A further constraint in SDH Dev Edition might be access to esri-like geo data for mapping the positioning of the vehicles. Just thinking through how to apply this MQTT source for SDH experiments...

      Regardless, I wanted to highlight this excellent public MQTT topic source!

      Doug

      Author's profile photo Witalij Rudnicki
      Witalij Rudnicki
      Blog Post Author

      Thank, Douglas Maltby for sharing!

      I did request earlier this year to update the documentation and I see different schemes tcp, ssl and ws are listed in https://help.sap.com/viewer/97fce0b6d93e490fadec7e7021e9016e/Cloud/en-US/709a1b65b93d4f5f90c992b3442f92ab.html.

      I think you should be able to use wss://mqtt.hsl.fi:443/ in the operator as well, even if wss is not listed.

      Author's profile photo Douglas Maltby
      Douglas Maltby

      Excellent! I was just now able to get broker tcp://mqtt.hsl.fi:1883/ to work with the delivered SDH MQTT consumer client (after the pipeline engine restarted itself). I see wss and ssl are supported in Data Intelligence, but not sure if that also applies to SDH 2.4 Dev Ed.

      I had been considering trying to install the eclipse-mosquitto in the datahub container to create a new mosquitto client MQTT operator, but ultimately just got broker tcp://mqtt.hsl.fi:1883/ to work with a currently running train (01077) to Lentoasema- Helsinki (the airport). Here's the same via the mosquitto one-shot container

      docker run -it --rm eclipse-mosquitto mosquitto_sub -h mqtt.hsl.fi -t /hfp/v2/journey/ongoing/vp/train/0090/01077/+/1/#

      Now to figure out how plot the MQTT feed info on a map in SDH (google maps with http client?) or SAC...or maybe just save the data to hdfs or vora and plot speed over the journey time of the train.