The Docker Health Check command. It was a new feature added in 1.12, which came out mid 2016, the
same time that Swarm Kit and Swarm Mode were available in Docker. It was added really as a part of
that toolkit, but it still works in all the different files like the Dockerfile, the Compose file,
the docker run command uses it, the Stack files support it, the service update and service create command
support it.
It's everywhere.
I highly recommend that when you're going production, you do engage in testing options for this health
check command.
It's going to work right out of the box with an exec.
It's going to execute that command inside the container just like if you were running your own exec
command.
So, it's not running it from outside the container; it's just running it inside which means that even
simple workers that don't have exposed ports, you can run a simple command in them to validate whether
they're returning good data or whatever.
It's a simple execution of a command, which means it gets a simple return. It expects a 0 or a 1.
In Linux and Windows, you have exit codes from commands and a 0 is a good thing.
It means everything was fine.
Anything other than a 0 is going to be an error in most applications.
But in Docker, we need that application to exit a 1 specifically. We'll show in a minute how you
do that.
There's only three states to a healthcheck in Docker. It starts out with starting. Starting is the
first 30 seconds, by default, where it hasn't run a healthcheck command yet.
Then it's going to run one.
If that returns a 0, it'll start with the healthy. It'll change to the healthy option.
It'll take that command and it'll run it every 30 seconds by default again. If it ever receives an unhealthy
return, like an exit 1, then it marks it as an unhealthy container.
We have options for controlling all of this including retries.
We'll see that in a minute.
This is a much better option than we've had in the past because Docker, until now, was just making
sure the application was still running.
It didn't have any insight into whether that application was doing what it was supposed to.
Now we can do that inside the Docker container itself.
But this isn't a replacement for your third party monitoring solution.
This isn't going to give you graphs, or status over time, or any sort of third party tooling that you
would expect out of a monitoring solution.
This is about Docker understanding if the container itself has a basic level of healthy.
So, in a Nginx, it might return a localhost of the root index file. A return of 200 or 300 is fine
and gives it an exit code of 0, and it considers it healthy.
That's not a super advanced uh, you know, monitoring tool.
But if it did return a 404 or 500 error, it would then consider it unhealthy and we can do something about
that.
Where are we going to see this Docker healthcheck in the GUI?
The first place is in container ls.
It'll just see it as this new option.
It's in the middle. We'll see in a second where it'll show us one of the three states if the health check
is running, and that's how we actually know that there's a healthcheck.
That's the easiest way, at least, to know. We'll see the history, the last five of that healthcheck,
show up in the inspect for that container. And we can see some basic trend over time there.
But the docker run command does not take action on an unhealthy container.
Once the healthcheck considers a container unhealthy, docker run is just going to indicate that in the ls
command, and in the inspect, but it's not going to take action.
That's where we expect the Swarm Services to take action.
So the stacks and services will actually replace that container with a new task, on a new host possibly,
depending on the scheduler.
Even in the update command, we see a little extra bonus by using the healthchecks because the updates
will consider the healthcheck as a part of the readiness for that container before it goes and changes
the next one.
If a container comes up, but it doesn't pass its health check, then the service update won't go to
the next one. Or it'll take action based on the changes you give it.
Let's look at a few examples before we go to the command line.
This is one that we're using on docker run.
This allows us to use an existing image that doesn't have a health check in it, and we're adding
the health check in at runtime.
In this case, we're using the Elasticsearch image. You can see the command is a cURL localhost
9200, which is the port that the Elasticsearch is running on inside the container, not the published port,
but inside the container. For Elasticsearch,
there is an actual health URL.
So, we can use that here.
You'll notice the two pipes with the false at the end of that command.
And that's going to be pretty common
if using something like cURL or another tool that will send out an error code that's other than 1.
Remember when I mentioned that while ago? We need it to exit with 1 if there's a problem. Because that's
the one error code that Docker is going to do something about.
We need to make sure that in this case, a shell will always return the false 1 exit code
if there's anything coming out of that command other than 0.
It's a nice way to get around that problem.
It just so happens with cURL, cURL will give other potential error codes and we don't want it to
do that.
In the actual Docker files, we can add the same command. The format's a little bit different. But you see
that we have these options here. We have the interval, the timeout, the start period (which is new), and retries.
The interval is what you would think it is. It's, by default, every 30 seconds.
How often it's going to run this health check. The time out is how long it's going to wait before it errors
out and returns a bad code, if maybe the app is slow. The start period is a new feature that allows us now
in 17.09 and newer, to give a longer wait period than the first 30 seconds of the duration. Before, it
would always just wait the long...the interval time before it started the healthcheck.
But maybe you have a Java app, or database, or something that takes a lot longer to start.
Maybe it takes five minutes.
You could add that start period in there. It'll still do healthchecks. But what it will do is it won't
alarm on an unhealthy check until that time has elapsed.
So if you set two minutes in there, even though it's health checking every 30 seconds, it's going to only
consider it unhealthy once it's past that two minute mark. The last one there, retries, means that
we will try this health check x number of times before we consider it unhealthy.
That gives maybe a potentially unstable app a chance to come back with a healthy and recover on
its own before we consider this a truly unhealthy container.
The basic healthcheck command you would use in a Dockerfile is called HEALTHCHECK,
all capital letters there. The same format exists where if we're just doing a simple cURL of the localhost
because maybe it's PHP app or something. We can do that.
This is how you would add all those options in to a Dockerfile so you would see how I add the
timeout interval and the retries before the command itself.
The first one there for the basic command, notice I don't have to put in a CMD if I'm just giving it the
command to run. But if I want to show options, if I want to give it custom options out of the box with
the timeout and so on, then I have to specify which one is the command.
Now these aren't two different lines.
Notice the back slash on the end of the first line there.
So don't get that confused.
Here we have a simple example of what it might be like if you had a static application running inside
an Nginx server.
You could set the interval and the time out from your Dockerfile, and you would just have it simply
do a cURL command on the localhost.
If it returns a 200 or 300, it considers that fine. If it returns a 4, or 5, or something else, it considers
that an error.
You notice here that I have an exit 1, which is the same thing as a false.
I did that just to show you that certain examples on the Internet will have a false. Certain examples
will have an exit 1.
They both do the same thing.
Here's a little bit more advanced example.
In this case, we're using a PHP app that's combined with Nginx.
What I've done is, in the resources, you'll find a link to this PHP example. I've added in a custom
Nginx config file that uses Nginx and PHP-FPM status URLs.
Both of those applications have their own status page and sort of a healthcheck ping URL.
You can use those in your apps if you're using PHP or Nginx. There are two different URLs,
but you can use both of them inside the same healthcheck.
In this case, we're using just one of them, and we're throwing in the localhost/ping, which is
actually a PHP-FPM
status command, but you have to enable that inside your PHP-FPM. Again, in the resources of this lecture,
there's a link to a PHP Docker Good Defaults.
You can go check that out on a GitHub where I've shown in this example in a little bit more detail.
Next we have a Postgres example so in the Dockerfile I can use a different URL.
Here we have a Postgres application where in the healthcheck command, I'm using a command of pg isready.
Now, with different apps, there's different tools. With Postgres, it comes with a built-in tool,
that's a very simple testing of a connection to a Postgres server.
It doesn't validate that you have good data, or that your database is mounted properly. It's simply
going to say, 'Does this database server allow connections? Yes or no?'
That's a neat one that you can do out of the box.
Here's what it would look like in a composer/stack file.
Very similar.
You'll notice that the start period down there requires a different version.
Since the healthcheck command came out in 1.12, it was actually supported in 2.1 of this Compose
file.
But, if you're going to use the start period, that means you have to update your Compose file to version
3.4 in order to support that. Because the start period came out over a year later after the healthcheck
command did.
Let's start out with some simple run commands.
What we're gonna do here is we're going to start a Postgres database server without the healthcheck
because by default, it doesn't come with one.
Then we're going to run it again with a manual healthcheck command that will add at the command
line, and we'll see the difference.
Here, we're just going to call the first one p1. We'll run it detached from the official Postgres
image.
If I do a docker container ls, you'll see that there's nothing indicating a healthcheck here.
If we do that same command again, and call it p2 this time, we're going to add a health command.
This time, we're going to use the pg isready, which we talked about earlier,
to test that the connections are available on this Postgres server. We're going to tell it that the
user we need is the postgres user. We don't actually need to give it a password. It's not going to try
to log in. It's just going to try to validate. We'll use the Postgres image.
Now if we do a docker container ls, and I zoom out a little bit, you'll see that it says, 'Up 4 seconds
health is starting.'
Now we get this additional feature in our status of our ls command. It will stay in the starting
state for the default
30 seconds until it runs the healthcheck command for the first time.
Now that we've waited over 30 seconds, you'll see that it's changed to status of healthy.
If we do a docker container inspect on that p2, we'll see at the very top of that that we had
this new health status option. In this case, I've only been able to run it twice.
You can see the output there, that it's showing it's accepting connections.
All right. Let's do some service create commands to that same database, in that same test healthcheck.
What we'll see here when we do this is that there are three different states that a service goes through
on starting up.
It's preparing, which usually means it's downloading the image. It's starting, which means it's executing
the container and bringing it up. Then it's running.
Without the healthcheck command, the starting and running are very quick. They're almost instantaneous.
We'll see that here with a docker service create name p1 postgres.
Once it's done preparing by downloading the image, you'll see that it goes immediately from starting
to running, because there is no healthcheck. It doesn't have anything else to do other than start the
container and say, 'Yep. The binary is running.'
But if we do that same command, docker service create, and call it p2 like before, and give it that same
health command.
We start this service with the healthcheck command built in.
What we'll see is that it'll go from preparing to starting, and it will sit at the starting state for
the default 30 seconds until the first healthcheck runs.
This is now the Docker service expecting a healthy state before it considers this service fully
running. After the 30 seconds is over, it'll shift to the running state. Then we get the last little
verify there, just to make sure that it's considered stable, and then we're done.
You can already see, out of the box, that with services, as well as service updates,
we're going to get this extra bonus of health concept if we use these commands whenever we can.