Hello World! Docker allows us to save the checkpoints of the running image with the current state of the container including process state (CPU registers, memory dumps and etc), and also it can be used to continue the container from that checkpoint. This is different from the docker commit, which only saves the state of the filesystem as an image. For more information, read this.

To get hands-on practice, I will be discussing the following two labs from AttackDefense

• Recovering File Contents from the Image (Link to Lab)
• Recovering Credentials from the Memory of Program in Checkpoint (Link to Lab)

You can use this lab to test your knowledge –  https://attackdefense.com/challengedetails?cid=1037

Recovering File Contents from the Image

In this lab you are provided with an archived image of the container which was then used as the parent image of the container and the flag file was deleted from it. So this image of course contains the flag.

Container-diff tool is developed by Google to provide diff'ing for docker images or language-specific packages. The subcommand analyze can be used to perform the operations on a single image with different modes defined by --type argument.

Let's use analyze with --type=history to get a list of all the commands executed while building the final image.

container-diff analyze -t history challenge-image.tar

As you can see there are a lot of commands used and earlier the entry point was /bin/bash, then that parent image is used to create another image starting from ENM name=docker-forensics-challenges.

See how verbose is this. Not only the current image, but you also got details and commands executed for the previous image as well. This is not the point here to discuss right now, I know that.

If you see the 5th command from the bottom, there are some changes in the filesystem with a message "C H E C K S U M  I S  F L A G", this means you have to get the file content from /bin/shell which was written into system logs.

Now export the image using tar in a new folder as shown below

Layers in the manifest.json will contain the ordered entries of the filesystem changes which will be then read by overlayfs to create the volume for the containers. The CMD command did not change any content in the file system so you can ignore it. The 4th entry from the bottom would be the file system which might contain our flag.

Extract the layer.tar archive in the new folder and you will find the flag in the ./bin/shell file as seen in the history using the container-diff command.

Recovering Credentials from the Memory of Program in Checkpoint

Docker's experimental checkpoint feature allows you to save the running state of a container and to resume the container from the same state later. This feature comes handy during docker service reboot, host machine upgrade etc. This feature is still in beta and is only allowed on Linux machines as of now.

In this lab, a stopped WordPress container is present. This container uses an "authenticator" process to authenticate with a third party database. It takes user credentials using standard input. The user has provided the credentials a few hours back to this process. Can you retrieve those credentials?

You can check the stopped contains using docker ps -a command and will find wordpress container in Exit state. To keep checkpoints for the container, they must be created, and when you delete, checkpoints are also deleted with them.

Now start the container using docker start wordpress or docker start 82973dde3ff6 command. This will start the container by executing commands provided in the entrypoint.

Have you noticed something? You won't find the authenticator process running on the container. This is because, when the container is stopped all the states of the processes except changes on the filesystem are vanished.

Stop the container and look for any checkpoint for the wordpress container using docker checkpoint ls wordpress.

You need to now start the same container while providing information about the checkpoint from where to resume. So use --checkpoint checkpoint2-initial argument in the start command.

Find the process id of the authenticator process in the host machine (because gdb is installed in the host), it will be required for dumping memory in the following steps.

You can find the following code from the ServerFault question (link attached in the reference section). I have created a bash script for you to copy and paste in the terminal. It will create the memory_dump.sh script and execute for 2256 process

cat << 'EOF' > memory_dump.sh
#!/bin/bash

grep rw-p /proc/$1/maps \
| sed -n 's/^\([0-9a-f]*\)-\([0-9a-f]*\) .*$/\1 \2/p' \
| while read start stop; do \
    gdb --batch --pid $1 -ex \
        "dump memory $1-$start-$stop.dump 0x$start 0x$stop"; \
done
EOF
bash ./memory_dump.sh 2256

The dump files will have format {PID}-{START-ADDR}-{STOP-ADDR}.dump. Sometimes file creation can be failed based on the permissions of the current user with current directory. Check whether dump files are created or not.

These files will contain binary data as well as some ASCII data. I am sure you are not interested in the binary data, so use strings tool on all the files at once to get the human readable strings from them. This will contain the password you are supposed to submit.

References

• https://stackoverflow.com/questions/34864987/committing-a-running-docker-container-with-pause-then-run-it-whats-the-imag
• https://github.com/GoogleContainerTools/container-diff
• https://serverfault.com/a/408929
• https://docs.docker.com/engine/reference/commandline/checkpoint/#description