Effective Use and Debugging of .gcloudignore, .dockerignore, and .gitignore

Ignore files are a tool to withhold certain files from being sent when transferring code to a different place.

This protects sensitive data like cryptographic keys, and reduces the transfer load to just the code needed by the recipient.

In this article, I explore the relationship between three common ignore files — .gitignore, .dockerignore, and .gcloudignore — and how they impact builds in Docker and uploads to GCloud. These three ignore files can be used together if, for example, you want to build a Docker container from an architecture different than the one on your local machine: .gitignore is a must, .dockerignore dictates which files go into the container, and (using Cloud Builds to build the container for you) you can use .gcloudignore to reduce data transfer load.

I’ll use a sample folder structure to illustrate how these different contexts interact with specific files and folders. My aim is to help deepen your understanding of these files by seeing the effects of modifying them in certain ways. I’ll also share simple tools to help you visualize which files are NOT ignored in each of those three contexts. If you're a software developer eager to learn more about how these files interact with each other, you've come to the right place!

To visualize the impact of these files, take a look at the sample folder structure that will be used throughout this post:

• The /vendor folder is excluded by all three contexts.

• The Makefile is included in all three contexts.

• From the /special folder, each context will only include a single file, excluding the rest.

To illustrate how the different contexts interact with specific files and folders, imagine an application with this structure:

.gitignore

.gitignore files specify which files should be shared in version control when uploading to a remote codebase.

Start by opening the .gitignore file, which tells git to ignore everything under `vendor/`, `dist/`, and `special/`; and not ignore `special/only-git.txt`:

To see which files are available to be included or not ignored by .gitignore, run this command to show an ordered list of new and committed files of the git repository, taken from StackOverflow:

And you should see this output:

.dockerignore

Containerizing your application with Docker allows you to build isolated systems by copying files from your project to the container, and .dockerignore prevents the files you define from being accidentally included in that process.

When containerizing the example application, I only want to use files in the dist/ folder, so it is safe to exclude the src/ folder. You can also exclude .git and .gitignore since they will no longer be useful for a deployed app. Then create a .dockerignore file that excludes `vendor/`, `src/`, and `special/` folders, but includes `special/only-docker.txt`.

The most reliable way to see which files are included is to create a Dockerfile copying everything in its context and printing the resulting list of files. Other alternatives (like using `rsync`) do not take into account the exception syntax of `!/some/path`.

As taken from this StackOverflow answer, run the following script in your terminal:

And you should see this output (trimmed):

The `special/only-docker.txt` file should be available, but not the `special/only-git.txt` nor `special/only-gcloud.txt`.

.gcloudignore

Several tools in `gcloud` involve uploading the contents of a directory to Google Cloud. In many cases, you will not want to upload certain files. If there is a `.gcloudignore` file in the root of your project, the files specified there will not be uploaded.

The `.gcloudignore` file has added functionality beyond glob patterns. It also allows you to “include” the definition of another ignore file, like `.gitignore`. This means that if you “include” the `.gitignore` file in `.gcloudignore`, everything that is not part of the git repository will not be uploaded to Google Cloud — basically borrowing the patterns of `.gitignore`.

I’ll illustrate how it works with a definition for `.gcloudignore`. In this definition, the `.gcloudignore` file ignores itself, the `.git/` hidden folder, and the `.gitignore` file; and then includes the definition of the `.gitignore` file. It also excludes the `special/` folder, but includes `special/only-gcloud.txt`. In the end it re-adds the `dist/*` folder, which was ignored as defined in the imported `.gitignore`.

This way, the only files uploaded are part of the git repository and have not been excluded by `.gitignore`. You can further specify which files to include or exclude in the upload process.

:spark: Note: the `#!include:` directive is not recursive. If the included file is itself including another one, that last one will be skipped.

You can see which files will be uploaded to google cloud by running the following command, taken from gcloud topic gcloudignore:

And you should see this output:

You should be able to only see the `only-cloud.txt` file under the `special/` folder. Nothing from the `vendor/` folder is shown, and there was no need to ignore it from the `.gcloudignore` file: it is instead borrowing that functionality from `.gitignore`.

On your project, it might make more sense to `#!include:.dockerignore` from your `.gcloudignore`; or you may have multiple `.gcloudignore` configuration files for different builds and a general one to follow DRY principles. You might even find it easier to not include any other file and to think of it as starting from scratch. Or you may have a different provider, like `.helmignore` for helm. Regardless, the mental model should be fairly similar.

Conclusion

You should now have a deeper understanding of the relationship between .gitignore, .dockerignore, and .gcloudignore files, and how they impact Docker and GCloud. By exploring these files and their interactions with specific files and folders, you’ve seen how you can use them to your advantage. With the help of the simple tools in this article, you can now easily visualize which files are being ignored in each of those three contexts. Remember to use these files wisely, and to ensure that they’re working together in harmony with your code.