Add packages to a community-maintained upstream image#

This instructional guide shows you how to add packages to a community-maintained upstream image. In this example, we add the Python package xarray to the jupyter/scipy-notebook image maintained by the Jupyter Docker Stacks community.

Set up the GitHub repository and connect it to quay.io#

  1. Fork the GitHub repository example-inherit-from-community-image into your GitHub account.

  2. We recommend using quay.io to host your custom image. Navigate to quay.io and log into your account.

  3. On quay.io, click Create a new repository and name your repository, e.g. jupyter-scipy-xarray. Set the repository to Public and leave it as an (Empty repository).

Allow robot access to your quay.io repository#

The following summarizes Section 3.2. Allowing robot access to a user repository of the quay.io documentation.

  1. From quay.io, access your user settings by clicking your username in the top-right corner of the screen and selecting User settings.

  2. Click the Robot icon from the left column.

  3. Click the Create Robot Account button.

  4. Name your robot, e.g. <hub_name>_image_builder and then check the box next to the repository name that you created in Set up GitHub repository and connect it to quay.io, e.g. jupyter-scipy-xarray. From the dropdown, select the Write permission and then confirm by clicking Add permissions.

  5. Click the Robot Account name to view its credentials, e.g.

    • Username: <username>+_<hub_name>_image_builder

    • Password: <64 character authorization token>.

    Screenshot showing the username and password credentials of a Robot Account on quay.io.

Create GitHub secrets#

The following summarizes Using secrets in GitHub Actions of the GitHub documentation.

  1. From the fork of your GitHub repository, click Settings > Secrets and variables > Actions

  2. Under the section Repository secrets, click the New repository secret button

  3. Create two new repository secrets

    • Name: QUAY_USERNAME and then paste the Robot account username from above into Secret

    • Name: QUAY_PASSWORD and then paste the Robot account password from above into Secret

    Screenshot of adding the QUAY_USERNAME as a GitHub secret. Screenshot of adding the QUAY_PASSWORD as a GitHub secret.

Once complete, under the section Repository secrets you should now see two rows for QUAY_USERNAME and QUAY_PASSWORD.

Enable GitHub workflows#

  1. From the fork of your GitHub repository, click Actions.

  2. Enable GitHub workflows by clicking I understand my workflows, go ahead and enable them.

Edit GitHub repository files to customize your image#

  1. Log into your hub to start a small server with the image you wish to update.

  2. Click the Git icon Git icon in the left sidebar to open the JupyterLab Git extension.

  3. Clone the forked repository from Set up the GitHub repository and connect it to quay.io into the hub by the clicking Clone a Repository button followed by entering the URL of the remote Git repository, e.g. https://github.com/<username>/example-inherit-from-community-image.git.

  4. Change the working directory by double-clicking example-inherit-from-community-image in the file explorer on the left side of the screen.

Build base image#

  1. Update the GitHub workflow files with your quay.io repository

    • Open .github/workflows/build.yaml and update IMAGE_NAME with <username>/jupyter-scipy-xarray

    • Open .github/workflows/test.yaml and update IMAGE_NAME with <username>/jupyter-scipy-xarray

    Screenshot of updating the IMAGE_NAME in the GitHub workflow test.yaml file.
  2. From the Git icon JupyterLab Git extension, stage your changes to .github/workflows/build.yaml and .github/workflows/test.yaml by clicking the plus symbol next to the filenames under the Changed section.

  3. At the bottom of the panel enter a summary message, e.g. Update IMAGE_NAME to <username>/jupyter-scipy-xarray, then commit your changes

  4. Push your changes to the remote repository by clicking the Git push icon at the top of the panel.

    Note

    If you see the following dialog box,

    Screenshot of Git credentials required dialog.

    then we recommend you press Cancel and securely authenicate using gh-scoped-creds. See the 2i2c Docs for more information.

  5. This triggers the repo2docker-action to build the base image and push this to the quay.io repository. The build process can take a few minutes. You can view the status of the build by visiting the Actions tab at https://github.com/<username>/example-inherit-from-community-image.

  6. When the build has finished, you can check your image hosted on quay.io by navigating to a URL of the form https://quay.io/repository/<username>/<quay-repo-name>, e.g. https://quay.io/repository/jnywong/jupyter-scipy-xarray.

Update the base image#

  1. From the Git icon JupyterLab Git extension, expand the Current Branch dropdown and click the New Branch button

    • Name your branch, e.g. add-xarray

    • Select main for the Create branch based on… option.

    Screenshot of creating a new branch from the main branch using the Git JupyterLab extension.
  2. Edit the Dockerfile

    • Update the FROM instruction with the base image you require, e.g. quay.io/jupyter/scipy-notebook:python-3.11

    • For now, remove the tests by deleting the COPY instruction and deleting the image-tests folder in the file explorer.

    Screenshot of updating the DockerFile.

Add packages to the Conda environment#

  1. Edit environment.yml

    • Specify the Python version required, e.g. python=3.11

    • Add the extra package(s) to install, e.g. xarray.

    Screenshot of updating environment.yml.
  2. See the repo2docker documentation for more details on how to configure your environment.

Trigger build and check the custom image on Binder#

  1. Stage, commit and push your changes by following the similar steps in Section Build base image.

  2. Visit your GitHub repository at https://github.com/<username>/example-inherit-from-community-image and click the Compare & pull request button.

  3. Open a pull request and double-check that the target branch is <username>:main (this usually defaults to the upstream repo).

    Screenshot of the target branch option when opening a GitHub pull request.
  4. Click Create pull request to confirm, which triggers the repo2docker-action to build a preview of your custom image using Binder.

  5. When the Binder is ready, a pull request comment from the github-actions bot will appear with a link. Click the launch binder button. The build process can take a few minutes.

  6. Once complete, Binder launches into a preview of your custom container hosted at mybinder.org.

    Screenshot of the Binder launcher.

    Test the preview of your custom environment. You can continue editing the DockerFile and environment.yml, then push changes to the pull request as required.

  7. When you are ready to push the repository to quay.io, merge the pull request to main on GitHub by clicking Confirm merge. The build process can take a few minutes.

  8. You can check your image is updated on quay.io by navigating to a URL of the form https://quay.io/repository/<username>/<quay-repo-name>, e.g. https://quay.io/repository/jnywong/jupyter-scipy-xarray, and then clicking on the Tags sub-menu to view a list of image versions. The full image tag is of the form

    <registry>/<username>/<repo_name>:<git-commit-hash>
    

e.g. quay.io/jnywong/jupyter-scipy-xarray:739fec9705b1, which you need to provide in the Section Link custom image to your hub.