Birdhouse Components ¶

Scheduler ¶

The scheduler component runs specific jobs on a schedule. This is similar to using the cron service but this runs in docker containers and is specifically designed to interact with the Birdhouse stack.

Available jobs ¶

Scheduler jobs can be configured by enabling optional components. Birdhouse comes with a variety of these Scheduler Jobs in the optional-components directory. To enable any of these jobs, add the relevant component directory to the BIRDHOUSE_EXTRA_CONF_DIRS variable in your local environment file.

Custom jobs ¶

To add custom jobs to the scheduler component, create a new component that mounts a .yml configuration file to the /scheduler-job-configs/ directory inside the scheduler container.

See the components listed above as examples on how to build a similar scheduler job component. For more information about the syntax of the configuration files see the documentation.

The old way to add additional jobs is to update the BIRDHOUSE_AUTODEPLOY_EXTRA_SCHEDULER_JOBS environment variable in the local environment file to contain a YAML string that describes the job to run.

Note that this method is deprecated and may be removed in the future. Please update all jobs defined in the BIRDHOUSE_AUTODEPLOY_EXTRA_SCHEDULER_JOBS variable to components.

For example a simple additional job might look like:

if [ -z "$(echo "$BIRDHOUSE_AUTODEPLOY_EXTRA_SCHEDULER_JOBS" | grep 'example job')" ]; then
  export BIRDHOUSE_AUTODEPLOY_EXTRA_SCHEDULER_JOBS="
$BIRDHOUSE_AUTODEPLOY_EXTRA_SCHEDULER_JOBS
- name: example job
  comment: basic job that echos 'something' every hour
  schedule: '1 * * * *'
  command: 'echo something'
  dockerargs: >-
    --rm --name example
"
fi

Note in the example above, the code first checks to make sure that there isn’t already a job named example job. This is because the local environment file may be read multiple times when it is loaded so it is crucial to ensure that jobs are not accidentally duplicated.

Automated Deployment ¶

This component provides automated unattended continuous deployment for the “Birdhouse stack” (all the git repos in var BIRDHOUSE_AUTODEPLOY_EXTRA_REPOS), for the tutorial notebooks on the Jupyter environment and for the automated deployment itself.

It can also be used to schedule other tasks on the Birdhouse physical host.

Everything is dockerized, the deployment runs inside a container that will update all other containers.

Automated unattended continuous deployment means if code change in the remote repo, matching the same currently checkout branch (ex: config changes, docker-compose.yml changes) a deployment will be performed automatically without human intervention.

The trigger for the deployment is new code change on the server on the current branch (PR merged, push). New code change locally will not trigger deployment so local development workflow is also supported.

Multiple remote repos are supported so the “Birdhouse stack” can be made of multiple checkouts for modularity and extensibility. The autodeploy will trigger if any of the checkouts (configured in BIRDHOUSE_AUTODEPLOY_EXTRA_REPOS) is not up-to-date with its remote repo.

A suggested “Birdhouse stack” is made of at least 2 repos, this repo and another private repo containing the source controlled env.local file and any other docker-compose override for true infrastructure-as-code.

Note: there are still cases where a human intervention is needed. See note in script deployment/deploy.sh.