genesis info command will get the general information that you
$ genesis info my-env ================================================================================ CONCOURSE Deployment for Environment 'my-env' Last deployed more than a month ago (06:52PM on Oct 14, 2019 UTC) by you to BOSH my-env based on kit concourse/3.7.0 using Genesis v2.6.16 with manifest .genesis/manifests/my-env.yml (redacted) -------------------------------------------------------------------------------- Web Client URL: https://10.128.80.32 username: concourse password: ... ================================================================================
If you have a need to see the find all the IPs in your concourse deployment, use bosh directly to do that.
$ bosh -e my-env -d my-env-concourse vms Using environment 'https://10.128.80.0:25555' as user 'admin' Task 1416. Done Deployment 'my-env-concourse' Instance Process State AZ IPs VM CID VM Type Active db/537db50b-e548-4af2-b799-7c2d5d2b38aa running z1 10.128.80.37 vm-d0402cb0-2a6f-4c60-a400-a0aabc20d2fe small true haproxy/8cc09e62-9ac2-44cc-8594-acca42113895 running z1 10.128.80.32 vm-b7e74ce1-50d5-4a03-9001-0b14880970c8 small true web/1543f581-167b-4791-b044-ce7eea0dd3fe running z1 10.128.80.33 vm-876eaf63-ad86-4789-87d5-93aaae99f6ec small true worker/6dcc7c21-c976-41fd-ad5a-9c86b63a95fb running z1 10.128.80.35 vm-13346811-d7f0-41af-b963-4dc0bd9c2752 concourse-worker true worker/a2d8882c-f649-4318-880d-e29612489a6c running z1 10.128.80.34 vm-71a21afa-8773-4e5a-b93c-9ff86b5b03d3 concourse-worker true worker/e4d3d23b-469c-40a0-ad16-a99fa7a8e092 running z1 10.128.80.36 vm-3b3509f3-0567-4246-8f5b-6d52105fcc51 concourse-worker true 6 vms Succeeded
haproxy instance IP matches the Web Client URL in the
output above because
haproxy is the one terminating inbound API access.
If specified, the Web Client URL will be the external domain.
There are a number of helpful Genesis addons as well:
$ genesis do my-env list Running list addon for my-env The following addons are defined: visit Open the Concourse Web User Interface in your browser (requires macOS) download-fly Get the version of fly compatible with this Concourse login Login to this Concourse deployment with fly logout Logout of this Concourse deployment with fly fly Run fly commands targetting this Concourse Deployment setup-approle Create the necessary Vault AppRole and policy for Genesis Concourse deployments.
fly, The Concourse CLI
Quickest way is with the genesis addon
genesis do my-env download-fly Running download-fly addon for my-env Downloading darwin/amd64 version of fly from https://10.128.80.32... % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 20.2M 100 20.2M 0 0 18.6M 0 0:00:01 0:00:01 --:--:-- 18.6M Download successful - written to ./fly
As you upgrade Concourse, you will also need to update your local
fly. Assuming you have logged into Concourse, you can
easily update by running
To log in from the command-line, you’ll need to have
installed. Then you can use the
$ genesis do my-env login Running login addon for my-env Logging in to Concourse deployment my-env as user 'concourse'. logging in to team 'main' target saved
You can use this addon when your token expires as well.
$ fly targets name url team expiry my-env https://10.128.80.32 main n/a: Token is expired $ genesis do my-env login Running login addon for my-env Logging in to Concourse deployment my-env as user 'concourse'. logging in to team 'main' target saved $ fly targets name url team expiry my-env https://10.128.80.32 main Sat, 23 Nov 2019 19:48:30 UTC
You can use
fly directly using the
-t my-env option to specify the target
deployment, or use
genesis do my-env fly to execute fly commands and genesis
will specify which deployment to target.
$ fly -t my-env status logged in successfully $ genesis do my-env fly status Running fly addon for my-env Running fly against my-env logged in successfully
Similarly, there is a
logout addon as well.
$ genesis do my-env logout Running logout addon for my-env logged out of target: my-env
If you are on the same machine as your browser instance (not on a jumpbox),
you can use the
visit command to get to the web UI. Or you can reach it
directly via the Web Client URL from
genesis do my-env visit Running visit addon for my-env You will need to enter the following credentials once the page opens: username: concourse password: ... Press any key to open the web console...
Select login. For Github authentication, click the login with Github button.
The first time you do this, you will need to authorize the Github endpoint to access your account information. On subsequent authentication attempts, it should be seamless.
If you are trying to access concourse on a machine without Genesis, or give someone access to Concourse without giving them access to Genesis
First thing is to install
fly. Easiest way to get the correct version
fly is from the Apple, Windows, or Linux buttons in the concourse UI.
They can be found on the bottom bar of most concourse screens (usually
towards the right). They can also be found on the welcome screen prior
fly where your Concourse is:
$ fly -t my-env login -c https://$IP -k logging in to team 'main' navigate to the following URL in your browser: https://$IP/auth/github?team_name=main&fly_local_port=$SOME_PORT or enter token manually:
If you are executing
fly from the same machine that runs your
browser instance, you can just visit the link and the Concourse
server will communicate to the
fly process through the browser.
If you are running
fly on a jumpbox or bastion host, the port
connection won’t be properly wired up (your browser can’t
communicate with the jumpbox instance of
fly). You can either
omit the trailing
&fly_local_port=... on the URL you put in the
browser, or leave it and let it fail. In either case, the browser
will display a bearer token that looks like this:
Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJjc3JmIjoiZjhkZTEzYzUxMTAzYWYyNTMzN WE3MGE3ZGQwNzdlYzUxMGQxMzA2ZjlhNGZkZGZmNzAwOGEwMzU4YzQwYzAwOSIsImV4cCI6MTU yMjE3MDU1OCwiaXNBZG1pbiI6dHJ1ZSwidGVhbU5hbWUiOiJtYWluIn0.cbhjKKzDr7V0UjpuA F0yrpr7VFdc7baKH8gJ1w5WaLYpy0oeUjdgMPbM9Th04OsOWMKxAi6yQcHDj6mWDlKNaqjJrkn yMjE3MDU1OCwiaXNBZG1pbiI6dHJ1ZSwidGVhbU5hbWUiOiJtYWluIn0.cbhjKKzDr7V0UjpuA F0yrpr7VFdc7baKH8gJ1w5WaLYpy0oeUjdgMPbM9Th04OsOWMKxAi6yQcHDj6mWDlKNaqjJrkn 4lqHk3NcudKUW-qxJatQf49iIDcIyJK4isX2qRiQMYGXEY44m3etbDd2VGoLMDsNNrUE-JHL73 OS42qZJAD-hNz2jl8zxPek8md-oRA
(newlines added for clarity)
You must copy the entire string, including the "Bearer " prefix, into the waiting prompt in terminal.
fly targets to verify:
$ fly targets name url team expiry my-env https://10.128.80.32 main Sat, 23 Nov 2019 19:48:30 UTC
From now on, all of your
fly commands will need the
option, to target correctly.
From a browser, the list of pipelines is accessed via the three-bar icon in the top-left corner of the interface, which shows the sidebar.
Clicking on a pipeline in the sidebar brings up the main page, which has all of the inputs, outputs, and jobs that comprise the workflow.
From the terminal (assuming you have installed
fly and are
logged in), you can run:
$ genesis do my-env fly pipelines name paused public blacksmith-genesis-kit no no bosh-genesis-kit no no concourse-genesis-kit no no jumpbox-genesis-kit no no shield-genesis-kit no no squid-genesis-kit no no vault-genesis-kit no no shieldproject.io no no prometheus-genesis-kit no no ... etc ...
You can list the jobs on a single pipeline as well:
$ genesis do my-env fly jobs -p concourse-genesis-kit name paused status next testflight no succeeded n/a rc no succeeded n/a minor no succeeded n/a major no n/a n/a shipit no succeeded n/a
Jobs that have never run have a status of
NOTE: This task requires that
fly is installed and that you are
To list off all of the workers that have registered with your
Concourse TSA, and their health, use
$ genesis do my-env fly workers name containers platform tags team state version 7cfecc9b-e0cd-4f8d-b771-6c26b8782de3 6 linux none none running 1.2 ac17d59e-e92e-4304-a09e-9a5356996296 19 linux none none running 1.2 e52c1b40-1171-464e-89ac-07c60cb09d5d 17 linux none none running 1.2
In sequestered network environments, you will have one contingent of workers for each separate network environment. Those workers will have tags defined that help to partition the overall concourse into zones for purposes of deployment scheduling.
From the browser, you can select a job and click on the big
+ icon on any job to
force it to run right now:
From the command-line, you can use
fly (assuming it is installed
and you are already logged in).
First, find the pipeline, and inspect its jobs:
$ genesis do my-env fly pipelines $ genesis do my-env fly jobs -p my-pipeline
Then, trigger the pipeline/job:
$ genesis do my-env fly trigger -j my-pipeline/my-job
If you pause a pipeline, it will not execute, even if one of its inputs triggers it to. This can be useful if you want to make sure Concourse isn’t building software or doing deployments in a chagne blackout window.
You can pause a pipeline from the web UI by clicking on the ⏸ icon in the top right next to your user:
To unpause, click the ▶ icon in the same location. Paused pipelines show up with a blue header in the web UI:
To pause a pipeline from the command-line, you will need to
fly and make sure you are logged in.
$ genesis do my-env fly pause-pipeline -p my-pipeline
$ genesis do my-env fly unpause-pipeline -p my-pipeline
Note: Genesis always unpauses its pipelines after you repipe them.
Concourse has a feature called hijacking or (more politically correct) intercepting, by which an operator can gain a remote shell on a check or task container and poke around.
This is handy for troubleshooting failing pipelines, but the execution is hit or miss. Concourse tends to only keep failed containers around for a brief period of time, to facilitate debugging. Successful containers are destroyed immediately.
Start by finding the job you are interested in:
$ genesis do my-env fly pipelines $ genesis do my-env fly jobs -p m-pipeline
Then, specify the pipeline/job in a call to
$ genesis do my-env fly hijack -j my-pipeline/testflight 1: build #51, step: git, type: get 2: build #51, step: notify, type: get 3: build #51, step: notify, type: put 4: build #51, step: testflight, type: task choose a container: 4 [email protected]:/tmp/build/778af108# ls / bin boot dev etc home lib lib64 media mnt opt proc root run sbin scratch srv sys tmp usr var [email protected]:/tmp/build/778af108# exit
You will note that containers in Concourse are often very bare and stripped
down. For example, in the above container, there is no
A common cause of “drive-by” failures in CI is a misbehaving worker, often caused by an out-of-disk scenario. In these cases, the easiest recourse is to forcibly recreate the workers via BOSH.
Workers have no persistent state. All of the interesting bits of Concourse are stored in the database node, which does have a persistent disk attached. Deleting and rebuilding workers has no lasting ill effects on a Concourse installation, although it does temporarily reduce capacity. Of course, if your workers are unable to perform, you are already at reduced capacity.
To recreate workers:
$ bosh -e your-env -d your-env-concourse recreate worker
This will redeploy just the
worker instance group, assuming the rest of
the Concourse deployment is healthy.
If you operate in a distributed infrastructure with sequestered networks served by workers-only deployments, you will need to target the appropriate environment BOSH director and worker deployment, but the command is otherwise identical.
It can be helpful to know what input triggered a given job to run in a pipeline, especially with deployment pipelines that seem to run for no apparent reason.
Concourse does give you a visual cue, but you have to know to look for it, and what it is. Any input that triggers a job will have a yellow/gold down arrow as opposed to a white down arrow.
Here, even though we have two inputs to this job, it was the
git resource that triggered the
rc job, not the
And now you know.
Concourse has a complete manifest that it uses to run your
pipeline. You can use
fly to retrieve this manifest.
$ genesis do my-env fly get-pipeline -p name-of-pipeline
This will dump a (probably very large) YAML document, to standard output. This YAML document describes the pipeline, in whole.
To create a new pipeline, assuming you have already written the YAML definition file (the “manifest”), all you need to do is:
$ genesis do my-env fly set-pipeline -p name-of-pipeline path/to/def.yml
Note: for Genesis deployments, you should refer to the Genesis
genesis actually manages the pipeline
definition, and has first-class support for configuring Concourse
on your behalf.
If you wish to use the pipeline features of genesis, you first need to create an approle within vault. The easiest way to do this is by running:
$ genesis do my-env setup-approle