Providers and hosts

A provider runs jobs: the VM runner's VMs and the lanes' runs. A host runs the dispatcher. Both implement a trait from ciburst_core::types. Scaleway is the only provider and host (D65); crates/scaleway is the template for another: the API clients, in-process API fakes under src/testing, and the contract tests. The config's provider.type and host.type select them.

Register a provider

1. Crate crates/<provider>, listed in members of the root Cargo.toml.Checkcargo build --workspace passes.
2. CLI, crates/cli/src/lib.rs: the provider in Modules::main_provider and PROVIDERS, with its credentials from <local>/<provider>.env or, with --ci, from CIBURST_<VAR> variables that pipeline mode reads and never writes.Checkcargo run -p ciburst -- plan --config <config> with provider.type set to it lists its steps.
3. Function: a host's function crate builds the provider from its environment, as crates/dispatch-scaleway does in Env::backends_on; crates/scaleway/build.rs vendors every path dependency of the function into the zip.Checkcargo test -p ciburst-dispatch-scaleway passes.
4. Config: vm.runtime accepts only instances today (ciburst_core::config::validate); a provider with another VM runtime adds its name there.Checka config with the new runtime validates.
5. docs/RUNBOOK.md: the <provider>.env row of the .local/ table, facts, console checks and failure modes.Checkthe developer setup works from the runbook alone.

A host registers in Modules::host and HOSTS of the CLI and needs its own function crate.

Provider: trait Provider

Step ids, run ids, run specs

• Step ids are stable across plans. Lane definitions are definition:<project id>-<lane tag>; shared resources have fixed ids such as project, bucket, runner-binary, vm:agent, vm:security-group; an older agent object to delete is vm:agent:<sha256>.
• plan and apply handle the host's lanes from the config. A definition whose project is not the host's belongs to a member and follows that member's /join; apply leaves it alone.
• A run spec (config::Pool) is either the VM runner (project None, tag the VM runner's tag) or one lane (project set, tag the lane key <project id>-<lane tag>, runner_tag() the lane tag).

Outputs

Provisioner::apply calls apply once per non-noop step and writes that call's outputs with overwrite, then calls apply(&[]) and writes its outputs without overwrite. Keys of keys::DISPATCHER_ENV (CIBURST_AGENT_SHA256) are overwritten from every call and reach the function as plain environment variables (provision::secrets_from_outputs).

• Return a secret only from the call that created or rotated it; a later call without it keeps the stored value.

Runner object

Step runner-binary: the object ciburst/runner/<ciburst_core::cloud_init::RUNNER_SHA256> in the cache bucket, public-read. apply downloads cloud_init::RUNNER_URL, refuses a SHA-256 other than RUNNER_SHA256 and uploads the binary; the step is noop when the object exists. VMs and lane runs download gitlab-runner from it and check the hash; shared_env() hands its URL to the VM's user data as CIBURST_RUNNER_URL.

VMs

StartRequest.env of the VM runner holds RUNNER_TOKEN (the host's VM runner), CIBURST_VM_SECRET, CIBURST_DISPATCH_URL and, with the S3 cache, CACHE_S3_ACCESS_KEY and CACHE_S3_SECRET_KEY.

The provider adds CIBURST_RUN_ID (known before the user data is set) and CIBURST_VM_TOKEN, which is ciburst_core::vm::token(<CIBURST_VM_SECRET>, <run id>), and renders the user data with ciburst_core::cloud_init::render. The user data downloads the runner object and the agent object and checks each against its SHA-256. The VM secret never enters user data.

/etc/ciburst/vm.env, read by the agent: CIBURST_RUN_ID, CIBURST_VM_TOKEN, CIBURST_DISPATCH_URL, CI_SERVER_URL, RUNNER_TOKEN, CIBURST_RUNNER (the VM runner's tag), CIBURST_SLOTS, CIBURST_DEFAULT_IMAGE (vm.image), CIBURST_TIMEOUT_S; with the S3 cache also CACHE_TYPE=s3, CACHE_S3_SERVER_ADDRESS, CACHE_S3_BUCKET_NAME, CACHE_S3_BUCKET_LOCATION, CACHE_S3_ACCESS_KEY, CACHE_S3_SECRET_KEY, CACHE_SHARED. The agent exits 64 naming a missing key. Members' VM runners reach the agent through the report answer, not through vm.env (spec 6.2).

A VM's state, free slots, last report and drain time live with the server (tags on Scaleway), so the function stays stateless.

Lane runs

A lane is a one-job runtime that runs the lane's image (D45, runtime jobs on Scaleway). Its definition carries:

plan and sync_lanes update a definition when any of these changes. StartRequest.env of a lane holds RUNNER_TOKEN (the lane's runner in its own project) and, with the S3 cache, CACHE_S3_ACCESS_KEY and CACHE_S3_SECRET_KEY, never CIBURST_VM_SECRET; the run also gets the labels as CIBURST_PROJECT, CIBURST_PIPELINE, CIBURST_JOB and CIBURST_STAGE.

The bootstrap needs CI_SERVER_URL, RUNNER_TOKEN, CIBURST_RUNNER_URL and CIBURST_RUNNER_SHA256, and with CACHE_TYPE=s3 the cache location and key. As root it installs missing git, curl and CA certificates with apk or apt-get (spec section 7.3); then it downloads and checks the runner, unsets the token and cache keys and execs gitlab-runner run-single with the shell executor. Optional: CIBURST_BUILDS_DIR (/builds, for a non-root user $HOME/builds), CIBURST_CACHE_DIR (/cache, $HOME/cache); CIBURST_LANE_DRY_RUN=1 installs, skips the download and prints the command with secrets masked. A failed check prints ciburst-lane: <reason> to stderr and exits 64 (docs/RUNBOOK.md, Failure modes).

Tests

• The feature testing of ciburst-core exposes ciburst_core::testing: the contract harnesses and gitlab_fake::FakeGitlab. A provider crate gets its own testing feature that enables ciburst-core/testing and holds its API fakes behind ciburst_core::http::HttpClient; its dev-dependency on itself turns the feature on (crates/scaleway/Cargo.toml).
• testing::contract::run_provider_contract fits lane runtimes. Implement ProviderFactory: backend() returns an empty backend, make(config, &backend) binds a provider to it, so a changed config is planned against existing resources. It checks plan and idempotent apply, definition:<project id>-<tag> updates and deletes, key rotation, start, list, stop and destroy (crates/core/tests/contract_fake.rs).
• testing::vm_contract::run_vm_provider_contract fits VM runtimes: start with taken gives free = slots - taken; list and capacity show the run; claim lowers free; report sets state, free and seen; a silent VM is swept by vm::sweep_reason; other runs are ignored; stop ends the run. Scaleway Instances and the core fake run it.
• sync_lanes: a join with a lane change updates its definition, a removed lane deletes it, and another project's definitions stay (crates/core/src/dispatch/members/tests.rs on the core fake; crates/scaleway/tests/provider_lanes.rs on the Serverless Jobs fake, with the 63-character name limit); apply leaves member definitions alone (crates/scaleway/tests/provider_instances.rs).

Host: trait Host

Secrets { hook_token, runner_tokens, dispatch, cache, vm_secret, sweep_token, dispatcher_env, fleet_key } comes from ciburst_core::provision::secrets_from_outputs; the host stores dispatcher_env as plain and the rest as secret environment variables, and compares the config by Config::deployment_view, so a routing change alone needs no deploy (D58).

The host's function hands every request to Dispatcher::handle_request(method, path, headers, body, now) from ciburst_core::dispatch and answers with its status and JSON body (no body for Value::Null):

A wrong or missing token answers 401 without a body. The host also needs a schedule that posts the sweep body every 30 minutes.

Apply order

preflight, provider steps, GitLab runners (the VM runner's, enabled in every further project of gitlab.projectIds, and one per host lane in the host project), fleet state (config and member records), host, GitLab webhooks, replay of waiting jobs. Outputs go to .local/outputs.env, or with --ci to project variables.