Skip to content
Rust

Add callmessage discriminant to logs where possible #9836

Sign in to view logs

Add callmessage discriminant to logs where possible

Add callmessage discriminant to logs where possible #9836

Workflow file for this run

.github/workflows/rust.yml at c15b2d9
name: Rust
# On Rust, GitHub Actions, and caching
# ===========
# Here's a list of things to keep in mind if you find yourself maintaining this
# CI:
#
# https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows#matching-a-cache-key
#
# - Always install and select the desired Rust toolchain *before* running
# `Swatinem/rust-cache`. This is because the active Rust toolchain is used as
# a cache key.
# - You can use `rustup show` to install and select the right Rust toolchain if
# you have a `rust-toolchain.toml` file:
# https://github.com/rust-lang/rustup/issues/1397.
# - When caching Rust compilation artifacts, keep in mind that different `cargo`
# commands will use different profiles
# (https://doc.rust-lang.org/cargo/reference/profiles.html). Learn what you
# can reuse between one job and another and don't assume two commands will
# just share caches without conflicts.
# - Be extremely aware of cache thrashing a.k.a. churning. GitHub Actions' cache
# allows for 10GiB of data which is easily exceeded if not careful.
# Sometimes it's better not to cache than cache excessively.
# Disabling cache writes for non-default branches altogether if cache churning
# is unacceptably high is supposed to help with this.
# - Learn cache invalidation rules of `Swatinem/rust-cache` before making
# changes, e.g. what happens when `rustc --version` changes or `Cargo.lock`
# changes (or is missing).
# - The jobs dependency tree is the way it is to accommodate for sharing caches,
# not necessarily because it makes logical sense to run one job after the
# other. This is due to the fact that we can't share caches between jobs that
# run in parallel.
# - `sccache` is a good alternative to `Swatinem/rust-cache`, but it behaves
# poorly with GHA and often incurs into cache requests rate limits. We should
# probably explore `sccache` with a different backend.
# - If a job makes good use of extra cores, consider give it a bigger machine.
# GHA larger runners increase in cost linearly with the number of cores
# (https://docs.github.com/en/billing/managing-billing-for-github-actions/about-billing-for-github-actions),
# so you're not wasting money unless several cores are sitting idle for long.
on:
# Relevant docs:
# - https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/configuring-pull-request-merges/managing-a-merge-queue#how-merge-queues-work
# - https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#merge_group
merge_group:
types: ["checks_requested"]
push:
branches: ["nightly", "stable", "dev"]
pull_request: {}
workflow_dispatch: {}
env:
CARGO_TERM_COLOR: always
CARGO_NET_GIT_FETCH_WITH_CLI: true
NO_COLOR: 1
# Automatically cancels a job if a new commit if pushed to the same PR, branch, or tag.
# Source: <https://stackoverflow.com/a/72408109/5148606>
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
# Except in `dev`, `nightly` and `stable` branches! Any cancelled job will cause the
# CI run to fail, and we want to keep a clean history for major branches.
cancel-in-progress: ${{ (github.ref != 'refs/heads/nightly') && (github.ref != 'refs/heads/stable') && (github.ref != 'refs/heads/dev') }}
jobs:
check-aggregate:
# Follows the guide at <https://github.com/re-actors/alls-green>.
runs-on: nscloud-ubuntu-22.04-amd64-2x4
timeout-minutes: 5
if: always()
needs:
# If you're modifying this workflow file and you're adding/removing a job
# which should be required to pass before merging a PR, don't forget to
# update this list!
- check
- bench_check
- rollup_tps_validation
- doctests
- hack
- hack_default_targets
- nextest
- nextest_all_features
- coverage
- cargo-deny-check-licenses
# - cargo-doc-artifact
# - deploy-github-pages
- check-demo-rollup-table-of-contents
- check-demo-rollup-bash-commands-mock-da
- check-demo-rollup-bash-commands
- check-constant-overriding-is-disabled-in-release-mode
- validate-packages-to-publish-yml
- cargo-switcheroo-default
- check-web3-js-sdk-integration
steps:
- name: Compute whether the needed jobs succeeded or failed
uses: re-actors/alls-green@release/v1
with:
# allowed-skips: deploy-github-pages
jobs: ${{ toJSON(needs) }}
check:
name: check
runs-on: namespace-profile-sov-ubuntu-24-04-amd64-16x32-dev-250gb-1-88
timeout-minutes: 60
env:
RUSTFLAGS: "-D warnings"
steps:
- uses: actions/checkout@v5
- uses: namespacelabs/nscloud-cache-action@v1
with:
cache: rust
path: ./crates/fuzz/target
# CI always operates on the entire workspace
- run: cargo switcheroo disable
shell: bash
- name: Run lint
env:
CARGO_NET_GIT_FETCH_WITH_CLI: true
run: |
if ! make lint ; then
echo "Linting or formatting errors detected, please run 'make lint-fix' to fix it";
exit 1
fi
- name: Exit if working tree is dirty
# A dirty working tree likely means that `sov-modules-schemas` was
# out of date.
# Ignore the ELF's, because of how they are exposed in SP1 with include_bytes.
run: cargo switcheroo set default && git diff --exit-code -- . ':(exclude)**/elf/**'
- name: Check exit code and print custom message
if: ${{ failure() }}
run: |
echo "Dirty working tree means probably that some of the Cargo.lock files haven't been updated. Run `make lint` to update them"
bench_check:
name: bench_check
needs: check
# Privileged feature is needed for proper `io_uring` functionality needed by NOMT
runs-on: namespace-profile-sov-ubuntu-24-04-amd64-16x32-bench-250gb-1-88;container.privileged=true;container.host-pid-namespace=true
timeout-minutes: 120
env:
RUSTFLAGS: "-D warnings"
steps:
- uses: actions/checkout@v5
- uses: namespacelabs/nscloud-cache-action@v1
with:
cache: rust
# CI always operates on the entire workspace
- run: cargo switcheroo disable
shell: bash
- run: cargo bench --no-run
env:
SOV_BENCH_BLOCKS: 1
SOV_BENCH_TXNS_PER_BLOCK: 10
SOV_BENCH_NUM_PUB_KEYS: 100
rollup_tps_validation:
name: rollup_tps_validation
needs: check
# Privileged feature is needed for proper `io_uring` functionality needed by NOMT
runs-on: namespace-profile-sov-ubuntu-24-04-amd64-16x32-bench-250gb-1-88;container.privileged=true;container.host-pid-namespace=true
timeout-minutes: 120
steps:
- uses: actions/checkout@v5
- uses: namespacelabs/nscloud-cache-action@v1
with:
cache: rust
- name: rollup bench check
env:
SOV_BENCH_BLOCKS: 10
SOV_BENCH_TXNS_PER_BLOCK: 10000
SOV_BENCH_NUM_PUB_KEYS: 100
run: sh scripts/ci_rollup_count_check.sh ##
# Check that every combination of features is working properly.
hack:
name: features
# `cargo-hack` uses the same profile as `cargo check` and doesn't require
# building dependencies, only checking them, so we can share caches
# effectively.
needs: check
runs-on: namespace-profile-sov-ubuntu-24-04-amd64-16x32-dev-250gb-1-88
strategy:
matrix:
partition: [1, 2, 3]
timeout-minutes: 120
env:
RUSTFLAGS: "-D warnings"
SKIP_GUEST_BUILD: "1"
SP1_SKIP_PROGRAM_BUILD: "true"
CARGO_HACK_PARTITION_N: ${{ matrix.partition }}
CARGO_HACK_PARTITION_M: 3
steps:
- uses: actions/checkout@v5
- uses: namespacelabs/nscloud-cache-action@v1
with:
cache: rust
# CI always operates on the entire workspace
- run: cargo switcheroo disable
shell: bash
# intentionally no target specifier; see https://github.com/jonhoo/rust-ci-conf/pull/4
- name: cargo hack
run: make check-features
# Check that every combination of features is working properly, but without
# test or bench targets so dev-dependencies are not required.
hack_default_targets:
name: features_default_targets
# `cargo-hack` uses the same profile as `cargo check` and doesn't require
# building dependencies, only checking them, so we can share caches
# effectively.
needs: check
runs-on: namespace-profile-sov-ubuntu-24-04-amd64-16x32-dev-250gb-1-88
strategy:
matrix:
partition: [1, 2, 3]
timeout-minutes: 120
env:
RUSTFLAGS: "-D warnings"
SKIP_GUEST_BUILD: "1"
SP1_SKIP_PROGRAM_BUILD: "true"
CARGO_HACK_PARTITION_N: ${{ matrix.partition }}
CARGO_HACK_PARTITION_M: 3
steps:
- uses: actions/checkout@v5
- uses: namespacelabs/nscloud-cache-action@v1
with:
cache: rust
# CI always operates on the entire workspace
- run: cargo switcheroo disable
shell: bash
- name: cargo hack
run: make check-features-default-targets
nextest:
name: nextest
# Privileged feature is needed for proper `io_uring` functionality needed by NOMT
runs-on: namespace-profile-sov-ubuntu-24-04-amd64-16x64-test-250gb-1-88;container.privileged=true;container.host-pid-namespace=true
timeout-minutes: 60
env:
RISC0_DEV_MODE: "true"
SP1_PROVER: "mock"
RUST_BACKTRACE: 1
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
steps:
- uses: actions/checkout@v5
- uses: namespacelabs/nscloud-cache-action@v1
with:
cache: rust
# CI always operates on the entire workspace
- run: cargo switcheroo disable
shell: bash
- run: make test-default-features
nextest_all_features:
name: nextest_all_features
# Privileged feature is needed for proper `io_uring` functionality needed by NOMT
runs-on: namespace-profile-sov-ubuntu-24-04-amd64-16x64-test-250gb-1-88;container.privileged=true;container.host-pid-namespace=true
timeout-minutes: 60
env:
RISC0_DEV_MODE: "true"
SP1_PROVER: "mock"
RUST_BACKTRACE: 1
steps:
- uses: actions/checkout@v5
- uses: namespacelabs/nscloud-cache-action@v1
with:
cache: rust
# CI always operates on the entire workspace
- run: cargo switcheroo disable
shell: bash
- run: make test-all
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
doctests:
name: doctests
runs-on: namespace-profile-sov-ubuntu-24-04-amd64-16x32-test-250gb-1-88
timeout-minutes: 60
env:
RUSTFLAGS: "-D warnings"
SKIP_GUEST_BUILD: "1"
SP1_SKIP_PROGRAM_BUILD: "true"
steps:
- uses: actions/checkout@v5
- uses: namespacelabs/nscloud-cache-action@v1
with:
cache: rust
# CI always operates on the entire workspace
- run: cargo switcheroo disable
shell: bash
# `cargo-nextest` does not support doctests (yet?), so we have to run them separately.
# TODO: https://github.com/nextest-rs/nextest/issues/16
- run: make doctest
coverage:
name: coverage
# TODO:try coverage 32x64
runs-on: namespace-profile-sov-ubuntu-24-04-amd64-16x64-coverage-250gb-1-88;container.privileged=true;container.host-pid-namespace=true
timeout-minutes: 90
env:
SKIP_GUEST_BUILD: "1"
SP1_SKIP_PROGRAM_BUILD: "true"
RUST_BACKTRACE: 1
CARGO_BUILD_JOBS: 4
steps:
- uses: actions/checkout@v5
- uses: namespacelabs/nscloud-cache-action@v1
with:
cache: rust
# Coverage is intentionally run without heavy crates, like demo-rollup.
- name: cargo generate-lockfile
if: hashFiles('Cargo.lock') == ''
run: cargo generate-lockfile
- name: cargo llvm-cov
run: make coverage
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Upload to codecov.io
uses: codecov/codecov-action@v3
with:
# When retry is implemented, we can enable it again: https://github.com/codecov/codecov-action/issues/926
fail_ci_if_error: false
cargo-deny-check-licenses:
runs-on: nscloud-ubuntu-22.04-amd64-2x4
timeout-minutes: 30
steps:
- uses: actions/checkout@v5
- uses: taiki-e/install-action@cargo-deny
with:
tool: [email protected]
# CI always operates on the entire workspace
- run: cargo switcheroo disable
shell: bash
- run: make cargo-deny-check-licenses
# Temporarily disabled until AWS credentials can be re-issued
# cargo-doc-artifact:
# runs-on:
# - nscloud-ubuntu-22.04-amd64-8x32-with-cache
# - nscloud-cache-tag-cache-dev
# - nscloud-cache-size-100gb
# timeout-minutes: 90
# env:
# RUSTDOCFLAGS: "-D warnings"
# SKIP_GUEST_BUILD: "1"
# SP1_SKIP_PROGRAM_BUILD: "true"
# S3_BUCKET_NAME: "sovlabs-ci-rustdoc-artifacts"
# AWS_REGION: "us-east-1"
# steps:
# - uses: actions/checkout@v5
# - uses: namespacelabs/nscloud-cache-action@v1
# with:
# cache: rust
# - uses: ./.github/actions/setup
# with:
# github_token: ${{ secrets.GITHUB_TOKEN }}
# install_zkvm: "0"
# # The docs' artifact tends to become quite large with all the
# # dependencies, so we don't include them.
# - run: make docs-generate
# - name: Add index.html
# # We're inside a Cargo workspace, so there's no index.html by default.
# # We must redirect to one of the workspace member crates' documentation.
# # <https://dev.to/deciduously/prepare-your-rust-api-docs-for-github-pages-2n5i>
# run: echo '<meta http-equiv="refresh" content="0; url=sov_rollup_interface/index.html">' > target/doc/index.html
# # https://github.com/actions/deploy-pages/issues/188
# - name: Fix assets' file permissions
# run: chmod -c -R +rX "target/doc"
# - name: Upload artifacts
# uses: actions/upload-pages-artifact@v3
# with:
# path: target/doc
# - run: mkdir "test-upload-dir"; echo "This is a test file from CI" > test-upload-dir/test_ci.txt
# - name: Configure AWS credentials
# if: github.event.pull_request.head.repo.full_name == github.repository
# uses: aws-actions/configure-aws-credentials@v4
# with:
# aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
# aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
# aws-region: ${{ env.AWS_REGION }}
# - name: Upload to doc artifacts to S3
# if: github.event.pull_request.head.repo.full_name == github.repository
# env:
# GITHUB_HEAD_REF_SAFE: ${{ github.head_ref }}
# GITHUB_REF_SAFE: ${{ github.ref }}
# run: |
# # Sanitize inputs by removing any non-alphanumeric characters
# SAFE_HEAD_REF=$(echo "$GITHUB_HEAD_REF_SAFE" | tr -cd '[:alnum:]-_')
# SAFE_REF=$(echo "$GITHUB_REF_SAFE" | tr -cd '[:alnum:]-_')
# # Use the sanitized variables
# TARGET_DIR="${SAFE_HEAD_REF:-${SAFE_REF#refs/heads/}}-doc"
# # Use quoting to prevent word splitting and globbing
# aws --quiet s3 sync target/doc "s3://${S3_BUCKET_NAME}/${TARGET_DIR}" && \
# echo "Docs are available at https://${S3_BUCKET_NAME}.${AWS_REGION}.amazonaws.com/${TARGET_DIR}/index.html";
# deploy-github-pages:
# runs-on: nscloud-ubuntu-22.04-amd64-2x4
# needs: cargo-doc-artifact
# timeout-minutes: 5
# if: github.ref == 'refs/heads/stable'
# # Grant GITHUB_TOKEN the permissions required to make a Pages deployment
# permissions:
# pages: write # to deploy to Pages
# id-token: write # to verify the deployment originates from an appropriate source
# # Deploy to the github-pages environment
# environment:
# name: github-pages
# url: ${{ steps.deployment.outputs.page_url }}
# steps:
# - name: Deploy to GitHub Pages
# id: deployment
# uses: actions/deploy-pages@v4
check-demo-rollup-table-of-contents:
runs-on: nscloud-ubuntu-22.04-amd64-2x4
steps:
- uses: actions/checkout@v5
- uses: actions/setup-node@v3
- run: npm install -g doctoc
- name: Update table of contents for mock-da
run: doctoc README.md --github --notitle
working-directory: ./examples/demo-rollup
- name: Update table of contents for celestia-da
run: doctoc README_CELESTIA.md --github --notitle
working-directory: ./examples/demo-rollup
- name: Check table of contents
# Exit status 0 means no changes were made, so the table of contents is
# up-to-date.
# You can run `make fix-readme-toc` from `examples/demo-rollup`.
run: git diff --exit-code
check-demo-rollup-bash-commands:
runs-on: namespace-profile-sov-ubuntu-24-04-amd64-32x64-demo-rollup-250gb-1-88
needs:
- check
timeout-minutes: 60
env:
SP1_SKIP_PROGRAM_BUILD: "true"
SKIP_GUEST_BUILD: "sp1"
RUST_LOG: "debug,sov_sequencer=trace,sov-celestia-adapter=trace,hyper=info,risc0_zkvm=warn,jmt=info,jsonrpsee-server=info,jsonrpsee-client=info,reqwest=info,sqlx=warn,tiny_http=warn,tower_http=info,tungstenite=info,risc0_circuit_rv32im=info,risc0_zkp::verify=info,h2=info,tower=info"
steps:
- uses: actions/checkout@v5
# CI always operates on the entire workspace
- run: cargo switcheroo disable
shell: bash
- name: Login to Docker Hub
if: github.event.pull_request.head.repo.full_name == github.repository
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- uses: namespacelabs/nscloud-cache-action@v1
with:
cache: rust
- name: Compile README.md to Bash
run: bashtestmd --input examples/demo-rollup/README_CELESTIA.md --output demo-rollup-readme.sh --tag test-ci
- run: cat demo-rollup-readme.sh
- run: chmod +x demo-rollup-readme.sh && ./demo-rollup-readme.sh
check-demo-rollup-bash-commands-mock-da:
runs-on: namespace-profile-sov-ubuntu-24-04-amd64-32x64-demo-rollup-250gb-1-88
needs:
- check
timeout-minutes: 60
env:
SP1_SKIP_PROGRAM_BUILD: "true"
SKIP_GUEST_BUILD: "sp1"
RUST_LOG: "debug,sov_stf_runner=trace,sov_sequencer=trace,hyper=info,risc0_zkvm=warn,jmt=info,jsonrpsee-server=info,jsonrpsee-client=info,reqwest=info,sqlx=warn,tiny_http=warn,tower_http=info,tungstenite=info,risc0_circuit_rv32im=info,risc0_zkp::verify=info,h2=info,tower=info"
steps:
- uses: actions/checkout@v5
- uses: namespacelabs/nscloud-cache-action@v1
with:
cache: rust
# CI always operates on the entire workspace
- run: cargo switcheroo disable
shell: bash
- name: Compile README.md to Bash
run: bashtestmd --input examples/demo-rollup/README.md --output demo-rollup-readme.sh --tag test-ci
- run: cat demo-rollup-readme.sh
- run: chmod +x demo-rollup-readme.sh && ./demo-rollup-readme.sh
check-constant-overriding-is-disabled-in-release-mode:
runs-on: namespace-profile-sov-ubuntu-24-04-amd64-16x32-test-250gb-1-88
timeout-minutes: 30
steps:
- uses: actions/checkout@v5
- uses: namespacelabs/nscloud-cache-action@v1
with:
cache: rust
- run: make check-constant-overriding-is-disabled-in-release-mode
validate-packages-to-publish-yml:
runs-on: nscloud-ubuntu-22.04-amd64-2x4
steps:
- uses: actions/checkout@v5
- run: ./scripts/validate_packages_to_publish_yml.sh
shell: bash
working-directory: .
cargo-switcheroo-default:
runs-on: namespace-profile-sov-ubuntu-24-04-amd64-16x32-dev-250gb-1-88
steps:
- uses: actions/checkout@v5
- uses: namespacelabs/nscloud-cache-action@v1
with:
cache: rust
- run: cargo switcheroo set default
# Exit status 0 means no changes were made.
- run: git diff --exit-code
check-web3-js-sdk-integration:
runs-on: namespace-profile-sov-ubuntu-24-04-amd64-32x64-demo-rollup-250gb-1-88
needs:
- check
timeout-minutes: 60
env:
SP1_SKIP_PROGRAM_BUILD: "true"
SKIP_GUEST_BUILD: "1"
CARGO_NET_GIT_FETCH_WITH_CLI: true
steps:
- uses: actions/checkout@v5
- uses: namespacelabs/nscloud-cache-action@v1
with:
cache: |
rust
pnpm
path: |
./typescript/.turbo
- uses: actions/setup-node@v4
with:
node-version: 20.x
- uses: pnpm/action-setup@v4
with:
package_json_file: ./typescript/package.json
- name: Run web3-js SDK tests against demo-rollup
# See the script for more details about what to do if job
run: |
cd examples/demo-rollup
bash ./web3-js-sdk-test-runner.sh