Skip to content

Conversation

@kichristensen
Copy link
Contributor

@kichristensen kichristensen commented Nov 18, 2025

What does this change

This PR introduces an experimental feature flag optimized-bundle-build that reduces bundle image sizes by approximately 54% (from ~446MB to ~204MB) by restructuring how bundles are built.

How it works

The optimization changes the build context and eliminates duplicate layers:

Before (legacy build):

  • Build context: project root directory
  • Creates duplicate layers: copies .cnab directory twice (once as . then again as .cnab)
  • Uses RUN commands to delete files and set permissions (creates additional metadata layers)
  • Result: ~446MB bundle images

After (optimized build):

  • Build context: .cnab directory
  • Uses named build context userfiles to selectively copy user files
  • Uses COPY --chown and COPY --chmod to set permissions in a single layer
  • Eliminates duplicate 121MB layer from copying .cnab twice
  • Eliminates 121MB metadata layer from RUN chgrp/chmod commands
  • Result: ~204MB bundle images

Enabling the feature

Set the experimental flag via environment variable:

export PORTER_EXPERIMENTAL=optimized-bundle-build
porter build

Or in your Porter config (~/.porter/config.toml):

experimental = ["optimized-bundle-build"]

Migration for custom Dockerfiles

If you use a custom Dockerfile, you'll need to update the COPY syntax:

Legacy syntax:

COPY . ${BUNDLE_DIR}

Optimized syntax:

COPY --from=userfiles --link . ${BUNDLE_DIR}/

See updated documentation in docs/content/docs/bundle/custom-dockerfile.md for complete migration guide.

Example

# Enable the feature
export PORTER_EXPERIMENTAL=optimized-bundle-build

# Build a bundle
porter build

# Check the resulting image size
docker images | grep mybundle
# Before: mybundle:v0.1.0  446MB
# After:  mybundle:v0.1.0  204MB

What issue does it fix

This is an optimization enhancement that doesn't fix a specific issue. Bundle images can be unnecessarily large due to duplicate layers in the Dockerfile build process. This change provides an opt-in experimental feature to reduce image sizes significantly.

Notes for the reviewer

  • The feature is opt-in via experimental flag to maintain backward compatibility
  • All existing tests pass in legacy mode (default behavior unchanged)
  • Added comprehensive test coverage:
    • Unit tests for both build modes (pkg/build/dockerfile-generator_test.go)
    • Smoke tests validate both modes end-to-end (tests/smoke/*_test.go)
    • Custom Dockerfile handling tested with both modes
  • Documentation updated to explain both build modes and migration path
  • The optimization eliminates two major sources of bloat:
    1. Duplicate .cnab directory copy (121MB saved)
    2. Metadata layer from RUN chgrp/chmod (121MB saved)

Breaking change considerations:

  • Users with custom Dockerfiles will need to update COPY syntax when enabling the flag
  • Migration is straightforward and documented
  • Legacy mode remains the default until feature is proven stable

Checklist

  • Did you write tests?
  • Did you write documentation?
  • Did you change porter.yaml or a storage document record? Update the corresponding schema file.
  • If this is your first pull request, please add your name to the bottom of our Contributors list. Thank you for making Porter better! 🙇‍♀️

@kichristensen kichristensen marked this pull request as ready for review November 26, 2025 09:24
@kichristensen kichristensen requested a review from a team as a code owner November 26, 2025 09:24
Change build context from project root to .cnab directory and use
named build context for user files. This eliminates duplicate COPY
layers and unnecessary RUN commands for file deletion.

Key changes:
- Build from .cnab directory instead of project root
- Add 'userfiles' named context for accessing user bundle files
- Use COPY --from=userfiles to selectively copy user files
- Use COPY --chown and --chmod flags to set permissions in one step
- Remove redundant RUN rm commands (handled via .dockerignore)
- Eliminate 121MB duplicate layer from copying .cnab twice
- Eliminate 121MB metadata layer from chgrp/chmod RUN command

Result: Reduces bundle image size by ~54% (446MB → 204MB)

Signed-off-by: Kim Christensen <kimworking@gmail.com>
Signed-off-by: Kim Christensen <kimworking@gmail.com>
Update documentation and templates to reflect the change from
project root to .cnab directory as the Docker build context,
with userfiles named context for bundle source files.

Changes:
- Update template.buildkit.Dockerfile to use COPY --from=userfiles
- Add Build Context section to custom-dockerfile.md explaining:
  * How .cnab is used as build context
  * The userfiles named context for source files
  * Examples of copying from both contexts
- Update BUNDLE_DIR documentation with correct COPY syntax
- Update CLI help for --build-context flag
- Update architecture-buildtime.md to explain build structure

These changes help users understand the new build context model
and provide migration guidance for custom Dockerfiles.

Signed-off-by: Kim Christensen <kimworking@gmail.com>
Add new experimental feature flag to enable optimized bundle
builds that reduce image size by ~54%.

When enabled:
- Uses .cnab directory as build context
- Adds userfiles named context for bundle source files
- Uses COPY --chown and --chmod flags to avoid extra layers
- Eliminates redundant RUN commands

When disabled (default):
- Preserves backward compatibility
- Uses project root as build context
- Works with existing custom Dockerfiles

Signed-off-by: Kim Christensen <kimworking@gmail.com>
Implement conditional logic to support both legacy and optimized
bundle build processes based on the optimized-bundle-build flag.

Changes in buildx.go:
- Check feature flag to determine build context path
- When enabled: use .cnab directory with userfiles named context
- When disabled: use project root (legacy behavior)

Changes in dockerfile-generator.go:
- buildCNABSection() now generates different instructions based on flag
- When enabled: optimized COPY with --chown/--chmod flags
- When disabled: legacy COPY with separate RUN commands for permissions

This ensures backward compatibility while enabling opt-in optimization.

Signed-off-by: Kim Christensen <kimworking@gmail.com>
Revert test golden files and default template to match legacy
(non-optimized) build behavior, which is now the default when
the optimized-bundle-build feature flag is disabled.

This ensures backward compatibility and that tests pass with
the default configuration.

Updated files:
- pkg/build/testdata/*.Dockerfile (all test golden files)
- pkg/templates/templates/build/buildkit.Dockerfile

These files now generate Dockerfiles with:
- COPY .cnab /cnab (instead of COPY . /cnab)
- RUN chgrp/chmod commands for permissions
- No userfiles named context

Signed-off-by: Kim Christensen <kimworking@gmail.com>
Add test cases to verify the optimized bundle build behavior
when the optimized-bundle-build experimental flag is enabled.

New test files:
- buildkit-optimized.Dockerfile: Expected output with optimized build
- custom-dockerfile-optimized-expected-output.Dockerfile: Custom
  dockerfile with optimized build

New test cases:
- TestPorter_buildDockerfile_WithOptimizedBuild: Tests default
  Dockerfile generation with optimized build flag
- TestPorter_buildCustomDockerfile_WithOptimizedBuild: Tests custom
  Dockerfile generation with optimized build flag

These tests verify that when the flag is enabled:
- Build context uses .cnab directory
- COPY uses --from=userfiles for bundle files
- COPY uses --chown and --chmod for permissions
- No separate RUN commands for permissions

Signed-off-by: Kim Christensen <kimworking@gmail.com>
Update buildPorterSection() to conditionally copy user files based
on the optimized-bundle-build feature flag:

- When optimized build enabled: Use COPY --from=userfiles
- When disabled (legacy): Use COPY . with RUN rm porter.yaml

Also updated default template to remove hardcoded COPY line,
as it's now generated dynamically by buildPorterSection().

Updated test golden files to match new output format.

All build tests now pass successfully.

Signed-off-by: Kim Christensen <kimworking@gmail.com>
Update documentation to explain both build modes and the
experimental optimized-bundle-build feature flag.

Updated files:
- custom-dockerfile.md: Added sections explaining default vs
  optimized build modes, migration requirements, and usage
  examples for both modes
- template.buildkit.Dockerfile: Added comments explaining when
  to use each COPY syntax based on feature flag
- architecture-buildtime.md: Updated to document both build
  modes, their differences, and benefits

Key documentation points:
- Feature flag is experimental and opt-in
- Optimized build reduces image size by ~54%
- Migration guide for existing custom Dockerfiles
- Clear distinction between legacy and optimized behavior

Signed-off-by: Kim Christensen <kimworking@gmail.com>
Convert TestHelloBundle to run with both default and optimized
build modes using table-driven test cases.

The test now runs twice:
1. With default build mode (backward compatible)
2. With optimized-bundle-build experimental flag enabled

This ensures that both build modes work correctly end-to-end,
building and executing the same bundle successfully.

The optimized build flag is set via PORTER_EXPERIMENTAL environment
variable, which is the standard way to enable experimental features
in Porter.

Signed-off-by: Kim Christensen <kimworking@gmail.com>
Update all smoke tests to validate both default and optimized
build modes using table-driven test patterns. The tests now
ensure the experimental optimized-bundle-build feature flag
works correctly in end-to-end scenarios.

- Convert TestDesiredState to use table-driven tests with both
  build modes
- Extend TestAirgappedEnvironment test cases to include both
  build modes for each registry configuration scenario
- Use PORTER_EXPERIMENTAL environment variable to enable the
  optimized build mode during test execution

Signed-off-by: Kim Christensen <kimworking@gmail.com>
Add support for switching between default and optimized custom
Dockerfiles when testing bundles with custom Dockerfiles. The
mybuns test bundle uses a custom Dockerfile that must be updated
to use the userfiles named context when the optimized build flag
is enabled.

Changes:
- Create Dockerfile-optimized.tmpl for mybuns test bundle
- Update smoke tests to dynamically switch which Dockerfile to use
  based on the enableOptimizedBuild flag
- Tests now properly handle custom Dockerfiles for both build modes
- Ensures backward compatibility while testing new optimized builds

Signed-off-by: Kim Christensen <kimworking@gmail.com>
The optimized-bundle-build refactoring inadvertently changed the
legacy build behavior to unconditionally remove porter.yaml from
the bundle directory. This fails when using -f to specify a manifest
outside the build context (e.g., -f testdata/bundles/signing/porter.yaml).

Restore the original logic that only removes the manifest if it exists
within the build context, using the relative path to the manifest.

Signed-off-by: Claude <noreply@anthropic.com>
Signed-off-by: Kim Christensen <kimworking@gmail.com>
Signed-off-by: Kim Christensen <kimworking@gmail.com>
Copy link
Member

@dgannon991 dgannon991 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple of questions, happy to be told they're not relevant :D

Rename the named context from 'userfiles' to
'porter-internal-userfiles' to reduce the risk of collision with
user-defined build contexts. Add validation to prevent users from
defining a build context with the reserved name when using the
optimized-bundle-build experimental feature.

Changes:
- Rename context from 'userfiles' to 'porter-internal-userfiles'
  across all code, tests, and documentation
- Add validation in buildx.go to detect and reject reserved name
  collision
- Add comprehensive test coverage for the validation logic
- Update all Dockerfile templates and golden test files
- Update documentation to reflect the new context name

The validation provides a clear error message directing users to
rename their build context if they attempt to use the reserved name.

Signed-off-by: Kim Christensen <kimworking@gmail.com>
// Use optimized build context when feature flag is enabled
if b.IsFeatureEnabled(experimental.FlagOptimizedBundleBuild) {
// Validate that user hasn't defined a "porter-internal-userfiles" named context
if _, exists := buildContexts["porter-internal-userfiles"]; exists {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if _, exists := buildContexts["porter-internal-userfiles"]; exists {
if _, exists := namedContexts["porter-internal-userfiles"]; exists {

Should this be checking namedContexts instead of buildContexts?

Signed-off-by: Kim Christensen <kimworking@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In progress

Development

Successfully merging this pull request may close these issues.

2 participants