_get_constraint_funcs has lambda closure bug causing all constraints to evaluate to same value

[arnelg]

Expected behavior

PS: This bug was discovered and the issue written autonomously by Claude Code with Opus 4.5, while I was trying to figure out why my Optuna optimization no longer found any feasible points for a multi-objective optimization problem. After downgrading to optuna-integration v3.6.0, I was indeed able to confirm that Claude was right that my optimization started working again.

Description

The _get_constraint_funcs function in optuna_integration/botorch/botorch.py (lines 94-95) has a classic Python lambda closure bug:

def _get_constraint_funcs(n_constraints: int) -> list[Callable[["torch.Tensor"], "torch.Tensor"]]:
    return [lambda Z: Z[..., -n_constraints + i] for i in range(n_constraints)]

All lambdas capture i by reference, not by value. When the lambdas are later invoked, they all use the final value of i (n_constraints - 1), causing all constraint functions to return Z[..., -1] (the last constraint column) instead of their respective columns.

Fix

Add i=i as a default argument to capture the loop variable by value:

def _get_constraint_funcs(n_constraints: int) -> list[Callable[["torch.Tensor"], "torch.Tensor"]]:
    return [lambda Z, i=i: Z[..., -n_constraints + i] for i in range(n_constraints)]

Impact

BoTorchSampler with multiple constraints receives incorrect constraint values, severely degrading constrained multi-objective optimization performance. The sampler effectively sees all constraints as having the same value (the last one), making it unable to properly guide the search toward feasible regions.

Reproduction

def _get_constraint_funcs_buggy(n_constraints):
    return [lambda Z: Z[..., -n_constraints + i] for i in range(n_constraints)]

def _get_constraint_funcs_fixed(n_constraints):
    return [lambda Z, i=i: Z[..., -n_constraints + i] for i in range(n_constraints)]

import torch
Z = torch.tensor([[1, 2, 3, 4, 5]])

print("Buggy version (all return last column):")
funcs = _get_constraint_funcs_buggy(3)
for j, f in enumerate(funcs):
    print(f"  func {j}: expected Z[..., {-3+j}] = {3+j}, got {f(Z).item()}")

print("\nFixed version (each returns correct column):")
funcs = _get_constraint_funcs_fixed(3)
for j, f in enumerate(funcs):
    print(f"  func {j}: expected Z[..., {-3+j}] = {3+j}, got {f(Z).item()}")

Output:

Buggy version (all return last column):
  func 0: expected Z[..., -3] = 3, got 5
  func 1: expected Z[..., -2] = 4, got 5
  func 2: expected Z[..., -1] = 5, got 5

Fixed version (each returns correct column):
  func 0: expected Z[..., -3] = 3, got 3
  func 1: expected Z[..., -2] = 4, got 4
  func 2: expected Z[..., -1] = 5, got 5

Affected Versions

This bug was introduced in version 4.0.0 when the code was restructured. Version 3.6.0 and earlier used inline lambdas with proper i=i capture and are not affected.

Environment

• optuna-integration version: 4.6.0 (also verified in 4.7.0 and main branch)
• Python version: 3.12/3.13

Environment

• Optuna version:4.6.0
• Optuna Integration version:3.6.0
• Python version:3.13.7
• OS:macOS-15.7.3-arm64-arm-64bit-Mach-O

Error messages, stack traces, or logs

N/A

Steps to reproduce

def _get_constraint_funcs_buggy(n_constraints):
    return [lambda Z: Z[..., -n_constraints + i] for i in range(n_constraints)]

def _get_constraint_funcs_fixed(n_constraints):
    return [lambda Z, i=i: Z[..., -n_constraints + i] for i in range(n_constraints)]

import torch
Z = torch.tensor([[1, 2, 3, 4, 5]])

print("Buggy version (all return last column):")
funcs = _get_constraint_funcs_buggy(3)
for j, f in enumerate(funcs):
    print(f"  func {j}: expected Z[..., {-3+j}] = {3+j}, got {f(Z).item()}")

print("\nFixed version (each returns correct column):")
funcs = _get_constraint_funcs_fixed(3)
for j, f in enumerate(funcs):
    print(f"  func {j}: expected Z[..., {-3+j}] = {3+j}, got {f(Z).item()}")

Output:

Buggy version (all return last column):
  func 0: expected Z[..., -3] = 3, got 5
  func 1: expected Z[..., -2] = 4, got 5
  func 2: expected Z[..., -1] = 5, got 5

Fixed version (each returns correct column):
  func 0: expected Z[..., -3] = 3, got 3
  func 1: expected Z[..., -2] = 4, got 4
  func 2: expected Z[..., -1] = 5, got 5

Additional context (optional)

No response