NumPy 2.0 support (AttributeError: "itemset")

[jwnimmer-tri]

Is your feature request related to a problem? Please describe.

NumPy 2.0 has been released. See https://numpy.org/devdocs/numpy_2_0_migration_guide.html.

Currently, importing certain pydrake versions crashes with this error:

AttributeError: `itemset` was removed from the ndarray class in NumPy 2.0.

Known to fail:

• stable releases 1.30 and earlier
• nightly builds 0.0.20240618 and earlier

Describe the solution you'd like

Drake should be compatible with both NumPy 1.x and 2.x.

Describe alternatives you've considered

In the meantime, pydrake users should depend on numpy < 2 in their requirements file.

Alternatively, you can use a nightly build (which itself depends on numpy < 2).

[jwnimmer-tri]

For starters we need to merge up our pybind11 fork to incorporate pybind/pybind11#5050, i.e., to v2.12.0. That means:

• Open a pull request to https://github.com/RobotLocomotion/pybind11/ that merges the upstream from 2.11 to 2.12.
  • To to this, make a local branch and do the merging.
• Open a pull request to Drake that bumps our COMMIT pin to your branch's git sha (so that we get CI coverage).

[jwnimmer-tri]

To unbreak our wheel builds, I believe can do something along these lines:

--- a/tools/wheel/image/provision-python.sh
+++ b/tools/wheel/image/provision-python.sh
@@ -42,7 +42,7 @@ ln -s /usr/local/bin/python /usr/bin/python
 # keeping them up to date if they are?
 pip install \
     matplotlib \
-    numpy \
+    'numpy < 2' \
     pyyaml \
     semantic-version \
     setuptools \

[ggould-tri]

https://drake-jenkins.csail.mit.edu/view/Production/job/linux-jammy-unprovisioned-gcc-wheel-nightly-release/111 is this issue; I am using the magic word //:install in this comment so that future buildcop searches will find this issue when they search for the failing target.

[jwnimmer-tri]

I think possibly the last remaining obstacle is to figure out how to rework RobotLocomotion/pybind11@0da7ac7 to no longer rely on itemset, which is related to pybind/pybind11#1181 and pybind/pybind11#1152.

[jwnimmer-tri]

The first thing to try is if we can call __setitem__ with a slice, instead of itemset.