Test rrules with finite differences

[gdalle]

Or ChainRulesTestUtils, but beware to keep the same seed.

Problem: our real-life function is an average over a finite number of samples, so its finite-differences (and true) gradient is almost zero. What we define as its gradient through the rrule is actually the Monte-Carlo approximation of the expectation (over an infinite number of samples).
Take a large enough finite difference step or enough samples to mitigate piecewise-constantness.

Our functions are not smooth, but for FYL + SPO + SSVM for instance we return a subgradient. Can we test that property in the convex case?