[SPARK-54269][PYTHON] Upgrade cloudpickle to 3.1.2 for Python 3.14

### What changes were proposed in this pull request?

This PR aims to upgrade `cloudpickle` to 3.1.2.

### Why are the changes needed?

To support Python 3.14 properly.
- https://github.com/cloudpipe/cloudpickle/releases/tag/v3.1.2
- https://github.com/cloudpipe/cloudpickle/blob/master/CHANGES.md#312

    > Fix pickling of abstract base classes containing type annotations for Python 3.14. (cloudpipe/cloudpickle#578)

### Does this PR introduce _any_ user-facing change?

No, Python 3.14 support is not announced yet.

### How was this patch tested?

Pass the CIs.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes apache#52964 from dongjoon-hyun/SPARK-54269.

Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>

Author: dongjoon-hyun

python/pyspark/cloudpickle/__init__.py

@@ -3,7 +3,7 @@
 
 __doc__ = cloudpickle.__doc__
 
-__version__ = "3.1.1"
+__version__ = "3.1.2"
 
 __all__ = [  # noqa
     "__version__",

python/pyspark/cloudpickle/cloudpickle.py

@@ -783,6 +783,12 @@ def _class_getstate(obj):
 
     clsdict.pop("__dict__", None)  # unpicklable property object
 
+    if sys.version_info >= (3, 14):
+        # PEP-649/749: __annotate_func__ contains a closure that references the class
+        # dict. We need to exclude it from pickling. Python will recreate it when
+        # __annotations__ is accessed at unpickling time.
+        clsdict.pop("__annotate_func__", None)
+
     return (clsdict, {})
 
 
@@ -1190,6 +1196,10 @@ def _class_setstate(obj, state):
         for subclass in registry:
             obj.register(subclass)
 
+    # PEP-649/749: During pickling, we excluded the __annotate_func__ attribute but it
+    # will be created by Python. Subsequently, annotations will be recreated when
+    # __annotations__ is accessed.
+
     return obj
 
 
@@ -1301,12 +1311,9 @@ def _function_getnewargs(self, func):
     def dump(self, obj):
         try:
             return super().dump(obj)
-        except RuntimeError as e:
-            if len(e.args) > 0 and "recursion" in e.args[0]:
-                msg = "Could not pickle object as excessively deep recursion required."
-                raise pickle.PicklingError(msg) from e
-            else:
-                raise
+        except RecursionError as e:
+            msg = "Could not pickle object as excessively deep recursion required."
+            raise pickle.PicklingError(msg) from e
 
     def __init__(self, file, protocol=None, buffer_callback=None):
         if protocol is None: