components/python/python26/patches/Python26-37-multiproc-race.patch
author Erik Trauschke <Erik.Trauschke@oracle.com>
Mon, 17 Nov 2014 15:12:43 -0800
changeset 2212 0b6092e8d4e7
permissions -rw-r--r--
20017740 multiprocessing module susceptible to race condition during exit
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
2212
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
     1
This patch comes from upstream: http://bugs.python.org/issue15881
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
     2
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
     3
It fixes a race condition in multiprocessing/util.py in which globals are
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
     4
deleted during exit but still used during the exit.
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
     5
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
     6
diff -ur a/Lib/multiprocessing/util.py b/Lib/multiprocessing/util.py
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
     7
--- a/Lib/multiprocessing/util.py	2014-11-12 08:02:39.431360950 -0800
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
     8
+++ b/Lib/multiprocessing/util.py	2014-11-12 08:03:56.970704872 -0800
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
     9
@@ -221,6 +221,12 @@
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    10
     Finalizers with highest priority are called first; finalizers with
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    11
     the same priority will be called in reverse order of creation.
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    12
     '''
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    13
+    if _finalizer_registry is None:
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    14
+        # This function may be called after this module's globals are
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    15
+        # destroyed.  See the _exit_function function in this module for more
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    16
+        # notes.
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    17
+        return
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    18
+    
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    19
     if minpriority is None:
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    20
         f = lambda p : p[0][0] is not None
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    21
     else:
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    22
@@ -252,21 +258,38 @@
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    23
 
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    24
 _exiting = False
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    25
 
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    26
-def _exit_function():
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    27
+def _exit_function(info=info, debug=debug, _run_finalizers=_run_finalizers,
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    28
+                   active_children=active_children,
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    29
+                   current_process=current_process):
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    30
+    # NB: we hold on to references to functions in the arglist due to the
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    31
+    # situation described below, where this function is called after this
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    32
+    # module's globals are destroyed.
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    33
+
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    34
     global _exiting
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    35
 
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    36
     info('process shutting down')
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    37
     debug('running all "atexit" finalizers with priority >= 0')
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    38
     _run_finalizers(0)
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    39
 
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    40
-    for p in active_children():
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    41
-        if p._daemonic:
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    42
-            info('calling terminate() for daemon %s', p.name)
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    43
-            p._popen.terminate()
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    44
-
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    45
-    for p in active_children():
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    46
-        info('calling join() for process %s', p.name)
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    47
-        p.join()
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    48
+    if current_process() is not None:
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    49
+        # NB: we check if the current process is None here because if
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    50
+        # it's None, any call to ``active_children()`` will throw an
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    51
+        # AttributeError (active_children winds up trying to get
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    52
+        # attributes from util._current_process).  This happens in a
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    53
+        # variety of shutdown circumstances that are not well-understood
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    54
+        # because module-scope variables are not apparently supposed to
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    55
+        # be destroyed until after this function is called.  However,
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    56
+        # they are indeed destroyed before this function is called.  See
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    57
+        # issues 9775 and 15881.  Also related: 4106, 9205, and 9207.
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    58
+
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    59
+        for p in active_children():
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    60
+            if p._daemonic:
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    61
+                info('calling terminate() for daemon %s', p.name)
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    62
+                p._popen.terminate()
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    63
+
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    64
+        for p in active_children():
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    65
+            info('calling join() for process %s', p.name)
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    66
+            p.join()
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    67
 
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    68
     debug('running the remaining "atexit" finalizers')
0b6092e8d4e7 20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff changeset
    69
     _run_finalizers()