author | Erik Trauschke <Erik.Trauschke@oracle.com> |
Mon, 17 Nov 2014 15:12:43 -0800 | |
changeset 2212 | 0b6092e8d4e7 |
permissions | -rw-r--r-- |
2212
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
1 |
This patch comes from upstream: http://bugs.python.org/issue15881 |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
2 |
|
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
3 |
It fixes a race condition in multiprocessing/util.py in which globals are |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
4 |
deleted during exit but still used during the exit. |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
5 |
|
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
6 |
diff -ur a/Lib/multiprocessing/util.py b/Lib/multiprocessing/util.py |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
7 |
--- a/Lib/multiprocessing/util.py 2014-11-12 08:02:39.431360950 -0800 |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
8 |
+++ b/Lib/multiprocessing/util.py 2014-11-12 08:03:56.970704872 -0800 |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
9 |
@@ -221,6 +221,12 @@ |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
10 |
Finalizers with highest priority are called first; finalizers with |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
11 |
the same priority will be called in reverse order of creation. |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
12 |
''' |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
13 |
+ if _finalizer_registry is None: |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
14 |
+ # This function may be called after this module's globals are |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
15 |
+ # destroyed. See the _exit_function function in this module for more |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
16 |
+ # notes. |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
17 |
+ return |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
18 |
+ |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
19 |
if minpriority is None: |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
20 |
f = lambda p : p[0][0] is not None |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
21 |
else: |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
22 |
@@ -252,21 +258,38 @@ |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
23 |
|
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
24 |
_exiting = False |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
25 |
|
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
26 |
-def _exit_function(): |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
27 |
+def _exit_function(info=info, debug=debug, _run_finalizers=_run_finalizers, |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
28 |
+ active_children=active_children, |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
29 |
+ current_process=current_process): |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
30 |
+ # NB: we hold on to references to functions in the arglist due to the |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
31 |
+ # situation described below, where this function is called after this |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
32 |
+ # module's globals are destroyed. |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
33 |
+ |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
34 |
global _exiting |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
35 |
|
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
36 |
info('process shutting down') |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
37 |
debug('running all "atexit" finalizers with priority >= 0') |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
38 |
_run_finalizers(0) |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
39 |
|
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
40 |
- for p in active_children(): |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
41 |
- if p._daemonic: |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
42 |
- info('calling terminate() for daemon %s', p.name) |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
43 |
- p._popen.terminate() |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
44 |
- |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
45 |
- for p in active_children(): |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
46 |
- info('calling join() for process %s', p.name) |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
47 |
- p.join() |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
48 |
+ if current_process() is not None: |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
49 |
+ # NB: we check if the current process is None here because if |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
50 |
+ # it's None, any call to ``active_children()`` will throw an |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
51 |
+ # AttributeError (active_children winds up trying to get |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
52 |
+ # attributes from util._current_process). This happens in a |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
53 |
+ # variety of shutdown circumstances that are not well-understood |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
54 |
+ # because module-scope variables are not apparently supposed to |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
55 |
+ # be destroyed until after this function is called. However, |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
56 |
+ # they are indeed destroyed before this function is called. See |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
57 |
+ # issues 9775 and 15881. Also related: 4106, 9205, and 9207. |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
58 |
+ |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
59 |
+ for p in active_children(): |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
60 |
+ if p._daemonic: |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
61 |
+ info('calling terminate() for daemon %s', p.name) |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
62 |
+ p._popen.terminate() |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
63 |
+ |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
64 |
+ for p in active_children(): |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
65 |
+ info('calling join() for process %s', p.name) |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
66 |
+ p.join() |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
67 |
|
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
68 |
debug('running the remaining "atexit" finalizers') |
0b6092e8d4e7
20017740 multiprocessing module susceptible to race condition during exit
Erik Trauschke <Erik.Trauschke@oracle.com>
parents:
diff
changeset
|
69 |
_run_finalizers() |