PSARC 2015/535 OpenStack service updates for Kilo
PSARC 2015/458 aioeventlet - asyncio event loop scheduling callbacks in eventlet
PSARC 2015/460 msgpack - C/Python bindings for MessagePack (de)serializer data
PSARC 2015/466 openstackclient - OpenStack Command-line Client
PSARC 2015/467 oslo.versionedobjects - Oslo Versioned Objects library
PSARC 2015/468 pint - A physical quantities module
PSARC 2015/469 pysaml2 - A pure Python implementation of SAML2
PSARC 2015/471 semantic_version - A library implementing the 'SemVer' scheme
PSARC 2015/472 testresources - PyUnit extension for managing expensive test resources
PSARC 2015/473 testscenarios - Extensions to Python unittest to support scenarios
PSARC 2015/474 trollius - Port of the Tulip project (asyncio module, PEP 3156) on Python 2
PSARC 2015/475 urllib3 - HTTP library with thread-safe connection pooling, file post, and more
PSARC 2015/520 oslo.concurrency - Oslo Concurrency library
PSARC 2015/521 oslo.log - Oslo Logging Configuration library
PSARC 2015/529 oslo.policy - Oslo Policy library
PSARC 2015/530 psutil - Python system and process utilities
PSARC 2015/538 fixtures - Python module to support reusable state for writing clean tests
PSARC 2015/539 sqlparse - An SQL parser module for Python
PSARC 2016/017 extras - Useful extra utilities for Python
PSARC 2016/018 linecache2 - Port of the standard linecache module
PSARC 2016/019 python-mimeparse - Basic functions for parsing mime-types
PSARC 2016/020 testtools - Extensions to the Python unit testing framework
PSARC 2016/021 traceback2 - Port of the standard traceback module
PSARC 2016/014 OpenStack Cinder NFS driver for Solaris
22384068 OpenStack service updates for Kilo (Umbrella)
21974208 The Python module msgpack should be added to Userland
22010630 The Python trollius module should be added to Userland
22011755 The Python module pint should be added to Userland
22012256 The Python aioeventlet module should be added to Userland
22012282 The Python oslo.versionedobjects module should be added to Userland
22012317 The Python semantic_version module should be added to Userland
22012321 The Python testresources module should be added to Userland
22012329 The Python testscenarios module should be added to Userland
22012336 The Python urllib3 module should be added to Userland
22012343 The Python openstackclient module should be added to Userland
22299389 The Python oslo.concurrency module should be added to Userland
22299409 The Python oslo.log module should be added to Userland
22299418 The Python oslo.policy module should be added to Userland
22299469 The Python psutil module should be added to Userland
22337793 The Python sqlparse module should be added to Userland
22338325 The Python fixtures module should be added to Userland
22535728 The Python testtools module should be added to Userland
22535739 The Python extras module should be added to Userland
22535748 The Python linecache2 module should be added to Userland
22535753 The Python traceback2 module should be added to Userland
22535760 The Python python-mimeparse module should be added to Userland
18961001 Image filtering does not function as expected
21678935 NFS for Cinder in Solaris OpenStack
22548630 derived manifest should not enforce presence of global when installing from UAR
22629795 problem in SERVICE/KEYSTONE
This fix will be included in future 2015.1.3 (Kilo) and 5.0.1 (Liberty)
releases.
From fa19a617a79fd1cb0d892bb8ea87c4b9f6398c34 Mon Sep 17 00:00:00 2001
From: Zane Bitter <[email protected]>
Date: Tue, 24 Nov 2015 12:29:38 -0500
Subject: Load template files only from their known source
Modify get_class to ensure that user-defined resources cannot result in
reads from the local filesystem. Only resources defined by the operator
in the global environment should read local files.
To make this work, this patch also adds a separate
get_class_to_instantiate() method to the Environment.
We were previously using get_class for two different purposes - to get a
resource plugin on which we could perform introspection to obtain the
properties and attributes schema, and to get a resource plugin we could
instantiate to create a Resource object. These are both the same except in
the case of a TemplateResource, where having two different use cases for
the same piece of code was adding considerable extra complexity. Combining
the use cases in this way also made the error handling confusing (leading
to bug 1518458).
This change separates out the two cases.
Change-Id: I845e7d23c73242a4a4c9c40599690ab705c75caa
Closes-Bug: #1496277
Related-Bug: #1447194
Related-Bug: #1518458
Related-Bug: #1508115
(cherry picked from commit 06a713c4456203cd561f16721dc8ac3bcbb37a3
and 26e6d5f6d776c1027c4f27058767952a58d15e25)
---
--- heat-2015.1.2/heat/engine/environment.py.~1~ 2015-10-13 09:51:53.000000000 -0700
+++ heat-2015.1.2/heat/engine/environment.py 2016-01-25 20:50:15.096875593 -0800
@@ -112,6 +112,12 @@ class ResourceInfo(object):
def matches(self, resource_type):
return False
+ def get_class(self):
+ raise NotImplemented
+
+ def get_class_to_instantiate(self):
+ return self.get_class()
+
def __str__(self):
return '[%s](User:%s) %s -> %s' % (self.description,
self.user_resource,
@@ -140,10 +146,20 @@ class TemplateResourceInfo(ResourceInfo)
def get_class(self):
from heat.engine.resources import template_resource
+ if self.user_resource:
+ allowed_schemes = template_resource.REMOTE_SCHEMES
+ else:
+ allowed_schemes = template_resource.LOCAL_SCHEMES
+ data = template_resource.TemplateResource.get_template_file(
+ self.template_name,
+ allowed_schemes)
env = self.registry.environment
- return template_resource.generate_class(str(self.name),
- self.template_name,
- env)
+ return template_resource.generate_class_from_template(str(self.name),
+ data, env)
+
+ def get_class_to_instantiate(self):
+ from heat.engine.resources import template_resource
+ return template_resource.TemplateResource
class MapResourceInfo(ResourceInfo):
@@ -398,6 +414,13 @@ class ResourceRegistry(object):
return match
def get_class(self, resource_type, resource_name=None):
+ info = self.get_resource_info(resource_type,
+ resource_name=resource_name)
+ if info is None:
+ raise exception.ResourceTypeNotFound(type_name=resource_type)
+ return info.get_class()
+
+ def get_class_to_instantiate(self, resource_type, resource_name=None):
if resource_type == "":
msg = _('Resource "%s" has no type') % resource_name
raise exception.StackValidationFailed(message=msg)
@@ -414,7 +437,7 @@ class ResourceRegistry(object):
if info is None:
msg = _("Unknown resource Type : %s") % resource_type
raise exception.StackValidationFailed(message=msg)
- return info.get_class()
+ return info.get_class_to_instantiate()
def as_dict(self):
"""Return user resources in a dict format."""
@@ -521,6 +544,10 @@ class Environment(object):
def get_class(self, resource_type, resource_name=None):
return self.registry.get_class(resource_type, resource_name)
+ def get_class_to_instantiate(self, resource_type, resource_name=None):
+ return self.registry.get_class_to_instantiate(resource_type,
+ resource_name)
+
def get_types(self, support_status=None):
return self.registry.get_types(support_status)
--- heat-2015.1.2/heat/engine/resource.py.~1~ 2015-10-13 09:51:53.000000000 -0700
+++ heat-2015.1.2/heat/engine/resource.py 2016-01-25 20:50:15.097540727 -0800
@@ -137,14 +137,11 @@ class Resource(object):
# Call is already for a subclass, so pass it through
ResourceClass = cls
else:
- from heat.engine.resources import template_resource
-
registry = stack.env.registry
- try:
- ResourceClass = registry.get_class(definition.resource_type,
- resource_name=name)
- except exception.NotFound:
- ResourceClass = template_resource.TemplateResource
+ ResourceClass = registry.get_class_to_instantiate(
+ definition.resource_type,
+ resource_name=name)
+
assert issubclass(ResourceClass, Resource)
return super(Resource, cls).__new__(ResourceClass)
--- heat-2015.1.2/heat/engine/resources/openstack/heat/resource_group.py.~1~ 2015-10-13 09:51:53.000000000 -0700
+++ heat-2015.1.2/heat/engine/resources/openstack/heat/resource_group.py 2016-01-25 20:50:15.098020940 -0800
@@ -193,11 +193,7 @@ class ResourceGroup(stack_resource.Stack
val_templ = template.Template(test_tmpl)
res_def = val_templ.resource_definitions(self.stack)["0"]
# make sure we can resolve the nested resource type
- try:
- self.stack.env.get_class(res_def.resource_type)
- except exception.NotFound:
- # its a template resource
- pass
+ self.stack.env.get_class_to_instantiate(res_def.resource_type)
try:
name = "%s-%s" % (self.stack.name, self.name)
--- heat-2015.1.2/heat/engine/resources/template_resource.py.~1~ 2015-10-13 09:51:53.000000000 -0700
+++ heat-2015.1.2/heat/engine/resources/template_resource.py 2016-01-25 20:50:15.098440251 -0800
@@ -26,8 +26,11 @@ from heat.engine.resources import stack_
from heat.engine import template
-def generate_class(name, template_name, env):
- data = TemplateResource.get_template_file(template_name, ('file',))
+REMOTE_SCHEMES = ('http', 'https')
+LOCAL_SCHEMES = ('file',)
+
+
+def generate_class_from_template(name, data, env):
tmpl = template.Template(template_format.parse(data))
props, attrs = TemplateResource.get_schemas(tmpl, env.param_defaults)
cls = type(name, (TemplateResource,),
@@ -74,9 +77,9 @@ class TemplateResource(stack_resource.St
self.template_name = tri.template_name
self.resource_type = tri.name
if tri.user_resource:
- self.allowed_schemes = ('http', 'https')
+ self.allowed_schemes = REMOTE_SCHEMES
else:
- self.allowed_schemes = ('http', 'https', 'file')
+ self.allowed_schemes = REMOTE_SCHEMES + LOCAL_SCHEMES
return tri
--- heat-2015.1.2/heat/engine/service.py.~1~ 2015-10-13 09:51:53.000000000 -0700
+++ heat-2015.1.2/heat/engine/service.py 2016-01-25 20:50:15.099200696 -0800
@@ -978,8 +978,6 @@ class EngineService(service.Service):
"""
try:
resource_class = resources.global_env().get_class(type_name)
- except exception.StackValidationFailed:
- raise exception.ResourceTypeNotFound(type_name=type_name)
except exception.NotFound as ex:
raise exception.StackValidationFailed(message=ex.message)
@@ -1010,8 +1008,6 @@ class EngineService(service.Service):
try:
return resources.global_env().get_class(
type_name).resource_to_template(type_name)
- except exception.StackValidationFailed:
- raise exception.ResourceTypeNotFound(type_name=type_name)
except exception.NotFound as ex:
raise exception.StackValidationFailed(message=ex.message)
--- heat-2015.1.2/heat/tests/test_provider_template.py.~1~ 2015-10-13 09:51:54.000000000 -0700
+++ heat-2015.1.2/heat/tests/test_provider_template.py 2016-01-25 20:50:15.099763200 -0800
@@ -613,7 +613,11 @@ class ProviderTemplateTest(common.HeatTe
env_str = {'resource_registry': {'resources': {'fred': {
"OS::ResourceType": test_templ_name}}}}
- env = environment.Environment(env_str)
+ global_env = environment.Environment({}, user_env=False)
+ global_env.load(env_str)
+ with mock.patch('heat.engine.resources._environment',
+ global_env):
+ env = environment.Environment({})
cls = env.get_class('OS::ResourceType', 'fred')
self.assertNotEqual(template_resource.TemplateResource, cls)
self.assertTrue(issubclass(cls, template_resource.TemplateResource))
@@ -640,10 +644,6 @@ class ProviderTemplateTest(common.HeatTe
self.assertTrue(test_templ, "Empty test template")
self.m.StubOutWithMock(urlfetch, "get")
urlfetch.get(test_templ_name,
- allowed_schemes=('file',)
- ).AndRaise(urlfetch.URLFetchError(
- _('Failed to retrieve template')))
- urlfetch.get(test_templ_name,
allowed_schemes=('http', 'https')).AndReturn(test_templ)
parsed_test_templ = template_format.parse(test_templ)
self.m.ReplayAll()
--- heat-2015.1.2/heat/tests/test_resource.py.~1~ 2015-10-13 09:51:54.000000000 -0700
+++ heat-2015.1.2/heat/tests/test_resource.py 2016-01-25 20:50:15.100592773 -0800
@@ -67,12 +67,13 @@ class ResourceTest(common.HeatTestCase):
self.patch('heat.engine.resource.warnings')
def test_get_class_ok(self):
- cls = resources.global_env().get_class('GenericResourceType')
+ cls = resources.global_env().get_class_to_instantiate(
+ 'GenericResourceType')
self.assertEqual(generic_rsrc.GenericResource, cls)
def test_get_class_noexist(self):
self.assertRaises(exception.StackValidationFailed,
- resources.global_env().get_class,
+ resources.global_env().get_class_to_instantiate,
'NoExistResourceType')
def test_resource_new_ok(self):