Access Protocol : Cleanup Protocol : Implicit Access : Weak References : Interface : Examples : Structure : Support : Download : Copyright & License : History : Home | Version 2.0.3 |
A while ago, I ran into a security problem within Python: there was no apparent way to hide attributes from unauthorized access other than running in an restricted environment and using the standard lib's Bastion object wrapper.
Since I couldn't run the code in restricted mode (e.g. it does assignments to __class__ and __dict__ in a few vital places), the only reasonable way seemed building a new type that hides a few attributes in the C implementation. These attributes are not accessible from within Python. Of course, a debugger will give you access and some special extension module that knows about the internal structure of the types could too. But for my purpose this level security suffices.
Instead of creating a special Bastion like type I figured that it would be a good idea to provide a more generic and extendable version of a Proxy object. The object itself doesn't provide too much funtionality. It basically hides the wrapped object and provides controlled access to its attributes.
Another subject that comes up every now and then in the Python community is that of weak references, i.e. references to objects that don't prevent them to be garbage collected. Since Python uses a reference count garbage collection scheme, circular references cause memory leakage as soon as the reference to the cycle is dropped (the involved objects are not garbage collected because their reference count never drops to zero).
Proxy objects can use two flavors of references starting
with version 0.2: strong references which act just like any
other reference you use to the object in Python and weak
references. The
To control access to the wrapped objects attributes (these also
include its methods), the Proxy provides two features:
The access scheme first checks the interface list. If given,
only attributes with names appearing in it can be set or
fetched. All other accesses are cancelled by raising an
The presence of the interface list also indicates whether
methods that are to be returned to the requesting object should
be Proxy-wrapped on-the-fly or not. Wrapped methods only have
the callable slot enabled, thus inhibiting access to the
enclosed object reference.
In case an attribute it to be fetched, the Proxy checks the
availability of a
Setting attributes is done in a similar way: the Proxy checks
the availability of a
When dealing with large object systems circular references are
sometimes introduced. Since these reference loops cannot
automatically be broken by the system, the user has to provide
means of breaking the circles at the proper places. One such place
is the object destructor of objects that explicitly contain such
references.
The Proxy object can be used to provide an indirect pointer into
such a circle of objects that reference each other. When the
system deletes the Proxy object, its destructor tries to find a
special method
Errors raised in
On many occasions object attributes are not explicitly accessed
by e.g. using
As of version 0.2, Proxy objects also allow managing the type
slot interface for most applications. This means that you can
transparently use the Proxy to wrap builtin types such as lists
or tuples and use the Proxy just like you would use the
referenced builtin object itself.
Since Proxies are about access management, you can also restrict
access to these slots. For simplicity, the names you have to use
for different slots are exactly those you would define for
Python classes, e.g. the length slot is named '__len__', the
sequence and mapping get item slot are grouped under the name
'__getitem__'. These names have to be explicitly stated in the
interface list you pass to the Proxy constructor when creating
the proxy in case you do define an interface list. If you don't
specify such a list, no direct interface restriction is applied.
These type slot names are defined:
Omissions currently are: __coerce__, __hex__ and
__oct__. __repr__ is handled by the Proxy object itself since it
would otherwise possibly expose address information about the
underlying object.
Proxies also work transparently for Python instances. To achieve
this, another proxy-like object which is a real Python instance
is to be put in front of the Proxy object. Access to this
instance then gets translated into getattr-calls by the
interpreter, these calls are filtered through the Proxy and the
wrapped instance object's attribute then gets accessed in the
usual way. The result is passed back to the requesting object.
Proxy objects work in two modes: keeping a strong or a weak reference
to the object. The
Weak references are called weak because they don't keep the
object alive by incrementing the reference count on the
referenced object. Since objects get garbage collected when this
reference count falls down to 0, a weak reference can become
invalid at any time. The mxProxy implementation raises a
This may sound like a pretty flaky feature at first, but the
main pro argument for these weak references is that you can
build up circular references without having to fear about them
not being properly garbage collected. Using strong references
(which do increment the reference count and thus keep the object
alive as long as the reference is around) would produce cycles
in the referencing scheme which the Python garbage collection
(GC) mechanism cannot automatically break causing the cycle to
become unreachable from the Python namespaces: a severe memory
leak.
Weak references in mxProxy work by using a global dictionary of
all objects handled through weak reference Proxies. This
dictionary is checked prior to every action on a weak Proxy and
after its deletion. You can also force a check by calling the
The dictionary holds a strong reference to the object keeping it
alive until the next check is done. During the check all handled
objects are inspected to see if their reference count has gone
down to 1 (phantom objects: only the dictionary
references them). If this is the case, all weak proxies are
marked defunct and the object is removed from the dictionary
causing it to be garbage collected by the Python GC
mechanism. All subsequent actions on the weak references to this
object will then cause a WeakProxy()
constructor must be
used for the latter. Details are described below.
Access Protocol
__public_getattr__
and/or __public_setattr__
.
AccessError
exception (which is a subclass of the
standard AttributeError
).
__public_getattr__
method. If
found, this method is called with the name as parameter and
whatever this method returns is returned to the requesting
object. Otherwise, the Proxy uses the standard
getattr()-functionality to fetch the attribute.
__public_setattr__
method. If
found, this method is called with parameters name and value.
Otherwise, the standard setattr() functionality is used.
Cleanup Protocol
__cleanup__
in the wrapped
object and calls it before continuing the destruction. This
allows the object to break the reference circles, e.g. by
performing a self.__dict__.clear()
or something
similar.
__cleanup__
calls are
ignored. Warnings are printed to stderr in case Python is run in
verbose mode (python -v) and tracebacks printed if run in debug
mode (python -d; note that printing tracebacks can sometimes
cause core dumps e.g. during finalization of the interpreter).
Implicit Access
object.attribute
, but indirect
through builtin functions or the interpreter itself. This poses
a problem to the Proxy, since there are different ways to reach
an objects implementation. For Python instances all important
hooks are reachable via getattr()
, e.g. __len__ and
__add__. This is different for extension and builtin types: they
use a system of slots for providing access to their data.
Note: number coercion does not yet
work, so most of these are currently useless !
Weak References
Proxy()
constructor returns a
Proxy object using a strong reference, the WeakProxy()
constructor one
using a weak reference.
LostReferenceError
in case a weak reference to such
an object is used.
checkweakrefs()
anytime you like, e.g. at regular
intervals.
LostReferenceError
exception to be raised.
These constructors are available in the package:
Proxy(object,interface=None,passobj=None)
interface can be given as sequence of strings and/or objects with __name__ attribute or as dictionary with string keys (only the keys are currently used) and tells the Proxy to only allow access to these names. If not given or None, no filtering is done by the Proxy (see above on how access is managed).
passobj can be provided to retrieve the wrapped object
from the Proxy at a later point using the
.proxy_object()
method.
InstanceProxy(object,interface=None,passobj=None)
This makes the Proxy transparent for access to wrapped Python instances, meaning that the Proxy will act as if it were the wrapped object itself (with the added features mentioned above).
CachingInstanceProxy(object,interface=None,passobj=None)
Note that this may introduce circular references if not used properly. Cached attributes are not looked up in the wrapped instance after the first lookup -- if their value changes, this won't be noticed by objects that access the object through this wrapper.
SelectiveCachingInstanceProxy(object,interface=None,passobj=None)
The cached types are defined by the
.proxy_cacheable_types
attribute. It
defaults to only cache Python methods.
MethodCachingProxy(object,interface=None,passobj=None)
ReadonlyInstanceProxy(object,interface=None,passobj=None)
AccessError
being raised
ProxyFactory(Class,interface=None)
interface is passed to the Proxy constructor, pass-objects are not supported.
InstanceProxyFactory(Class,interface=None)
WeakProxy(object,interface=None,passobj=None)
interface can be given as sequence of strings and/or objects with __name__ attribute or as dictionary with string keys (only the keys are currently used) and tells the Proxy to only allow access to these names. If not given or None, no filtering is done by the Proxy (see above on how access is managed).
passobj can be provided to retrieve the wrapped object
from the Proxy at a later point using the
.proxy_object()
method.
For details on weak references and how they work, see the above discussion.
A Proxy instance proxy
defines these
methods in addition to the ones available through restricted
access to the wrapped object:
proxy_object(passobj)
Simple equality is not enough -- it has to be the same object.
proxy_getattr(name)
proxy_setattr(name,value)
proxy_defunct()
Note that all attribute names starting with
'proxy_
' are interpreted as being Proxy
attributes and are not passed to the wrapped object. Access
to these attributes is not subject to the
restrictions explained above.
Proxy instances do not provide any instance variables themselves. They do provide restricted access to the variables of the wrapped object though.
Note that all attribute names starting with
'proxy_
' are interpreted as being Proxy
attributes and are not passed to the wrapped object. Also,
access to these attributes is not subject to the
restrictions explained above.
These functions are available:
checkweakrefs()
Weak references cause the objects to stay alive until either a proxy is used on them (causing an exception), a proxy referencing them is deleted or this function is called. To ensure the timely garbage collection of the objects, call this function at regular intervals.
initweakrefs()
finalizeweakrefs()
Calling this function after finalization is not an error.
These constants are available:
AccessError
Here is a very simple one:
import Proxy class DataRecord: a = 2 b = 3 # Make read-only: def __public_setattr__(self,what,to): raise Proxy.AccessError,'read-only' # Cleanup protocol def __cleanup__(self): print 'cleaning up',self o = DataRecord() # Wrap the instance: p = Proxy.InstanceProxy(o,('a',)) # Remove o from the accessible system: del o print 'Read p.a through Proxy:',p.a # This will cause an exception, because the object is read-only p.a = 3 # This will cause an exception, because no access is given to .b print p.b # Clear all traces of the provious exceptions (they might contain # references to p) by issuing another one. Note that not doing # so will cause the following 'del p' to not destroy the final # reference to p... (don't ask me why). 1/0 # Deleting the Proxy will also delete the wrapped object, if there # is no other reference to it in the system. It will invoke # the __cleanup__ method in that case. del p # If you want to have the wrapping done automatically, you can use # the InstanceProxyFactory: DataRecord = Proxy.InstanceProxyFactory(DataRecord,('a',)) # This gives the same behaviour... p = DataRecord() print p.a p.a = 3 print p.b
More examples will eventually appear in the Examples
subdirectory of the package.
Entries enclosed in brackets are packages (i.e. they are
directories that include a __init__.py file). Ones
without brackets are just simple subdirectories that are not
accessible via
The package imports all symbols from the Proxy sub module
which in turn imports the extension module, so you only need
to '
eGenix.com is providing commercial support for this
package. If you are interested in receiving information
about this service please see the eGenix.com
Support Conditions.
© 1998-2000, Copyright by Marc-André Lemburg;
All Rights Reserved. mailto: mal@lemburg.com
© 2000-2001, Copyright by eGenix.com Software GmbH,
Langenfeld, Germany; All Rights Reserved. mailto: info@egenix.com
This software is covered by the eGenix.com Public
License Agreement. The text of the license is also
included as file "LICENSE" in the package's main directory.
By downloading, copying, installing or otherwise using
the software, you agree to be bound by the terms and
conditions of the eGenix.com Public License
Agreement.
Things that still need to be done:
There were no significant changes between 2.0.0 and 2.0.3.
Changes from 0.2.0 to 2.0.0:
Changes from 0.1.0 to 0.2.0:
Version 0.1.0 was the intial release.
Package Structure
[Proxy]
Doc/
Examples/
[mxProxy]
Proxy.py
import
.
import Proxy
' to start working.
Support
Copyright & License
History & Future