Using the observer model to remove an observer when a topic has been deleted & rdquo; in Python

advertisements

I am attempting to implement a very simple observer pattern in python.

Here is my Observer class (it's really just an interface and I guess I don't actually need it):

class Observer():
    def update(self,subject,message): pass

And my Subject class (aka Observable, but I prefer Subject):

class Subject():
    def __init__(self):
        self.observers = []
    def registerObserver(self, observer):
        if observer not in self.observers:
            self.observers.append(observer)
    def removeObserver(self, observer):
        self.observers.remove(observer)
    def notifyObservers(self, message = None):
        for observer in self.observers:
            observer.update(self,message)

Class A contains a nested DelNotifier class, which is a subclass of Subject. The idea that when a class A object is deleted (actually garbage collected, since it is in the __del__ method), A.DelNotifier will notify all of its observers of the deletion.

class A():
    def __init__(self, name):
        self.name = name
        self.delNotifier = A.DelNotifier(self)
    class DelNotifier(Subject):
        def __init__(self, outer):
            super(A.DelNotifier,self).__init__()
            self.outer = outer
        def notifyObservers(self):
            Subject.notifyObservers(self,"This is Class A object " + self.outer.name + ": I'm dying!")
    def registerB(self,observer):
        if not isinstance(observer,B): raise ValueError("Can only register Class B objects with Class A.")
        self.delNotifier.registerObserver(observer.Aobserver)
    def deleteme(self):
        print("Must notify observers of my impending doom first...")
        self.delNotifier.notifyObservers()
    def __str__(self):
        return "Class A object " + self.name
    def __del__(self):
        self.deleteme()
        print("Done notifying everyone, time to go gentle into that good night.")

Class B contains a nested AObserver class, which is a subclass of Observer and will receive the message from the class A.DelNotifier subject when an A has been deleted (again, in actuality this happens when the A object has been garbage collected):

class B():
    def __init__(self, name, a):
        self.name = name
        self.Aobserver = B.AObserver(self)
        a.registerB(self)
    class AObserver(Observer):
        def __init__(self,outer):
            super(B.AObserver,self).__init__()
            self.outer = outer
        def update(self,subject,message):
            print(str(self.outer) + " received message: '" + str(message) + "'")
            print("Time for", self.outer, "to die, too.")
            self.outer.__del__()
    def __str__(self):
        return "Class B object " + self.name
    def __del__(self):
        print("This is " + str(self) + ": now I'm dying, too!")

This design works when I call __del__() directly, however, some of the objects seem to be gc'd a second time when the session exits:

>>> a = A('a')
>>> b1 = B('b1', a)
>>> b2 = B('b2', a)
>>> a.__del__()
Must notify observers of my impending doom first...
Class B object b1 received message: 'This is Class A object a: I'm dying!'
Time for Class B object b1 to die, too.
This is Class B object b1: now I'm dying, too!
Class B object b2 received message: 'This is Class A object a: I'm dying!'
Time for Class B object b2 to die, too.
This is Class B object b2: now I'm dying, too!
Done notifying everyone, time to go gentle into that good night.
>>> exit()
Must notify observers of my impending doom first...
Class B object b1 received message: 'This is Class A object a: I'm dying!'
Time for Class B object b1 to die, too.
This is Class B object b1: now I'm dying, too!
Class B object b2 received message: 'This is Class A object a: I'm dying!'
Time for Class B object b2 to die, too.
This is Class B object b2: now I'm dying, too!
Done notifying everyone, time to go gentle into that good night.
This is Class B object b1: now I'm dying, too!
This is Class B object b2: now I'm dying, too!

Another problem, and I think this is more important, is that when I del a class A item from a list, the item is not immediately garbage collected and I cannot be sure that any registered B items have been deleted:

>>> b1 = B('b1',a[0])
>>> b2 = B('b2',a[0])
>>> del a[0]
## Note that items are not deleted until session exits
>>> exit()
Must notify observers of my impending doom first...
Class B object b1 received message: 'This is Class A object a: I'm dying!'
Time for Class B object b1 to die, too.
This is Class B object b1: now I'm dying, too!
Class B object b2 received message: 'This is Class A object a: I'm dying!'
Time for Class B object b2 to die, too.
This is Class B object b2: now I'm dying, too!
Done notifying everyone, time to go gentle into that good night.
##Note that the class B objects get gc'd a second time....???
This is Class B object b1: now I'm dying, too!
This is Class B object b2: now I'm dying, too!

In addition to these problems, I am aware of the many many problems inherent in relying on the __del__ method to do anything other than cleaning up an object after it has been gc'd, and that it should probably be avoided for the purposes I am trying to employ. But I don't know of another way.

What would be a better way to do this? I have thought about trying to use a context manager (with) to delete things after I'm done using them, but I have no experience doing that. If that would be a good option, how would I go about doing that? What would it look like?

EDIT: Clarification of desired behavior

I'll attempt to clear up some of the (understandable) confusion.

I've simplified the code quite a bit above, but B is an object that depends on an object A. If a B's A goes away, that B should go away. I'll have some container (using a list here) of As and of Bs:

As = [A('a'+str(i)) for i in range(10)]
Bs = [B('b'+str(i),As[i]) for i in range(10)] #Bs made of As
del As[0] #whoops, don't need As[0] anymore
assert Bs[0] is None #ERROR!
#or using pop:
As.pop(0)
assert Bs[0] is None #ERROR!

Also see my previous question from the other day which helped lead me to this whole idea of using the observer pattern in the first place.


This is a significant amount of code difference to account for your requirement of automatically maintaining a list of references, and cleaning up when the references are deleted. I've added a Manager class to accomplish this, and a second pass deleted() event that is a hook for the Manager to clean up the lists it is maintaining. I'm re-posting the full, modified code here as it wasn't trivial to update my previous answer.

I believe this fully satisfies the question you've asked. Perhaps not the reason why you need this in the first place, but I think you asked that in the other question.

We need weak references to make this work:

import weakref

The Observer interface gets a new method to be called after deleted()

class Observer(object):
    def update(self,subject,message): pass

    def deleting(self,subject):
        ''' the subject is being deleted '''
        pass

    def deleted(self,subject):
        pass

A manager class maintains the lists of subjects and observers

class Manager(Observer):

    def __init__(self, ):
        self._subjects = []
        self._observers = []

    def monitorSubject(self, subject):
        self._subjects.append( weakref.ref(subject) )

        # observe the subject for deletion to
        # trigger list maintenance on "deleted"
        subject.registerObserver(self)

    def monitorObserver(self, observer):
        self._observers.append( weakref.ref(observer) )

    def deleted(self, subject):
        ''' a subject was deleted, remove it from the list.
        deleting() is called first, and the observers delete themselves.
        deleted() is called next, and is a hook for the manager to
        cleanup any dead subjects and observers '''

        # calling a weakref returns the original object, and `None` when the
        # reference is dead
        def isNotDead(r):
            return not r()==None

        # remove any dead subjects
        print 'Removing dead subjects...'
        self._subjects = filter(isNotDead, self._subjects)

        # remove any dead observers
        print 'Removing dead observers...'
        self._observers = filter(isNotDead, self._observers, )

    def __str__(self,):
        return "[Manager] Subjects:{0}, Observers:{1}".format(
            ','.join([str(r()) for r in self._subjects]),
            ','.join([str(r()) for r in self._observers])
        )

The differences in the subject are noted in the comments, but primarily there is a second pass to call deleted. The first pass of deleting notifies the observers, while the second pass of deleted notifies the manager. Also, the __del__ routine uses weak refs to iterate over, as some of them get deleted along the way.

class Subject(object):
    def __init__(self):
        self.observers = []

    def __del__(self, ):
        ''' on deletion, notify the observers '''
        print "{0}.__del__".format(self)

        # copy the observer list, then remove all references to the observers
        # NEW - use weakrefs here, or they will not be properly deleted later
        obs = [weakref.ref(o) for o in self.observers]

        # clear all strong references to the observers
        self.observers = []

        # notify all observers that we were deleted
        # ** only if they are not already deleted **
        for o in obs:
            if not o() == None:
                o().deleting(self)

        # NEW - second pass to allow the Manager to cleanup
        # ** only if they are not already deleted **
        for o in obs:
            if not o() == None:
                o().deleted(self)

    def registerObserver(self, observer):
        if observer not in self.observers:
            self.observers.append(observer)

    def removeObserver(self, observer):
        self.observers.remove(observer)

    def notifyObservers(self, message = None):
        for observer in self.observers:
            observer.update(self,message)

Same as before, with simplified string formatting

class A(Subject):
    ''' A is just a subject '''
    def __init__(self, name):
        super(A,self).__init__()
        self.name = name

    def __str__(self):
        return "A[ {0} ]".format(self.name)

B is the same as before

class B(Observer):
    ''' class B is an observer of A '''

    def __init__(self, name, a):
        self.name = name

        # don't keep a reference to 'a', or 'a' will not be deleted!
        a.registerObserver(self)

    def __str__(self):
        return "B[ {0} ]".format(self.name)

    def __del__(self):
        print("{0}.__del__".format(self))

    def deleting(self, subject):
        ''' notification from the subject (A) that it is being deleted. I'm
        assuming here that the subject is actually the one we registered in
        __init__, since I couldn't store a reference or else __del__ would
        not have been called! '''
        print "B[{0}].deleting, subject={1}".format(self.name, subject)

        del self

Some code to execute the file:

if __name__ == '__main__':

    mgr = Manager()

    # keep strong references to the subjects, because
    # we will delete them explicitly
    a1 = A('a1')
    a2 = A('a2')
    mgr.monitorSubject(a1)
    mgr.monitorSubject(a2)

    # monitor observers directly, and do NOT keep
    # strong references or they will not be deleted
    mgr.monitorObserver( B('b1', a1) )
    mgr.monitorObserver( B('b2', a1) )
    mgr.monitorObserver( B('b3', a2) )
    mgr.monitorObserver( B('b4', a2) )

    # show the starting state
    print mgr

    print "Deleting a1..."
    del a1
    print mgr

    print "Deleting a2..."
    del a2
    print mgr

And the output:

    #  OUTPUT (some newlines added)
    #
    #  [Manager] Subjects:A[ a1 ],A[ a2 ], Observers:B[ b1 ],B[ b2 ],B[ b3 ],B[ b4 ]
    #
    #  Deleting a1...
    #  A[ a1 ].__del__
    #  B[ b1 ].__del__
    #  B[ b2 ].__del__
    #  Removing dead subjects...
    #  Removing dead observers...
    #  [Manager] Subjects:A[ a2 ], Observers:B[ b3 ],B[ b4 ]
    #
    #  Deleting a2...
    #  A[ a2 ].__del__
    #  B[ b3 ].__del__
    #  B[ b4 ].__del__
    #  Removing dead subjects...
    #  Removing dead observers...
    #
    #  [Manager] Subjects:, Observers: