durable writer problem

Hey,

The problem of consistency in distributed systems is pretty well known.

Your scenario highlights just one of the problems.

Say that writer A writes instance z with state x and writer B writes instance z with state y.

Can you guarantee that all readers agree on which state is the latest state?

These kinds of problems arise when a single "object" (in this example, an instance) is being maintained by multiple writers.

One way to solve this problem is to avoid it: make sure different writers do not modify the same instances.

How can that be achieved?

You could have an application that will be the sole writer (per instance / per topic / for a set of instances / for a set of topics) and have A and B send it updates on a more dynamic topic (for example, A will send the new application an update that a new instance z should be added then later B will send the new application an update that the instance z should be removed).

You could split your data set between your applications (so that if writer A is responsible for instance z, then B will not be responsible for it) although I realize for your use case this must be a no-can-do situation (for example, if they update different parts of an instance and could potentially uncover reasons to have it removed). One way you could still use this option is of you remodel your data so that it becomes possible.

You could add a reader on both applications on the same topic which you could use to synchronize them on disposes (for example, the reader will have a listener and when instances are disposed by another writer for an instance that this application has written, this application will also dispose it).

Hopefully one of these options is good for you,

Roy.