tl;dr: Because event processing software doesn't do distributed transactions / 2-PC, put an event queue for each microservice in the database where the microservice puts it events, so you get transactions on your events. Then have a subordinate messaging nanoservice empty the locally transactional microservice event queue into the globally untransactional primary event queue.
By formerly_proven 4 years ago
> The pattern also puts an additional burden on the database. Therefore, we should consider the potential performance hit on the database
Massive understatement, one of the downsides that we've found with this pattern is the increase increase in binlog size for our database replication topology. One must be very careful with this pattern as to not create downstream issues with replication delay and such.
Folks looking to implement the outbox pattern with Apache Kafka may find the support provided for it by Debezium useful: https://debezium.io/documentation/reference/stable/transform....
Debezium is implementing log-based change data capture, i.e. there's no polling on the outbox table.
> But what happens if the subscription record is deleted from the DB right before emitting the event due to a bug?
Isn't this still inside an uncommitted transaction? What sort of bug would lead to an uncommitted record being deleted that was not a major bug in the database?
If the actual concern is that the record is deleted before the event is processed (not before it is sent as the article seems to be talking about), why not put foreign keys between the outbox and the users to either prevent that or make sure that both are deleted simultaneously?
By SahAssar 4 years ago
I have never worked on a project where the db being “down” was anything less than a catastrophe, as seemingly compared to a regular hiccup in this article.
By sparker72678 4 years ago
The article just seemed to me like it does not focus on that error case because it's not interesting to the pattern.
You tell your client that the request failed and that's it, right? Or there's certainly other patterns designed to help avoid hard fails when the DB is down, but they seem orthogonal to this one, use them if they're helpful to your project, skip if not?
By kadoban 4 years ago
That's my favorite part about using messaging in Vitess. I can emit events in a real transaction. It eliminates an entire class of bugs.
By formerly_proven 4 years ago
By earless1 4 years ago
By gunnarmorling 4 years ago
By daigoba66 4 years ago
By SahAssar 4 years ago
By sparker72678 4 years ago
By kadoban 4 years ago
By derekperkins 4 years ago