In the last couple of videos we've looked at a bunch of diagrams that laid out a bunch of different
solutions to this concurrency stuff.
It's now in this video.
We're gonna try to come up with some kind of solution to this problem.
Can tell you right now this is going to be a long video but I've kind of planned it with a ton of diagrams
so hopefully it should capture your interest for some amount of time.
Let me first begin by telling you what we're really going to do in this video and how we're gonna solve
this concurrency stuff.
So I got to make a claim here.
I'm going to say that this entire accounting application that we're trying to fix up with this concurrency
stuff is just poorly designed.
And in all the solutions we've looked at so far we've been trying to figure out how we could somehow
use nets to save this concurrency problem or somehow fix it.
In other words we've been looking at Nats and trying to imagine how we could use those sequence numbers
and so on to somehow properly order these events.
I'm going to make a big claim here and say that that was actually a really poor approach.
It's the system itself that is poorly designed.
In other words we can't really use Nats to somehow save this application or this accounting stuff.
We need to figure out how to redesign the application itself.
We need to figure out how to redesign the services inside of it.
And if we provide a better designed to the system I think you're going to see that a better solution
for this concurrency stuff is actually going to present itself without even worrying about any of the
internal features of Nats whatsoever.
So long story short what we're really going to do in this video is try to redesign this accounting service
and then at the very end after we redesign the entire thing we're going to very quickly see that there's
actually a pretty easy and straightforward way to solve all this concurrency problem so let's get to
it.
We're going to first try to redesign this accounting application to get started.
We're going to look at something that looks very unrelated right now.
I know this looks really unrelated.
I want to think back in time to that blog post application.
We worked on in the first section of this course so in that application we had a request handler that
would allow us to create a new post so we'd make a request to create a post that would go to the postal
service the Postal Service would save that post internally.
It was in memory but we'll call it a database whatever.
We then created an event to describe that post being created.
We sent that off to a event boss that we had built and that event eventually flowed onto our query service.
And the goal of that query service was to take a lot of these different events and build up a complete
picture of all the different posts and comments we had inside of our app the comment service was very
similar.
We made a request of the common service.
We saved it to some database we emitted an event and then it flowed onto the query service.
So I now want to take this diagram right here and to simplify it a little bit.
Apply some easier to understand terminology in some critical locations.
OK.
So very similar diagram I've just simplified it a pretty good amount.
I also removed the comments related stuff so here's what was really going on.
We made a request to create post that went to the postal service and that was a service that was 100
percent responsible for what a post was.
It knew exactly all the attributes that a post had and was the canonical source to define what a post
was we stored that information side of a database.
And we then emitted some event describing the fact that the post was just created.
We didn't actually send it over to Nats that went to our customer event boss but essentially that eventually
got related over to the query service where that event was consumed not going to even further simplify
this diagram.
I got to move the idea of post entirely.
I'm going to move the idea of query service and just put on some more generic terminology.
It's now in here we've got some really generic terms that's going to describe what we're going to try
to do over on the accounting service or that account counting application as well.
So we really want to have in general in a micros services application.
We want to somehow make a request to modify a resource that request should always go to some service
that completely owns that resource in this case we're imagining it's a resource called X Y Z.
So that service is going to own the definition of what x y z is when receives a request to create update
or delete this resource.
Well those changes will be made in some accompanying database.
Well then you met an event describing the fact that the x y z resource has changed and that event can
flow to all the other services inside of our app that need to modify its data.
Based upon this event so I now want to take this kind of flow right here and apply it to the accounting
application.
So when we go back over to the accounting application diagrams we've been working with there's a couple
of things that immediately pop up to me.
We've been putting tremendous focus on the nat string server and the service that somehow listening
to events coming out of it.
And so respectively that's really reference to this part of diagram right here this yellow box and this
gray box down here and our entire design for this accounting application we've been working with has
a 100 percent neglected everything over here.
In other words we have not even remotely begun to think about what this publisher thing really is where
these events are coming from.
And the underlying data that these events correspond to.
So here's what I want to do.
I want to get a better idea of what service is actually managing at these events right here.
And like I said as soon as we start to get a better idea of what these service is that's actually meeting
these events and handling this concurrency stuff is going to get a lot easier so when I look at this
publisher I'm seeing that it's emitting some events kind of concerned with account transactions of sorts
a transaction let's imagine is something that tries to exchange money in or out of an account.
So if we try to follow the same pattern of something like this right here to describe this accounting
application we might end up with something like this right here so down here at the bottom right hand
side we still have an account service.
We still have the Nats event bus but now we've got this entire idea of a service that is somehow going
to receive requests to create a transaction thing a service that's going to record all these transactions
a database for storing them and then anytime someone tries to create a transaction chances are we're
going to emit some kind of event describing the fact that a transaction was just created.
And so these events right here that we're going to emit are essentially used to these events right here
inside this diagram.
Now let's start to think about how we would actually design the service and have some database to store
the transactions that people are trying to create.
So we might end up with something like this right here.
So in this diagram we're imagining that someone's logging onto maybe some banking Web site as some particular
user will imagine that it's a user with an I.D. of C Z Q And then this user on our banking Web site
is going to use the online banking stuff and eventually try to deposit 70 dollars then 40 and then withdraw
100.
Now we really start to follow this pattern of the pattern side this diagram right here.
Imagine what would really go on behind the scenes.
Each of these requests would probably fall off to some transaction service and then from there the transaction
service would want to store those in some kind of database and maybe in this database we would store
the user I.D. who made the transaction or who created it and then it would associate with that would
be a list of all the transactions that they have created over time.
So the first request would maybe create a record like this right here maybe the second request would
it create a record like so and then the third request or withdraw one would create a record like that
right there and then once again if we really follow the pattern laid out in this diagram as we store
all these different records inside this transaction database we'd probably want to also you met some
kind of event describing the fact that we had just emitted or created these transactions.
So it's going to walk through that process as well.
So here's our transaction database transaction service we're going to want to emit some events describing
the fact that these things were just created and maybe send them over to say our customer event boss
or this case let's say we're sending off to the Nats streaming server so I want to imagine what an event
would look like that would describe these things being created.
Well maybe you would look something like this.
Maybe for the first object out there the first transaction we would emit an object or an event with
a type of transaction related
and then inside of here we would probably want to list some information about the transaction that just
been created as well.
So maybe we would say a was a deposit of 70 dollars maybe the transaction itself has an I.D. of K SJ
f I.D. K SJ F and then maybe we also put on say the user's I.D. as well.
So like user I.D. and that is C C Q And then maybe kind of interestingly maybe it would be handy for
some reason I don't know.
Let's just imagine if we also included in this event a description of what this transactions number
was for this user.
So let's say that the very first transaction that the user creates has a number of one and then the
next one has a number of two then a number of three and so on.
So for this first event I would give it a number of one.
So we'll say this is very similar to a sequence number but it's kind of being determined by this database
as opposed to gnats or something like that.
So now we can repeat that process for these other two transactions as well I would create another one.
This one would be a deposit of 40 an idea of 5 J.
Twenty five and a number of two.
And then finally this last one would be a withdrawal
of one hundred.
Maybe we'll give it an idea as well as well.
One Kayo one and this would be at number three.
Like so now.
These are the three events that we can imagine we would throw over to prevent us and we want those to
eventually flow on to any services.
That want to somehow watch for new transactions being created and so in our case that would be the account
service.
So let's now imagine how that account service would really be put together.
So we've once again got our transactions database we've got the Nat string server maybe it has a channel
of transaction created and we can imagine maybe there are two account services listening for events
from that channel and then these account services very similar to the query service we had put together
earlier probably would have some kind of database for it that would take in a list of all these transactions
and then prevent or present or calculate a total sum for each user's account balance I'm going to go
and pull those events we just defined over and we'll imagine what these things would do to process these
events so we can imagine the first event would flow maybe to a right here let's just take the information
out of this event and store it inside this accounts database and here's the part where since we've gone
through this really and have better design and the transaction service you're going to see that these
concurrency issues start to fall away and the current currency issue is going to fall away without having
to rely upon any fancy features of Nats server so we're going to imagine that we start to process this
event right here.
OK.
So let's imagine sidebar accounts database we've got our user sees queue.
That's the person we care about and maybe we've got some other users inside of here as well.
We don't really care about them whatever but they exist inside of here right now.
Susie Q has a balance of presumably zero.
I also want to try to record the last transaction number.
So this last transaction number is going to be essentially the number of the last transaction that we
processed when we first create sees.
Q they'll start off the balance of zero and no last transaction number because we've not processed any
transactions related to them yet so now we'll process the thing okay.
It's deposit 70 so we'll increment by 70 and the number of this transaction was number one.
So put into one like so Okay that is simple enough.
So now we can move on to our next transaction the one of number two so once again we're looking at season
Q It's a deposit of 40 so we'll go up to 110 and now our last transaction that's created was to sell
update lands at last transaction number to do as well.
Now we can move on to the third one.
In this case still the user Susie Q It's a withdrawal of one hundred and a last transaction number is
three in this case or this transaction is number three so we'll update Lancer last transaction number
to three and subtract one hundred off the balance and so we're left with 10 and so in this scenario
it definitely looks like everything still works as expected.
Right.
We still at the end the day end up with some user with the balance and assuming that all these events
flow through the way we expect.
In other words nothing crashes.
We don't get any double issue of an event or anything like that.
It appears that everything works as expected.
So now let's kind of go through this flow again.
Well let's imagine that some of these events are not properly processed.
In other words let's imagine that maybe this first event goes to service a service say in that instant
crashes entirely.
So you and I know based upon what we've done with Nats at this point in time we know that if this thing
crashes and we don't acknowledge this event for 30 seconds eventually Nats will reissue it to someone
else inside this cougar.
But for right now let's just imagine that this thing gets sent into service day and it just does not
get processed right away and it simultaneously maybe event number two right here flows in as well.
So at this point this event not going be processed.
This event will be processed.
Now what would happen.
Well let's add in a new rule here as well.
Let's say that because each of these transaction events right here have a number associated with them
we only want to process this thing.
If the last transaction number for this user stored inside of our database is equal to that number minus
one so once again this thing is crashed.
We're just going to forget about it.
We can't process that right now.
We have to wait 30 seconds.
So then this event comes into service be so service B is going to take a look at its can say okay let's
see.
All right deposit forty four user C.C. Q And it's a number of two that's the number of the transaction.
Now we could set up the service.
So it looks at its database finds user C.C. Q takes a look at the last transaction number in this case
it doesn't have one yet so ideally we want to say take a look at this number of to subtract one and
that's what user sees.
Q Should have for the last transaction number but there's nothing there.
We don't have a one and because of that we're gonna have service be set up to say oh well I guess we
haven't processed transaction one yet.
I'm just going to forget this event I'm going to pass over it entirely.
So what just happened.
Well we failed to process event number one it will be processed or we will attempt to process it process
it again 30 seconds in the future but critically we didn't process that thing before the second one
and that was okay because since we have this number of field on here and we are recording the last transaction
number inside of our current database we knew not to attempt to process this second transaction even
though it was handled by a totally different service than the one that rejected the transaction number
one so we can now imagine that we basically now sit around and we're gonna wait 30 seconds for both
these things to time out and eventually be reissued by Nats so maybe 30 seconds goes by and the Nats
sees that this message never got acknowledged.
So it's then going to take it and pass it on to service be now service B is going to take a look at
this thing it's going say OK deposit seventy to user season Q And it's a number of one.
Okay let's go find Susie Q Oh they don't have a last transaction number well that's okay because this
is the very first transaction so we'll set last transaction to 1 and we'll do our deposit of 70 and
that's it.
We've now successfully process that event.
Now we can imagine that after those 30 seconds now maybe event number two right here finally gets reissued
because it timed out during those 30 seconds.
Now once again we're going to take a look at the thing user season.
Q Last transaction recorded of one.
This one has a number of two.
Okay well two minus one is one fantastic it lines up.
It means we process the first transaction.
We are now on the second one.
Let's process this thing and commit to our database.
So in this case we can deposit 40 dollars.
We're now up to 110 and then as usual we've taken this last event I forgot to update the last transaction
number on that second one.
So we've taken this last event okay user Susie Q last transaction number of two we are on now on number
3.
Yep all lines up Geils go ahead and do that withdrawal of one hundred.
And now we're down to 10 so by adding on this number field right here we have effectively solved all
the different concurrency issues we ran through it in the last couple videos.
This solves the issue where Nats thinks the decline is still alive when it's actually dead.
This solves the issue where maybe one listener might run more quickly than another.
This also solves the issue where a listener can fail to process an events and we have to wait a 30 second
time out for that event to be reissued.
This also solve some issues we ran through in the last video as well where we might have different users
trying to use the database or commit events at the same time.
So if we imagine there are other users or other users creating transactions no problem.
If there's something wrong with this transaction right here.
In other words if we're still on transaction number one and waiting for number two to be committed.
That is not going to prevent user JP for or user Cayenne from creating transactions as well.
It is a per user lockout or again in this case.
We don't want to think about this as per user.
This entire idea can be used with any arbitrary kind of resource.
So this entire idea would have worked with say comments as well.
Now at this point I just want to kind of reflect and say that a lot of this stuff was solved really
by adding on just that no field but we were not able to add on that number field until we started to
really understand the service that all these events were going to come from which in this case was the
transactions database so the transaction service you might merely say Well Steven this is only going
to work if we somehow number each of these transactions and that would have been easy because presumably
we're sticking these transactions in array tied to user C.C. Q That is true.
But as you'll see very shortly we can apply the same methodology to normal records as well so we could
use the same methodology with say our tickets service.
So we just worked on a little bit ago with our ticket service.
We don't have an array to rely upon or anything like that.
But eventually we might want to emit some kind of event like say ticket related until the outside world
that hey we just created a ticket and here's some information about it.
So again we don't really have the benefits or anything like that of working with some numbering of these
tickets because a ticket is one standalone record but there are some ways so we can have a very similar
approach or the similar idea of numbering updates to a ticket over time and then making sure that those
updates are applied sequentially to some other service that relies upon its ok.
So at this point you might be sitting there thinking I'm not quite seeing this stuff don't sweat it
we're going to pause right here we're gonna write out a little bit more code just to work with our little
test example that we've had going on around this accounting stuff we're then going to start to go back
over to events inside of our actual ticketing application and you're going to see how this idea of recording
a last transaction number of sorts is going to work to solve concurrency issues inside of our ticketing
application as well.
So as usual quick pause right here and I'll see you in just a minute.