Our application is now working pretty well but at the end of last video we saw a little issue and this
issue might be something that's been kind of nagging in the back of your head throughout this entire
course.
What happens to your application when a service goes down for some period of time.
We just saw that last video at the moderation service going down.
We created a comment while the moderation service was down and now this comment is going to be forever
stuck in the pending state.
So how do we deal with this.
Let's take a look at a diagram to better understand what the real issue here is OK.
So in this diagram kind of hard to understand what's going on.
This is kind of a time sequence diagram.
So we start off time of our application starting with the very first time right up here and we have
time flowing down or vertically in our application we've got the event plus the green lines throughout
all these different services indicate times in which the service was successfully running.
So we're gonna say that our event bus was running the whole time host was running comments and query.
Those are all running for the entire time that our application has been up and running.
And during that time maybe we've had a couple of events flow out into the event one went out to post
comments query and moderation successfully because all those services were up at the time that event
was sent out.
Well let's imagine that for some period of time indicated by this red line right here maybe the moderation
service was down.
So event two and three might have been emitted from the event bus gone over to our post successfully
went to comments and queries successfully but then failed to be delivered over to the moderation service
and then maybe sometime later moderation service came up and it received even four.
But events two and three have now been lost to time.
We don't currently have any way to tell moderation about these events and how they just occurred.
Now this is the scenario in which a service goes down for some little period of time but there's another
scenario that you've probably been wondering about again throughout the first hour.
This course already what happens if we're using this kind of event style approach and we don't bring
up or we don't even create a service until maybe sometime in the future.
So in this diagram let's imagine that we did not initially create our query service and so maybe we've
been running post comments in moderation for days maybe even years and they have tons of data inside
them and they've already received tons of events.
But maybe we only created that query service like one year down the line.
And so we're creating this great service but it's already missed out on all these different events.
How do we somehow get query into sync.
How do we get all the existing posts and comments and results of moderation over to the query service
well as usual.
There's a couple of different ways to handle all these different cases.
So let's take a look at some possible solutions.
So here's option number one perhaps.
Number one we're going to imagine that we've had our post service and common service running for the
entire lifetime of our application and then maybe one year into the future we create the query service
the query service of course needs to know about all of the posts that exist and all of the comments.
So we somehow need to get query into sync with everything else one way or option number one would be
to use synchronous requests.
So the instant we launch query maybe we'd have some code inside of the query service to make a direct
network requests over to posts and say Give me a list of all your posts.
It could get all the different post that exist inside the post service the plane network requests once
it's stored all this post it could then turn round to the com and service and also make a sync request
directly over and say give me all the comments you have and then I could store all those comments with
the associated posts.
Now the downside to this approach is pretty clear we are falling back to synchronous requests.
The real downside here is that we would have to have some code inside of posts and comments just to
service or kind of handle this new service that is coming online.
Right now we do not have any real bulk kind of endpoint where we can say give me all the posts for all
eternity per say well okay technically our blog application does right now but you can imagine that
in a production application we would probably not have an endpoint that just says Give me all posts
that exist probably would not implement something like that so if we did not already have some endpoint
like that we would have to implement it for both posts and comments now one thing to be clear about
after getting all the posts and comments and synchronizing all this data then from that point in time
into the future then the query service would start to receive any events or remitted from posts and
comments.
So eventually query would kind of revert back to our expected behavior but just to get it online and
up and running it could make these initial synchronous requests.
So that's option number one option number two is the one exception to that rule that we discussed way
earlier on INSIDE THE COURSE around every service having its own private database.
So with option number two which is kind of similar to option number one we can say that the instant
that query came online we could give it direct access to the data store or essentially the database
for all the post that we have.
And same thing for comments as well.
So rather than relying upon synchronous network requests directly to the service we could just say you
know what query needs all the information on a database.
Let's just give it access to the database.
Now the upside to this is now the query Service could run its own queries and figure out all the different
data that needs to get out of the Post Database and Commons database after synchronizing all this data.
Once again the query service could start listening to different events.
And the downside to this approach is once again we are making sync requests over which means we are
going to have to implement some code inside of query to work directly with whatever this database is
or this databases.
Let's imagine that maybe the post data store over here was my ask you all and maybe the Commons data
store was Mongo DB.
Now all the sudden we need to write some code inside of query to interface with a mystical database
and a Mongo DB database.
That's a lot of extra code to potentially have to write.
So onto option number three which as you can guess is what we are going to be doing.
All right.
So option number three really falls into the same category that we've gone through several times in
the course where I'm going to tell you this and you're going to say no way not possible.
Totally inefficient can't do it.
Not going to happen.
But once again yes this is how people actually implement micro services.
People do this.
Not making this stuff up with option number three.
We're going to once again say that we are implementing or bringing query online at some point time in
the future.
Like right here now in theory query could work if it had access to all the events that had been emitted
in the past.
So the post service for example is going to emit events maybe one two and three and each of these might
be a post creation event.
Those are all events that query cares about if query just had access to those events.
It can work just fine.
And that's what the strategy is going to rely upon.
We're going to say that whenever any of our services emits any event whatsoever over to the event us
the event buzz will send that event out to all the other services.
But it's going to do something else at the same time.
It's going to store that event internally.
So we can imagine that after event one gets emitted the event buzz is going to store that event internally
in some kind of data structure probably not in memory probably in some kind of database or something
similar because this data store is going to grow to be very very large over time we can then imagine
that maybe at the Postal Service immense event.
Number two.
And so we would add that on now event bus has a record of event one event too.
And then finally event three as well.
And so we can imagine that at that point time the event bus now knows about event 1 2 and 3.
Again I want to repeat these events are still going out to all the other services as they currently
are.
We are just adding one extra little step here.
The event bus is going to store all these events now the upside to this approach is that if query comes
along or online at some point in time the future where we can say over to the event bus Hey give me
access to all the events you have stored all them just throw them over and I'll decide whether or not
I care about them.
And so we can send over event one an event to and event three and now the crew service will be totally
up to speed and we can use all the same code that we've probably already written to handle these is
the exact events so we don't have to write any extra code.
The only extra burden we really have here is storing all these events inside of our event bus data store.
Now admittedly this data store could grow to be extremely large over time but as you saw from that calculation
we did a little bit ago where we said Hey we could pay like 15 dollars a month for 100 million products.
Well even though it might cost some amount of money it probably costs way less than you might think
so that's what we were going to do inside this course.
We are going to be using option number three.
We're going to say that whenever we emit into any event we're going to store it with our event bus so
that if we ever bring a service online in the future we can get access to all those events that occurred
in the past.
This also solves the issue with a service going down for some point in time.
Let's imagine I don't have a diagram prepared for this but I'll just improvise really quick so let's
imagine that moderation misses out on events 2 and 3 but it did receive event one when moderation comes
back on line right here.
It can take a look at one of the last event was that it received maybe moderation would know that it
had received event one and it could say to the event bus Hey give me all the events give me everything
and that occurred during the time that I was down and so event but can say OK you received event one
we'll give you everything since then.
So we can take you back to throw it over to moderation and event three throw it over.
And now moderation is all cut up.
It's received two events two and three even though it was down for some period of time you can see that
this approach actually ends up working out pretty well not only does it solve the issue of bringing
services online in the future but it also solves this problem of events possibly being missed while
a service is experiencing some amount of downtime.
All right.
So let's take a pause right here.
I apologize for the long video.
We'll continue in just a moment.