1 00:00:01,300 --> 00:00:05,260 In the last video we saw that we could read deliver all the events that have ever been emitted but that 2 00:00:05,260 --> 00:00:07,570 was probably not the best strategy in the world. 3 00:00:07,570 --> 00:00:13,120 So in this video we're gonna take a look at a very complimentary option to this idea of reading shivering 4 00:00:13,270 --> 00:00:16,060 all events that have been emitted in the past. 5 00:00:16,060 --> 00:00:21,880 The option we're going to take a look at is something called a durable subscription a durable subscription 6 00:00:22,150 --> 00:00:26,000 is going to be created when we give an identifier to a subscription. 7 00:00:26,020 --> 00:00:28,210 Let's first go and take a look at how we do that. 8 00:00:28,300 --> 00:00:33,520 Back inside my editor I'm going to once again find this option stuff and I'll chain on another method 9 00:00:33,520 --> 00:00:34,770 called right here of set. 10 00:00:34,840 --> 00:00:36,730 Durable name. 11 00:00:36,850 --> 00:00:39,090 This is how we create a durable subscription. 12 00:00:39,310 --> 00:00:44,320 When we call set durable name we're gonna put in a string that's going to serve as the name or the identifier 13 00:00:44,350 --> 00:00:45,780 for the subscription. 14 00:00:45,910 --> 00:00:51,840 We're usually going to provide a name that kind of names or is the same name as this overall service. 15 00:00:51,880 --> 00:00:58,240 So I inside of here would put in something like say order service or accounting service or whatever 16 00:00:58,240 --> 00:01:00,850 we are trying to build out. 17 00:01:00,890 --> 00:01:03,410 So what does a durable subscription actually do. 18 00:01:03,410 --> 00:01:09,410 Well let's imagine we created our subscription with a name or I.D. of ABC one two three. 19 00:01:09,560 --> 00:01:14,540 When we create that subscription Nats internally inside of the channel that we are subscribing to or 20 00:01:14,540 --> 00:01:20,120 listening to is going to create a record listing all the different durable subscriptions that we have. 21 00:01:20,890 --> 00:01:26,590 Then whenever we emit an event that's is going to record whether or not this subscription has received 22 00:01:26,650 --> 00:01:29,150 and successfully process that event. 23 00:01:29,240 --> 00:01:31,320 Let's imagine that we emit event right here. 24 00:01:31,320 --> 00:01:33,480 Number one or sequence number one. 25 00:01:33,490 --> 00:01:39,340 That's gonna go over to our service and our service might then acknowledge this event and say OK I've 26 00:01:39,340 --> 00:01:45,040 successfully processed this as soon as that has been successfully processed NATS is going to store a 27 00:01:45,040 --> 00:01:51,400 little record next to this terrible subscription and is going to say OK ABC 1 2 3 has processed 28 00:01:55,200 --> 00:01:57,720 Event number one successfully. 29 00:01:57,750 --> 00:02:02,370 So now let's imagine that for some reason our account service goes offline temporarily for some point 30 00:02:02,430 --> 00:02:10,760 in time and maybe while it is off line we then emit events 2 and 3 so that service is no longer online 31 00:02:10,760 --> 00:02:16,010 to receive this but internally NATS is going to keep a record of all the different events that ABC one 32 00:02:16,010 --> 00:02:17,870 two three has missed out on. 33 00:02:17,960 --> 00:02:20,310 So it's going to say OK number two right here. 34 00:02:20,450 --> 00:02:23,440 ABC 1 2 3 has not processed this and 3. 35 00:02:23,540 --> 00:02:27,050 ABC 1 2 3 has not processed this then. 36 00:02:27,080 --> 00:02:33,500 Whenever our account service comes back online and reconnects with the exact same idea right there it 37 00:02:33,500 --> 00:02:38,570 has to be the same NATS is gonna take a look at that subscription I.D. or the double name and it's going 38 00:02:38,580 --> 00:02:44,750 to say okay for ABC once you three you have already processed Event number one but you have not processed 39 00:02:44,750 --> 00:02:50,450 two and three so it will take two and throw it over and we'll take three and throw it over and then 40 00:02:50,450 --> 00:03:02,350 once the account service processes those they will be marked as processed and processed so journal subscription 41 00:03:02,350 --> 00:03:07,420 is really fantastic and making sure that our services never miss out an event and also make sure that 42 00:03:07,420 --> 00:03:13,000 we do not erroneously reprocess events in the future which was kind of the downside of this whole set 43 00:03:13,000 --> 00:03:15,160 deliver all available option. 44 00:03:15,310 --> 00:03:16,600 Now quick question here. 45 00:03:16,600 --> 00:03:22,330 Do we still need an option of set deliver all available if we are using set Durban name. 46 00:03:22,330 --> 00:03:24,170 The answer is yes. 47 00:03:24,250 --> 00:03:30,820 So setter or something set deliver all available is going to make sure that the very first time that 48 00:03:30,820 --> 00:03:40,560 our subscription is created like let's say that we emit one and two before our services ever been created. 49 00:03:40,620 --> 00:03:47,590 Let's imagine those are kind of sitting in history as kind of records hey we already emitted these things 50 00:03:47,590 --> 00:03:49,270 at some point in the past. 51 00:03:49,270 --> 00:03:56,420 If we then bring account service online for the very very first time and it has that option of set deliver 52 00:03:56,450 --> 00:04:01,700 all available then for the first time only NATS is going to make sure that it takes all those events 53 00:04:01,700 --> 00:04:08,030 from the past and deliver them to that service and add them as being delivered to ABC one two three 54 00:04:08,680 --> 00:04:12,960 and that is only going to happen for the very first time we create this subscription. 55 00:04:13,070 --> 00:04:17,450 So this set deliver all available is still a necessity because the very first time we bring the service 56 00:04:17,480 --> 00:04:22,010 up online we're gonna get all the events that have been emitted in the past which is fantastic because 57 00:04:22,010 --> 00:04:24,090 we can make sure that we don't miss out any data. 58 00:04:24,290 --> 00:04:28,700 But then because we also have the double name here those events we had delivered in the past will be 59 00:04:28,700 --> 00:04:30,110 marked as delivered. 60 00:04:30,110 --> 00:04:35,750 So then whenever we restart the account service let's say we restart this for some reason or we bring 61 00:04:35,780 --> 00:04:38,030 another subscription online. 62 00:04:38,030 --> 00:04:40,610 Well we have already delivered one in two. 63 00:04:40,640 --> 00:04:46,210 So even though we have set deliver all available that will be ignored on any restart. 64 00:04:46,240 --> 00:04:51,770 So this option right here is really just used for the very first time we bring the durable name subscription 65 00:04:51,830 --> 00:04:54,190 online. 66 00:04:54,210 --> 00:04:54,520 All right. 67 00:04:54,540 --> 00:04:55,580 Let's now save this file. 68 00:04:55,590 --> 00:04:58,100 We're gonna go over to our terminal and test this out. 69 00:04:58,170 --> 00:05:02,730 I want you to know that when we go and test this out even though we should only see all of our things 70 00:05:02,730 --> 00:05:07,020 get the BRI deliver it just one time and then after that we should not get any redo livery because all 71 00:05:07,020 --> 00:05:12,030 the previously admitted events have been marked as Red by this terrible subscription that is not quite 72 00:05:12,030 --> 00:05:13,170 what we're going to see. 73 00:05:13,170 --> 00:05:15,210 So let's just go take a look. 74 00:05:15,210 --> 00:05:17,420 All right so back over here I'm going to restart this. 75 00:05:17,460 --> 00:05:21,810 You'll notice that every single time I restarted I get all these different events re delivered and that 76 00:05:21,810 --> 00:05:25,090 is very much contrary to what I just said was going to happen. 77 00:05:25,140 --> 00:05:29,900 I just said that the very first time we start this thing up we're going to deliver all available. 78 00:05:29,910 --> 00:05:34,980 We're then going to set up the durable subscription name of accounting service and then for any restart 79 00:05:35,160 --> 00:05:40,110 because the subscription of accounting service has already gotten everything available it should not 80 00:05:40,110 --> 00:05:41,460 get any reader livery. 81 00:05:41,490 --> 00:05:43,380 So what's going on here. 82 00:05:43,380 --> 00:05:45,930 Well this is a little bit of a gotcha. 83 00:05:45,930 --> 00:05:51,610 Remember every single time that we restart this terminal window we are closing out our connection to 84 00:05:51,610 --> 00:05:57,060 nets and we are reconnecting in that very short window where we close out the entire connection. 85 00:05:57,090 --> 00:05:59,630 That's a saying oh that client disconnected. 86 00:05:59,770 --> 00:06:05,030 That was the clients with this subscription or this journal subscription since they disconnected. 87 00:06:05,050 --> 00:06:09,370 I guess they're probably never going to come back at any point time in the future so I'm going to wipe 88 00:06:09,370 --> 00:06:11,620 out that history of that terrible name. 89 00:06:11,730 --> 00:06:15,350 So for that just because that very very small disconnect window. 90 00:06:15,460 --> 00:06:19,490 NASA's going to say let's just dump this history about that durable subscription. 91 00:06:19,600 --> 00:06:21,020 They're probably not going to come back. 92 00:06:21,040 --> 00:06:22,180 Right. 93 00:06:22,210 --> 00:06:25,400 So that's obviously kind of a big downside to this approach. 94 00:06:25,630 --> 00:06:30,640 Just because our service goes down for half a second we're going to throw out that journal subscription. 95 00:06:30,640 --> 00:06:35,230 Well turns out that we can kind of use one other option we've already spoken about to solve this whole 96 00:06:35,230 --> 00:06:36,250 issue. 97 00:06:36,250 --> 00:06:41,230 So we're going to reintroduce our Q group inside of that stand subscribe to remember. 98 00:06:41,260 --> 00:06:50,250 That's gonna be the second argument that's going to be a string and I'll put in my Q Group Name like 99 00:06:50,250 --> 00:06:51,060 so. 100 00:06:51,220 --> 00:06:57,240 So by adding in the Q Group right here it's going to make sure that even if we very temporarily disconnect 101 00:06:57,360 --> 00:07:04,170 all clients are all subscriptions with this terrible name it will not dump the entire dribble subscription. 102 00:07:04,170 --> 00:07:09,460 So just by adding on the Q group even if all of our different services inside this Q Group go down NASA's 103 00:07:09,480 --> 00:07:13,880 going to persist that journal name subscription let's start saving this now. 104 00:07:14,200 --> 00:07:17,410 I'll then go back to my terminal the very first time we start up. 105 00:07:17,430 --> 00:07:22,920 We do get all of our events delivered but if now I restart this we're not going to get any re delivery 106 00:07:23,160 --> 00:07:28,670 because again by adding in that Q Group along with the verbal name it makes sure that every time or 107 00:07:28,680 --> 00:07:34,680 for some very brief period in time that everything goes off line the durable subscription does not get 108 00:07:34,680 --> 00:07:41,580 completely dumped now at this point I just want to really highlight how the set deliver all available 109 00:07:42,020 --> 00:07:44,570 this durable name in the Q group name. 110 00:07:44,730 --> 00:07:50,940 These three options really all meshed together extremely well and they give us the exact behavior that 111 00:07:50,940 --> 00:07:52,940 we are really really looking for. 112 00:07:52,950 --> 00:07:56,760 So we just need to remember that for every subscription we create we are probably always going to use 113 00:07:56,790 --> 00:08:01,250 such a lever all available that we can get all the events that's been emitted in the past we're going 114 00:08:01,250 --> 00:08:06,060 to use separable name to make sure that we keep track of all the different events that have gone to 115 00:08:06,060 --> 00:08:10,020 this subscription or the SKU group even if it goes off line for a little bit. 116 00:08:10,020 --> 00:08:14,850 And then finally we're going to use this Q Group to make sure that we do not accidentally dumped the 117 00:08:14,910 --> 00:08:19,040 durable name even if all of our services restart for a very brief period of time. 118 00:08:19,260 --> 00:08:23,970 And to make sure that all these emitted events only go off to one instance of our services even if we 119 00:08:23,970 --> 00:08:26,480 are running multiple instances so again. 120 00:08:26,480 --> 00:08:29,580 Q Group Name Satya liver all available and set terrible name. 121 00:08:29,580 --> 00:08:32,180 Very very tightly coupled options. 122 00:08:32,370 --> 00:08:36,740 Now very last thing on go and restart the second listener when we restart the second one. 123 00:08:36,750 --> 00:08:42,180 It's not tried to get a list of all previous events because all previous events have already been emitted 124 00:08:42,180 --> 00:08:47,830 into this durable queue group and finally I should be able to publish another one. 125 00:08:48,240 --> 00:08:50,570 Yep it still gets received over here. 126 00:08:50,730 --> 00:08:51,040 All right. 127 00:08:51,050 --> 00:08:51,970 So that's pretty much it. 128 00:08:52,020 --> 00:08:54,610 These three options totally key deliver all of. 129 00:08:54,640 --> 00:08:59,170 Terrible name and the Q Group let's take a pause right here and continue in just a moment.