1 00:00:00,950 --> 00:00:05,620 In the last video we saw that we can get into some really big trouble if we ever successfully save a 2 00:00:05,620 --> 00:00:09,610 record to the database and then fail to emit an event describing that record. 3 00:00:09,610 --> 00:00:15,460 We just saved so we would not update this other database or this other service over here and a user 4 00:00:15,460 --> 00:00:18,450 would see some really strange data inside of our app. 5 00:00:18,520 --> 00:00:19,960 So how can we solve this. 6 00:00:19,960 --> 00:00:24,280 Well luckily there's a somewhat straightforward approach to this pretty straightforward and it's going 7 00:00:24,280 --> 00:00:28,880 to work vast majority the time to make sure that we don't run into any really big issues. 8 00:00:28,880 --> 00:00:34,780 So here's the idea though as in the diagram we were just looking at in the code that we currently have. 9 00:00:34,950 --> 00:00:40,140 We have seen that whenever we save record to the database we're then going to immediately emit an event 10 00:00:40,170 --> 00:00:45,480 or publish it over to our event bus to solve this whole issue rather than emitting an event over to 11 00:00:45,480 --> 00:00:46,530 the event bus. 12 00:00:46,530 --> 00:00:53,570 We're going to instead save that event into our database so we're going to still take some record in 13 00:00:53,570 --> 00:00:59,240 this case a transaction or whatever a ticket whatever it is save it into our database and we're going 14 00:00:59,240 --> 00:01:04,970 to simultaneously take the event that we want to emit and rather than immediate we're gonna save it 15 00:01:04,970 --> 00:01:08,520 into some kind of events collection inside of our database as well. 16 00:01:09,980 --> 00:01:14,000 Then along with that event we're going to record some kind of flag on whether the event has actually 17 00:01:14,000 --> 00:01:15,110 been published. 18 00:01:15,320 --> 00:01:19,020 When he first say this thing into the database no the event has not yet been published. 19 00:01:19,100 --> 00:01:26,890 So you would say no it has not been set then inside of some separate process or some separate piece 20 00:01:26,890 --> 00:01:29,200 of code something outside of our root handler. 21 00:01:29,350 --> 00:01:35,730 We can then have some code that is watching this event call it collection and it can take note of any 22 00:01:35,730 --> 00:01:38,180 time we save a new event into it. 23 00:01:38,290 --> 00:01:42,910 Any time we save a new event into this event collection this separate piece of code can then extract 24 00:01:42,910 --> 00:01:49,270 that event essentially pull it out of that collection and publish it off to nets once it successfully 25 00:01:49,270 --> 00:01:49,950 gets published. 26 00:01:49,990 --> 00:01:52,530 We can then update this scent flag over too. 27 00:01:52,540 --> 00:01:52,950 Yes. 28 00:01:52,960 --> 00:02:00,040 This event has been published for this kind of two stage approach is going to go a long way to solving 29 00:02:00,040 --> 00:02:04,340 this entire issue if there's ever some issue connecting to our database. 30 00:02:04,360 --> 00:02:07,900 Well then we will fail to save this transaction and the event. 31 00:02:07,900 --> 00:02:13,580 And so we will not have any data integrity issues if is down or if our connection to Nats is down for 32 00:02:13,580 --> 00:02:19,100 some crazy reason we can still save the ticket to the database or save the transaction to the database 33 00:02:19,340 --> 00:02:24,180 save the event and then hopefully at some future point time when our Nats connection comes back up this 34 00:02:24,190 --> 00:02:28,990 separate piece of code or process or whatever else can still pull this event out and publish it often 35 00:02:29,000 --> 00:02:34,040 that's this approach really makes sure that we're not going to run into this issue where we save a record 36 00:02:34,070 --> 00:02:37,450 and fail to publish some data around it. 37 00:02:37,460 --> 00:02:39,200 There is one other thing to imagine. 38 00:02:39,200 --> 00:02:45,260 However there might be some kind of crazy possibility where we successfully save this transaction in 39 00:02:45,710 --> 00:02:48,140 and then maybe fail to save that event. 40 00:02:48,320 --> 00:02:53,470 Maybe fail to save the event because there is some kind of database constraint or something like that. 41 00:02:53,660 --> 00:02:58,940 So we really need to make sure that whenever we save a transaction though if that gets saved we have 42 00:02:58,940 --> 00:03:01,920 to make sure that the event gets successfully saved as well. 43 00:03:02,150 --> 00:03:07,370 And if saving either these things fail then we need to make sure that we undo all the changes that we 44 00:03:07,370 --> 00:03:08,800 have made to the database. 45 00:03:08,810 --> 00:03:13,870 In other words if we successfully save the transaction but not the event we need to make sure sure we 46 00:03:13,870 --> 00:03:15,760 roll back saving the transaction. 47 00:03:15,980 --> 00:03:21,350 And if we save the transaction or something save the event but not the transaction we need to make sure 48 00:03:21,350 --> 00:03:23,780 we roll back saving the event as well. 49 00:03:24,320 --> 00:03:28,620 So to do something like this a lot of databases have something like this already built in. 50 00:03:28,800 --> 00:03:30,670 Even Mongo DB does. 51 00:03:31,020 --> 00:03:33,960 This is the idea something called a database transaction. 52 00:03:33,970 --> 00:03:38,220 Now let me pause right here and say that we have the word transaction on this diagram twice and these 53 00:03:38,220 --> 00:03:40,760 are two very different transactions. 54 00:03:40,860 --> 00:03:45,840 So inside of this database this transactions collection is talking about a user depositing some money 55 00:03:45,840 --> 00:03:47,130 or something like that. 56 00:03:47,460 --> 00:03:50,510 That is some very application specific behavior. 57 00:03:50,520 --> 00:03:54,690 A lot of databases on the other hand so this is the other use of the word transaction. 58 00:03:54,690 --> 00:04:00,390 A lot of databases implement a feature called a transaction and a database transaction is something 59 00:04:00,390 --> 00:04:04,610 that allows us to say there is a set of changes that we want to make to our database. 60 00:04:04,800 --> 00:04:11,040 And if any of these changes fail do not make any of the changes so we can wrap up saving this record 61 00:04:11,670 --> 00:04:14,410 and saving the event inside of a database transaction. 62 00:04:14,490 --> 00:04:19,170 And if saving either of these things fails the database will automatically roll back all these changes 63 00:04:20,310 --> 00:04:25,660 so a database transaction will ensure that we always save the record and the event along with it successfully 64 00:04:25,870 --> 00:04:29,230 or both them will fail. 65 00:04:29,320 --> 00:04:31,320 So again that's how we're going to solve this issue. 66 00:04:31,320 --> 00:04:35,680 Now you might recall earlier on inside this course I said that I wanted to build this application as 67 00:04:36,040 --> 00:04:41,500 realistically and production great as possible but that we were going to cut one or two corners so this 68 00:04:41,500 --> 00:04:46,360 entire system right here of saving an event into the database and publishing it later on. 69 00:04:46,360 --> 00:04:48,550 And this entire idea of a database transaction. 70 00:04:48,640 --> 00:04:50,940 This is where I'm kind of drawing the line for the course. 71 00:04:50,990 --> 00:04:55,600 We are not going to implement this inside our application just because it adds in a pretty good amount 72 00:04:55,600 --> 00:04:56,530 of complexity. 73 00:04:56,650 --> 00:05:01,990 And to be honest with you I don't really anticipate us to ever really have to deal with an issue around 74 00:05:02,050 --> 00:05:07,840 saving a record to the database and then possibly not successfully saving some person and publishing 75 00:05:07,840 --> 00:05:09,730 some information over to nets. 76 00:05:09,760 --> 00:05:15,790 So I just don't really think we are going to experience this issue inside of application and I don't 77 00:05:15,790 --> 00:05:21,070 really expect that we will ever even be able to effectively test that or test this realistically as 78 00:05:21,070 --> 00:05:21,430 well. 79 00:05:21,460 --> 00:05:26,160 We could definitely write some kind of automated test to make sure that this thing works expect as expected. 80 00:05:26,440 --> 00:05:31,030 But I just don't really expect us to be able to test it in any realistic scenario. 81 00:05:31,030 --> 00:05:33,750 So we are not going to implement this inside this application. 82 00:05:33,860 --> 00:05:37,990 But I just wanted you to know that this is a concern and it's something that you might want to write 83 00:05:37,990 --> 00:05:39,590 some code to handle. 84 00:05:39,670 --> 00:05:46,130 Again we would save it or deal with this issue by saving the record an event at the same time having 85 00:05:46,230 --> 00:05:51,580 a separate process to pull events out at events collection publish them often that's and we would probably 86 00:05:51,580 --> 00:05:57,930 want to also make sure that we wrap up saving the record and event inside of some transaction as well. 87 00:05:58,020 --> 00:05:58,410 OK. 88 00:05:58,410 --> 00:06:00,360 So that's that as usual. 89 00:06:00,390 --> 00:06:01,290 Quick pause right here. 90 00:06:01,290 --> 00:06:02,300 Continue in just a minute.