Make Your Apps Resilient Using Finite State Machines - With David Khourshid
David Khourshid talks about how finite-state machines can make our code stronger.
In this episode, David Khourshid gives the rundown on how finite state machines can make your app more testable, more resilient to bugs, and easier to refactor.
David's initial interest in finite state machines stemmed from his background in music. With music, there is a universal notation that crosses genre boundaries. David thought what if there could be music notation for logic? Well, it ends up people have been trying to figure this out for the last thirty years.
A finite state machine can only be in exactly one state out of a limited number of possible states. The machine can transition to another state through explicitly defined events.
David also chats with Kent about extended finite state machines, how state machines can be used to simplify integration testing, the differences between xstate and redux.
This episode's call to action is to take whatever feature you are working on and model it out in your head as a finite state machine, this can help you find potential bugs and also could lead you to implement a finite state machine style solution to avoid those bugs.
Kent C. Dodds: Hey everyone, it is your friend Kent C. Dodds here and I'm joined by my friend Dave Khourshid.
David Khourshid: Hey, how are you doing?
Kent C. Dodds: Good, good. Hey, so I say that you're my friend, but I know I have to ask. Do you go by Dave or David?
David Khourshid: David.
Kent C. Dodds: David.
David Khourshid: There's a special type of person who's a Dave and I'm not that type of person. I don't even know what that type of person is, but I'm mostly David.
Kent C. Dodds: Cool. Cool. My brother's name is David and I grew up always calling him David, but then when he got married his in-laws and that family started calling him Dave and now everybody in my family is calling him Dave now too, which is kind of weird, but.
David Khourshid: Yeah. I'm either David or DK.
Kent C. Dodds: Dk. All right.
David Khourshid: Donkey Kong.
Kent C. Dodds: Oh, nice. Nice. Or piano man you might-
David Khourshid: Actually, yeah. Yeah.
Kent C. Dodds: Cool. Well, so David, was it like two years ago that you spoke at React Rally about xstate? Has that been that long?
David Khourshid: It was two years ago. Yeah. I went to React Rally last year, but just to attend, which was a great time, but two years ago was pretty much the first time I talked about finite state machines in general at a conference. So I was really happy to be able to do that at React Rally.
Kent C. Dodds: Yeah, that's ... I remember that talk because of course I was there. I never miss, if I can snag a ticket, which is not easy. But yeah, by the way, React Rally is my favorite conference. So those listening, try to get that ticket. It's so good.
David Khourshid: Yeah.
Kent C. Dodds: But yeah, so I remember that talk. I was really impressed. I thought it was super cool, like, conceptually. I still have yet to actually build something with xstate, but every time I think about managing complex state I always go back to xstate as probably like the best solution. I need to test that theory, but that's what I'm working with right now.
Kent C. Dodds: So yeah, let's talk about more than just xstate. Let's talk about the general idea of what a finite state machine really is. And I'm kinda curious how you got into all of this too, so why don't we start there. I like to kind of start with the why behind the topic. So what made you interested in this, and why do you think other people should dedicate some of their brain cycles to it?
David Khourshid: It's going to sound weird, but it actually stems from my musical background. You know, if you play an instrument or anyone who plays an instrument really, they know that there's music notation that you would read either from sheet music or maybe guitar tabs or like a lead sheet or whatever that basically tells you in a very visual way how something is going to be performed. I realized like, when I studied music in college we had to study so many different genres and different types music, but they all use the same music notation. This music notation was expressive enough to be able to inform the player how to play these many different styles of music. So as I grew in my development career, I was thinking, "What if there was some sort of music notation for logic?" Like application logic, application behavior, things that change over time.
David Khourshid: Of course, naively at first I tried to make my own, but then I discovered there's this really old computer science concept called state machines. It was presented in a very theoretical sense, but I tried to dig deeper and think about how it could be used in user interfaces and in applications and things that weren't strictly theoretical or strictly academic. So that's when I stumbled into state charts, and I realized that people have been asking the same questions that I've been asking for the last 30 or so years and some people much, much smarter than me have already developed this idea of state charts and using state machines and state charts in user interfaces and other types of applications like embedded electronics and aircraft control, all of these advanced use cases.
David Khourshid: Then I started seeing them everywhere. I think that might be where ... Like, I've heard a lot of people, you included, just seeing these state machines in the software rewrite. Then we stop and think about like, "Oh, you know, this could be expressed as a finite series of states instead of just random booleans," so.
Kent C. Dodds: Dude, that's really interesting. You know, there really is nothing new under the sun, but it's cool that it kind of came out of your thoughts around music notation. So finite state machines, I think maybe they're like ... we can understand each of those terms in isolation, but together maybe we could get a more clear definition. So can you help us understand what exactly is a finite state machine?
David Khourshid: So, the main idea behind finite state machines is that you are in only one of a finite number of states at any given time, so a promise cannot be both resolved and rejected. That makes absolutely no sense. It's impossible, nor could it be pending and rejected at the same time. The things that cause these changes in states are events because it doesn't just happen for no reason. Everything happens based on an event. So finite state machines are these finite states, these finite number of events that either a user could do or some other external system could do and how these states change from one state to another based on those events. In a nutshell, that's all finite state machines are. And of course there's an initial state, you have to start from somewhere.
Kent C. Dodds: Yeah, that makes a lot of sense. So like conceptually we can think of many examples, like stoplights is I think probably a pretty worldwide understood example. But in code this is represented by you have your object and that object may have a certain number of properties. Like if we were to take your promise example, it would have the resolved or rejected. Like, there's the response and there's the error if you're doing a fetch call or something like that. So it would be, like, there could be multiple properties in our state, but we can only have one, a set of those properties in our state at a time. Am I kind of going off the rails there or is that kind of ... does that make any sense what I just said?
Kent C. Dodds: Let me take a step back. So like maybe a good thing to define is what state even is. I define it as something ... it's data or a value that changes over time. So like you have static data, like we have data.JSON in your project or whatever. It's just static data. And that could be like the user's, you know, first name and last name that maybe that'll, you know, never change in the lifetime of our application anyway. That would be just data. But then there's state which could be represented by the position of the mouse on the page or something like that. That's something that can change over time. So that would be state, and that's something that could be represented in a finite state machine, right?
David Khourshid: Yeah. So you could think about state in two ways. There's the discrete finite states and there's sort of continuous or infinite states. So if you think about a cup of water, the water could have maybe a hundred milligrams of water or two cups of water or three quarts of water, something like that. Like just an infinitely large number of different measurements that you could have in that cup of water. But you could also separate it into distinct categories of the cup is empty, the cup is filling, or the cup is full. So finite state machines can represent the finite states of that where you could have empty filling or full, and with extended finite state machines or state charts, which is an extension that [David Herro 00:09:34] made in 1987 I'm pretty sure.
Kent C. Dodds: That was like a year before I was born, man.
David Khourshid: Oh, Wow. Yeah. There's the notion of extended states or [inaudible 00:09:48] called context, but this idea that we're separating finite state from this infinitely many values of states. So, for example, your first name, your last name, my date of birth, just things that could be very, very numerous and we can't really express all possible combinations of that. So in the state charts view state is a mixture of this finite state and this infinite state, if that makes sense.
Kent C. Dodds: It kind of makes sense. So like when I'm building a finite state machine with xstate, for example, I'm going to take my objects that represents all of the states, all the finite states, and I'm going to feed that into my state machine with xstate. But then there are like dynamic pieces of data or information. That was actually a question I wanted to ask about during our chat was how do you represent those kinds of pieces of information where maybe the database, like there's some sort of bad code on the back end so my data is in a corrupt situation where like it is an impossible state or something like that. So, we can't really represent that up front anyway.
Kent C. Dodds: Like, as I'm writing out my state machine configuration or whatever. So like where do those two places meet when I'm like in a typical practical application that I'm building with something like xstate?
David Khourshid: So you're talking about like if you want to represent something as finite states but the infinite state might be corrupt or something?
Kent C. Dodds: Well yeah, that may have been a little tangential. I'm really interested in and like I do have those two different things where I know all of the states and I know what the values should be. You know, given my promises in the resolve state, I know what these values should be or, or whatever the case may be. I can represent the state of my component based off of this type of event or whatever. But then we have the dynamic data that's coming from the server. So where do those two things mesh in a like a practical, real world scenario?
David Khourshid: So this is sort of the difference between redux and xstate, is that xstate you get back this state object, which it has a value, which is your finite state. As in, let's take a promise for example, it's either idle, loading, success, or failure, right? Those are the four possible states that could be in, and when it's in success you have this data, which of course could be anything. It's whatever payload comes from that and, you know, failure of course. Whatever error message comes from that as well. So it combines the two in this object.
David Khourshid: You know, when I talk about this, I actually keep thinking about the stopwatch example that you made with Hooks, like you and Dan were going back and forth between this stopwatch example. The one thing I noticed, and so this sort of speaks to the practical aspect of this, is that in the stopwatch example there was no notion of stopped or playing or reset or any of these discrete events. It was just, and this is even in Dan's example when he tried to show you the right way ... I mean, of course his way is correct.
Kent C. Dodds: It works.
David Khourshid: It works. Yeah. So technically correct, but it was trying to determine state and to a set state based on those infinite values. For example, when you press reset, all it said was like, oh yeah, set the current time to zero and set the play rates to zero. But those aren't finite states. They were deriving finite states from that but it's not really explicit. So xstate, if I were to rewrite a stopwatch example in xstate, not only would I have some part of this state saying, "Here's the current time," I would have a part of the state saying, "It's playing," or, "It's paused," or, "It's reset," or something like that.
Kent C. Dodds: Yeah. I actually, I love that idea. I'm totally going to rebuild the stopwatch with xstate now to figure out, because I've been like thinking of, well, I need to find a good example or a good use case or scenario for me-
David Khourshid: Stopwatch would be a perfect example. Yeah.
Kent C. Dodds: Yeah. Then once I figured that out, I'll ask you about it and then I'll probably do a live stream or something explaining how it works. That'd be pretty cool. I love that.
Kent C. Dodds: Cool. So we do have these, like in the stopwatch example you have that lapse time that can be calculated from the time that the stopwatch was started minus the current time or the current time minus that, so like our state machine would only represent the finite states like it is running, it is stopped.
Kent C. Dodds: Now, when it's paused that state needs to live somewhere as well. Do you have ... when you're representing this finite state machine, though, those actual values are not something that you know ahead of time. Like, I don't know when the start time is going to be and I don't know when the pause time is going to be. So those wouldn't be stored in my state machine configuration object, I don't know what you call that. But those values wouldn't be stored there. So do I separate my finite states from this more ephemeral, like you know, these values that I'm going to find out in the future, or?
David Khourshid: So they don't necessarily separate them. Where they live in the state is in those two pockets. So the value, which is the finite state, and the context, which is your more infinite [inaudible 00:15:59] so those come back in the same object. So inherently they're together. So what you've define in your state machine is ... let's say you have a start event where it tells you the current time that you started or you have a lapse event where it says here's how long the lap was or stop event telling you what time it stopped. Those events carry data, so they could be ... and Redux does the same thing. You have a discrete type for the event such as start, lap, reset, stop and you also have these potentially infinite values. So, start time, the person's name, something like that. So, you would represent that in the state machine as well, like assigning those values to the context or the infinite state. That actually prevents a lot of bugs that we don't realize are present in so many of our applications.
David Khourshid: Like, you've worked with Redux a lot, I'm sure.
Kent C. Dodds: Yeah, sort of. I worked with it a little bit a couple of years ago, long enough for me to realize that it wasn't my thing. So I haven't done a lot of Redux but I definitely understand it, so we can chat about Redux.
David Khourshid: Right, so I'm sure many listeners are familiar with Redux or at least the concepts behind Redux too. So something I see very often is that we're basing what the next state is just on the event, not in the current state. If it is a current state and they really need to use a current state, they might add an if statement, like: if this is this state, then do this as well.
David Khourshid: So if you think about like a simple loading example where you're trying to load async data and you do it based on event, where it's like I have a success event with some data, put it in there, and then change this and change that. That's not adequate because what if I ask you, "Can this success event be done without me even loading, without me even going to the loading state?" And the developer might say, "No, that would never happen."
David Khourshid: Then I would go and I would tell them, "Can you prove it? Like, if we put $100 on the table, you know, guarantee me that, that will never happen." And then they would say like, "Oh, well, I mean the button is disabled." But yeah, it's like there's no [crosstalk 00:18:37]-
Kent C. Dodds: It's impossible.
David Khourshid: Yeah, there's no formal way. There's no guarantee that it's impossible for that to happen. So any event can happen at any time and change data at any time. And that increases the surface area of application behavior that we really want to narrow it down and make impossible states impossible.
Kent C. Dodds: Oh, that makes so much sense. And that leads into the idea that like the finite state represents not only the state that it could be in, but also the states that I can travel to from that state.
David Khourshid: Exactly. Yeah. Which is a very important concept.
Kent C. Dodds: Yeah, that makes a lot of sense. And actually that leads into a great discussion I wanted to get into as well is Redux is typically used for like entire application state, and I'm curious about xstate and where it plays in with that kind of thing. Would I be able to use xstate or some other finite state machine implementation? Am I able to use this concept for my application state? Is it typically better for a single component state? Where is the best application of this kind of thing? Like, would I use it ... Here's where I use my component state and I have Redux for my application state. Like, where's the best practical scenario there?
David Khourshid: That's also one of the big differences between Redux and xstate or just using finite state machines, and it's that Redux embodies, this whole global nature. There's one single atomic state, events can happen from anywhere, it updates the state, and then you use a bunch of fancy selectors and memoization to make sure that updating the global state doesn't make everything fire off and change, because that would be a performance-
Kent C. Dodds: Yeah, and we're experiencing that with Hooks, because you can't [crosstalk 00:20:36] early. Yeah.
David Khourshid: Exactly. So xstate is sort of a different idea. xstate really embodies the idea of local state. Now, of course you could have like an xstate, like a state machine at the very top and then state machines for each one of your components and then they could send events to each other, and by having this distributed local state, you have more of what's actually known as the actor model, which is also ... it's actually an older idea in the state charts. So the actor model, which I know this is going on a tangent, but I promise it's related.
Kent C. Dodds: Oh, no it's fine. No, let's go into the actor model.
David Khourshid: Yeah. Actor model is mainly used for concurrency among distributed systems, but it can also be used in user interfaces to simplify how different components work together. So a real quick explanation. Let's say you're an actor, I'm an actor. The way that we're communicating with each other as the way that actors communicate with each other, we're sending events to each other. Like I'm sending you in event with my voice, you're sending me an event back with your voice and we change our internal state based on those events.
David Khourshid: Now, the way we code is sort of antithetical to that. It's more like we could read each other's minds and we could hope that I could read your mind or something. But with the actor model it's like think of each of your components having local states and that's it. And the way components change their state with relation to other components is they communicate with each other based on events. Now, in React we sort of do that naturally without thinking. Like when we have on change, on whatever, on disabled, just basically all of these event handler props, that's a way of a parents component communicating with the child component, or a child component communicating with a parent component. So, to answer your question, it doesn't matter whether you use Redux or xstate or whatever, it's just the idea that you do need local state, and local state is not something that should be avoided.
David Khourshid: I think that, that was the thing that Redux might've unintentionally encouraged. And I wrote a form library called React Redux form, which I don't really maintain anymore. I just tell people to go Formik and I [crosstalk 00:23:07] and I review them and everything, but it was a form library based on Redux. I found that people were having very significant performance issues once they had forms with maybe 50, 70, 100 fields. This is very easy to envision, like if you make some sort of spreadsheet app or an app where you have many line items, things like that, it easily gets to that many inputs and that many components. So I had to make some sort of makeshift local React Redux form for each component, which it works, but it sort of felt like going against the grain with Redux. So that's why local component state is great.
Kent C. Dodds: Yeah, I'm definitely a big fan of that. So if I recall correctly with xstate I can like have nested state charts, right? Like so I can build like just like I have a nested component tree, I could have these nested state charts where, oh you know what ... like in React, if you need two components to communicate with each other, they're siblings, you're going to lift state. Is that the same concept when I'm using xstate that I'm going to ... say I have these two places that need to communicate state, so I'm going to lift that state to this, you know, state chart above or how does that work?
David Khourshid: Sort of. So what you're describing would be more of having a parent state chart where two components just communicate with that parent and that parent communicates back with the two components. Now, nested state machines, it's more for refining states. So let's say that we have a loading application where you want to fetch some data, but it might be fetching multiple parts of data. So you would have three states, let's say idle, loading, and success, but that loading state you might want to refine more, like, load user, load group, load friends, load metadata, like just all these sub states of that main state. So it's just a way of refining state and grouping states in a logical way that prevents you from having to create so many different states and transitions. That's what its purpose is.
Kent C. Dodds: Cool. Okay, cool. So we're coming down on our time, but I did want to ask you about a couple of other things because I happened to know that you are doing some really interesting things with testing and finite state machines that have nothing to do with whether your application is actually using xstate or finite state machines, but is actually this concept of model based testing. Can you tell us a little bit about that?
David Khourshid: Yeah. So actually just going to your article, you wrote an article a while ago about the testing trophy and it is an important point because it shows that unit tests, you could write 100, 1000 unit tests, but your application could still have serious flaws, whether it's in behavior or edge cases, that you might not catch in unit testing. But the problem is that with integration tests, even though integrating and end to end tests are extremely important, and they actually test how your application is going to behave in real life, these tests are difficult to author. Just authoring one takes a chunk of time. Then for every single other possible thing your user could do you would have to author another one. Then if application functionality changes, then you have to rewrite or refactor every single one of those integration tests. I think that's one of the reasons that people tend to avoid writing too many integration tests, which of course makes your application's reliability suffer.
David Khourshid: So there's this other old concept. I'm all about old concepts, but there's this whole concept called model based testing. So what it is, is you would describe your application as a state machine, even in a vague way. It doesn't matter if your application is written with a state machine or not. You would just say, "My application behaves like this." It starts in this state, it moved to this state and so on. So you describe your application as this and then you write heuristics saying, "When my application is in this loading state, there's a spinner available." So if the spinners on the screen then just assume it's on a loading state. So you would write all these heuristics and you would also write things that say, "The action of submitting the form is physically going to the submit button and clicking it."
David Khourshid: With the combination of this you could take your state machine and you could trace every single path from the start states to any other state and you generate hundreds and maybe thousands of different test plans. And each of those test plans says, "In order to get to this state you have to go here. Do this, go here, do that." So it's sort of like Google Maps. Long story short, it's a way for you to automatically generate integration in end to end tests without having to write any of them by hand, which is great.
Kent C. Dodds: Yeah, I think ... and the key element there is the fact that the state machine encodes the transitions that you can go from one state to another, and that's what enables that kind of power. Right?
David Khourshid: Right, exactly. And so you could use the same test plans, I call them, for writing both integration and end to end tests too.
Kent C. Dodds: Cool. Yeah. Yeah. That makes a lot of sense. Where integration, maybe you mock out the backend [crosstalk 00:29:08] calls, but then end to end just let them happen.
David Khourshid: Exactly.
Kent C. Dodds: Yeah, that's a very interesting and a cool concept. And the fact that ... and I think maybe this is a really good takeaway for people or a call to action for people is to say like, take the feature that you're working on right now and take a step back and see if you can model it in your head as a finite state machine where, okay, so the user enters my form or they enter the checkout cart or whatever. Maybe it's a checkout flow. Then they transition from that into they enter their payment information. Then if they enter it wrong, then they go into a state of error. I feel like that exercise could really help, not only in you identifying potential bugs like, "Oh, I didn't think about what could happen if they hit the back button here," or something, you know?
Kent C. Dodds: Then it can also have the benefit of you thinking, "Hey, maybe I could apply some finite state machine solution to this situation so that I avoid those kinds of bugs."
David Khourshid: Right. Even if not, if you're just ... this is something I tell everyone, it's like if you don't need to use my library, don't use it. Take a pencil and paper, draw out each state in each arrow between the states and you're going to quickly realize, I forgot to add that, I didn't consider this edge case, and more just by thinking of your application as a finite state machine.
Kent C. Dodds: Yeah, I think that's super great advice and a good thing to kind of end on here. Yeah. Like as you're drawing that out, I'm just picturing this in my mind, like you're drawing arrows connecting these different states and then you realize, oh, I actually, I think in my application it's right now possible to transition from this state to that one and that should not be possible. So I got to go fix that bug.
David Khourshid: Right.
Kent C. Dodds: Yeah. Very cool. So is there anything else that you'd like to just give to our audience to take away from this conversation? Any other calls to action or anything?
David Khourshid: Well if you want, please check out my xstate library. By the time of publishing this, there might even be a new version with new features and some extra functionality around testing analytics, simulation, and more. And I'm working on it constantly, and there's also some really great communities around it, especially on spectrum.chat/statecharts. And, yeah.
Kent C. Dodds: Cool. Cool. All right. We'll get people on that spectrum. Spectrum is fantastic, by the way. I just love that. Very cool. And also, just as a teaser for people to go to xstate.js.org, because the animation is just great. I just refresh the page to watch that animation again.
David Khourshid: Thanks.
Kent C. Dodds: So people should check that out. Cool, David. Thank you so much for taking the time to chat with me about this. One last question: where can people find you on the Internet?
David Khourshid: I'm on the Internet everywhere @DavidKpiano.
Kent C. Dodds: Awesome.
David Khourshid: So, piano, the instrument I play. K, because my last name is too hard to spell.
Kent C. Dodds: Cool. All right, sounds good. That is, yeah, that's it for this episode. Thanks everybody, and we'll be in your ears next time. Good bye.
David Khourshid: All right, thanks.