Improve test error messages of your abstractions
Photo by Austin Neill
How to manipulate stack traces to get beautiful error messages with Jest and your test helper functions.
Let's say you've got this test:
1const add = (a, b) => a + b23if (add(1, 2) !== 4) {4 throw new Error('Expected 3 to be 4')5}
If you run that with node, here's the output you could expect:
1add.test.js:42 throw new Error('Expected 3 to be 4')3 ^45Error: Expected 3 to be 46 at add.test.js:4:97 at Script.runInThisContext (vm.js:116:20)8 at Object.runInThisContext (vm.js:306:38)9 at Object.<anonymous> ([stdin]-wrapper:9:26)10 at Module._compile (internal/modules/cjs/loader.js:959:30)11 at evalScript (internal/process/execution.js:80:25)12 at internal/main/eval_stdin.js:29:513 at Socket.<anonymous> (internal/process/execution.js:192:5)14 at Socket.emit (events.js:215:7)15 at endReadableNT (_stream_readable.js:1184:12)
That's a pretty standard stack trace for that error. The message is clear-ish, but we can do better and we do! If we write this same test with Jest, the resulting error is much more helpful:
1test('sums numbers', () => {2 expect(add(1, 2)).toBe(4)3})
That will fail with an error like this:
1FAIL ./add.test.js2 ✕ sums numbers (3 ms)34 ● sums numbers56 expect(received).toBe(expected) // Object.is equality78 Expected: 49 Received: 31011 2 |12 3 | test('sums numbers', () => {13 > 4 | expect(add(1, 2)).toBe(4)14 | ^15 5 | })16 6 |1718 at Object.<anonymous> (src/__tests__/add.js:4:21)
It looks even better in the terminal:
Nice right? Especially that codeframe. Being able to see not only the error itself. Now, I'm going to keep things contrived here to make it simple, but stick with me here. What if I like that assertion so much (or I have a collection of assertions) that I want to abstract it away into a function so I can use it in a bunch of different tests? Let's try that:
1const add = (a, b) => a + b23function assertAdd(inputs, output) {4 expect(add(...inputs)).toBe(output)5}67test('sums numbers', () => {8 assertAdd([1, 2], 4)9})
Please keep in mind, I am not recommending you create useless abstractions like the one above. As with everything, you should be applying AHA Programming (and for testing). This blog post is just useful for situations where the abstraction is clear and you want to include assertions in your abstraction.
Alright, with this little abstraction, here's the error we get:
1FAIL ./add.test.js2 ✕ sums numbers (3 ms)34 ● sums numbers56 expect(received).toBe(expected) // Object.is equality78 Expected: 49 Received: 31011 2 |12 3 | function assertAdd(inputs, output) {13 > 4 | expect(add(...inputs)).toBe(output)14 | ^15 5 | }16 6 |17 7 | test('sums numbers', () => {1819 at assertAdd (add.test.js:4:26)20 at Object.<anonymous> (add.test.js:8:3)
What!? That's not nearly as helpful! What if we had a bunch of places we're
calling assertAdd
? What good is that codeframe going to do us? How do we know
which one failed. Oh, there it is, I we do get a line in the stack trace,
but... like... talk about a step backward. I'd much rather have the line that
called assertAdd
be what shows up in the codeframe.
Well, there's no API into Jest for this (yet?), but you can force Jest to give you a codeframe where you want. So what I'm going to show you next is how we can make this error output like this:
1FAIL ./add.test.js2 ✕ sums numbers (3 ms)34 ● sums numbers56 expect(received).toBe(expected) // Object.is equality78 Expected: 49 Received: 31011 14 |12 15 | test('sums numbers', () => {13 > 16 | assertAdd([1, 2], 4)14 | ^15 17 | })16 18 |1718 at Object.<anonymous> (add.test.js:16:3)
Interested? Cool. Let's dive in.
Actually, it's pretty simple. Remember the full stack trace we had with regular
node? Well, when the expect
library throws an error, we get a full stack trace
as well. Let's take the contents of our assertAdd
function and put it in a
try/catch
so we can check out the error.stack
:
1function assertAdd(inputs, output) {2 try {3 expect(add(...inputs)).toBe(output)4 } catch (error) {5 console.log(error.stack)6 throw error7 }8}
Here's what's logged with that:
1Error: expect(received).toBe(expected) // Object.is equality23Expected: 44Received: 35 at assertAdd (/Users/kentcdodds/code/kentcdodds.com/add.test.js:5:28)6 at Object.<anonymous> (/Users/kentcdodds/code/kentcdodds.com/add.test.js:17:3)7 at Object.asyncJestTest (/Users/kentcdodds/code/kentcdodds.com/node_modules/jest-jasmine2/build/jasmineAsyncInstall.js:100:37)8 at /Users/kentcdodds/code/kentcdodds.com/node_modules/jest-jasmine2/build/queueRunner.js:47:129 at new Promise (<anonymous>)10 at mapper (/Users/kentcdodds/code/kentcdodds.com/node_modules/jest-jasmine2/build/queueRunner.js:30:19)11 at /Users/kentcdodds/code/kentcdodds.com/node_modules/jest-jasmine2/build/queueRunner.js:77:4112 at processTicksAndRejections (internal/process/task_queues.js:93:5)
That error.stack
has already gotten some helpful treatment from Jest's
expect
assertion library (it's even got helpful colors at this point).
Note that
error.stack
is actually a combination of theerror.message
+ the stack trace, so the error message thatexpect
provides is everything above the first "at" line which is where the stack trace actually starts.
Ok, so you'll notice that the stack trace we've got here is very different from
the one that Jest shows us. This is because most of the stuff in there is pretty
useless to developers. It's just noise. Why do developers need to know that
their code ran through mapper
function at queueRunner.js:30:19
? Yeah, pretty
useless. So
when Jest formats the stack trace,
it
filters out a bunch of the noise,
and we're left with:
1Error: expect(received).toBe(expected) // Object.is equality23Expected: 44Received: 35 at assertAdd (/Users/kentcdodds/code/kentcdodds.com/add.test.js:5:28)6 at Object.<anonymous> (/Users/kentcdodds/code/kentcdodds.com/add.test.js:17:3)
Definitely more helpful. The next thing Jest does is it takes the first line in the remaining stack trace lines and creates the codeframe for the first line. Then it formats filepaths and we're left with the relatively useless error + codeframe + stack trace shown above.
So, understanding that, the solution is pretty simple: ensure that the first relevant line in our stack trace is the one we want in the codeframe!
So, what we need to do, is filter out the one that includes the function
assertAdd
and we're off the races:
1function assertAdd(inputs, output) {2 try {3 expect(add(...inputs)).toBe(output)4 } catch (error) {5 error.stack = error.stack6 // error.stack is a string, so let's split it into lines7 .split('\n')8 // filter out the line that includes assertAdd (you could make this more robust by using your test utils filename instead).9 .filter(line => !line.includes('assertAdd'))10 // join the lines back up into a single (multiline) string11 .join('\n')12 throw error13 }14}
And with that we get the stack trace I previewed to you above. Here's a screenshot of that:
The problem with this is we actually don't want to just filter out our
utility. What if that utility function is built on top of other functions. So
really, we want to remove everything above our utility as well. This is
actually what Jest's expect
does and
it uses Error.captureStackTrace
.
Let's try that:
1function assertAdd(inputs, output) {2 try {3 expect(add(...inputs)).toBe(output)4 } catch (error) {5 Error.captureStackTrace(error, assertAdd)6 throw error7 }8}
Wow, that's a lot cleaner. So we pass the error
we want updated and the
function we want removed from the stack trace. That argument is called the
constructorOpt
.
According to the Node.js docs:
The optional
constructorOpt
argument accepts a function. If given, all frames aboveconstructorOpt
, includingconstructorOpt
, will be omitted from the generated stack trace.
It's almost as if this were created for our exact use case!
Conclusion
So here it is all together:
1const add = (a, b) => a + b23function assertAdd(inputs, output) {4 try {5 expect(add(...inputs)).toBe(output)6 } catch (error) {7 Error.captureStackTrace(error, assertAdd)8 throw error9 }10}1112test('sums numbers', () => {13 assertAdd([1, 2], 4)14})
And here's the output:
1FAIL ./add.test.js2 ✕ sums numbers (3 ms)34 ● sums numbers56 expect(received).toBe(expected) // Object.is equality78 Expected: 49 Received: 31011 11 |12 12 | test('sums numbers', () => {13 > 13 | assertAdd([1, 2], 4)14 | ^15 14 | })16 15 |1718 at Object.<anonymous> (add.test.js:13:3)
And here's what that looks like visually:
One other thing to note is that Jest automatically knows to not make a codeframe
out of a line that's coming from node_modules
. So if you publish your
utilities to npm
, you probably don't need to bother filtering things out
yourself. This is really only useful for those testing abstractions you find
yourself writing in a testbase at scale.
But manipulating the stack trace for improved error messages can be good
knowledge to have, even for things you publish to a registry. For example,
DOM Testing Library does this in waitFor
to make sure failures of asynchronous utilities (like find*
queries and
waitFor
itself) have beautiful errors and sensible stack traces (async stack
traces are pretty useless).
1● waitFor works23 TestingLibraryElementError: Unable to find an element with the text: /nothing matches this/. This could be because the text is broken up by multiple elements. In this case, you can provide a function for your text matcher to make your matcher more flexible.45 <body />67 2 |8 3 | test('waitFor has a nice stack trace', async () => {9 > 4 | await waitFor(() => {10 | ^11 5 | screen.getByText(/nothing matches this/)12 6 | })13 7 | })1415 at waitForWrapper (node_modules/@testing-library/dom/dist/wait-for.js:94:27)16 at Object.<anonymous> (add.test.js:4:9)
I hope that helps you understand how to make the error messages better for custom utilities you make for your tests! Good luck.