Victor Schubert’s personal page

Don’t use Javascript numbers for your IDs

Under the hood, Javascript numbers are double floats, even in cases where conceptually we are dealing with integers. And in the vast majority of cases this is perfectly fine. Actually we can define “the vast majority of cases”. This is: as long as our integers fall in the inclusive range from Number.MIN_SAFE_INT and Number.MAX_SAFE_INT, which are equal to −(253−1) and 253−1, respectively. (Important, yet easy to miss: the exponent is 53, not the 63 you usually see when measuring the bounds of 64-bit integers.) But what happens when we start dealing with integers larger than that? Things get quite silly indeed.

> Number.parseInt("10000000000000001")
10000000000000000
> Number.parseInt("9999999999999999")
10000000000000000
> 10000000000000000 + 1
10000000000000000
> 10000000000000000 + 1 === 10000000000000000
true

What is happening here is that at this magnitude, beyond the safe integer range, floating point numbers are not precise enough to represent each integer. In other words, the gap between one float and the next becomes large enough to skip over some integers. Then we are forced to round up or down to the nearest representable integer.

This is not an issue if we are representing a quantity. After all, these are just rounding errors we are talking about, and when we get to the quadrillons and quintillions being off by one simply will not matter in most cases. But numerical IDs, the ones we use to identify rows in a SQL table for example, are a different kind of numbers. They are not quantities, and it is essential that they always be exact values. Otherwise, what is a rounding error when measuring a quantity, morphs into pulling data for the wrong customer.

Now, this is rarely a problem in practice because your auto-incrementing IDs won’t reach those magnitudes any time soon. If you trust that your IDs will always work the same, ever counting away from one, then you’ll be fine. 253 is so mind-boggingly large already that no matter how big your project becomes, I can confidently affirm that no counter in there will ever reach this number. But I invite you to consider, that systems and the practices that surround them evolve, and that your IDs and how they are generated can change in such ways that you can reach this dangerous point where you can no longer trust your IDs. I have seen two real-life situations which could have triggered this.

The first, was a coworker designing a new SQL table. I unfortunately cannot recall the specifics. The table in question was going to have an int64 ID. Pretty typical so far. But this ID was not going to be your usual auto-incrementing ID. Instead, it was to be randomly chosen in the [1, 263) range for each new record. This would have meant that 99.9% of all generated IDs would have fallen outside of the safe integer range. The design later was changed to use a UUIDv4 instead.

The second didn’t get as close, but highlights a general way that this can happen. The company was growing, and one table in particular that had an int32 ID was reaching into the two billion records, nearing the int32 limit. We migrated the ID to int64 in time, but unfortunately another table that had an int32 foreign key to the bigger table was forgotten. Eventually 231−1 was reached and the foreign keys broke. I found the fix to be clever: the auto-increment counter of the big table was reset to −231, exploiting the usually ignored negative range of int32 IDs to give us time to migrate the forgotten table before setting it back to a high positive value. Now if we had been using int64 from the start we wouldn’t have been in this situation and so I cannot claim that we would have reset the counter to −263 and fallen outside the safe integer range. And yet, this highlights that the number an auto-incrementing counter is counting from, can be changed to extreme values, including those that Javascript numbers won’t be able to handle.

And so I claim: even for the most basic auto-incrementing integer IDs, the appropriate type to use in Javascript is the string, rather than the number. You won’t be able to do math on your IDs, but that’s not normally something you do. This also has the potential to make a migration easier if in the future you choose to replace those numerical IDs with UUIDs or some other ID not normally represented by integers. BigInts would also be a practical solution if it was possible to have JSON.parse produce them, but in my opinion this makes things more complicated, marginally more efficient and the added ability to do math on IDs is just not useful.