Unit Tests Are Not Real Tests

The rhetoric on web application testing frustrates me.

Maybe I've been living in a bubble, but a significant number of critical applications are like 80% databases. The rest is just service layers for features like emails or caching.

Naturally, we ask: how do you handle testing applications that are heavily dependent on databases? (StackOverflow). But then the answer is "write unit tests because integration/e2e/tests-that-touch-the-db are slow". There is this non-sensical idea that somehow 50-90% of your test suite should be composed of unit tests. I don't mean to call anyone out -- this is advice with good intensions -- but my application is like 80% database code? Why is >=50% of my test suite banned from touching the DB? (By database code I mean SQL, your ORM calls or the JSON-as-a-DSL for MongoDB.)

Over-simplifying, many many applications can be reduced to the pattern below. The only things you can test here are:

app.get('/todos', async (req, res) => {
  const body = parseTodoHttpBody(res);  // 1. Can you parse the HTTP body right?
  await heyDatabase.runThis(generateSql(body, req.session));  // 2. Is the SQL you generate correct
  res.send('We done it!');
});

But how do you unit test without touching the database? I guess we could snapshot test the SQL, but that's no different than:

# app.py

def add(x, y):
  return x - y

# app_test.py

def test_add():
  with open("app.py") as file:
    source = file.read()

  assert source === """
def add(x, y):
  return x - y
"""

Antithesis: Unit Tests are Great Actually

Ultimately the problem with unit tests vs. integration tests is that integration tests are slow. When unit tests are useful, they are pretty great.

Some parts of our applications can be contained in a small box. Think about utility functions, small abstractions, input parsing. When you put unit tests here, they are a superpower. I've worked in codebases where incrementally adding new features is smooth because you can test your way through the process instead of writing a pile of code before running anything.

Writing code this way feels like a video game. Feedback is instant and the entire codebase grows in a way that encourages you to write idiomatic code.

Conclusion

I don't think we know how to effectively test a large fraction of applications.

When it comes to database-heavy web apps, the only option seems to be slow tests. Some recommendations I've seen are:

But these don't fix the speed problem. Even if you could completely teardown and restore a DB in 1 second, you can do, at most, 60 tests a minute.

I wish I could write tests against my real database whilst both running as fast as unit tests and enjoying strong isolation between tests.

If you know how to do this, please let me know. I'd be happy to learn that I've been living in a cave :)