The value of fast feedback loop for developers

We all wanna get from 0 to 1 (with many trailing zeros), the sooner the better.

So, what’s the problem?

You build something and you’re brave enough to put it out there, now what? Now you start the endless feedback loop cycle, you look at the data, analytics, user recordings, product inputs, and decide whats the next thing that’ll make your product better then ever, it’s an iterative process that never ends. The faster you iterate, the faster you’ll get to your target.

How fast can this be done?

It depends on so many things, lets try to focus on the dev part, suppose the product is a ninja with an endless queue of things he wants the R&D department to execute, what does it take? what kind of feedback loop do we need as developers?

The simplest way to explain what need to be done: “Write the correct code, according to product spec, put it on production.”

Writing the correct code - takes time and that’s fine, what kind of feedback loop do we have when writing code?

It starts with a developer coding on his IDE, gets immediate feedback from it, auto lint/compile issue/warnings etc. This is the most basic and immediate feedback out there, and it should be seamlessly fast (as fast as typing)

Second layer of feedback are unit tests, fastest way to run your new code, in a neutral environment allowing to identify and fix issues quickly. This is less basic and immediate, but, an optimal developer experience, allow changing code, running tests almost instantly get results and iterate. In a world of SOA/Micro-services and remote dev environment, this might not be so fast, but it should. Let’s take an example, company with 200 great developers, each one constantly coding amazing things, writing tests to make sure their code works, after each test written, developer need to wait 30sec to see the test results, lets say each developer write 20 tests a day. 20X30X200=120K seconds = 33.3 hours, meaning every day, almost 2% of productivity go’s on waiting for tests to run. let’s continue to the next loop.

Third layer - running the real product on dev env, mandatory for engineers who make user experience, how much time does it take to make an html/css/js change and see it reflects on the app? it should be instant, if it’s 30 sec, we lost another 2% for the waiting, how much time does it take to spin the env? is it fast? stable?

Note - each time a developer need to wait more then 1 second, she might lose her focus to something else, which is a multiplier for the waiting time, not to mention the frustration when things doesn’t work locally.

Forth layer - Pull request, this is an async process, which by definition get’s everything stuck, and a time to work on something else. If we need to move super fast, this process should be synchronized, and conducted f2f right after the pr was created, this is also an iterative process, get comment, fix them, re review.

Fifth layer - QA, Again async process, we wanna get here as ready as possible, to avoid another loop, cause cycles are slow here.

We just went over 5 layers of feedback loop, from the fastest to slowest. now think about it this way, the more iterations we do on the fast layers (1,2,3) the less chances we’ll have many iterations on the slow layers (4-5). To encourage developers to do more iterations, we need a strong infra to allow developers to write and run tests fast, play with their local env, make changes and check them fast. If it’s not fast they’ll do less iterations, and we’ll get to later layers less ready.

So in a hypergrowth company, how to maintain fast feedback loop?

It should start with observability over the metrics.

1. How much time does it take to run a single test?
2. How much time does it take to spin a new dev env with latest code?
3. How much time does it take to make a code change on dev/staging/production env and see it live?
4. How much time does a pr wait in review on average?

Most of the time, when you start, everything is super fast(If you are at the beginning and something is slow, I urge you to stop and optimize), and gradually gets slower and slower, you got chunky modules, more dependencies, files, tooling, scripts and the whole shabang. Keeping things optimal when running fast is almost impossible. At some point you need to stop and think, What feedback cycle is too slow? What can I do about it?

Possible solution is to have a lab mode where you work in isolation from the entire project, have a thin and lean project that has a shell wrapper that simulates the entire app with stubs.

Another one is lot’s of separation and decoupling that allows you to work on relatively small parts of the system that can run fast (for example run a test that loads/compiles 15 files of 1 module, instead of all the 3000 files the project need in order to fully run).

What’s your feedback loop time? what are you doing to improve it? please share on comment!