Visit Agile Thoughts and register to receive free development, analysis, or leadership and management materials and learn to excel at developing software. I’ll also send information on my low cost email courses you can take via the internet.

The Test automation Pyramid series Starts on episode 1 and runs to episode 8.

Continuous Integration coordinates contents of the test pyramid like a conductor does of a Symphony.

 

Recall the pyramid generally has three floors: the penthouse filled with UI macro tests, the middle is inhabited by subcutaneous macro tests, and the ground floor–a plethora of micro tests. The goal of CI is to give valuable feedback to a team as fast as possible.

 

The conductor should execute the tests floor by floor starting from the ground level. The word “continuous” in continuous integration means that whenever any code is checked in, the conductor lifts her baton, signaling to the compiler to make a clean build with what was just checked in. [cue symphony]. If the build succeeds, she waves to trigger the fastest tests at the ground floor–micro tests [cue glitch music]. If the tests report green, then she signals stage hands to prepare the middle floor, which may entail deploying the server into a clean environment or launching a service processes. When the stage hands work is complete, the conductor, with a flourish of her baton, signals the middle floor tests to start [cue music]. If the result of their performance is again, green bar, then, what happens next depends on the sophistication of the infrastructure available to the macro tests in the penthouse: if the application can deploy and run macro tests within an hour, then the conductor goes for it [cue music] returning feedback to the team on pass or fail. Having macro tests this fast requires a number of build agents running in parallel, such as a test cloud. If there aren’t enough build agents to go that fast, then usually the conductor takes her seat, only executing the first two floors of the pyramid in a continuous fashion, leaving the macro tests to execute upon a scheduled build of every 12 or 24 hours [cue slow music]. Upon a new checkin, the conduct stands and repeats the process over again. Although executing UI macro tests continuously should be every team’ goal, most organizations haven’t invested in infrastructure to do so. Waiting more than 24 hours to execute slow macro tests isn’t recommended as responding to problems found by tests with such a slow feedback loop is quiet costly. If a test fails in this situation, a lot of time must be spent studying source code changes and using the debugger in order to simply find what code caused a regression. Even keeping the macro tests in good working order is expensive.