A browser is an incredibly complex piece of software. With such enormous complexity, the only way to maintain a rapid pace of development is through an extensive CI system that can give developers confidence that their changes won’t introduce bugs. Given the scale of our CI, we’re always looking for ways to reduce load while maintaining a high standard of product quality. We wondered if we could use machine learning to reach a higher degree of efficiency.
At Mozilla we have around 85,000 unique test files. Each contain many test functions. These tests need to run on all our supported platforms (Windows, Mac, Linux, Android) against a variety of build configurations (PGO, debug, ASan, etc.), with a range of runtime parameters (site isolation, WebRender, multi-process, etc.).
While we don’t test against every possible combination of the above, there are still over 90 unique configurations that we do test against. In other words, for each change that developers push to the repository, we could potentially run all 85k tests 90 different times. On an average work day we see nearly 300 pushes (including our testing branch). If we simply ran every test on every configuration on every push, we’d run approximately 2.3 billion test files per day! While we do throw money at this problem to some extent, as an independent non-profit organization, our budget is finite.
To continue reading this article, click here.
Great news from Microsoft! Check out Kelly Office Solutions for even more amazing technology innovations. Here, everyone will find something that meets their business needs.
That’s right! Testing Firefox More Efficiently With Machine Learning pizza tower