Over the last few months, several large companies cited Android performance problems for not upgrading some of their products. In order to benchmark and stress-test any hardware and OS component, a sound performance test and benchmarking toolset is required. The proliferation of Android based systems, as well as the rather rapid release cycle of new Android OS versions, further intensifies the necessity for a sound Android performance evaluation and stress-testing toolset. In other words, it is paramount to hardware, as well as OS design and development engineers to have access to a comprehensive Android toolset that allows evaluating and quantifying performance under real-world conditions at any level of the systems hierarchy.
To illustrate, actual systems performance consists of an application driven workload, the speed and availability of the accessed OS components, as well as the speed and availability of the involved hardware resources. Every OS is basically an abstraction provider, a reactive entity whose performance depends on the pattern of the application load imposed to it. More specifically, OS performance depends on both, the capabilities of the hardware interface that is being abstracted, and the way the OS abstractions are utilized by an application. In reality, not every application is using the same OS abstractions, as each application has its own access (code) path through the OS. It is a fact that prior to releasing new systems/products, virtually no company truly stress-tests (benchmarks) the product from a HW AND an OS perspective.
The DHTDroid v3.2 toolset addresses the above discussed issues in a comprehensive, pragmatic, and scientific manner. The DHTDroid v3.2 toolset allows HW engineers and Android SW developers alike to understand how the major OS abstractions depend on the speed and availability of the underlying hardware platform. From the start, the goal for the DHTDroid project was to implement a set of Android based systems benchmarks that generate an OS and HW abstraction vector that can be mapped to an actual application load (hence the toolset is universally applicable). Further, the performance behavior of different Android OS versions can be compared, and HW cross-comparison studies can be conducted. The DHTDroid v3.2 toolset consists of 9 individual macro benchmarks that stress-test the CPU, the TLB, the cache, the memory, the IO, and the NW subsystems, respectively. Some of the macro benchmarks can be individually customized to reflect an actual application workload behavior. Further, DHTDroid v3.2 provides 4 systems/support tools that aid Android developers in engineering high-quality products and solutions.
The earlier version of DHTDroid (v2.8) has been very well received by the Android community. Several HW and SW companies already use DHTDroid to baseline, stress-test, and benchmark their Android based products. The new version (v3.2) significantly augments on the status-quo, by streamlining the existing benchmarks while adding additional tools. To illustrate, based on feedback from the Android community, additional memory and IO benchmarks were developed. Further, a tool that allows aging a file system (for benchmark purposes) was added as well. In version 3.2, a script is provided to illustrate how to bundle several DHTDroid benchmarks into a single execution entity. DHTechnologies (www.dhtusa.com) released the DHTDroid v3.2 Android performance and benchmark toolset to the community in February 2011. The toolset and the documentation can be downloaded from the Resource page on www.dhtusa.com