# Power testing *[go/cros-plt-doc][11]* ## Description Modern mobile computers are sold with an advertised battery life, ranging from a few to tens of hours. Nonetheless, when these devices are used on a daily basis, many users report that their actual battery life doesn’t match up with the advertised numbers. For Chrome OS devices, we wanted to try and report battery life that is as close as possible to what an average user experiences. Thus `power_LoadTest` was created to emulate average user behavior and measure the resultant battery life. This test is as an [open source][1] Chrome extension anyone can install and run. The `power_LoadTest` runs as a series of one hour long iterations until battery exhaustion. Within each iteration, a load mix known as **60/20/10/10** is run: * **[First 60%]** Browsing: a new website is loaded every minute. The web page loaded is scrolled down one page (600px) every 10 seconds, then scrolled back up at the same rate. * **[Next 20%]** Email: Gmail is loaded in the foreground tab and audio is streamed from a background tab. * **[Next 10%]** Documents: Various Google Docs are loaded. * **[Final 10%]** Video: A full screen 720p YouTube Video is played. The parameters of the device under test (DUT) are as follows: * Backlight * Screen: Set to platform default brightness on battery or 40% if default can not be determined via [backlight_tool --get_initial_brightness --lux=150][2]. * Keyboard: Depends on existance of ALS / Hover sensor | ALS | hover | keyboard backlight level | |-----|-------|----------------------------------------| | No | No | default | | Yes | No | 40% of default | | No | Yes | System with this config does not exist | | Yes | Yes | 30% of default | * Power management: * Dimming, blanking the screen and transitions to standby state are disabled. * Ambient light sensor readings are ignored. * Battery: * Device only powered by battery (no AC power). * Battery charged to 100% prior to initializing test. Test continues in 1 hour iterations until battery passes low threshold (typically set at 3%). Initial & remaining battery charge is recorded. * USB: No external devices connected * Network: Device is associated with a wireless access point via WiFi. * Audio: Built-in speakers at 10% volume, Built-in microphone at 10% volume. Throughout the duration of the test, there are 5 sites loaded in background tabs. These sites were chosen to represent typical actions of a user on a daily basis: * Searching * Reading news * Checking on finance * Shopping * Communication ## Running ### Via cros_sdk & autotest If you are interested in running `power_LoadTest` on a Chrome OS system, you will need a Chromium OS test image that can be built by following [Build your own Chromium image][3] instruction with `./build_image --board=${BOARD} test` command. After the test image is built, you can follow the [Installing Chromium OS on your device][4] instruction to install the test image to your DUT. Since running `power_LoadTest` requires that the device is disconnected from the wired Ethernet (including USB-Ethernet) as well as from the AC power source, it is trickier to run it compared to [running other autotests][5] if you do not have both the build machine and the DUT under a same private WiFi network connected to the Internet. If your build machine and the DUT are in the same WiFi network, you can run `power_LoadTest` by running the following command in scripts directory inside [chroot][6]. (Make sure you have battery fully charged with AC power source and Ethernet disconnected from the DUT before running the test.) ```bash test_that ${DUT_ipaddr} power_LoadTest ``` If your build machine is not on the same private WiFi network as the DUT (applicable for most Googlers and any who have the build machine connected to a corporate network), follow the instructions below. * Keep the AC power source plugged and the wired Ethernet connected to the same network as your build machine. * Run the same command as above. It is expected to fail as the pure purpose is to copy the test suite over to the DUT from the build machine. ```bash test_that ${DUT_ipaddr} power_LoadTest ``` * Disconnect the wired Ethernet. * On the DUT, enter VT2 by pressing `Ctrl + Alt + F2` (right arrow or refresh key on the Chrome keyboard), and login as `root` (May require password. Default is likely `test0000`. See additional details [here][7] if that doesn't work. ```bash cd /usr/local/autotest* # remove your AC power source right before running the test. bin/autotest tests/power_LoadTest/control* ``` * Re-enter VT1 by pressing `Ctrl + Alt + F1` (left arrow on the Chrome keyboard). * Now you just need to wait until the battery runs out. (Don't worry, the test result is stored on the DUT.) * Reconnect the AC power source, boot up, and enter VT2 as `root`. ```bash cd /usr/local/autotest/results/default/power_LoadTest/results/ ``` ### Via extension only As mentioned earlier, `power_LoadTest` uses [chrome extension][8] to drive the various workloads. As such it can be run directly on a 'normal mode' machine once the extension is installed. To run with this method, * Download the `power_LoadTest` [extension][9] to your device and unpack tarball. * Navigate to `chrome://extensions` and click *load unpacked extension*. * Choose the *extension* directory from tarball. You should now have the extension installed and clicking on it will start the test but before you do that read the following caveats & hints to make this run go smoother. * Before starting, note the battery state-of-charge (SOC). The extension will only run workload for 1 hour so you'll need to extrapolate total runtime from that. * If device has keyboard backlight be sure its off. See [keyboard backlight][10] documentation for details on manually controlling keyboard backlight. * In order to remove impact of ambient light changes, use brightness keys to set the panel brightness to your preferred brightness first. Note, autotest typically sets brightness to ~80nits. * When test completes be sure to note battery SOC again for calculating battery life. Estimate your total battery life by this calculation, 100 / (battery_soc_start - battery_soc_end) For example if you started test at 50% SOC and it ended with 40% SOC your battery life would be, 100 / (50 - 40) = 10 hours ## Interpreting Results If you ran via cros_sdk & autotest there will be a keyvals file at `power_LoadTest/results/keyval`. The test will publish *minutes_battery_life* which we use to track platforms battery life. However that only tells part of the story. As with any other real world test, the results have other measurements that should be examined to ensure the battery life estimate is genuine. Keyvals of particular interest beyond *minutes_battery_life* to judge quality of test results are: * ext_*\_failed_loads * Any non-zero values for these keyvals can be problematic. The keyval itself is a string with counts separated by underscores. Each count is the failures logged by the particular loop. For example ‘0_1_2’ would mean no failures for loop0, 1 for loop1 and 2 for loop2. While failures aren’t good if the subsequent ext_*\_successful_loads equals the correct quantity then data may be deemed ok. * percent_cpuidle_* * These numbers typically stay roughly the same for a particular platform so be cogniscent that they roughly match previously qualified PLT runs or the differences should be investigated. * percent_cpufreq_* * Similar to cpuidle above this represents the P-state the processor frequency runs in. These too stay roughly the same and should be compared to previously qualified numbers. * loop*\_system_pwr_avg * Each loop should be roughly the same average power. Often the first loop (loop0) consumes more as webpage caching hasn’t occurred yet. * percent_usb_suspended_* * Should be ~100% unless run is being done with expected USB device that has been attached externally or has had selective suspend disabled for some functional reason. ## Conclusion While the initial version of `power_LoadTest` seems to emulate well what users experience every day on Chrome OS devices, this test will be constantly improved. As we learn more about how users use Chrome OS devices and how experienced battery life differs from tested battery life, we will use this data to refine the test, potentially changing the load mix or the parameters of the test. Our goal is to ensure that when you purchase a device, you know - with reasonable certainty - how long that device will last in your daily use. [1]: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/master/client/site_tests/power_LoadTest/ [2]: https://chromium.googlesource.com/chromiumos/platform2/+/master/power_manager/tools/backlight_tool.cc#154 [3]: http://www.chromium.org/chromium-os/developer-guide#TOC-Build-a-disk-image-for-your-board [4]: http://www.chromium.org/chromium-os/developer-guide#TOC-Installing-Chromium-OS-on-your-Device [5]: http://www.chromium.org/chromium-os/testing/autotest-user-doc#TOC-Running-tests [6]: https://chromium.googlesource.com/chromiumos/docs/+/master/developer_guide.md#enter-the-chroot [7]: https://www.chromium.org/chromium-os/how-tos-and-troubleshooting/debugging-features [8]: https://developer.chrome.com/extensions/getstarted [9]: https://chromium.googlesource.com/chromiumos/third_party/autotest/+archive/master/client/site_tests/power_LoadTest.tar.gz [10]: https://chromium.googlesource.com/chromiumos/platform2/+/master/power_manager/docs/keyboard_backlight.md [11]: http://go/cros-plt-doc