Baseline Profile, Where Art Thou?
NOTE: While not being an actual continuation of my previous medium post, it’ll be great if you read that as well to get an info on other issues that you might face while measuring the impact of Baseline Profiles on OPPO, OnePlus or Xiaomi devices.
Also as there are already many articles and documentation about different app startup states and Baseline Profiles I’m not planning to cover it here, instead if you’re curious you can scroll to the bottom of this post to check the reference links.
Recently I was on a mission to improve App Cold Start performance for one of my client. I was able to generate the Baseline Profiles without any issues as I have done similar exercises a few times already in the past. I check the impact of it on my iQOO 7 (specs: SD870G + 8GB RAM + 128GB & Android 13) device and I was able to see a decent improvement where the time spent by Jit thread pool during app cold start got reduced to 41ms, which was 212ms earlier i.e. 80% reduction 🙂.
Then I was very eager to do the same exercise on a budget device OPPO A15 (specs: Helio P35 + 2GB RAM + 32GB & Android 10) as here the time spent by Jit thread pool was 2180ms (i.e. ~2.2s) so I was expecting to save atleast 1000ms (i.e. 1s). So I did the benchmarking with a baseline profile build and to my surprise, I saw no reduction in app cold start time 🤔. So I opened one of the systrace generated during the profiling using macrobenchmark and checked the time spent by Jit thread pool which was still very similar i.e. ~2048ms.
My first thought was that maybe there was an issue in the test execution so I ran the tests 2 more times but still there was no improvement. Then I thought maybe because Android 10 it’s not working properly, so I checked the documentation and found that it should work on all Android API24+ devices. Finally, I came to the conclusion that it was time to put the Sherlock hat and get the magnifying glass ready to investigate the issue. 🕵
Note: I used AGP version 7.3.1, MacroBenchmark version 1.1.1 & ProfileIntsaller bersion 1.2.2 for measuring Baseline Profile impact.
Investigation
As I can’t show the actual project that I’m working on for many reasons I’m going to use Now in Android App to demonstrate the same issue. I ran Cold Start tests with both with and without Baseline Profile on the same Oppo A15 device. As you can see in the below screenshots the time spent by Jit thread pool with and without Baseline Profiles was nearly same i.e. 367ms and 337ms respectively.
For reference you can also see the result of the same tests on iQOO 7 device below where the time spent by Jit thread pool with and without Baseline Profiles was 75ms and 0.6ms 😲 respectively.
Then I started digging into the logs generated during the BaselineProfile tests to see if I can find some clue. After spending few hours I found few statements (same count as the test iteration count) saying “Skipping profile installation for com.google.samples.apps.nowinandroid.benchmark” as shown below, which made it concrete that for some reason Baseline Profiles were not being installed properly.
Then I checked all the known issues with Baseline Profiles and finalized two of the points mentioned there that can interfere with Baseline profile installation.
- Battery optimizations may interfere with profile installation. To ensure that your profiles are installed effectively, disable any battery optimizations on your benchmark devices.
- Android Studio Profilers don’t install Baseline Profiles when profiling the app (issue).
I verified that there was no battery optimization enabled on Oppo A15 and battery was also at 100%. About the second issue mentioned above, from the good UX perspective, I thought I should’ve gotten an error if it was the case, similar to the ones we get when doing profiling on device with low battery or an emulator. Finally, I thought about trying latest releases of macrobenchmark (i.e. 1.2.0-alpha09) and profileinstaller (1.3.0-alpha03) to verify that it’s not an issue with the dependencies that I was using.
I bumped the versions and then ran the tests again but this time the test threw an error saying “Check failed”. This at least made me happy as now I know that there is an issue with baseline profile tests, unlike earlier where if the User doesn’t check the logs then they don’t get to know why the baseline profiles were not working. Then I followed the stacktrace, which led me to cmdPackageCompile() function where the result of “cmd package compile -f -m $compileArgument $packageName” was being matched to “Success” string as shown below.
This made me wonder why I this error didn’t happen earlier so I reverted the version bump and checked the same cmdPackageCompile() function and saw that in earlier versions the result of the same package compile command was not being asserted 💡, which made it clear why there was no error earlier. For reference you can see the source code of cmdPackageCompile() function till androidx.benchmark:benchmark-*:1.2.0-alpha06 & above in below screenshots.
To verify if this was an OEM i.e. Oppo specific issue, I ran same tests on both Xiaomi and OnePlus devices. On Xiaomi device the tests ran fine but faced a similar issue on the OnePlus device (owned by Oppo now), so finally I concluded that it’s OEM specific issue.
After this investigation now I know that in case I don’t see any improvement in benchmarks with Baseline Profiles (i.e. JIT is still running at full throttle) then I can either execute below command to verify that if Baseline Profile is even supported on the device or check the logs in the BaselineProfile test. 🙂
"adb shell cmd package compile -f -m speed-profile $YOUR_PACKAGE_NAME"Hope this post saves some time while debugging in case the Baseline Profiles are not showing any improvement on Oppo & OnePlus devices atleast.
Reference Material Links and Credits:
- App Startup Time - https://developer.android.com/topic/performance/vitals/launch-time
- Baseline Profiles - https://developer.android.com/topic/performance/baselineprofiles/overview
- Rahul Ravikumar from Google does detailed discussion on Baseline Profiles - https://www.youtube.com/watch?v=m7oRRdud-mY
- ADS 2022 talk on Baseline Profiles - https://www.youtube.com/watch?v=yJm5On5Gp4c
- Baseline Profile article published on Google Developer’s Blog - https://android-developers.googleblog.com/2022/01/improving-app-performance-with-baseline.html
Hope you liked reading the article. Please feel free to reach me on Twitter or LinkedIn. Thank You!
