How We Fixed a Huge ANR in Our My11Circle App: A Fun Deep Dive Into Solving Android Vitals Challenge 📱💥
ANRs got you down? Here’s how we crushed them in My11Circle and improved our Android Vitals.
TL;DR: Our team at Games24x7 faced a monstrous ANR issue that pushed our app’s Google Play Console metrics into the “bad behavior” zone. After weeks of investigation and some "technical wizardry," we not only crushed the ANR but also tackled a react-native bug specific to Android 12. Here’s how we did it :-
The Problem: A mysterious ANR without a clear cause 🤔
It all started with a spike in ANRs in our My11Circle Android app. The situation got so bad that it pushed our metrics into the "bad behaviour" zone on Google Play Console 😱.The first issue? The root cause message was as cryptic as a riddle wrapped in an enigma. The error log read:
android.os.MessageQueue.nativePollOnce
libandroid_runtime.so (android::android_os_MessageQueue_nativePollOnce + 44) (BuildId: cebd92605c5c36dc0013f1ca7b00edbd)
at android.os.MessageQueue.nativePollOnce(Native method)
at android.os.MessageQueue.next(MessageQueue.java:335)
at android.os.Looper.loopOnce(Looper.java:186)
at android.os.Looper.loop(Looper.java:313)
at android.app.ActivityThread.main(ActivityThread.java:8663)
at java.lang.reflect.Method.invoke(Native method)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:571)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1135)
I mean, seriously — what does that even mean? 🧐
The Breakthrough: Crashes leading to ANR 💡
Then, after countless hours of stress testing (and probably some coffee-fueled breakdowns ☕💀), we got lucky. While trying to reproduce the ANR, we triggered a crash instead. But wait... instead of crashing as expected, the app entered an ANR state.
BINGO! 🎯 This was a crucial breakthrough. The issue wasn’t just a random crash; it was related to the crash handling mechanism. We were getting closer to the cause... and closer to saving the day (saving the week TBH 😅🙈).
Uncovering the Root Cause: The faulty exception handler 🔧
As we dug deeper, we discovered the villain behind the ANRs: our custom uncaught exception handler. Now, this handler was supposed to be our trusty sidekick 🦸♂️ — stepping in when ever things went wrong, catching exceptions, and showing the user an error screen with a helpful message and a “relaunch” button.
In theory, it was brilliant. In practice, not so much. 🤷♂️
This was our implementation of exception handler:
Thread.setDefaultUncaughtExceptionHandler(new Thread.UncaughtExceptionHandler() {
@Override
public void uncaughtException(Thread thread, Throwable throwable) {
String stackTraceString = Log.getStackTraceString(throwable);
callbackHolder.invoke(stackTraceString);
activity = getCurrentActivity();
Intent i = new Intent();
i.setClass(activity, errorIntentTargetClass);
i.putExtra("stack_trace_string",stackTraceString);
i.setFlags(Intent.FLAG_ACTIVITY_NEW_TASK);
activity.startActivity(i);
activity.finish();
}
});
The problem here is that when the app encountered a fatal exception, the handler wasn’t checking the activity state properly, and would try to launch anew activity even if the app was in the middle of a critical state (like destroying the current activity). This caused an ANR because the app couldn’t decide what to do next.
The Fix: Refactoring the exception handler 🔨
With the root cause identified, it was time to put on our problem-solving cap and refactor our handler. We were determined to stop the app from freezing and to handle uncaught exceptions like pros.💼
Here’s what we did:
1. Refactored the exception handler 🔄
The handler got a little “refactor therapy” 💆♂️.We updated the logic to make sure we check the current activity state before launching a new one. And if the handler couldn’t manage the exception, we passed it to the original handler.
With that, the ANRs were reduced dramatically, and the app no longer froze when things went wrong. Victory! 🏆
This is our refactored implementation:
originalHandler = Thread.getDefaultUncaughtExceptionHandler();
Thread.setDefaultUncaughtExceptionHandler(new Thread.UncaughtExceptionHandler() {
@Override
public void uncaughtException(Thread thread, Throwable throwable) {
String stackTraceString = Log.getStackTraceString(throwable);
callbackHolder.invoke(stackTraceString);
activity = getCurrentActivity();
if (activity == null || activity.isFinishing) {
originalHandler?.uncaughtException(thread, throwable);
return;
}
Intent i = new Intent();
i.setClass(activity, errorIntentTargetClass);
i.putExtra("stack_trace_string",stackTraceString);
i.setFlags(Intent.FLAG_ACTIVITY_NEW_TASK);
activity.startActivity(i);
activity.finish();
}
});
2. Testing &Validation ✅
We didn't just stop there .We made sure to stress test the app to simulate crashes and see if the fix worked. We also added logs and breadcrumbs to track issue on Firebase Crashlytics. We wanted to make sure our app was no longer hanging around. 🤸♂️
The Aftermath: From ANRs to crashes 💥😱
As the ANRs dwindled, a new problem arose: crash reports started flooding in. It turns out, many of the errors that were causing ANRs were now being logged as crashes. But hey, at least they were being logged now! 💬
This meant we had taken a step forward, but also uncovered another layer of issues. We weren’t done yet.
The React Native Bug: The ConcurrentLinkedQueue conundrum 🧵
As the new crashes rolled in, we discovered that the real culprit was a bug in the React Native Android library that was only causing issues on Android 12 devices. Specifically, the issue was tied to ConcurrentLinkedQueue, a data structure that was being used in the react-native and react-native-reanimated.
The Issue: Dropped items on Android 12 🚨
On Android 12, the ConcurrentLinkedQueue was be having unpredictably. It would drop items under certain conditions, which would eventually lead to crashes. The behavior was very hard to reproduce as it was not happening on all the devices and it was a very edge case scenario, making it even harder to track down. 😓
The Solution: Replacing ConcurrentLinkedQueuewith LinkedBlockingQueue 🛠️To fix this, we decided to “queue” up a plan. Our solution? Replace the rogue ConcurrentLinkedQueue with the much more reliable LinkedBlockingQueue. We took the issue into our own hands and built a custom version of react-android with this fix. Talk about “queue-trolling” the problem! 😂
Here’s the plan:
1. Cloning and forking the react native android library:
We forked the React Native Android repo from GitHub and checked out the commit that matched our version.
2. Implementing the patch:
Here's how we made it work:
Android version check: We added a check to see if the app was running on Android 12 (API level 31). If it was, we used LinkedBlockingQueue instead of ConcurrentLinkedQueue.
private final Queue<UIThreadOperation> mQueue;
String versionAndroid = Build.VERSION.RELEASE;
if (versionAndroid.equals("12")) {
mQueue = new LinkedBlockingQueue<>();
} else {
mQueue = new ConcurrentLinkedQueue<>();
}
Patch implementation: We patched the React Native Android library by updating the source code wherever ConcurrentLinkedQueuewas used, making it conditional based on the Android version.
3. Building and hosting the custom library:
We had to set our own little “code bakery” to serve the freshest fix! 🍞.Since React Native uses a separate react-android library (and not the one from node_modules),we built and hosted it on our S3 server. Refer this for building react native from source: https://github.com/facebook/react-native/wiki/Building-from-source/29169d93c6ffc43cb7454bebc94761a859c1c450
4. Fixing dependency issues:
The fun didn’t stop there! Even after mentioning our version in dependency it was picking the old version of react-android. Then after some debugging we found that React Native’s gradle.properties file was locking the library to an old version, so we had to override it to point to our patched version. Here’s how we made it “snap” into place:
.
.
diff --git a/node_modules/react-native/ReactAndroid/gradle.properties b/node_modules/react-native/ReactAndroid/gradle.properties
index 2ab4e65..7792733 100644
--- a/node_modules/react-native/ReactAndroid/gradle.properties
+++ b/node_modules/react-native/ReactAndroid/gradle.properties
@@ -1,4 +1,4 @@
-VERSION_NAME=0.71.13
+VERSION_NAME=0.80.11-SNAPSHOT-1
GROUP=com.facebook.react
.
Patching react-native-reanimated:
Our version of react-native-reanimated was fetching pre-built AAR files, so simply applying the patch wasn’t working. Nothing was coming easy to us—no mercy! 💥 So, we had to dive deeper and make changes to its build.gradle file to enable building from source. Once we got that working, we finally applied the final patch with the ConcurrentQueue fixes and build fixes. It felt like finally getting past the toughest level of a game. 🎮🏆. This is how our reanimated patch looks like:
.
.
.
-def aar = detectAAR(REACT_NATIVE_MINOR_VERSION, JS_RUNTIME)
-boolean BUILD_FROM_SOURCE = shouldBuildFromSource(aar, JS_RUNTIME)
+def aar = null
+boolean BUILD_FROM_SOURCE = true
.
.
.
+import java.util.concurrent.LinkedBlockingQueue;
+import android.util.Log;
+import android.os.Build;
public class NodesManager implements EventDispatcherListener {
@@ -110,7 +113,7 @@ public class NodesManager implements EventDispatcherListener {
private RCTEventEmitter mCustomEventHandler;
private List<OnAnimationFrame> mFrameCallbacks = new ArrayList<>();
- private ConcurrentLinkedQueue<CopiedEvent> mEventQueue = new ConcurrentLinkedQueue<>();
+ private Queue<CopiedEvent> mEventQueue;
private boolean mWantRunUpdates;
public double currentFrameTimeMs;
@@ -187,6 +190,14 @@ public class NodesManager implements EventDispatcherListener {
mUIManager.getEventDispatcher().addListener(this);
mAnimationManager = new AnimationsManager(mContext, mUIImplementation, mUIManager);
+
+ String versionAndroid = Build.VERSION.RELEASE;
+ Log.i("VERSION21", "NodesManager RNANIMATED" + versionAndroid);
+ if (versionAndroid.equals("12")) {
+ mEventQueue = new LinkedBlockingQueue<>();
+ } else {
+ mEventQueue = new ConcurrentLinkedQueue<>();
+ }
}
.
.
.
Testing the fix 🔍🔨
Testing was a real “hunt the bug” situation! Since this was an elusive issue, we couldn’t just rely on automation tests alone. We needed a backup plan:
Automation:
We ran all the automation suites on our build, performed regression testing, and verified that everything is working. Since this was not an easily reproducible scenario, we could not rely only on these results, as these test suits were working earlier as well. That’s why we had to figure out another approach; let's see what it was.
Decompiled APK:
We wanted to make sure that the correct versionof react-android is beingused in the final build and has all our changes for the fix. This is how we didit:
1. Decompiled our APK using apktool, which extracted DEX files from our APK:
apktool d -r -s universal.apk
2. Converted these DEX files to JAR using dex2jartool:
d2j-dex2jar universal/classes.dex
3. Explored these JAR files using JD-GUI tool, searching for all the classes in which we made changes and verified that those changes are there.
After testing with both automation and APK decompiling, we were confident the fix was solid. It was like finding the final puzzle piece and slotting it into place. 🧩 With that, we rolled out our next release, and the issue was finally laid to rest. 🛌
The Result: A smoother experience for our users 🎉
After deploying the patch, we saw a huge drop in ANRs. The app was running smoothly, and users on Android 12 experienced fewer crashes. Our android vitals really improved and it felt like we’d finally crossed the finish line. 🏁 Win!
Conclusion: The journey is never really over 🚀
What started as a vague ANR message turned into a full-blown investigation that required a series of fixes, patches, and some caffeine. While we successfully reduced ANRs and solved a major third-party library issue, we’re not stopping here. 🚀
In the next phase, we’ll focus on optimising app start times and squashing performance bottlenecks. Stay tuned for more posts on our journey to make My11Circle even more smooth and seamless. 🏃♂️💨
About the Author:
I'm an Android developer with 5+ years of experience wrangling Kotlin, dodging NullPointerExceptions, and surviving Gradle builds with my sanity(mostly) intact. When I’m not building apps, I’m sipping chai and explaining to friends for the hundredth time that I don’t actually fix phones.
📬 Connect with me on LinkedIn
Explore More
Discover the latest insights from the world of Games24x7
