We’ve been using Crashlytics by Fabric for many years in our company. When Google have acquired Fabric and asked Crashlytics users to migrate to Firebase, we started thinking what can happen with Crashlytics: will it require Play Services as other Firebase tools do (e.g. Firebase App Distribution), will it work in China (huge market for us), etc.

Actually, above situation was one of many reasons why we decided to implement our own crash catcher system. Having crash information in your own database opens a lot of opportunities. We do a lot of A/B testing, all new features are covered by feature flags, remote configurations, etc. Using the crash info such as a filename, line number we can identify which feature/experiment causes this crash and automatically & remotely turn it off. This is just one use case. We can identify what team or developer this file or change belongs to using git blame and automatically fire notify them. Or even train ML models using all collected crashes. Let’s just imagine, you got a crash and in a minite you received an email with PR that fixes the crash. Infinite opportunities.

To be fair, we can acrhieve all things we imagined using Google’s BigQuery to export data from Firebase Crashlytics. But it’s not real-time and not free. Other libraries we didn’t like for one reason or another. We’d like to have 100% control.

How

Android (excluding JNI) is the easiest part of this huge system. Backend will be deobfuscating (proguard, R8), saving, analyzing, showing, notifying, etc. On client side, we should just catch exceptions and send them to the backend side. In order to do that, we must use setDefaultUncaughtExceptionHandler method of Thread class. Do not forget to “save” previously set handler and pass caught data to it once we’re done. Otherwise, Android won’t kill the process.

val defaultHandler = Thread.getDefaultUncaughtExceptionHandler()

Thread.setDefaultUncaughtExceptionHandler { thread, throwable ->
    try {
        crashLogger.log(thread, throwable)
    } finally {
        defaultHandler.uncaughtHandler(thread, throwable)
    }
}

This code can be called from Application#onCreate() or ContentProvider which is called even earlier.

CrashLogger interface might look like this. We have to save stacktrace somewhere locally and next app launch it’s need to be sent to the server. We can’t immediatelly make network call once we caught a crash because it will be too slow.

interface CrashLogger {
    fun log(thread: Thread, throwable: Throwable)
}

class FileCrashLogger : CrashLogger { ... }
class SqliteCrashLogger : CrashLogger { ... }

Easy way of converting throwable stacktrace to string:

val sw = StringWriter()
throwable.printStackTrace(PrintWriter(sw))
val str = sw.toString()

If you want to get all stacktraces from all threads:

val stackTraces = mutableMapOf<String, String>()

Thread.getAllStackTraces().forEach { (thread, stackTrace) ->
    val stringBuilder = StringBuilder(thread.name)

    stackTrace.forEach { element ->
        stringBuilder
            .append("\n\tat ")
            .append(element.className)
            .append('.')
            .append(element.methodName)
            .append('(')
            .append(element.fileName)
            .append(':')
            .append(element.lineNumber)
            .append(')')
    }

    stackTraces[thread.name] = stringBuilder.toString()
}

If you’d like to know only minimum information you can use the code below:

throwable.stackTrace?.firstOrNull()?.let { crash ->
    crash.fileName // sample.kt
    crash.lineNumber // 42
    crash.className // Sample
}

Remember, Throwable#getStackTrace() method can return an empty array if writableStackTrace flag is false (e.g. ArithmeticException).

NOTE: This code is just a sample. If you really want to create your own Crashlytics and make it production ready there are many things to keep in mind, at least Multi processes, JNI (if you use it).

Story

(can be skipped)

Long time ago in our company, we decided to use code coverage level as one of the must requirements to merge PR into develop branch. We have automated merge bot which automatically merges if the branch meets all requirements. We came up with magic number 80%, so every new/modified class must be covered by Unit tests as minimum 80%.

There are many discussions about code coverage, 80%, JaCoCo, etc. This article is not about that.

We were happy with this requirement. Teams started to write more testable code, think about architecture, app stability went up, etc. We understood it’s impossible to cover all classes in our project, there are many reasons: legacy code (lots of LOC need to be refactored; Business just won’t give time for that; “if it works don’t touch!”), auto-generated code (why we need to test 3rd party libs, etc), some code frankly does not need to be tested. Therefore, we need a way to filter out specific classes from JaCoCo report and allow devs/tools to merge branches.

Even more problems started to appear after moving to Kotlin from Java. JaCoCo, as you may know, works as java-agent with java bytecode only. Kotlin is a great language which reduced boilerplate code a lot but underhood the hood (in case of JVM) it generates a lot of helper methods, lines of code. For example, data class becomes a class with auto-generated methods such as equals, hashCode, component1, etc; even one line of code like s?.foo() or just s.foo() (where s defined as lateinit) will have 1 if in the generated code. Obviuosly, we do not have to test the generated code. All this generated stuff dramatically decreased our code coverage numbers. Keep in mind, those numbers can be KPI for some teams (yeah, bloody enterprise…).

Ideas

1. Custom Gradle task with filter

This way is suitable for most teams. Create a task using JacocoReport and make it depend on test & generate report tasks. Then define an array of classes that you want to ignore and pass it as excludes parameter of fileTree method.

task jacocoTestReport(type: JacocoReport, dependsOn: [...]) {
    // filter rules
    def fileFilter = [
        '**/R.class',
        '**/R$*.class',
    ]

    def debugTree = fileTree(
        dir: "${buildDir}/intermediates/classes/debug", 
        excludes: fileFilter // <-- exclude listed classes
    )

    classDirectories = files([debugTree])

    // some code...
}

Pros:

• All ignored classes in 1 place

Cons:

• Easy to make a mistake. One day we have found a rule **/R*.class which supposed to filter out R class & nested classes (known by every Android developer) but JaCoCo was ignoring all classes started with R :)

2. Annotation processor

Since all developers are lazy and we rather spend 1 day on automating routine job, my colleague came up with Annotation processor approach. He has created NoCoverage annotation and a processor which saves all annotated classes into a file. Later on the file was used by jacocoTestReport instead of fileFilter array. Genius!

Pros:

• Automation!
• Less manual work

Cons:

• APT is not suitable for this kind of jobs. Especially, if it doesn’t support incremental processing and you have many modules.

3. Forked Kotlin compiler

We found out that JaCoCo team in order to support Lombok added a filter which doesn’t count coverage of @Generated annotated classes/methods. Without thinking twice, another colleague has forked Kotlin compiler (!) and modified so it started adding Generated annotation to every generated method. Of course, we haven’t used this approach in prod code. Later, JaCoCo added @kotlin.Metadata filter too.

… and finally!

Solution: Annotation + Kotlin typealias

While investigating JaCoCo code I found that they changed AnnotationGeneratedFilter behaviour. Now instead of checking exact match of annotation name it just should contain Generated word. It means we can create any annotation with proper name and JaCoCo will automatically ignore all annotated classes and methods.

In case of Kotlin you can use typealias to make your code more readable:

@Retention(AnnotationRetention.BINARY)
annotation class NoCoverageGenerated

typealias NoCoverage = NoCoverageGenerated

// or just

typealias NoCoverage = Generated // from javax.annotations package

Pros:

• Easy to use
• No code generation

Cons:

• Workaround. From JaCoCo Wiki:

  Remark: A Generated annotation should only be used for code actually generated by compilers or tools, never for manual exclusion.

• Useless annotation in prod code. But if you use code obfuscator such as Proguard/Dexguard/R8 (Android) that annotation will be cut out.

1. Override toString() by yourself

The simplest way how you can hide sensitive information is overriding toString. But it always needs an additional work in every class. What if you want to show other properties? You will have to concatenate strings by yourself.

data class Data(
    val login: String,
    val password: String 
) {
    override fun toString() = ""
}

2. Define variable out of Data class

In this case the property won’t be included to any of autogenerated functions of data classes (toString, hashCode, equals, components, etc)

data class Data(
    val login: String
) {
    var password: String
}

// Usage - instantiate and set
val data = Data("login")
data.password = "password"

3. Use a wrapper class

This one is more elegant solution but when you create/read a data you will have 1 more level of abstraction.

// Wrapper
class Secret<T>(val data: T) {
    override fun toString() = ""
}

data class Data(
    val login: String,
    val password: Secret<String>
)

// Usage - create / read
val data = Data("login", Secret("password"))
data.password.data

4. Do not use Data classes ¯\_(ツ)_/¯

In this case you will not have all advantages of data classes.

class Data(
    val login: String,
    val password: String
)

Kotlin compiler automatically generates equals, hashCode, componentN, copy and toString functions for Data classes. It’s super useful because you as a developer don’t have to spend time on writing boilerplate code and testing. But there is one problem (at least for me) — a result of toString() will contain all properties of Data class.

Let me explain why it can be a problem. For example in Android development, if you use MVI pattern, Views tend to have a single render() that accepts a state to render on the screen:

data class ViewState(
    val username: String,
    val password: String,
    val isButtonEnabled: Boolean
)

// PS: never keep passwords in String

Great, now you may want to add some logging system to able to understand what your customer’s behavior or what they have seen before a crash or for just debugging purpose:

fun render(viewState: ViewState) {
    logger.info("render $viewState")
    ...
}

Logger class can use Log.d() internally and just prints on *logcat. *Or it can be your own analytics system which sends data to web or just saves in files. It can even be Google Analytics, Crashlytics.log or any other 3rd party vendor. Your application can be a reason for PII, passwords, credit card info or any other sensitive data leakage. I agree that the logging system should be much smarter than just saving everything and everywhere.

It’s not only about Android, but it can also be an issue everywhere especially on microservices architecture where every component is independent and logs all in / out data. It can happen in Repository with sealed class results + data classes, Actions from View to Presenter, State of StateMachine, etc…

Solution

There are many ways how we can exclude our properties from toString() result. But all of them needs manual work. Since developers are lazy I’ve implemented a tool which helps with this. Sekret is Kotlin compiler plugin which changes generated Java bytecode of toString() method during the compilation process. You just need to annotate your property with a special annotation and that’s it!

data class Data(
    val username: String,
    @Secret val password: String
)

print(Data("james.bond", "123456"))

// prints out
// Data(username=james.bond, password=■■■)

More info on Github: https://github.com/aafanasev/sekret

Perhaps JetBrains will allow changing toString() body with an out-of-box solution like Transient annotation for serialization.

This article was originally posted on Medium.

data class Entity(val id: Int)
val json = """{ "id": null }"""
val entity = gson.fromJson(json, Entity::class.java)
entity.id // <- Throws NPE

In order to solve this issue in Java, I used to use AutoValue with AutoGson extension but didn’t find any library for Kotlin (at that time). So I’ve decided to implement my own library and called it KSON.

Simply annotate your data class with Kson annotation and annotation processor will generate null-safe GSON type adapter for your entity class:

@Kson
data class Entity(val id: Int)

Also, you can use KsonFactory annotation and register auto-generated type adapter factory which contains all generated adapters:

@KsonFactory
object FactoryProvider {
    get() = KsonFactoryProvider() // generated class
}

val gson = GsonBuilder()
    .registerTypeAdapterFactory(FactoryProvider.get())
    .create()

As you see, KSON doesn’t need any extra work, just need to add 1 annotation to your class and that’s it! If you don’t want to use for some reason you can simply delete 1 line of code.

More info on Github: https://github.com/aafanasev/kson

This article was originally posted on Medium.