4.3 Embed jacocoagent

I. Current Situation and Issues

With the continuous iteration of requirements, the business code of business systems has made great progress. When you are proud of your high code output, have you taken a look back at the real customer usage of the online systemquantityHow many more are there?

~ It takes a lot of effort and human resources to develop features that may not be used by anyone for a year. If they are not properly decommissioned, it will increase the system maintenance cost, and at this point, it is necessary to plan to delete unnecessary code. But how do we know whether a line of code in the real online environment is actually being used, or whether it is really not being used,How can we safely delete the offline function!

Two. Analyze the reasons

In fact, most business systems will have this common problem: zombie code online

• It may be that the business scenario was not analyzed well in the early stage of the product
• It may be that the demand function deviated from the correct direction during the development period
• It may be due to the decrease in customer business volume after going online due to external factors
• ······

Three. Take measures

Ask the product manager which can be taken offline?NONo one dares to promise

Observe whether there is traffic on the UMP interface?NOOnly knowing the interface dimension, does the traffic of the interface mean that all the code is useful?

Use jacoco (Java Code Coverage) for online code analysis and slim down the system.

Jacoco is essentially a test coverage tool that adds probes to the source code through the ASM bytecode enhancement technology to obtain code coverage. Jacoco mainly records whether the code has been executed by adding probes to the code being executed before the main function is executed through the specified method by the Java agent.

Java agent is a startup parameter provided by Java, which is different from the dynamic enhancement of proxy methods and the compile-time enhancement of annotation processors. This parameter calls the premain method in the jar package specified by the path before the main method is executed to enhance the source code. By implementing this method, we can modify the source code of the loaded class files for enhancement. Most APM tools also use this technology.

https://www.jacoco.org/jacoco/trunk/doc/index.html

Four. Practice steps

4.1 Dependency on jacoco.ant

Introduce jar dependency in the pom within the project

<dependency>
    <groupId>org.jacoco</groupId>
    <artifactId>org.jacoco.ant</artifactId>
    <version>0.8.3</version>
</dependency>
<dependency>
     <groupId>org.apache.ant</groupId>
     <artifactId>ant</artifactId>
     <version>1.9.9</version>
 </dependency>

4.2 Enable Rest request

Add a URL address, execute the dump task for Dump Coverage file through ant, and avoid the problem of using configuration files and requiring the operation assistance from operation and maintenance colleagues at the same time.

@RestController
@RequestMapping("/coverage")
public class CoverageController {

    @PostMapping("dump")
    @NoCheckMenuPermission
    public Result<Boolean> dumpCoverageFile() {
        DumpTask dumpTask = new DumpTask();
        // Dump file address
        dumpTask.setDestfile(new File("/export/Data/coverage/code-cover.exec"));
        // Multiple dump append format
        dumpTask.setAppend(true);
        // Select an idle interface
        dumpTask.setPort(8840);
        // Default to the local machine
        dumpTask.setAddress("127.0.0.1");
        dumpTask.execute();
        return Result.succeed(true);
    }
}

4.3 Embed jacocoagent

Since jacoco needs an enhanced jar package by jacocoagent on the server side, to avoid the need to trouble the operations and maintenance colleagues, we can find through Maven dependencies that the jar package org.jacoco.agent contains the package jacocoagent, so by adding the following command to the deployment startup script, we can obtain the jar package in this way through decompression!

Add the following parameters to the java startup: If there are multiple javaagents such as pfinder, just add them after.

#decompress file Decompress dependencies to obtain the jacocoagent.jar package to avoid the need to contact operations and maintenance for package upload
jar -xvf $BASEDIR/lib/org.jacoco.agent-0.8.3.jar

-javaagent:$BASEDIR/bin/jacocoagent.jar=includes=com.jdwl.*,output=tcpserver,port=8840,address=127.0.0.1 -Xverify:none

In the premain method, we can add the implementation class of the ClassFileTransformer interface through Instrumention's addTransformer. The interface has only one method as follows. By implementing ClassTransformer, we can define our own code enhancement methods. We can use ASM or javasist and other advanced class libraries.

Related practice: Diving Into Bytecode Manipulation: Creating an Audit Log With ASM and Javassist | New Reli

4.4 JDOS resource reservation

Resource reservation /export directory custom processing

• Add configuration script /home/admin/clean_export.sh(The default content of the script has been increased && $9 != "coverage")

The output file path is /export/Data/coverage/code-cover.exec

#! /bin/bash

ls -lh /export | awk 'NR >1 {print}' | awk '{if ($9 != "Data") print $9}' | xargs -i /bin/rm -rf /export/{} > /dev/null 2>&1;
ls -lh /export/Data | awk 'NR >1 {print}' | awk '{if ($9 != "jdos.jd.com" && $9 != "coverage") print $9}' | xargs -i /bin/rm -rf /export/Data/{} > /dev/null 2>&1;

4.5 Download cover file

/export/Data/coverage/code-cover.exec

Log in to the bastion host terminal

cd /export/Data/coverage

jdos download file

curl -s up.bastion.jd.com/file/up | bash

4.6 Analyze the code

Open idea -> run -> show coverage data to select the corresponding exec file to obtain the code coverage status of the server.

Green covered (active code)

Red not covered (zombie code)

Reference

1. JaCoCo - Documentation
2. javaagent usage guide - rickiyang - CSDN Blog (cnblogs.com
3. Using Jacoco to count the coverage of server-side code in practice - M104 - CSDN Blog (cnblogs.com
4. Diving Into Bytecode Manipulation: Creating an Audit Log With ASM and Javassist | New Reli

Five. Efficiency improvement

5.1 Improve demand delivery efficiency

5.1.1 Shorten the demand delivery cycle

Because of the deletion of zombie code, the scope of development requirements is reduced, the cognitive cost of old code is reduced, and the regression test cost is reduced.

The overall demand delivery cycle shows a shortening trend! The implementation in January 2023, the previous demand delivery cycle was about 15 days, and after that, about 12 days.

5.1.2 Reduce the duration of the development phase

A large amount of zombie code exists, the research and development cognitive demand for changes is very high, and a lot of time cost needs to be consumed.

After the implementation in January 2023, the development phase duration has been shortened to less than 4 days (from 4.54 to 3.11, a reduction of about 31%)., showing a clearShortening trend!

5.2 Improve labor productivity

5.2.1 Reduce research and development cognitive load

Delete unnecessary zombie code,The cyclomatic complexity will be greatly reduced, and the number of repeated code blocks will also decrease, then the research and development cognitive load will also decrease!

The average number of repeated code blocks in the system has decreased from 31 to about 27, reducing the system maintenance cost!

5.2.2 Improve per capita demand throughput

Because of the reduction in human cognitive cost and the narrowing of the demand scope, it will be directlyimprovementDemand throughput!

Since the implementation in January 2023, the per capita demand throughput has also been greatly improved, from 1.5 before to about 2.5.

5.3 Process quality improvement

5.3.1 Reduce the number of automated bugs

As the existing zombie code decreases, the overall rollback use cases and scenarios become streamlined, and the golden process will not be disturbed by zombie code, so the number of automated bugs has also increased significantlya downward trend!

With the continuous practice since January 2023, the number of bugs discovered by automation has also decreased month by month, from 11 per month -> 9 per month -> 6 per month -> 5 per month.

5.3.2 Improve single test coverage

Since the implementation in January 2023, with the removal of a large amount of zombie code, the overall code volume has decreased, ineffective code has been mercilessly taken offline, while the single test code coverage has also been improved, showingTrending upward!Unit test line coverage from 51.33% -> 52.28%, improve system quality!

Six. Brief Summary

• With the continuous iteration and delivery of requirements, business code is bound to accumulate continuously, and the operation and maintenance cost will continue to rise. If the code of the online useless function is always left, it is a huge burden for R&D! For such code, there is a common convention called Zombie code.
• Quickly use jacoco probe to deeply analyze each line of code in the system, see the most real side of the online function running, according to the code coverage, selectively disable and delete zombie code,Slim down the system, lighten the burden of R&D!

Author: JD Logistics, Zhou Yiru

Source: JD Cloud Developer Community, From Monkey to Man, Tech