Author: Wang Beiyong, Yao Zaiyi, JD Logistics

1 Background

In the process of daily development, especially in the DDD process, we often encounter the interconversion of domain models such as VO/MODEL/PO. At this time, we will set the set|get one field at a time. Either use a tool class for violent property copying, during which a good tool can better improve the running efficiency of the program, otherwise, it may cause extreme situations such as low performance, hidden details, and OOM, etc.

2 Existing Technology

Direct set|get method: it is okay when there are few fields, but when there are a large number of fields, the workload is huge, and repeated operations are time-consuming and labor-intensive.
Implementation through reflection and introspection to achieve value mapping: for example, many open-source tools such as apache-common, spring, and hutool provide such implementation tools. The disadvantage of this method is that it has low performance and black-box property copying. The processing of different tool classes also differs: spring's property copying will ignore type conversion but not report an error, hutool will automatically perform type conversion, and some tools will throw exceptions, etc. When production problems occur, it is relatively difficult to locate them.
mapstruct: needs to manually define the converter interface before use, automatically generates the implementation class according to the interface class annotations and method annotations, with clear attribute conversion logic, but the conversion between different domain objects still needs to write a separate conversion interface or add a conversion method.

3 Extension Design

3.1 Introduction to mapstruct

Based on AbstractProcessor, MapStruct is extended to automatically generate entity mapping utility classes

This extension component is based on mapstruct for expansion, and simply introduces the implementation principle of mapstruct.

mapstruct is implemented based on JSR 269, which is a specification introduced by JDK. With it, it can realize the processing of annotations during the compilation period and read, modify, and add content in the abstract syntax tree. JSR 269 uses Annotation Processor to process annotations during the compilation period, and Annotation Processor is equivalent to a plugin of the compiler, therefore it is also called insertable annotation processing.

We know that the class loading mechanism of Java needs to be run through the compilation period and runtime period. As shown in the figure below

mapstruct is exactly in the process of compiling the source code during the compilation period, through the modification of the syntax tree to generate bytecode twice, as shown in the figure below

The above can be summarized as follows: several steps:

1. Generate the abstract syntax tree. The Java compiler compiles the Java source code to generate the abstract syntax tree (Abstract Syntax Tree, AST).

2. Call the program that implements the JSR 269 API. As long as the program implements the JSR 269 API, the annotation processor implemented will be called during the compilation period.

3. Modify the abstract syntax tree. In the program that implements JSR 269 API, the abstract syntax tree can be modified, and your own implementation logic can be inserted.

4. Generate bytecode. After modifying the abstract syntax tree, the Java compiler will generate the bytecode file corresponding to the modified abstract syntax tree.

From the perspective of the mapstruct implementation principle, we find that the mapstruct attribute conversion logic is clear and has good extensibility. The problem is that a separate conversion interface or a conversion method needs to be written. Can the conversion interface or method be automatically expanded?

3.2 Improved Solution

The mapstruct solution mentioned above has a drawback. That is, if there is a new domain model conversion, we have to manually write a layer of conversion interface. If there are A/B two models that need to be converted to each other, generally, four methods need to be defined: A->B, B->A, List<A>->List<B>, List<B>->List<A>

For this reason, this solution uses the original mapstruct defined in the conversion interface class annotation and conversion method annotation, through mapping, to form a new wrapper annotation. Define this annotation directly on the class or field of the model, and then directly compile the conversion interface according to the custom annotations on the model. Then, mapstruct generates the specific conversion implementation class based on the automatically generated interface.

Note: The annotations of the classes and methods in the automatically generated interfaces are the annotations of the original mapstruct, so the original functions of mapstruct are not lost. Detailed adjustment is shown in the figure below:

4 Implementation

4.1 Technical Dependencies

Compile-time annotation processor AbstractProcessor: Annotation Processor is a plugin for the compiler, so it is also called an insert annotation processing. To implement JSR 269, there are mainly the following steps.

1) Inherit the AbstractProcessor class and override the process method, and implement your own annotation processing logic in the process method.

2) Create it under the META-INF/services directory
Register the implementation of javax.annotation.processing.Processor file

2) Google AutoService: AutoService is an open-source library from Google that makes it easy to generate files that comply with the ServiceLoader specification. It is very simple to use. Just add annotations, and it can automatically generate the standard constraint files.

Knowledge points:The benefits of using AutoService are that it helps us not to manually maintain the META-INF file directory and file content required for Annotation Processor. It will automatically generate it for us, and the usage is also very simple. Just add the following annotation to the custom Annotation Processor class: @AutoService(Processor.class)

mapstruct: Helps implement custom plugin automatic generation of conversion interfaces and inject them into the spring container (the existing solution has been explained).
javapoet: JavaPoet is an open-source library for dynamically generating code. It helps us generate java class files quickly and easily. Its main features are as follows:

1) JavaPoet is a third-party dependency that can automatically generate Java files.

2) Simple and easy-to-understand API, easy to get started.

3) Make complex and repetitive Java files automatically generated, improve work efficiency, and simplify the process.

4.2 Implementation Steps

The first step: automatically generate the enumeration required for the interface class, named AlpacaMap and field annotation AlpacaMapField.

1) AlpacaMap: Defined on the class, the attribute target specifies the target model to be converted; the attribute uses specifies the external objects dependent on the conversion process.

2) AlpacaMapField: Pack all annotations supported by the original mapstruct as aliases, using the AliasFor annotation provided by Spring.

Knowledge points:@AliasFor is an annotation in the Spring framework used to declare the alias of annotation properties. It has two different application scenarios:

Alias within annotation

Alias of metadata

The main difference between the two is whether they are within the same annotation.

The second step: Implementation of AlpacaMapMapperDescriptor. This class mainly loads all model classes defined in the first step, and then saves the class information and class Field information for direct use later. The logic of the fragment is as follows:

AutoMapFieldDescriptor descriptor = new AutoMapFieldDescriptor();
            descriptor.target = fillString(alpacaMapField.target());
            descriptor.dateFormat = fillString(alpacaMapField.dateFormat());
            descriptor.numberFormat = fillString(alpacaMapField.numberFormat());
            descriptor.constant = fillString(alpacaMapField.constant());
            descriptor.expression = fillString(alpacaMapField.expression());
            descriptor.defaultExpression = fillString(alpacaMapField.defaultExpression());
            descriptor.ignore = alpacaMapField.ignore();
             ..........

The third step: The AlpacaMapMapperGenerator class mainly generates the corresponding class information, class annotations, class methods, and annotations on methods using JavaPoet

Generate class information: TypeSpec createTypeSpec(AlpacaMapMapperDescriptor descriptor)
Generate class annotation information: AnnotationSpec buildGeneratedMapperConfigAnnotationSpec(AlpacaMapMapperDescriptor descriptor) {
Generate class method information: MethodSpec buildMappingMethods(AlpacaMapMapperDescriptor descriptor)
Generate method annotation information: List<AnnotationSpec> buildMethodMappingAnnotations(AlpacaMapMapperDescriptor descriptor){

During the process of implementing the generation of class information, it is necessary to specify the interface class AlpacaBaseAutoAssembler for the generated class. This class mainly defines the following four methods:

public interface AlpacaBaseAutoAssembler<S,T>{
    T copy(S source);

    default List<T> copyL(List<S> sources){
        return sources.stream().map(c->copy(c)).collect(Collectors.toList());
    }

    @InheritInverseConfiguration(name = "copy")
    S reverseCopy(T source);

    default List<S> reverseCopyL(List<T> sources){
        return sources.stream().map(c->reverseCopy(c)).collect(Collectors.toList());
    }
}

The fourth step: since the generated class converter is injected into the Spring container, a special annotation for generating mapstruct injection into the Spring container is required. This annotation is automatically generated by the class AlpacaMapSpringConfigGenerator, and the core code is as follows:

private AnnotationSpec buildGeneratedMapperConfigAnnotationSpec() {
        return AnnotationSpec.builder(ClassName.get("org.mapstruct", "MapperConfig"))
                .addMember("componentModel", "$S", "spring")
                .build();
    }

Fifth step: By the above steps, we have defined the relevant classes, methods of the relevant classes, annotations of the relevant classes, and annotations of the methods of the relevant classes. At this point, we will link them together through Annotation Processor to generate class files output, the core method is as follows

private void writeAutoMapperClassFile(AlpacaMapMapperDescriptor descriptor){
        System.out.println("Start generating interface: " + descriptor.sourcePackageName() + "."+ descriptor.mapperName());
        try (final Writer outputWriter =
                     processingEnv
                             .getFiler()
                             .createSourceFile(descriptor.sourcePackageName() + "."+ descriptor.mapperName())
                             .openWriter()) {
            alpacaMapMapperGenerator.write(descriptor, outputWriter);
        } catch (IOException e) {
            processingEnv
                    .getMessager()
                    .printMessage(ERROR, "Error while opening "+ descriptor.mapperName() + " output file: " + e.getMessage());
        }
    }

Knowledge points:In javapoet, the core classes are roughly as follows, for reference:

JavaFile JavaFile is used to construct a Java file containing a top-level class and is the abstract definition of the .java file

TypeSpec TypeSpec is the abstract type of classes/interfaces/enums

MethodSpec MethodSpec is the abstract definition of methods/constructors

FieldSpec FieldSpec is the abstract definition of member variables/fields

ParameterSpec ParameterSpec is used to create method parameters

AnnotationSpec AnnotationSpec is used to create marker annotations

5 Practice

The following example illustrates how to use it. Here we define a model Person and a model Student, which involve field conversion of plain strings, enums, time formatting, and complex type transformation. The specific steps are as follows.

5.1 Introduction of Dependencies

The code has been uploaded to the code repository, and you can pull the branch and package it again for specific requirements

<dependency>
            <groupId>com.jdl</groupId>
            <artifactId>alpaca-mapstruct-processor</artifactId>
            <version>1.1-SNAPSHOT</version>
        </dependency>

5.2 Object Definition

The uses method must be a normal bean in the spring container, this bean provides methods annotated with @Named that can be specified as a string in the qualifiedByName attribute of the AlpacaMapField class annotation, as shown in the figure below

@Data
@AlpacaMap(targetType = Student.class, uses = {Person.class})
@Service
public class Person {
    private String make;
    private SexType type;

    @AlpacaMapField(target = "age")
    private Integer sax;

    @AlpacaMapField(target="dateStr", dateFormat = "yyyy-MM-dd")
    private Date date;

    @AlpacaMapField(target = "brandTypeName", qualifiedByName ="convertBrandTypeName")
    private Integer brandType;

    @Named("convertBrandTypeName")
    public  String convertBrandTypeName(Integer brandType){
        return BrandTypeEnum.getDescByValue(brandType);
    }

    @Named("convertBrandTypeName")
    public  Integer convertBrandType(String brandTypeName){
        return BrandTypeEnum.getValueByDesc(brandTypeName);
    }
}

5.3 Generation Results

Use Maven packaging or compilation to observe that at this time in
Two files PersonToStudentAssembler and PersonToStudentAssemblerImpl are generated in the target/generated-source/annotations directory

The class file PersonToStudentAssembler is automatically generated by the custom annotation processor, and its content is as follows

@Mapper(
    config = AutoMapSpringConfig.class,
    uses = {Person.class}
)
public interface PersonToStudentAssembler extends AlpacaBaseAutoAssembler<Person, Student> {
  @Override
  @Mapping(
      target = "age",
      source = "sax",
      ignore = false
  )
  @Mapping(
      target = "dateStr",
      dateFormat = "yyyy-MM-dd",
      source = "date",
      ignore = false
  )
  @Mapping(
      target = "brandTypeName",
      source = "brandType",
      ignore = false,
      qualifiedByName = "convertBrandTypeName"
  )
  PersonToStudentAssemblerImpl is automatically generated by mapstruct based on the PersonToStudentAssembler interface annotations, as follows
}

@Component

public class PersonToStudentAssemblerImpl implements PersonToStudentAssembler {
@Autowired

    private Person person;
    public Person reverseCopy(Student arg0) {

    @Override
    if (arg0 == null) {
        person.setSax(arg0.getAge());
            return null;
        }
        Person person = new Person();
        try {
        if (arg0.getDateStr() != null) {
            person.setDate(new SimpleDateFormat("yyyy-MM-dd").parse(arg0.getDateStr()));
                catch (ParseException e) {
            }
        }
            throw new RuntimeException(e);
        }
        person.setBrandType(person.convertBrandType(arg0.getBrandTypeName()));
        person.setMake(arg0.getMake());
        person.setType(arg0.getType());
        return person;
    }

    @Override
    public Student copy(Person source) {
        if (source == null) {
            return null;
        }
        Student student = new Student();
        student.setAge(source.getSax());
        if (source.getDate() != null) {
            student.setDateStr(new SimpleDateFormat("yyyy-MM-dd").format(source.getDate()));
        }
        student.setBrandTypeName(person.convertBrandTypeName(source.getBrandType()));
        student.setMake(source.getMake());
        student.setType(source.getType());
        return student;
    }
}

5.4 Spring Container Reference

At this time, we can directly use @Autowired to introduce the instance of the interface PersonToStudentAssembler into our Spring container for the conversion of four types of data maintenance.

AnnotationConfigApplicationContext applicationContext = new  AnnotationConfigApplicationContext();
        applicationContext.scan("com.jdl.alpaca.mapstruct");
        applicationContext.refresh();
        PersonToStudentAssembler personToStudentAssembler = applicationContext.getBean(PersonToStudentAssembler.class);
        Person person = new Person();
        person.setMake("make");
        person.setType(SexType.BOY);
        person.setSax(100);
        person.setDate(new Date());
        person.setBrandType(1);
        Student student = personToStudentAssembler.copy(person);
        System.out.println(student);
        System.out.println(personToStudentAssembler.reverseCopy(student));
        List<Person> personList = Lists.newArrayList();
        personList.add(person);
        System.out.println(personToStudentAssembler.copyL(personList));
        System.out.println(personToStudentAssembler.reverseCopyL(personToStudentAssembler.copyL(personList)));

Console prints:

personToStudentStudent(make=make, type=BOY, age=100, dateStr=2022-11-09, brandTypeName=集团KA)
studentToPersonPerson(make=make, type=BOY, sax=100, date=Wed Nov 09 00:00:00 CST 2022, brandType=1)
personListToStudentList[Student(make=make, type=BOY, age=100, dateStr=2022-11-09, brandTypeName=集团KA)]
studentListToPersonList[Person(make=make, type=BOY, sax=100, date=Wed Nov 09 00:00:00 CST 2022, brandType=1)]

Note:

The qualifiedByName annotation attribute is not very user-friendly, if this attribute is used, a reverse type conversion function needs to be defined. Because in the abstract interface AlpacaBaseAutoAssembler we defined earlier, there is an annotation as shown in the figure, which performs reverse mapping from the destination object to the source object, because of Java's overloading, the same name with different parameters is not the same method, so when converting from S to T, this method cannot be found. Therefore, it is necessary to define the conversion function by yourself

@InheritInverseConfiguration(name = "copy")

For example, when converting from S to T, the first method will be used, and when converting from T to S, a method with the same name as the Named annotation must be defined, with the method parameters and the previous method being input parameters and output parameters, respectively.

@Named("convertBrandTypeName")
    public  String convertBrandTypeName(Integer brandType){
        return BrandTypeEnum.getDescByValue(brandType);
    }

    @Named("convertBrandTypeName")
    public  Integer convertBrandType(String brandTypeName){
        return BrandTypeEnum.getValueByDesc(brandTypeName);
    }

When using the qualifiedByName annotation, the Named annotation method specified must be defined as an object manageable by the spring container, and this object Class needs to be introduced through the model class annotation attribute used

Knowledge points:

The InheritInverseConfiguration feature is very powerful, it can perform inverse mapping, as seen in PersonToStudentAssemblerImpl above, the property sax can be mapped to sex in the forward direction, and the inverse mapping can automatically map sex to sax. However, the @Mapping#expression, #defaultExpression, #defaultValue, and #constant of the forward mapping will be ignored by the inverse mapping. In addition, the inverse mapping of a certain field can be ignored and overridden by expression or constant