How to make ConcurrentMapCacheManager support automatic deletion of expired items

Preface:In our application, there are some data that are obtained through rpc remote data, and this data does not change frequently, allowing the client to cache it locally for a certain period of time.

The logic of this scenario is simple, the cached data is small, and it does not require persistence, so it is not necessary to introduce other third-party caching tools to increase the burden on the application, which is very suitable for implementing with Spring Cache.

How to make ConcurrentMapCacheManager support automatic deletion of expired items

However, there is a problem that we hope to cache these rpc result data and automatically delete them after a certain period of time to realize obtaining the latest data after a certain period of time, similar to the expiration time of Redis.

Next are my research steps and development process.

What is Spring Cache?

Spring Cache is an abstraction layer of Spring, which automatically caches the returned results of method calls to improve system performance and response speed.
The goal is to simplify the use of caching, provide a consistent caching access method, and make it easy for developers to quickly add caching to the application.
Applied at the method level, when calling a method with the same parameters next time, the result can be obtained directly from the cache without executing the actual method body.

Application scenarios?

Including but not limited to:

Frequent method calls, improve performance by caching results
Database query results, cache the query results to reduce database access
External service call results, cache the response results of external services to reduce network overhead
Compute the result, and cache the computation result to accelerate the speed of subsequent calculations

Advantages and disadvantages

Advantages:

Improve the performance of the application, avoiding repeated computation or query.
Reduce access to underlying resources such as databases or remote services, thus reducing the load.
Simplify code by implementing caching logic through annotations, without the need to manually write caching code.

Disadvantages:

It needs to occupy a certain amount of memory space to store cached data.
May lead to data inconsistency issues. If the cached data changes but the cache is not updated in time, it may lead to dirty data issues. (Therefore, it is necessary to update the cache in a timely manner)
May cause cache penetration issues, when a large number of requests access a key that does not exist in the cache simultaneously, it may cause the requests to directly fall to the underlying resources, increasing the load.

Important components

CacheManager: A cache manager used to create, configure, and manage caching objects. It can be configured with specific cache implementations, such as Ehcache, Redis.
Cache: A caching object used to store cached data, which provides methods for reading, writing, and deleting cached data.
Common annotations:
- @Cacheable: When called, it will check if the result already exists in the cache. If it does, it will return the cached result directly; otherwise, it will execute the method and store the result in the cache, which is suitable for read-only operations.
- @CachePut: It will execute the method body each time and store the result in the cache, i.e., it will update the data in the cache each time, which is suitable for write operations.
- @CacheEvict: When called, Spring Cache will clear the corresponding cache data.

Usage method

Configure the cache manager (CacheManager): Use @EnableCachingAnnotations to enable caching functionality and configure specific cache implementations.
Add cache annotations to the method: Use @Cacheable,@CacheEvict,@CachePutAnnotations to mark methods that need to be cached.
Invoke the cached method: When calling a method marked as cached, Spring Cache will check if the cached result of the method already exists in the cache.
Return data based on cache results: If there is already a result in the cache, it will be returned directly from the cache; otherwise, the method will be executed and the result will be stored in the cache.
Clear or update the cache as needed: Use @CacheEvict,@CachePutAnnotations can clear or update the cache after method calls.
Through the above steps, Spring Cache can automatically manage cache read and write operations, thus simplifying the use and management of caching.

What is the default implementation used by Spring Boot and its advantages and disadvantages:

Spring Boot uses the defaultConcurrentMapCacheManagerAs an implementation of a cache manager, it is suitable for simple, single-machine, and applications with small cache capacity requirements.

Advantages:
1. Simple and lightweight: Without external dependencies, suitable for simple application scenarios.
2. Memory storage: Cache data is stored in theConcurrentMapIt is fast in reading and writing, suitable for fast access and frequent updates of data.
3. Support for multiple cache instances: Supports configuring multiple named cache instances, each using an independentConcurrentMapStore data, and multiple cache instances can be configured according to different needs.
Disadvantages:
1. Single-machine application limitations:ConcurrentMapCacheManagerIs suitable for single-machine applications, where cache data is stored in the application's memory and cannot implement distributed caching.
2. Limited capacity: Since the cache data is stored in memory,ConcurrentMapCacheManagerThe capacity is limited by the size of the application's memory, which may lead to insufficient capacity for large-scale data or high-concurrency access scenarios.
3. Lack of persistence support:ConcurrentMapCacheManagerDoes not support persisting cache data to disk or other external storage media; after the application restarts, the cache data will be lost.

How to make`ConcurrentMapCacheManager`Supports automatic deletion of expired data

As mentioned in the preface, our scenario logic is simple, the cache data is small, there is no need for persistence, and we do not want to introduce other third-party caching tools to increase the application burden, making it suitable for useConcurrentMapCacheManager. So let's expand on thatConcurrentMapCacheManagerPerhaps the simplest implementation.

Plan Design

For this reason, I have designed three plans:

Enable scheduled tasks to scan the cache and delete all cache data at regular intervals; this method is simple and forceful, deleting data at fixed intervals, but cannot perform expiration operations on individual data.
Enable scheduled tasks to scan the cache and delete expired cache data individually.
Before accessing the cache data, judge whether it has expired, and if it has expired, re-execute the method body and overwrite the original cache data with the result.

The above two and three plans are more closely aligned with the target, and both have a common difficulty, that is, how to determine whether the cache has expired? Or how to store the expiration time of the cache?
Since there is no good way, let's take a look at the source code to find some ideas!

Source Code Analysis

ConcurrentMapCacheManagerdefines acacheMap(As shown in the code below), it is used to store all cache names and corresponding cache objects.

private final ConcurrentMap<String, Cache> cacheMap = new ConcurrentHashMap<>(16);

cacheMapof which storesCacheThe specific type ofConcurrentMapCache,
WhileConcurrentMapCacheThe internal definition of astore(As shown in the code below), it is used to store all keys and values under this cache, that is, the actual cache data.

private final ConcurrentMap<Object, Object> store;

Its relationship diagram is:

The following is test code, adding a cache operation for a query: cacheName=getUsersByName, key is the value of the parameter name, and value is the query user collection.

@Service
public class UserServiceImpl implements UserService {
    @Autowired
    private UserMapper userMapper;

    @Override
    @Cacheable(value = "getUsersByName", key = "#name")
    public List<GyhUser> getUsersByName(String name) {
        return userMapper.getUsersByName(name);
    }
}

Before the program calls this method, it will automatically enter the cache interceptorCacheInterceptor, then enterConcurrentMapCacheManagerofgetCachemethod, get the corresponding cache instance, and generate one if it does not exist.

Then search for the cache data in the cache instance, return if found, and execute the target method if not found.

After executing the target method, the return result is placed in the cache.

to implement automatic expiration and deletion

According to the code above, it can be found that the cache data key/value is stored in the specific cache instanceConcurrentMapCacheofstoreAnd there is space for me to operate before and after get and put.

So, if I wrap the value again and encapsulate the cache time, and parse the actual cache data before and after get and put to provide it for developers to use, can it be realized? Let's get to work right away!

/**
 * Cache data wrapper class, ensuring cache data and insertion time
 */
public class ExpireCacheWrap {
    /**
     * Cache data
     */
    private final Object value;
    /**
     * Insertion time
     */
    private final Long insertTime;

    public ExpireCacheWrap(Object value, Long insertTime) {
        this.value = value;
        this.insertTime = insertTime;
    }

    public Object getValue() {
        return value;
    }

    public Long getInsertTime() {
        return this.insertTime;
    }
}

define a customCacheclass, inheritsConcurrentMapCache，extend get、put methods to record and parse cache time

/**
 * Cache expiration deletion
 */
public class ExpireCache extends ConcurrentMapCache {
    public ExpireCache(String name) {
        super(name);
    }

    @Override
    public ValueWrapper get(Object key) {
        // When parsing the cache object, obtain the value and remove the insertion time. The usage logic of the cache in the business is unaware and non-intrusive, so no adjustment of the related code is required
        ValueWrapper valueWrapper = super.get(key);
        if (valueWrapper == null) {
            return null;
        }
        Object storeValue = valueWrapper.get();
        storeValue = storeValue != null ? ((ExpireCacheWrap) storeValue).getValue() : null;
        return super.toValueWrapper(storeValue);
    }

    @Override
    public void put(Object key, @Nullable Object value) {
        // When inserting the cache object, encapsulate the object information: cache content + insertion time
        value = new ExpireCacheWrap(value, System.currentTimeMillis());
        super.put(key, value);
    }
}

Custom cache manager, using the customExpireCacheto replace the defaultConcurrentMapCache

/**
 * Cache manager
 */
public class ExpireCacheManager extends ConcurrentMapCacheManager {
    @Override
    protected Cache createConcurrentMapCache(String name) {
        return new ExpireCache(name);
    }
}

Add the custom cache managerExpireCacheManagerInject into the container

@Configuration
class ExpireCacheConfiguration {
    @Bean
    public ExpireCacheManager cacheManager() {
        ExpireCacheManager cacheManager = new ExpireCacheManager();
        return cacheManager;
    }
}

Enable scheduled task to automatically delete expired cache

/**
 * Schedule execution to delete expired cache
 */
@Component
@Slf4j
public class ExpireCacheEvictJob {

    @Autowired
    private ExpireCacheManager cacheManager;
    /**
     * Cache name and cache time
     */
    private static Map<String, Long> cacheNameExpireMap;
    // Can be optimized to configuration file or dictionary
    static {
        cacheNameExpireMap = new HashMap<>(5);
        cacheNameExpireMap.put("getUserById", 180000L);
        cacheNameExpireMap.put("getUsersByName", 300000L);
    }

    /**
     Run once every 5 minutes
     */
    @Scheduled(fixedRate = 300000)
    public void cacheEvict() {
        Long now = System.currentTimeMillis();
        // Get all caches
        Collection<String> cacheNames = cacheManager.getCacheNames();
        for (String cacheName : cacheNames) {
            // The expiration time set for this type of cache
            Long expire = cacheNameExpireMap.get(cacheName);
            // Get the collection of cache content for this cache
            Cache cache = cacheManager.getCache(cacheName);
            ConcurrentMap<Object, Object> store = (ConcurrentMap) cache.getNativeCache();
            Set<Object> keySet = store.keySet();
            // Loop to get cache key-value pairs, judge whether the key has expired based on the insertion time stored in the value, and delete it if expired
            keySet.stream().forEach(key -> {
                // Cache content wrapping object
                ExpireCacheWrap value = (ExpireCacheWrap) store.get(key);
                // Cache content insertion time
                Long insertTime = value.getInsertTime();
                if ((insertTime + expire) < now) {
                    cache.evict(key);
                    log.info("key={},insertTime={},expire={},delete expired", key, insertTime, expire);
                }
            });
        }

    }
}

Through the above operations, we have achievedConcurrentMapCacheManagerSupports automatic deletion of expired items and is basically invisible and non-intrusive to developers, you only need to configure the cache time in the configuration file.

But if my project has already supported third-party caching such as Redis, adhering to the principle of 'use it or lose it', how should this feature be integrated with Redis?

Just as our project has recently been introducing R2m, let's give it a try^-^.

To be continued...Thanks~