Java中LocalCache本地缓存实现代码

Java中LocalCache本地缓存实现代码

前言

本次分享探讨java平台的本地缓存，是指占用JVM的heap区域来缓冲存储数据的缓存组件。

一、本地缓存应用场景

localcache有着极大的性能优势：

1. 单机情况下适当使用localcache会使应用的性能得到很大的提升。

2. 集群环境下对于敏感性要求不高的数据可以使用localcache，只配置简单的失效机制来保证数据的相对一致性。

哪些数据可以存储到本地缓存？

1．访问频繁的数据；

2．静态基础数据（长时间内不变的数据）；

3．相对静态数据（短时间内不变的数据）。

二、java本地缓存标准

Java缓存新标准（javax.cache），这个标准由jsR107所提出，已经被包含在Java EE 7中。

特性：

1．原子操作，跟java.util.ConcurrentMap类似

2．从缓存中读取

3．写入缓存

4．缓存事件监听器

5．数据统计

6．包含所有隔离（ioslation）级别的事务

7．缓存注解（annotations）

8．保存定义key和值类型的泛型缓存

9．引用保存（只适用于堆缓存）和值保存定义

但目前应用不是很普遍。

三、java开源缓存框架

比较有名的本地缓存开源框架有：

1．EHCache

EHCache是一个纯java的在进程中的缓存，它具有以下特性：快速，简单，为Hibernate2.1充当可插入的缓存，最小的依赖性，全面的文档和测试。

BUG： 过期失效的缓存元素无法被GC掉，时间越长缓存越多，内存占用越大，导致内存泄漏的概率越大。

2．OSCache

OSCache有以下特点：缓存任何对象，你可以不受限制的缓存部分jsp页面或HTTP请求，任何java对象都可以缓存。拥有全面的API--OSCache API给你全面的程序来控制所有的OSCache特性。永久缓存--缓存能随意的写入硬盘，因此允许昂贵的创建(expensive-to-create)数据来保持缓存，甚至能让应用重启。支持集群--集群缓存数据能被单个的进行参数配置，不需要修改代码。缓存记录的过期--你可以有最大限度的控制缓存对象的过期，包括可插入式的刷新策略(如果默认性能不需要时)。

3．JCache

Java缓存新标准（javax.cache）

4．cache4j

cache4j是一个有简单API与实现快速的Java对象缓存。它的特性包括：在内存中进行缓存，设计用于多线程环境，两种实现：同步与阻塞，多种缓存清除策略：LFU, LRU, FIFO，可使用强引用。

5．ShiftOne

ShiftOne Java Object Cache是一个执行一系列严格的对象缓存策略的Java lib，就像一个轻量级的配置缓存工作状态的框架。

6．WhirlyCache

Whirlycache是一个快速的、可配置的、存在于内存中的对象的缓存。

四、LocalCache实现

1、LocalCache简介

LocalCache是一个精简版本地缓存组件，有以下特点：

1．  有容量上限maxCapacity；

2．  缓存达到容量上限时基于LRU策略来移除缓存元素；

3．  缓存对象的生命周期（缓存失效时间）由调用方决定；

4．  缓存对象失效后，将会有定时清理线程来清理掉，不会导致内存泄漏。

5．  性能比Ehcache稍强。

2、总体设计

LocalCache总体设计：

1．  缓存元素 CacheElement；

2．  缓存容器 LRULinkedHashMap；

3．  缓存接口 Cache；

4．  缓存组件实现 LocalCache。

3、详细设计

1．  CacheElement设计

/**

* 缓存元素

*

*/

public class CacheElement {

private Object key;

private Object value;

private long createTime;

private long lifeTime;

private int hitCount;

public CacheElement() {

}

public CacheElement(Object key ,Object value) {

this.key = key;

this.value = value;

this.createTime = System.currentTimeMillis();

}

public Object getKey() {

return key;

}

public void setKey(Object key) {

this.key = key;

}

public Object getValue() {

hitCount++;

return value;

}

public void setValue(Object value) {

this.value = value;

}

public long getCreateTime() {

return createTime;

}

public void setCreateTime(long createTime) {

this.createTime = createTime;

}

public int getHitCount() {

return hitCount;

}

public void setHitCount(int hitCount) {

this.hitCount = hitCount;

}

public long getLifeTime() {

return lifeTime;

}

public void setLifeTime(long lifeTime) {

this.lifeTime = lifeTime;

}

public boolean isExpired() {

boolean isExpired = System.currentTimeMillis() - getCreateTime() > getLifeTime();

return isExpired;

}

/*

* (non-Javadoc)

* @see java.lang.Object#toString()

*/

public String toString() {

StringBuffer sb = new StringBuffer();

sb.append("[ key=").append(key).append(", isExpired=").append(isExpired())

.append(", lifeTime=").append(lifeTime).append(", createTime=").append(createTime)

.append(", hitCount=").append(hitCount)

.append(", value=").append(value).append(" ]");

return sb.toString();

}

/*

* (non-Javadoc)

* @see java.lang.Object#hashCode()

*/

public final int hashCode(){

if(null == key){

return "".hashCode();

}

return this.key.hashCode();

}

/*

* (non-Javadoc)

* @see java.lang.Object#equals(java.lang.Object)

*/

public final boolean equals(Object object) {

if ((object == null) || (!(object instanceof CacheElement))) {

return false;

}

CacheElement element = (CacheElement) object;

if ((this.key == null) || (element.getKey() == null)) {

return false;

}

return this.key.equals(element.getKey());

}

}

2．  LRULinkedHashMap实现

import java.util.LinkedHashMap;

import java.util.Set;

import java.util.concurrent.locks.Lock;

import java.util.concurrent.locks.ReentrantLock;

/**

* 实现 LRU策略的 LinkedHashMap

*

* @param

* @param

*/

public class LRULinkedHashMap extends LinkedHashMap

{

protected static final long serialVersionUID = 2828675280716975892L;

protected static final int DEFAULT_MAX_ENTRIES = 100;

protected final int initialCapacity;

protected final int maxCapacity;

protected boolean enableRemoveEldestEntry = true;//是否允许自动移除比较旧的元素(添加元素时)

protected static final float DEFAULT_LOAD_FACTOR = 0.8f;

protected final Lock lock = new ReentrantLock();

public LRULinkedHashMap(int initialCapacity)

{

this(initialCapacity, DEFAULT_MAX_ENTRIES);

}

public LRULinkedHashMap(int initialCapacity ,int maxCapacity)

{

//set accessOrder=true, LRU

super(initialCapacity, DEFAULT_LOAD_FACTOR, true);

this.initialCapacity = initialCapacity;

this.maxCapacity = maxCapacity;

}

/*

* (non-Javadoc)

* @see java.util.LinkedHashMap#removeEldestEntry(java.util.Map.Entry)

*/

protected boolean removeEldestEntry(java.util.Map.Entry eldest)

{

return enableRemoveEldestEntry && ( size() > maxCapacity );

}

/*

* (non-Javadoc)

* @see java.util.LinkedHashMap#get(java.lang.Object)

*/

public V get(Object key)

{

try {

lock.lock();

return super.get(key);

}

finally {

lock.unlock();

}

}

/*

* (non-Javadoc)

* @see java.util.HashMap#put(java.lang.Object, java.lang.Object)

*/

public V put(K key, V value)

{

try {

lock.lock();

return super.put(key, value);

}

finally {

lock.unlock();

}

}

/*

* (non-Javadoc)

* @see java.util.HashMap#remove(java.lang.Object)

*/

public V remove(Object key) {

try {

lock.lock();

return super.remove(key);

}

finally {

lock.unlock();

}

}

/*

* (non-Javadoc)

* @see java.util.LinkedHashMap#clear()

*/

public void clear() {

try {

lock.lock();

super.clear();

}

finally {

lock.unlock();

}

}

/*

* (non-Javadoc)

* @see java.util.HashMap#keySet()

*/

public Set keySet() {

try {

lock.lock();

return super.keySet();

}

finally {

lock.unlock();

}

}

public boolean isEnableRemoveEldestEntry() {

return enableRemoveEldestEntry;

}

public void setEnableRemoveEldestEntry(boolean enableRemoveEldestEntry) {

this.enableRemoveEldestEntry = enableRemoveEldestEntry;

}

public int getInitialCapacity() {

return initialCapacity;

}

public int getMaxCapacity() {

return maxCapacity;

}

}

3．  Cache接口设计

/**

* 缓存接口

*

*/

public interface Cache {

/**

* 获取缓存

* @param key

* @return

*/

public T getCache(Object key);

/**

* 缓存对象

* @param key

* @param value

* @param milliSecond 缓存生命周期(毫秒)

*/

public void putCache(Object key, Object value ,Long milliSecond);

/**

* 缓存容器中是否包含 key

* @param key

* @return

*/

public boolean containsKey(Object key);

/**

* 缓存列表大小

* @return

*/

public int getSize();

/**

* 是否启用缓存

*/

public boolean isEnabled();

/**

* 启用 或 停止

* @param enable

*/

public void setEnabled(boolean enabled);

/**

* 移除所有缓存

*/

public void invalidateCaches();

/**

* 移除 指定key缓存

* @param key

*/

public void invalidateCache(Object key);

}

4．  LocalCache实现

import java.util.Date;

import java.util.Iterator;

import java.util.Random;

import org.slf4j.Logger;

import org.slf4j.LoggerFactory;

/**

* 本地缓存组件

*/

public class LocalCache implements Cache{

private Logger logger = LoggerFactory.getLogger(this.getClass());

private LRULinkedHashMap

protected boolean initFlag = false;//初始化标识

protected final long defaultLifeTime = 5 * 60 * 1000;//5分钟

protected boolean warnLongerLifeTime = false;

protected final int DEFAULT_INITIAL_CAPACITY = 100;

protected final int DEFAULT_MAX_CAPACITY = 100000;

protected int initialCapacity = DEFAULT_INITIAL_CAPACITY;//初始化缓存容量

protected int maxCapacity = DEFAULT_MAX_CAPACITY;//最大缓存容量

protected int timeout = 20;//存取缓存操作响应超时时间(毫秒数)

private boolean enabled = true;

private Thread gcThread = null;

private String lastGCInfo = null;//最后一次GC清理信息{ size, removeCount, time ,nowTime}

private boolean logGCDetail = false;//记录gc清理细节

private boolean enableGC = true;//是否允许清理的缓存(添加元素时)

private int gcMode = 0;//清理过期元素模式 { 0=迭代模式 ; 1=随机模式 }

private int gcIntervalTime = 2 * 60 * 1000;//间隔时间(分钟)

private boolean iterateScanAll = true;//是否迭代扫描全部

private float gcFactor = 0.5F;//清理百分比

private int maxIterateSize = DEFAULT_MAX_CAPACITY/2;//迭代模式下一次最大迭代数量

private volatile int iterateLastIndex = 0;//最后迭代下标

private int maxRandomTimes = 100;//随机模式下最大随机次数

protected final static Random random = new Random();

private static LocalCache instance = new LocalCache();

public static LocalCache getInstance() {

return instance;

}

private LocalCache(){

this.init();

}

protected synchronized void init() {

if(initFlag){

logger.warn("init repeat.");

return ;

}

this.initCache();

this.startGCDaemonThread();

initFlag = true;

if(logger.isInfoEnabled()){

logger.info("init -- OK");

}

}

private void startGCDaemonThread(){

if(initFlag){

return ;

}

this.maxIterateSize = maxCapacity /2;

try{

this.gcThread = new Thread() {

public void run() {

logger.info("[" + (Thread.currentThread().getName()) + "]start...");

//sleep

try {

Thread.sleep(getGcIntervalTime() < 30000 ? 30000 : getGcIntervalTime());

} catch (Exception e) {

e.printStackTrace();

}

while( true ){

//gc

gc();

//sleep

try {

Thread.sleep(getGcIntervalTime() < 30000 ? 30000 : getGcIntervalTime());

} catch (Exception e) {

e.printStackTrace();

}

}

}

};

this.gcThread.setName("localCache-gcThread");

this.gcThread.setDaemon(true);

this.gcThread.start();

if(logger.isInfoEnabled()){

logger.info("startGCDaemonThread -- OK");

}

}catch(Exception e){

logger.error("[localCache gc]DaemonThread -- error: " + e.getMessage(), e);

}

}

private void initCache(){

if(initFlag){

return ;

}

initialCapacity = (initialCapacity <= 0 ? DEFAULT_INITIAL_CAPACITY : initialCapacity);

maxCapacity = (maxCapacity < initialCapacity ? DEFAULT_MAX_CAPACITY : maxCapacity);

cacheMap = new LRULinkedHashMap

if(logger.isInfoEnabled()){

logger.info("initCache -- OK");

}

}

/*

* (non-Javadoc)

*/

@Supprehttp://ssWarnings("unchecked")

public T getCache(Object key) {

if(!isEnabled()){

return null;

}

long st = System.currentTimeMillis();

T objValue = null;

CacheElement cacheObj = cacheMap.get(key);

if (isExpiredCache(cacheObj)) {

cacheMap.remove(key);

}else {

objValue = (T) (cacheObj == null ? null : cacheObj.getValue());

}

long et = System.currentTimeMillis();

if((et - st)>timeout){

if(this.logger.isWarnEnabled()){

this.logger.warn("getCache_timeout_" + (et - st) + "_[" + key + "]");

}

}

if(logger.isDebugEnabled()){

String message = ("get( " + key + ") return: " + objValue);

logger.debug(message);

}

return objValue;

}

/*

* (non-Javadoc)

*/

public void putCache(Object key, Object value ,Long lifeTime) {

if(!isEnabled()){

return;

}

Long st = System.currentTimeMillis();

lifeTime = (null == lifeTime ? defaultLifeTime : lifeTime);

CacheElement cacheObj = new CacheElement();

cacheObj.setCreateTime(System.currentTimeMillis());

cacheObj.setLifeTime(lifeTime);

cacheObj.setValue(value);

cacheObj.setKey(key);

cacheMap.put(key, cacheObj);

long et = System.currentTimeMillis();

if((et - st)>timeout){

if(this.logger.isWarnEnabled()){

this.logger.warn("putCache_timeout_" + (et - st) + "_[" + key + "]");

}

}

if(logger.isDebugEnabled()){

String message = ("putCache( " + cacheObj + " ) , 耗时 " + (et - st) + "(毫秒).");

logger.debug(message);

}

if(lifeTime > defaultLifeTime && this.isWarnLongerLifeTime()){

if(logger.isWarnEnabled()){

String message = ("LifeTime[" + (lifeTime/1000) + "秒] too long for putCache(" + cacheObj + ")");

logger.warn(message);

}

}

}

/**

* key 是否过期

* @param key

* @return

*/

protected boolean isExpiredKey(Object key) {

CacheElement cacheObj = cacheMap.get(key);

return this.isExpiredCache(cacheObj);

}

/**

* cacheObj 是否过期

* @param key

* @return

*/

protected boolean isExpiredCache(CacheElement cacheObj) {

if (cacheObj == null) {

return false;

}

return cacheObj.isExpired();

}

/*

* (non-Javadoc)

*/

public void invalidateCaches(){

try{

cacheMap.clear();

}catch(Exception e){

e.printStackTrace();

}

}

/*

* (non-Javadoc)

*/

public void invalidateCache(Object key){

try{

cacheMap.remove(key);

}catch(Exception e){

e.printStackTrace();

}

}

/*

* (non-Javadoc)

*/

public boolean containsKey(Object key) {

return cacheMap.containsKey(key);

}

/*

* (non-Javadoc)

*/

public int getSize() {

return cacheMap.size();

}

/*

* (non-Javadoc)

*/

public Iterator

return cacheMap.keySet().iterator();

}

/*

* (non-Javadoc)

*/

public boolean isEnabled() {

return this.enabled;

}

/*

* (non-Javadoc)

*/

public void setEnabled(boolean enabled) {

this.enabled = enabled;

if(!this.enabled){

//清理缓存

this.invalidateCaches();

}

}

/**

* 清理过期缓存

*/

protected synchronized boolean gc(){

if(!isEnableGC()){

return false;

}

try{

iterateRemoveExpiredCache();

}catch(Exception e){

logger.error("gc() has error: " + e.getMessage(), e);

}

return true;

}

/**

* 迭代模式 - 移除过期的 key

* @param exceptKey

*/

private void iterateRemoveExpiredCache(){

long startTime = System.currentTimeMillis();

int size = cacheMap.size();

if(size ==0){

return;

}

int keyCount = 0;

int removedCount = 0 ;

int startIndex = 0;

int endIndex = 0;

try{

Object [] keys = cacheMap.keySet().toArray();

keyCount = keys.length;

int maxIndex = keyCount -1 ;

//初始化扫描下标

if(iterateScanAll){

startIndex = 0;

endIndex = maxIndex;

}else {

int gcThreshold = this.getGcThreshold();

int iterateLen = gcThreshold > this.maxIterateSize ? this.maxIterateSize : gcThreshold;

startIndex = this.iterateLastIndex;

startIndex = ( (startIndex < 0 || startIndex > maxIndex) ? 0 : startIndex );

endIndex = (startIndex + iterateLen);

endIndex = (endIndex > maxIndex ? maxIndex : endIndex);

}

//迭代清理

boolean flag = false;

for(int i=startIndex; i<= endIndex; i++){

flag = this.removeExpiredKey(keys[i]);

if(flag){

removedCount++;

}

}

this.iterateLastIndex = endIndex;

keys = null;

}catch(Exception e){

logger.error("iterateRemoveExpiredCache -- 移除过期的 key时出现异常: " + e.getMessage(), e);

}

long endTime = System.currentTimeMillis();

StringBuffer sb = new StringBuffer();

sb.append("iterateRemoveExpiredCache [ size: ").append(size).append(", keyCount: ").append(keyCount)

.append(", startIndex: ").append(startIndex).append(", endIndex: ").append(iterateLastIndex)

.append(", removedCount: ").append(removedCount).append(", currentSize: ").append(this.cacheMap.size())

.append(", timeConsuming: ").append(endTime - startTime).append(", nowTime: ").append(new Date())

.append(" ]");

this.lastGCInfo = sb.toString();

if(logger.isInfoEnabled()){

logger.info("iterateRemoveExpiredCache -- 清理结果 -- "+ lastGCInfo);

}

}

/**

* 随机模式 - 移除过期的 key

*/

private void randomRemoveExpiredCache(){

long startTime = System.currentTimeMillis();

int size = cacheMap.size();

if(size ==0){

return;

}

int removedCount = 0 ;

try{

Object [] keys = cacheMap.keySet().toArray();

int keyCount = keys.length;

boolean removeFlag = false;

int removeRandomTimes = this.getGcThreshold();

removeRandomTimes = ( removeRandomTimes > this.getMaxRandomTimes() ? this.getMaxRandomTimes() : removeRandomTimes );

while(removeRandomTimes-- > 0){

int index = random.nextInt(keyCount);

boolean flag = this.removeExpiredKey(keys[index]);

if(flag){

removeFlag = true;

removedCount ++;

}

}

//尝试 移除 首尾元素

if(!removeFlag){

this.removeExpiredKey(keys[0]);

this.removeExpiredKey(keys[keyCount-1]);

}

keys=null;

}catch(Exception e){

logger.error("randomRemoveExpiredCache -- 移除过期的 key时出现异常: " + e.getMessage(), e);

}

long endTime = System.currentTimeMillis();

StringBuffer sb = new StringBuffer();

sb.append("randomRemoveExpiredCache [ size: ").append(size).append(", removedCount: ").append(removedCount)

.append(", currentSize: ").append(this.cacheMap.size()).append(", timeConsuming: ").append(endTime - startTime)

.append(", nowTime: ").append(new Date())

.append(" ]");

this.lastGCInfo = sb.toString();

if(logger.isInfoEnabled()){

logger.info("randomRemoveExpiredCache -- 清理结果 -- "+ lastGCInfo);

}

}

private boolean removeExpiredKey(Object key){

boolean flag = false;

CacheElement cacheObj = null;

if(null != key){

try{

cacheObj = cacheMap.get(key);

boolean isExpiredCache = this.isExpiredCache(cacheObj);

if(isExpiredCache){

cacheMap.remove(key);

flag = true;

}

}catch(Exception e){

logger.error("removeExpired(" + key + ") -- error: " + e.getMessage(), e);

}

}

if(!flag && logGCDetail){

this.logger.warn("removeExpiredKey(" + key + ") return [" + flag + "]--" + cacheObj);

}

return flag;

}

public int getInitialCapacity() {

return initialCapacity;

}

public int getMaxCapacity() {

return maxCapacity;

}

public int getGcMode() {

return gcMode;

}

public void setGcMode(int gcMode) {

this.gcMode = gcMode;

}

public int getGcIntervalTime() {

return gcIntervalTime;

}

public void setGcIntervalTime(int gcIntervalTime) {

this.gcIntervalTime = gcIntervalTime;

}

public boolean isEnableGC() {

return enableGC;

}

public void setEnableGC(boolean enableGC) {

this.enableGC = enableGC;

}

public boolean isIterateScanAll() {

return iterateScanAll;

}

public void setIterateScanAll(boolean iterateScanAll) {

this.iterateScanAll = iterateScanAll;

}

public float getGcFactor() {

return gcFactor;

}

public void setGcFactor(float gcFactor) {

this.gcFactor = gcFactor;

}

/**

* gc 阀值

* @return

*/

public int getGcThreshold() {

int threshold = (int)( this.cacheMap.getMaxCapacity() * gcFactor );

return threshold;

}

public String getLastGCInfo() {

return lastGCInfo;

}

public void setLastGCInfo(String lastGCInfo) {

this.lastGCInfo = lastGCInfo;

}

public boolean isLogGCDetail() {

return logGCDetail;

}

public void setLogGCDetail(boolean logGCDetail) {

this.logGCDetail = logGCDetail;

}

public int getTimeout() {

return timeout;

}

public void setTimeout(int timeout) {

this.timeout = timeout;

}

public int getMaxIterateSize() {

return maxIterateSize;

}

public void setMaxIterateSize(int maxIterateSize) {

this.maxIterateSize = maxIterateSize;

}

public int getMaxRandomTimes() {

return maxRandomTimes;

}

public void setMaxRandomTimes(int maxRandomTimes) {

this.maxRandomTimes = maxRandomTimes;

}

public boolean isInitFlag() {

return initFlag;

}

public long getDefaultLifeTime() {

return defaultLifeTime;

}

public boolean isWarnLongerLifeTime() {

return warnLongerLifeTime;

}

public void setWarnLongerLifeTime(boolean warnLongerLifeTime) {

this.warnLongerLifeTime = warnLongerLifeTime;

}

//=http://======================= dynMaxCapacity ========================

private int dynMaxCapacity = maxCapacity;

public int getDynMaxCapacity() {

return dynMaxCapacity;

}

public void setDynMaxCapacity(int dynMaxCapacity) {

this.dynMaxCapacity = dynMaxCapacity;

}

public void resetMaxCapacity(){

if(dynMaxCapacity > initialCapacity && dynMaxCapacity != maxCapacity){

if(logger.isInfoEnabled()){

logger.info("resetMaxCapacity( " + dynMaxCapacity + " ) start...");

}

synchronized(cacheMap){

LRULinkedHashMap

cacheMap.clear();

cacheMap = cacheMap0;

this.maxCapacity = dynMaxCapacity;

}

if(logger.isInfoEnabled()){

logger.info("resetMaxCapacity( " + dynMaxCapacity + " ) OK.");

}

}else {

if(logger.isWarnEnabled()){

logger.warn("resetMaxCapacity( " + dynMaxCapacity + " ) NO.");

}

}

}

//======================== showCacheElement ========================

private String showCacheKey;

public String getShowCacheKey() {

return showCacheKey;

}

public void setShowCacheKey(String showCacheKey) {

this.showCacheKey = showCacheKey;

}

public Object showCacheElement(){

Object v = null;

if(null != this.showCacheKey){

v = cacheMap.get(showCacheKey);

}

return v;

}

}

版权声明：本文内容由网络用户投稿，版权归原作者所有，本站不拥有其著作权，亦不承担相应法律责任。如果您发现本站中有涉嫌抄袭或描述失实的内容，请联系我们jiasou666@gmail.com 处理，核实后本网站将在24小时内删除侵权内容。