【Android開發】深入理解硬碟快取類DiskLruCache:原始碼分析
1. 原理
DisKLruCache是一個用於硬碟儲存的cache的類,它不是Android API,其GitHub地址為【https://github.com/JakeWharton/DiskLruCache】,也有些部落格對其進行了介紹【Android DiskLruCache完全解析,硬碟快取的最佳方案:http://blog.csdn.net/guolin_blog/article/details/28863651】。與LruCache類似,實現依賴於LinkedHashMap<String,Entry> lruEntries。lruEntries儲存了鍵值對key-value,其中key是String型別,value是位元組序列,位元組序列最大為Integer.MAX_VALUE。
DisKLruCache本質上是將每個value儲存到對應的key生成的檔名的檔案中,方便本地索引。一旦總檔案超過設定的快取大小,就刪除隊標頭檔案。
DisKLruCache本身不是執行緒安全的,需要額外的執行緒安全處理。
2. 原始碼分析
2.1初始化
先來看一下成員變數。
private final File directory; //路徑 private final File journalFile; //日誌檔案 private final File journalFileTmp; //臨時日誌檔案 private final int appVersion; //app的版本 private final long maxSize; //快取的最大容量 private final int valueCount; //1個key對應多少個快取檔案 private long size = 0; private Writer journalWriter; //日誌檔案寫入 private final LinkedHashMap<String, Entry> lruEntries = new LinkedHashMap<String, Entry>(0, 0.75f, true); //快取佇列 private int redundantOpCount; /** * To differentiate between old and current snapshots, each entry is given * a sequence number each time an edit is committed. A snapshot is stale if * its sequence number is not equal to its entry's sequence number. */ private long nextSequenceNumber = 0
DisKLruCache採用靜態工廠的模式進行初始化。如果日誌檔案存在,就從檔案中讀取相關資訊,進一步完善DisKLruCache物件;否則就直接初始化DisKLruCache物件,並且重建日誌檔案。
private DiskLruCache(File directory, int appVersion, int valueCount, long maxSize) { this.directory = directory; this.appVersion = appVersion; this.journalFile = new File(directory, JOURNAL_FILE); this.journalFileTmp = new File(directory, JOURNAL_FILE_TMP); this.valueCount = valueCount; this.maxSize = maxSize; } /** * Opens the cache in {@code directory}, creating a cache if none exists * there. * * @param directory a writable directory * @param appVersion * @param valueCount the number of values per cache entry. Must be positive. * @param maxSize the maximum number of bytes this cache should use to store * @throws java.io.IOException if reading or writing the cache directory fails */ public static DiskLruCache open(File directory, int appVersion, int valueCount, long maxSize) throws IOException { if (maxSize <= 0) { throw new IllegalArgumentException("maxSize <= 0"); } if (valueCount <= 0) { throw new IllegalArgumentException("valueCount <= 0"); } // prefer to pick up where we left off DiskLruCache cache = new DiskLruCache(directory, appVersion, valueCount, maxSize); if (cache.journalFile.exists()) {//日誌檔案存在 try { cache.readJournal(); //讀取日誌檔案 cache.processJournal();// 處理日誌檔案 cache.journalWriter = new BufferedWriter(new FileWriter(cache.journalFile, true), IO_BUFFER_SIZE); return cache; } catch (IOException journalIsCorrupt) { // System.logW("DiskLruCache " + directory + " is corrupt: " // + journalIsCorrupt.getMessage() + ", removing"); cache.delete(); } } // create a new empty cache directory.mkdirs(); cache = new DiskLruCache(directory, appVersion, valueCount, maxSize); cache.rebuildJournal(); //日誌檔案不存在,就重建日誌檔案 return cache; }
第一次使用需要新建日誌檔案,日誌只寫入了固定的前5行,稍後會解釋該日誌檔案。
/**
* Creates a new journal that omits redundant information. This replaces the
* current journal if it exists.
*/
private synchronized void rebuildJournal() throws IOException {
if (journalWriter != null) {
journalWriter.close();
}
Writer writer = new BufferedWriter(new FileWriter(journalFileTmp), IO_BUFFER_SIZE);
writer.write(MAGIC);
writer.write("\n");
writer.write(VERSION_1);
writer.write("\n");
writer.write(Integer.toString(appVersion));
writer.write("\n");
writer.write(Integer.toString(valueCount));
writer.write("\n");
writer.write("\n");
for (Entry entry : lruEntries.values()) {
if (entry.currentEditor != null) {
writer.write(DIRTY + ' ' + entry.key + '\n');
} else {
writer.write(CLEAN + ' ' + entry.key + entry.getLengths() + '\n');
}
}
writer.close();
journalFileTmp.renameTo(journalFile);
journalWriter = new BufferedWriter(new FileWriter(journalFile, true), IO_BUFFER_SIZE);
}
2.2 Key-Value入隊
Value被儲存到本地檔案中,留在記憶體中充當Value角色的是該檔案的檔名。這裡將value的檔名以靜態內部類的形式封裝成Entry。Entry接收key為成員變數,並提供了File getCleanFile(int i)為檔案的返回值,而File getDirtyFile(inti)返回了臨時檔案。
private final class Entry {
private final String key;
/** Lengths of this entry's files. */
private final long[] lengths;
/** True if this entry has ever been published */
private boolean readable;
/** The ongoing edit or null if this entry is not being edited. */
private Editor currentEditor;
/** The sequence number of the most recently committed edit to this entry. */
private long sequenceNumber;
private Entry(String key) {
this.key = key;
this.lengths = new long[valueCount];
}
public String getLengths() throws IOException {
StringBuilder result = new StringBuilder();
for (long size : lengths) {
result.append(' ').append(size);
}
return result.toString();
}
/**
* Set lengths using decimal numbers like "10123".
*/
private void setLengths(String[] strings) throws IOException {
if (strings.length != valueCount) {
throw invalidLengths(strings);
}
try {
for (int i = 0; i < strings.length; i++) {
lengths[i] = Long.parseLong(strings[i]);
}
} catch (NumberFormatException e) {
throw invalidLengths(strings);
}
}
private IOException invalidLengths(String[] strings) throws IOException {
throw new IOException("unexpected journal line: " + Arrays.toString(strings));
}
public File getCleanFile(int i) {
return new File(directory, key + "." + i);
}
public File getDirtyFile(int i) {
return new File(directory, key + "." + i + ".tmp");
}
}
然後,另一個靜態內部類Editor封裝了Entry,提供隊Entry的編輯功能,並向外提供了Entry對應檔案的輸入輸出流物件,入隊時,呼叫了Editor的commit()函式。需要說明的一點,Editor中又有另一個靜態內部類FaultHidingOutputStream,用來封裝輸出流,這裡返回的臨時檔案的輸出流。
/**
* Edits the values for an entry.
*/
public final class Editor {
private final Entry entry;
private boolean hasErrors;
private Editor(Entry entry) {
this.entry = entry;
}
/**
* Returns an unbuffered input stream to read the last committed value,
* or null if no value has been committed.
*/
public InputStream newInputStream(int index) throws IOException {
synchronized (DiskLruCache.this) {
if (entry.currentEditor != this) {
throw new IllegalStateException();
}
if (!entry.readable) {
return null;
}
return new FileInputStream(entry.getCleanFile(index));
}
}
/**
* Returns the last committed value as a string, or null if no value
* has been committed.
*/
public String getString(int index) throws IOException {
InputStream in = newInputStream(index);
return in != null ? inputStreamToString(in) : null;
}
/**
* Returns a new unbuffered output stream to write the value at
* {@code index}. If the underlying output stream encounters errors
* when writing to the filesystem, this edit will be aborted when
* {@link #commit} is called. The returned output stream does not throw
* IOExceptions.
*/
public OutputStream newOutputStream(int index) throws IOException {
synchronized (DiskLruCache.this) {
if (entry.currentEditor != this) {
throw new IllegalStateException();
}
return new FaultHidingOutputStream(new FileOutputStream(entry.getDirtyFile(index)));
}
}
/**
* Sets the value at {@code index} to {@code value}.
*/
public void set(int index, String value) throws IOException {
Writer writer = null;
try {
writer = new OutputStreamWriter(newOutputStream(index), UTF_8);
writer.write(value);
} finally {
closeQuietly(writer);
}
}
/**
* Commits this edit so it is visible to readers. This releases the
* edit lock so another edit may be started on the same key.
*/
public void commit() throws IOException {
if (hasErrors) {
completeEdit(this, false);
remove(entry.key); // the previous entry is stale
} else {
completeEdit(this, true);
}
}
/**
* Aborts this edit. This releases the edit lock so another edit may be
* started on the same key.
*/
public void abort() throws IOException {
completeEdit(this, false);
}
private class FaultHidingOutputStream extends FilterOutputStream {
private FaultHidingOutputStream(OutputStream out) {
super(out);
}
@Override public void write(int oneByte) {
try {
out.write(oneByte);
} catch (IOException e) {
hasErrors = true;
}
}
@Override public void write(byte[] buffer, int offset, int length) {
try {
out.write(buffer, offset, length);
} catch (IOException e) {
hasErrors = true;
}
}
@Override public void close() {
try {
out.close();
} catch (IOException e) {
hasErrors = true;
}
}
@Override public void flush() {
try {
out.flush();
} catch (IOException e) {
hasErrors = true;
}
}
}
}
入隊時,commit()函式,呼叫了completeEdit(Editoreditor, boolean success)函式。該函式將臨時檔案更名為正式檔案,並調整緩衝區大小。
private synchronized void completeEdit(Editor editor, boolean success) throws IOException {
Entry entry = editor.entry;
if (entry.currentEditor != editor) {
throw new IllegalStateException();
}
// if this edit is creating the entry for the first time, every index must have a value
if (success && !entry.readable) {
for (int i = 0; i < valueCount; i++) {
if (!entry.getDirtyFile(i).exists()) {
editor.abort();
throw new IllegalStateException("edit didn't create file " + i);
}
}
}
for (int i = 0; i < valueCount; i++) {
File dirty = entry.getDirtyFile(i);
if (success) {
if (dirty.exists()) {
File clean = entry.getCleanFile(i);
dirty.renameTo(clean);
long oldLength = entry.lengths[i];
long newLength = clean.length();
entry.lengths[i] = newLength;
size = size - oldLength + newLength;
}
} else {
deleteIfExists(dirty);
}
}
redundantOpCount++;
entry.currentEditor = null;
if (entry.readable | success) {
entry.readable = true;
journalWriter.write(CLEAN + ' ' + entry.key + entry.getLengths() + '\n');
if (success) {
entry.sequenceNumber = nextSequenceNumber++;
}
} else {
lruEntries.remove(entry.key);
journalWriter.write(REMOVE + ' ' + entry.key + '\n');
}
if (size > maxSize || journalRebuildRequired()) {
executorService.submit(cleanupCallable);
}
}
調整緩衝區大小是通過仔細執行緒池中一個執行緒來呼叫trimToSize()函式的。
private void trimToSize() throws IOException {
while (size > maxSize) {
// Map.Entry<String, Entry> toEvict = lruEntries.eldest();
final Map.Entry<String, Entry> toEvict = lruEntries.entrySet().iterator().next();
remove(toEvict.getKey());
}
}
2.3 日誌檔案
Cache目錄下有一個檔名稱是journal,前5行是固定的,已經標註在程式碼中: * libcore.io.DiskLruCache //固定字串
* 1 //快取的版本號
* 100 //儲存的key-value個數
* 2 //應用的版本號
* //空行
* CLEAN 3400330d1dfc7f3f7f4b8d4d803dfcf6 832 21054
* DIRTY 335c4c6028171cfddfbaae1a9c313c52
//有效建立或更新的Key-Value的Key,更新包括刪除
* CLEAN 335c4c6028171cfddfbaae1a9c313c52 3934 2342
* REMOVE 335c4c6028171cfddfbaae1a9c313c52 //最新被刪除的Key-Value的Key
* DIRTY 1ab96a171faeeee38496d8b330771a7a
* CLEAN 1ab96a171faeeee38496d8b330771a7a 1600 234
* READ 335c4c6028171cfddfbaae1a9c313c52 //最不常使用的Key-Value的Key
* READ 3400330d1dfc7f3f7f4b8d4d803dfcf
日誌前5行的讀取是通過下面函式進行的。
private void readJournal() throws IOException {
InputStream in = new BufferedInputStream(new FileInputStream(journalFile), IO_BUFFER_SIZE);
try {
String magic = readAsciiLine(in);
String version = readAsciiLine(in);
String appVersionString = readAsciiLine(in);
String valueCountString = readAsciiLine(in);
String blank = readAsciiLine(in);
if (!MAGIC.equals(magic)
|| !VERSION_1.equals(version)
|| !Integer.toString(appVersion).equals(appVersionString)
|| !Integer.toString(valueCount).equals(valueCountString)
|| !"".equals(blank)) {
throw new IOException("unexpected journal header: ["
+ magic + ", " + version + ", " + valueCountString + ", " + blank + "]");
}
while (true) {
try {
readJournalLine(readAsciiLine(in));//讀取剩餘日誌
} catch (EOFException endOfJournal) {
break;
}
}
} finally {
closeQuietly(in);
}
}
剩餘日誌是什麼含義和作用呢?回到剛才的completeEdit(Editoreditor, boolean success)函式,檢視下面的程式碼段可以得知,當正確提交時,寫入了CLEAN,讀取出錯時寫入的是REMOVE:
private synchronized void completeEdit(Editor editor, boolean success) throws IOException {
……省略無關程式碼……
if (entry.readable | success) {
entry.readable = true;
journalWriter.write(CLEAN + ' ' + entry.key + entry.getLengths() + '\n');
if (success) {
entry.sequenceNumber = nextSequenceNumber++;
}
} else {
lruEntries.remove(entry.key);
journalWriter.write(REMOVE + ' ' + entry.key + '\n');
}
……省略無關程式碼……
}
2.4 Key-Value出隊
根據Key返回的是DisKLruCache內部快照類Snapshot的物件,Snapshot可以返回Key對應的Entry的輸入流和Editor物件,不存在就返回null。有了輸入流,就可以恢復出來Value。
/**
* A snapshot of the values for an entry.
*/
public final class Snapshot implements Closeable {
private final String key;
private final long sequenceNumber;
private final InputStream[] ins;
private Snapshot(String key, long sequenceNumber, InputStream[] ins) {
this.key = key;
this.sequenceNumber = sequenceNumber;
this.ins = ins;
}
/**
* Returns an editor for this snapshot's entry, or null if either the
* entry has changed since this snapshot was created or if another edit
* is in progress.
*/
public Editor edit() throws IOException {
return DiskLruCache.this.edit(key, sequenceNumber);
}
/**
* Returns the unbuffered stream with the value for {@code index}.
*/
public InputStream getInputStream(int index) {
return ins[index];
}
/**
* Returns the string value for {@code index}.
*/
public String getString(int index) throws IOException {
return inputStreamToString(getInputStream(index));
}
@Override public void close() {
for (InputStream in : ins) {
closeQuietly(in);
}
}
}
3. 使用
DisKLruCache可用於大檔案物件的硬碟快取,典型應用是BitMap。這裡以BitMap為例走一下DisKLruCache的構造和存取過程。抽取的程式碼位於Android6.0 SDK中的DisplayingBitmaps工程(路徑為:sdk\samples\android-23\ui\)。
3.1 DisKLruCache的構造
構造過程其實就1句話,這裡加入了同步鎖,以及一些必要條件。
private static final int DISK_CACHE_INDEX = 0;
private final Object mDiskCacheLock = new Object(); //用於同步硬碟快取的同步鎖
private boolean mDiskCacheStarting = true;
private DiskLruCache mDiskLruCache;
/**
* Initializes the disk cache. Note that this includes disk access so this should not be
* executed on the main/UI thread. By default an ImageCache does not initialize the disk
* cache when it is created, instead you should call initDiskCache() to initialize it on a
* background thread.
*/
public void initDiskCache() {
// Set up disk cache
synchronized (mDiskCacheLock) {
if (mDiskLruCache == null || mDiskLruCache.isClosed()) {
File diskCacheDir = mCacheParams.diskCacheDir; //檔案路徑
if (mCacheParams.diskCacheEnabled && diskCacheDir != null) {
if (!diskCacheDir.exists()) {
diskCacheDir.mkdirs();
}
//可用空間大於硬碟快取大小
if (getUsableSpace(diskCacheDir) > mCacheParams.diskCacheSize) {
try {
mDiskLruCache = DiskLruCache.open(
diskCacheDir, 1, 1, mCacheParams.diskCacheSize);//構造
} catch (final IOException e) {
mCacheParams.diskCacheDir = null;
Log.e(TAG, "initDiskCache - " + e);
}
}
}
}
mDiskCacheStarting = false;
mDiskCacheLock.notifyAll();
}
}
3.2 Key-Value入隊
入隊的過程就是通過Key獲得快照,通過快照檢視是否已經儲存過。如果沒有儲存過,再獲得Editor,從Editor獲得Key對應檔案的輸出流,最後將BitMap寫入輸出流。
/**
* Adds a bitmap to both memory and disk cache.
* @param data Unique identifier for the bitmap to store
* @param value The bitmap drawable to store
*/
public void addBitmapToCache(String data, BitmapDrawable value) {
synchronized (mDiskCacheLock) {//同步鎖
// Add to disk cache
if (mDiskLruCache != null) {
final String key = hashKeyForDisk(data);
OutputStream out = null;
try {
DiskLruCache.Snapshot snapshot = mDiskLruCache.get(key);//快照
if (snapshot == null) {//表明還沒有儲存該key
final DiskLruCache.Editor editor = mDiskLruCache.edit(key);
if (editor != null) {
out = editor.newOutputStream(DISK_CACHE_INDEX);//輸入流
value.getBitmap().compress(
mCacheParams.compressFormat, mCacheParams.compressQuality, out);//bitmap寫入輸入流,即寫入到了臨時檔案
editor.commit();//臨時檔案寫入正式檔案
out.close();
}
} else {//表明儲存過該key
snapshot.getInputStream(DISK_CACHE_INDEX).close();
}
} catch (final IOException e) {
Log.e(TAG, "addBitmapToCache - " + e);
} catch (Exception e) {
Log.e(TAG, "addBitmapToCache - " + e);
} finally {
try {
if (out != null) {
out.close();
}
} catch (IOException e) {}
}
}
}
}
3.3 Key-Value出隊
出隊過程相對簡單,就是獲得快照,從快照獲得輸入流,從輸入流恢復出Value。
/**
* Get from disk cache.
*
* @param data Unique identifier for which item to get
* @return The bitmap if found in cache, null otherwise
*/
public Bitmap getBitmapFromDiskCache(String data) {
final String key = hashKeyForDisk(data);
Bitmap bitmap = null;
synchronized (mDiskCacheLock) {
while (mDiskCacheStarting) {
try {
mDiskCacheLock.wait();
} catch (InterruptedException e) {}
}
if (mDiskLruCache != null) {
InputStream inputStream = null;
try {
final DiskLruCache.Snapshot snapshot = mDiskLruCache.get(key);
if (snapshot != null) {
if (BuildConfig.DEBUG) {
Log.d(TAG, "Disk cache hit");
}
inputStream = snapshot.getInputStream(DISK_CACHE_INDEX);
if (inputStream != null) {
FileDescriptor fd = ((FileInputStream) inputStream).getFD();
// Decode bitmap, but we don't want to sample so give
// MAX_VALUE as the target dimensions
bitmap = ImageResizer.decodeSampledBitmapFromDescriptor(
fd, Integer.MAX_VALUE, Integer.MAX_VALUE, this);
}
}
} catch (final IOException e) {
Log.e(TAG, "getBitmapFromDiskCache - " + e);
} finally {
try {
if (inputStream != null) {
inputStream.close();
}
} catch (IOException e) {}
}
}
return bitmap;
}
4. 總結
DisKLruCache相對LruCache來講,有多個內部類,所以較LruCache複雜。就使用來講,也需要考慮同步操作,更增添了複雜性。