Design and implement a data structure for Least Recently Used (LRU) cache. It should support the following operations: get and set.

get(key) – Get the value (will always be positive) of the key if the key exists in the cache, otherwise return -1.
set(key, value) – Set or insert the value if the key is not already present. When the cache reached its capacity, it should invalidate the least recently used item before inserting a new item.

下面列出幾種LRU Cache 的思路：

1 如果在面試題裡考到如何實現LRU，考官一般會要求使用雙鏈表 + hashtable 的方式。雙鏈表 + hashtable實現原理：

LRU的思想是基於“最近用到的資料被重用的概率比較早用到的大的多”這個設計規則來實現的。Cache中的所有塊位置都用雙向連結串列連結起來，當一個位置被命中後，就將通過調整連結串列的指向將該位置調整到連結串列的頭位置，新加入的內容直接放在連結串列的頭上。這樣，在進行過多次查詢操作後，最近被命中過的內容就向連結串列的頭移動，而沒有被命中的內容就向連結串列的後面移動。當需要替換時，連結串列最後的位置就是最近最少被命中的位置，我們只需要將新的內容放在連結串列前面，淘汰連結串列最後的位置就是想了。

將Cache的所有位置都用雙連表連線起來，當一個位置被命中之後，就將通過調整連結串列的指向，將該位置調整到連結串列頭的位置，新加入的Cache直接加到連結串列頭中。這樣，在多次進行Cache操作後，最近被命中的，就會被向連結串列頭方向移動，而沒有命中的，而想連結串列後面移動，連結串列尾則表示最近最少使用的Cache。當需要替換內容時候，連結串列的最後位置就是最少被命中的位置，我們只需要淘汰連結串列最後的部分即可。（轉自部落格 http://gogole.iteye.com/blog/692103 ) 

2.  直接使用SDK LinkedHashMap ， 一種是委託（delegate）方式，一種是繼承(inheritance)方式，兩種都是override了removeEldestEntry方法。

import java.util.LinkedHashMap;
import java.util.Collection;
import java.util.Map;
import java.util.ArrayList;

/**
* An LRU cache, based on <code>LinkedHashMap</code>.
*
* <p>
* This cache has a fixed maximum number of elements (<code>cacheSize</code>).
* If the cache is full and another entry is added, the LRU (least recently used) entry is dropped.
*
* <p>
* This class is thread-safe. All methods of this class are synchronized.
*
* <p>
* Author: Christian d'Heureuse, Inventec Informatik AG, Zurich, Switzerland<br>
* Multi-licensed: EPL / LGPL / GPL / AL / BSD.
*/
public class LRUCache<K,V> {

private static final float   hashTableLoadFactor = 0.75f;

private LinkedHashMap<K,V>   map;
private int                  cacheSize;

/**
* Creates a new LRU cache.
* @param cacheSize the maximum number of entries that will be kept in this cache.
*/
public LRUCache (int cacheSize) {
   this.cacheSize = cacheSize;
   int hashTableCapacity = (int)Math.ceil(cacheSize / hashTableLoadFactor) + 1;
   map = new LinkedHashMap<K,V>(hashTableCapacity, hashTableLoadFactor, true) {
      // (an anonymous inner class)
      private static final long serialVersionUID = 1;
      @Override protected boolean removeEldestEntry (Map.Entry<K,V> eldest) {
         return size() > LRUCache.this.cacheSize; }}; }

/**
* Retrieves an entry from the cache.<br>
* The retrieved entry becomes the MRU (most recently used) entry.
* @param key the key whose associated value is to be returned.
* @return    the value associated to this key, or null if no value with this key exists in the cache.
*/
public synchronized V get (K key) {
   return map.get(key); }

/**
* Adds an entry to this cache.
* The new entry becomes the MRU (most recently used) entry.
* If an entry with the specified key already exists in the cache, it is replaced by the new entry.
* If the cache is full, the LRU (least recently used) entry is removed from the cache.
* @param key    the key with which the specified value is to be associated.
* @param value  a value to be associated with the specified key.
*/
public synchronized void put (K key, V value) {
   map.put (key, value); }

/**
* Clears the cache.
*/
public synchronized void clear() {
   map.clear(); }

/**
* Returns the number of used entries in the cache.
* @return the number of entries currently in the cache.
*/
public synchronized int usedEntries() {
   return map.size(); }

/**
* Returns a <code>Collection</code> that contains a copy of all cache entries.
* @return a <code>Collection</code> with a copy of the cache content.
*/
public synchronized Collection<Map.Entry<K,V>> getAll() {
   return new ArrayList<Map.Entry<K,V>>(map.entrySet()); }

} // end class LRUCache

// Test routine for the LRUCache class.
public static void main (String[] args) {
   LRUCache<String,String> c = new LRUCache<String, String>(3);
   c.put ("1", "one");                           // 1
   c.put ("2", "two");                           // 2 1
   c.put ("3", "three");                         // 3 2 1
   c.put ("4", "four");                          // 4 3 2
   if (c.get("2") == null) throw new Error();    // 2 4 3
   c.put ("5", "five");                          // 5 2 4
   c.put ("4", "second four");                   // 4 5 2
   // Verify cache content.
   if (c.usedEntries() != 3)              throw new Error();
   if (!c.get("4").equals("second four")) throw new Error();
   if (!c.get("5").equals("five"))        throw new Error();
   if (!c.get("2").equals("two"))         throw new Error();
   // List cache content.
   for (Map.Entry<String, String> e : c.getAll())
      System.out.println (e.getKey() + " : " + e.getValue()); }

A cache is a mechanism by which future requests for that data are served faster and/or at a lower cost. This article describes a data structure to hold the cache data and an implementation in Java to service the  cache requests.

Requirements

1. Fixed size: The cache needs to have some bounds to limit memory usage.
2. Fast access: The cache insert and lookup operations need to be fast preferably O(1) time.
3. Entry replacement algorithm: When the cache is full, the less useful cache entries are purged from cache. The algorithm to replace these entries is Least Recently Used (LRU) - or the cache entries which have not been accessed recently will be replaced.

Design discussion

Since the lookup and insert operationed need to fast a HashMap would be a good candidate. The HashMap accepts an initial capacity parameter but it re-sizes itself if more entries are inserted. So we need to override the put() operation and remove (or purge) an entry before inserting.

How do we select the entry to be purged? One approach is to maintain  a timestamp at which the entry was inserted and select the entry with the oldest timestamp. But this search would be linear taking O(N) time.

So we need the entries to be maintained in a sorted list based on the order in which the entries were accessed. An alternate way to achieve this would be to maintain the entries in a doubly linked list using which everytime an entry is accessed ( a cache lookup operation), the entry is also moved to the end of the list. When we need to purge the entries it is done from the top of the list. In an ArrayList when an element is removed the rest of the entries need to be moved by one to fill the gap. A doubly linked list does not have this issue.

We have come up with a design that meets our requirements and guarantees O(1) insert and O(1) lookup operations and also has a configurable limit on the number of entries. Let's begin the implementation.

Lucky for us, JDK already provides a class that is very suitable for our purpose - LinkedHashMap. This class maintains the entries in a HashMap for fast lookup at the same time maintains a doubly linked list of the entries either inAccessOrder or InsertionOrder. This is configurable so use use AccessOrder as true. It also has a methodremoveOldestEntry() which we can override to return true when the cache size exceeds the specified capacity(upper limit). So here is the implementation. Enjoy.

import java.util.LinkedHashMap;
import java.util.Map.Entry;

public class LRUCache < K, V > extends LinkedHashMap < K, V > {

	private int capacity; // Maximum number of items in the cache.
	
	public LRUCache(int capacity) { 
		super(capacity+1, 1.0f, true); // Pass 'true' for accessOrder.
		this.capacity = capacity;
	}
	
	protected boolean removeEldestEntry(Entry entry) {
		return (size() > this.capacity);
	} 
}

3.  在Android的原始碼裡有一個完整的工業級實現，考慮了執行緒安全，值得學習。

Cache miss 有三類， 簡單來說:
Compulsory misses : 首次讀寫時,造成的miss
Capacity misses : 此時cache已滿,即超出了cache本身的能力,這時如果要讀取記憶體資料,而此資料還沒有移到cache裡面,就會造成cache miss,這是比較常見的一種.
Conflict misses : 這是一種可以避免的cache miss,主要由於我們的cache替換策略不當造成的.

下面的實現是Capacity Miss： 

import java.util.LinkedHashMap;
import java.util.Map;

/**
 * A cache that holds strong references to a limited number of values. Each time
 * a value is accessed, it is moved to the head of a queue. When a value is
 * added to a full cache, the value at the end of that queue is evicted and may
 * become eligible for garbage collection.
 *
 * <p>If your cached values hold resources that need to be explicitly released,
 * override {@link #entryRemoved}.
 *
 * <p>If a cache miss should be computed on demand for the corresponding keys,
 * override {@link #create}. This simplifies the calling code, allowing it to
 * assume a value will always be returned, even when there's a cache miss.
 *
 * <p>By default, the cache size is measured in the number of entries. Override
 * {@link #sizeOf} to size the cache in different units. For example, this cache
 * is limited to 4MiB of bitmaps:
 * <pre>   {@code
 *   int cacheSize = 4 * 1024 * 1024; // 4MiB
 *   LruCache<String, Bitmap> bitmapCache = new LruCache<String, Bitmap>(cacheSize) {
 *       protected int sizeOf(String key, Bitmap value) {
 *           return value.getByteCount();
 *       }
 *   }}</pre>
 *
 * <p>This class is thread-safe. Perform multiple cache operations atomically by
 * synchronizing on the cache: <pre>   {@code
 *   synchronized (cache) {
 *     if (cache.get(key) == null) {
 *         cache.put(key, value);
 *     }
 *   }}</pre>
 *
 * <p>This class does not allow null to be used as a key or value. A return
 * value of null from {@link #get}, {@link #put} or {@link #remove} is
 * unambiguous: the key was not in the cache.
 */
public class LruCache<K, V> {
    private final LinkedHashMap<K, V> map;

    /** Size of this cache in units. Not necessarily the number of elements. */
    private int size;
    private int maxSize;

    private int putCount;
    private int createCount;
    private int evictionCount;
    private int hitCount;
    private int missCount;

    /**
     * @param maxSize for caches that do not override {@link #sizeOf}, this is
     *     the maximum number of entries in the cache. For all other caches,
     *     this is the maximum sum of the sizes of the entries in this cache.
     */
    public LruCache(int maxSize) {
        if (maxSize <= 0) {
            throw new IllegalArgumentException("maxSize <= 0");
        }
        this.maxSize = maxSize;
        this.map = new LinkedHashMap<K, V>(0, 0.75f, true);
    }

    /**
     * Returns the value for {@code key} if it exists in the cache or can be
     * created by {@code #create}. If a value was returned, it is moved to the
     * head of the queue. This returns null if a value is not cached and cannot
     * be created.
     */
    public final V get(K key) {
        if (key == null) {
            throw new NullPointerException("key == null");
        }

        V mapValue;
        synchronized (this) {
            mapValue = map.get(key);
            if (mapValue != null) {
                hitCount++;
                return mapValue;
            }
            missCount++;
        }

        /*
         * Attempt to create a value. This may take a long time, and the map
         * may be different when create() returns. If a conflicting value was
         * added to the map while create() was working, we leave that value in
         * the map and release the created value.
         */

        V createdValue = create(key);
        if (createdValue == null) {
            return null;
        }

        synchronized (this) {
            createCount++;
            mapValue = map.put(key, createdValue);

            if (mapValue != null) {
                // There was a conflict so undo that last put
                map.put(key, mapValue);
            } else {
                size += safeSizeOf(key, createdValue);
            }
        }

        if (mapValue != null) {
            entryRemoved(false, key, createdValue, mapValue);
            return mapValue;
        } else {
            trimToSize(maxSize);
            return createdValue;
        }
    }

    /**
     * Caches {@code value} for {@code key}. The value is moved to the head of
     * the queue.
     *
     * @return the previous value mapped by {@code key}.
     */
    public final V put(K key, V value) {
        if (key == null || value == null) {
            throw new NullPointerException("key == null || value == null");
        }

        V previous;
        synchronized (this) {
            putCount++;
            size += safeSizeOf(key, value);
            previous = map.put(key, value);
            if (previous != null) {
                size -= safeSizeOf(key, previous);
            }
        }

        if (previous != null) {
            entryRemoved(false, key, previous, value);
        }

        trimToSize(maxSize);
        return previous;
    }

    /**
     * @param maxSize the maximum size of the cache before returning. May be -1
     *     to evict even 0-sized elements.
     */
    private void trimToSize(int maxSize) {
        while (true) {
            K key;
            V value;
            synchronized (this) {
                if (size < 0 || (map.isEmpty() && size != 0)) {
                    throw new IllegalStateException(getClass().getName()
                            + ".sizeOf() is reporting inconsistent results!");
                }

                if (size <= maxSize) {
                    break;
                }

                Map.Entry<K, V> toEvict = map.eldest();
                if (toEvict == null) {
                    break;
                }

                key = toEvict.getKey();
                value = toEvict.getValue();
                map.remove(key);
                size -= safeSizeOf(key, value);
                evictionCount++;
            }

            entryRemoved(true, key, value, null);
        }
    }

    /**
     * Removes the entry for {@code key} if it exists.
     *
     * @return the previous value mapped by {@code key}.
     */
    public final V remove(K key) {
        if (key == null) {
            throw new NullPointerException("key == null");
        }

        V previous;
        synchronized (this) {
            previous = map.remove(key);
            if (previous != null) {
                size -= safeSizeOf(key, previous);
            }
        }

        if (previous != null) {
            entryRemoved(false, key, previous, null);
        }

        return previous;
    }

    /**
     * Called for entries that have been evicted or removed. This method is
     * invoked when a value is evicted to make space, removed by a call to
     * {@link #remove}, or replaced by a call to {@link #put}. The default
     * implementation does nothing.
     *
     * <p>The method is called without synchronization: other threads may
     * access the cache while this method is executing.
     *
     * @param evicted true if the entry is being removed to make space, false
     *     if the removal was caused by a {@link #put} or {@link #remove}.
     * @param newValue the new value for {@code key}, if it exists. If non-null,
     *     this removal was caused by a {@link #put}. Otherwise it was caused by
     *     an eviction or a {@link #remove}.
     */
    protected void entryRemoved(boolean evicted, K key, V oldValue, V newValue) {}

    /**
     * Called after a cache miss to compute a value for the corresponding key.
     * Returns the computed value or null if no value can be computed. The
     * default implementation returns null.
     *
     * <p>The method is called without synchronization: other threads may
     * access the cache while this method is executing.
     *
     * <p>If a value for {@code key} exists in the cache when this method
     * returns, the created value will be released with {@link #entryRemoved}
     * and discarded. This can occur when multiple threads request the same key
     * at the same time (causing multiple values to be created), or when one
     * thread calls {@link #put} while another is creating a value for the same
     * key.
     */
    protected V create(K key) {
        return null;
    }

    private int safeSizeOf(K key, V value) {
        int result = sizeOf(key, value);
        if (result < 0) {
            throw new IllegalStateException("Negative size: " + key + "=" + value);
        }
        return result;
    }

    /**
     * Returns the size of the entry for {@code key} and {@code value} in
     * user-defined units.  The default implementation returns 1 so that size
     * is the number of entries and max size is the maximum number of entries.
     *
     * <p>An entry's size must not change while it is in the cache.
     */
    protected int sizeOf(K key, V value) {
        return 1;
    }

    /**
     * Clear the cache, calling {@link #entryRemoved} on each removed entry.
     */
    public final void evictAll() {
        trimToSize(-1); // -1 will evict 0-sized elements
    }

    /**
     * For caches that do not override {@link #sizeOf}, this returns the number
     * of entries in the cache. For all other caches, this returns the sum of
     * the sizes of the entries in this cache.
     */
    public synchronized final int size() {
        return size;
    }

    /**
     * For caches that do not override {@link #sizeOf}, this returns the maximum
     * number of entries in the cache. For all other caches, this returns the
     * maximum sum of the sizes of the entries in this cache.
     */
    public synchronized final int maxSize() {
        return maxSize;
    }

    /**
     * Returns the number of times {@link #get} returned a value that was
     * already present in the cache.
     */
    public synchronized final int hitCount() {
        return hitCount;
    }

    /**
     * Returns the number of times {@link #get} returned null or required a new
     * value to be created.
     */
    public synchronized final int missCount() {
        return missCount;
    }

    /**
     * Returns the number of times {@link #create(Object)} returned a value.
     */
    public synchronized final int createCount() {
        return createCount;
    }

    /**
     * Returns the number of times {@link #put} was called.
     */
    public synchronized final int putCount() {
        return putCount;
    }

    /**
     * Returns the number of values that have been evicted.
     */
    public synchronized final int evictionCount() {
        return evictionCount;
    }

    /**
     * Returns a copy of the current contents of the cache, ordered from least
     * recently accessed to most recently accessed.
     */
    public synchronized final Map<K, V> snapshot() {
        return new LinkedHashMap<K, V>(map);
    }

    @Override public synchronized final String toString() {
        int accesses = hitCount + missCount;
        int hitPercent = accesses != 0 ? (100 * hitCount / accesses) : 0;
        return String.format("LruCache[maxSize=%d,hits=%d,misses=%d,hitRate=%d%%]",
                maxSize, hitCount, missCount, hitPercent);
    }
}

補充材料：

在虛擬儲存器中常用的頁面替換演算法有如下幾種：
(1)隨機演算法，即RAND演算法（Random algorithm）。利用軟體或硬體的隨機數發生器來確定主儲存器中被替換的頁面。這種演算法最簡單，而且容易實現。但是，這種演算法完全沒有利用主儲存器中頁面排程情況的歷史資訊，也沒有反映程式的區域性性，所以命中率比較低。
(2)先進先出演算法，即FIFO演算法（First-In First-Out algorithm）。這種演算法選擇最先調入主儲存器的頁面作為被替換的頁面。它的優點是比較容易實現，能夠利用主儲存器中頁面排程情況的歷史資訊，但是，沒有反映程式的區域性性。因為最先調入主存的頁面，很可能也是經常要使用的頁面。
(3)近期最少使用演算法，即LFU演算法（Least Frequently Used algorithm）。這種演算法選擇近期最少訪問的頁面作為被替換的頁面。顯然，這是一種非常合理的演算法，因為到目前為止最少使用的頁面，很可能也是將來最少訪問的頁面。該演算法既 充分利用了主存中頁面排程情況的歷史資訊，又正確反映了程式的區域性性。但是，這種演算法實現起來非常困難，它要為每個頁面設定一個很長的計數器，並且要選擇 一個固定的時鐘為每個計數器定時計數。在選擇被替換頁面時，要從所有計數器中找出一個計數值最大的計數器。因此，通常採用如下一種相對比較簡單的方法。
(4)最久沒有使用演算法，即LRU演算法（Least Recently Used algorithm）。這種演算法把近期最久沒有被訪問過的頁面作為被替換的頁面。它把LFU演算法中要記錄數量上的"多"與"少"簡化成判斷"有"與"無"，因此，實現起來比較容易。
(5)最優替換演算法，即OPT演算法（OPTimal replacement algorithm）。上 面介紹的幾種頁面替換演算法主要是以主儲存器中頁面排程情況的歷史資訊為依據的，它假設將來主儲存器中的頁面排程情況與過去一段時間內主儲存器中的頁面排程 情況是相同的。顯然，這種假設不總是正確的。最好的演算法應該是選擇將來最久不被訪問的頁面作為被替換的頁面，這種替換演算法的命中率一定是最高的，它就是最 優替換演算法。
　　要實現OPT演算法，唯一的辦法是讓程式先執行一遍，記錄下實際的頁地址流情況。根據這個頁地址流才能找出當前要被替換的頁面。顯然，這樣做是不現實 的。因此，OPT演算法只是一種理想化的演算法，然而，它也是一種很有用的演算法。實際上，經常把這種演算法用來作為評價其它頁面替換演算法好壞的標準。在其它條件 相同的情況下，哪一種頁面替換演算法的命中率與OPT演算法最接近，那麼，它就是一種比較好的頁面替換演算法。