android 使用okhttp可能引發OOM的一個點
遇到一個問題: 需要給所有的請求加簽名校驗以防刷介面;傳入請求url及body生成一個文字串作為一個header傳給服務端;已經有現成的簽名檢驗方法String doSignature(String url, byte[] body);當前網路庫基於com.squareup.okhttp3:okhttp:3.14.2.
這很簡單了,當然是寫一個interceptor然後將request物件的url及body傳入就好.於是有:
public class SignInterceptor implements Interceptor { @NonNull @Override public Response intercept(@NonNull Chain chain) throws IOException { Request request = chain.request(); RequestBody body = request.body(); byte[] bodyBytes = null; if (body != null) { final Buffer buffer = new Buffer(); body.writeTo(buffer); bodyBytes = buffer.readByteArray(); } Request.Builder builder = request.newBuilder(); HttpUrl oldUrl = request.url(); final String url = oldUrl.toString(); final String signed = doSignature(url, bodyBytes)); if (!TextUtils.isEmpty(signed)) { builder.addHeader(SIGN_KEY_NAME, signed); } return chain.proceed(builder.build()); } }
okhttp的ReqeustBody是一個抽象類,內容輸出只有writeTo方法,將內容寫入到一個BufferedSink介面實現體裡,然後再將資料轉成byte[]也就是記憶體陣列.能達到目的的類只有Buffer,它實現了BufferedSink介面並能提供轉成記憶體陣列的方法readByteArray. 這貌似沒啥問題呀,能造成OOM?
是的,要看請求型別,如果是一個上傳檔案的介面呢?如果這個檔案比較大呢?上傳介面有可能會用到public static RequestBody create(final @Nullable MediaType contentType, final File file)方法,如果是針對檔案的實現體它的writeTo方法是sink.writeAll(source);而我們傳給簽名方法時用到的Buffer.readByteArray是將緩衝中的所有內容轉成了記憶體陣列, 這意味著檔案中的所有內容被轉成了記憶體陣列, 就是在這個時機容易造成OOM! RequestBody.create原始碼如下:
public static RequestBody create(final @Nullable MediaType contentType, final File file) { if (file == null) throw new NullPointerException("file == null"); return new RequestBody() { @Override public @Nullable MediaType contentType() { return contentType; } @Override public long contentLength() { return file.length(); } @Override public void writeTo(BufferedSink sink) throws IOException { try (Source source = Okio.source(file)) { sink.writeAll(source); } } }; }
可以看到實現體持有了檔案,Content-Length返回了檔案的大小, 內容全部轉給了Source物件。
這確實是以前非常容易忽略的一個點,很少有對請求體作額外處理的操作,而一旦這個操作變成一次性的大記憶體分配, 非常容易造成OOM. 所以要如何解決呢? 簽名方法又是如何處理的呢? 原來這個簽名方法在這裡偷了個懶——它只讀取傳入body的前4K內容,然後只針對這部分內容進行了加密,至於傳入的這個記憶體陣列本身多大並不考慮,完全把風險和麻煩丟給了外部(優秀的SDK!).
快速的方法當然是羅列白名單,針對上傳介面服務端不進行加簽驗證, 但這容易掛一漏萬,而且增加維護成本, 要簽名方法sdk的人另寫合適的介面等於要他們的命, 所以還是得從根本解決. 既然簽名方法只讀取前4K內容,我們便只將內容的前4K部分讀取再轉成方法所需的記憶體陣列不就可了? 所以我們的目的是: 期望RequestBody能夠讀取一部分而不是全部的內容. 能否繼承RequestBody重寫它的writeTo? 可以,但不現實,不可能全部替代現有的RequestBody實現類, 同時ok框架也有可能建立私有的實現類. 所以只能針對writeTo的引數BufferedSink作文章, 先得了解BufferedSink又是如何被okhttp框架呼叫的.
BufferedSink相關的類包括Buffer, Source,都屬於okio框架,okhttp只是基於okio的一坨, okio沒有直接用java的io操作,而是另行寫了一套io操作,具體是資料緩衝的操作.接上面的描述, Source是怎麼建立, 同時又是如何操作BufferedSink的? 在Okio.java中:
public static Source source(File file) throws FileNotFoundException { if (file == null) throw new IllegalArgumentException("file == null"); return source(new FileInputStream(file)); } public static Source source(InputStream in) { return source(in, new Timeout()); } private static Source source(final InputStream in, final Timeout timeout) { return new Source() { @Override public long read(Buffer sink, long byteCount) throws IOException { try { timeout.throwIfReached(); Segment tail = sink.writableSegment(1); int maxToCopy = (int) Math.min(byteCount, Segment.SIZE - tail.limit); int bytesRead = in.read(tail.data, tail.limit, maxToCopy); if (bytesRead == -1) return -1; tail.limit += bytesRead; sink.size += bytesRead; return bytesRead; } catch (AssertionError e) { if (isAndroidGetsocknameError(e)) throw new IOException(e); throw e; } } @Override public void close() throws IOException { in.close(); } @Override public Timeout timeout() { return timeout; } }; }
Source把檔案作為輸入流inputstream進行了各種讀操作, 但是它的read方法引數卻是個Buffer例項,它又是從哪來的,又怎麼和BufferedSink關聯的? 只好再繼續看BufferedSink.writeAll的實現體。
BufferedSink的實現類就是Buffer, 然後它的writeAll方法:
@Override public long writeAll(Source source) throws IOException { if (source == null) throw new IllegalArgumentException("source == null"); long totalBytesRead = 0; for (long readCount; (readCount = source.read(this, Segment.SIZE)) != -1; ) { totalBytesRead += readCount; } return totalBytesRead; }
原來是顯式的呼叫了Source.read(Buffer,long)方法,這樣就串起來了,那個Buffer引數原來就是自身。
基本可以確定只要實現BufferedSink介面類, 然後判斷讀入的內容超過指定大小就停止寫入就返回就可滿足目的, 可以名之FixedSizeSink.
然而麻煩的是BufferedSink的介面非常多, 將近30個方法, 不知道框架會在什麼時機呼叫哪個方法,只能全部都實現! 其次是介面方法的引數有很多okio的類, 這些類的用法需要了解, 否則一旦用錯了效果適得其反. 於是對一個類的瞭解變成對多個類的瞭解, 沒辦法只能硬著頭皮寫.
第一個介面就有點蛋疼: Buffer buffer(); BufferedSink返回一個Buffer例項供外部呼叫, BufferedSink的實現體即是Buffer, 然後再返回一個Buffer?! 看了半天猜測BufferedSink是為了提供一個可寫入的緩衝物件, 但框架作者也懶的再搞介面解耦的那一套了(唉,大家都是怎麼簡單怎麼來). 於是FixedSizeSink至少需要持有一個Buffer物件, 它作實際的資料快取,同時可以在需要Source.read(Buffer ,long)的地方作為引數傳過去.
同時可以看到RequestBody的一個實現類FormBody, 用這個Buffer物件直接寫入一些資料:
private long writeOrCountBytes(@Nullable BufferedSink sink, boolean countBytes) { long byteCount = 0L; Buffer buffer; if (countBytes) { buffer = new Buffer(); } else { buffer = sink.buffer(); } for (int i = 0, size = encodedNames.size(); i < size; i++) { if (i > 0) buffer.writeByte('&'); buffer.writeUtf8(encodedNames.get(i)); buffer.writeByte('='); buffer.writeUtf8(encodedValues.get(i)); } if (countBytes) { byteCount = buffer.size(); buffer.clear(); } return byteCount; }
有這樣的操作就有可能限制不了緩衝區大小變化!不過資料量應該相對小一些而且這種用法場景相對少,我們指定的大小應該能覆蓋的了這種情況。
接著還有一個介面BufferedSink write(ByteString byteString), 又得了解ByteString怎麼使用, 真是心力交瘁啊...
@Override public Buffer write(ByteString byteString) { byteString.write(this); return this; }
Buffer實現體裡可以直接呼叫ByteString.write(Buffer)因為是包名訪問,自己實現的FixedSizeSink宣告在和同一包名package okio;也可以這樣使用,如果是其它包名只能先轉成byte[]了, ByteString應該不大不然也不能這麼搞(沒有找到ByteString讀取一段資料的方法):
@Override public BufferedSink write(@NotNull ByteString byteString) throws IOException { byte[] bytes = byteString.toByteArray(); this.write(bytes); return this; }
總之就是把這些物件轉成記憶體陣列或者Buffer能夠接受的引數持有起來!
重點關心的writeAll反而相對好實現一點, 我們連續讀取指定長度的內容直到內容長度達到我們的閾值就行.
還有一個蛋疼的點是各種物件的read/write資料流方向:
Caller.read(Callee)/Caller.write(Callee),
有的是從Caller到Callee, 有的是相反,被一個小類整的有點頭疼……
最後上完整程式碼, 如果發現什麼潛在的問題也可以交流下~:
public class FixedSizeSink implements BufferedSink { private static final int SEGMENT_SIZE = 4096; private final Buffer mBuffer = new Buffer(); private final int mLimitSize; private FixedSizeSink(int size) { this.mLimitSize = size; } @Override public Buffer buffer() { return mBuffer; } @Override public BufferedSink write(@NotNull ByteString byteString) throws IOException { byte[] bytes = byteString.toByteArray(); this.write(bytes); return this; } @Override public BufferedSink write(@NotNull byte[] source) throws IOException { this.write(source, 0, source.length); return this; } @Override public BufferedSink write(@NotNull byte[] source, int offset, int byteCount) throws IOException { long available = mLimitSize - mBuffer.size(); int count = Math.min(byteCount, (int) available); android.util.Log.d(TAG, String.format("FixedSizeSink.offset=%d," "count=%d,limit=%d,size=%d", offset, byteCount, mLimitSize, mBuffer.size())); if (count > 0) { mBuffer.write(source, offset, count); } return this; } @Override public long writeAll(@NotNull Source source) throws IOException { this.write(source, mLimitSize); return mBuffer.size(); } @Override public BufferedSink write(@NotNull Source source, long byteCount) throws IOException { final long count = Math.min(byteCount, mLimitSize - mBuffer.size()); final long BUFFER_SIZE = Math.min(count, SEGMENT_SIZE); android.util.Log.d(TAG, String.format("FixedSizeSink.count=%d,limit=%d" ",size=%d,segment=%d", byteCount, mLimitSize, mBuffer.size(), BUFFER_SIZE)); long totalBytesRead = 0; long readCount; while (totalBytesRead < count && (readCount = source.read(mBuffer, BUFFER_SIZE)) != -1) { totalBytesRead = readCount; } return this; } @Override public int write(ByteBuffer src) throws IOException { final int available = mLimitSize - (int) mBuffer.size(); if (available < src.remaining()) { byte[] bytes = new byte[available]; src.get(bytes); this.write(bytes); return bytes.length; } else { return mBuffer.write(src); } } @Override public void write(@NotNull Buffer source, long byteCount) throws IOException { mBuffer.write(source, Math.min(byteCount, mLimitSize - mBuffer.size())); } @Override public BufferedSink writeUtf8(@NotNull String string) throws IOException { mBuffer.writeUtf8(string); return this; } @Override public BufferedSink writeUtf8(@NotNull String string, int beginIndex, int endIndex) throws IOException { mBuffer.writeUtf8(string, beginIndex, endIndex); return this; } @Override public BufferedSink writeUtf8CodePoint(int codePoint) throws IOException { mBuffer.writeUtf8CodePoint(codePoint); return this; } @Override public BufferedSink writeString(@NotNull String string, @NotNull Charset charset) throws IOException { mBuffer.writeString(string, charset); return this; } @Override public BufferedSink writeString(@NotNull String string, int beginIndex, int endIndex, @NotNull Charset charset) throws IOException { mBuffer.writeString(string, beginIndex, endIndex, charset); return this; } @Override public BufferedSink writeByte(int b) throws IOException { mBuffer.writeByte(b); return this; } @Override public BufferedSink writeShort(int s) throws IOException { mBuffer.writeShort(s); return this; } @Override public BufferedSink writeShortLe(int s) throws IOException { mBuffer.writeShortLe(s); return this; } @Override public BufferedSink writeInt(int i) throws IOException { mBuffer.writeInt(i); return this; } @Override public BufferedSink writeIntLe(int i) throws IOException { mBuffer.writeIntLe(i); return this; } @Override public BufferedSink writeLong(long v) throws IOException { mBuffer.writeLong(v); return this; } @Override public BufferedSink writeLongLe(long v) throws IOException { mBuffer.writeLongLe(v); return this; } @Override public BufferedSink writeDecimalLong(long v) throws IOException { mBuffer.writeDecimalLong(v); return this; } @Override public BufferedSink writeHexadecimalUnsignedLong(long v) throws IOException { mBuffer.writeHexadecimalUnsignedLong(v); return this; } @Override public void flush() throws IOException { mBuffer.flush(); } @Override public BufferedSink emit() throws IOException { mBuffer.emit(); return this; } @Override public BufferedSink emitCompleteSegments() throws IOException { mBuffer.emitCompleteSegments(); return this; } @Override public OutputStream outputStream() { return mBuffer.outputStream(); } @Override public boolean isOpen() { return mBuffer.isOpen(); } @Override public Timeout timeout() { return mBuffer.timeout(); } @Override public void close() throws IOException { mBuffer.close(); } }
以上就是本文的全部內容,希望對大家的學習有所幫助,也希望大家多多支援碼農教程。