效能測試工具
個人認為一個優秀的軟體工程師必須掌握一定的測試技能。可能有的開發工程師認為測試是測試工程師的事情,不是開發工程師的事情。這種開發工程師是缺一條腿的,離開了測試工程師,你如何證明你的程式沒有問題。
下面是我這兩天蒐集的效能測試工具,如果大家知道別的工具,可以在回覆中分享出來。
Web Bench
Web Bench is very simple tool for benchmarking WWW or proxy servers. Uses fork() for simulating multiple clients and can use HTTP/0.9-HTTP/1.1 requests. This benchmark is not very realistic, but it can test if your HTTPD can realy handle that many clients at once (try to run some CGIs) without taking your machine down. Displays pages/min and bytes/sec. Can be used in more aggressive mode with -f switch.
Web Bench是一個簡單的web或者web代理服務的基準測試工具。(它)使用fork()模擬多個客戶端並且可以發起HTTP/0/9-HTTP/1.1的請求。這個基準測試不是非常符合實際情況,但是可以測試出你的HTTP服務是否可以處理多個客戶同時訪問而不會把你的機器弄掛掉。這個工具可以獲得服務的兩個指標:每分鐘響應請求數量(pages/min)和每秒鐘傳輸資料量(bytes/sec)。使用-f選項可以執行在主動模式下(不等待伺服器響應直接傳送下一個請求)。
安裝方法參考:(總結)Web效能壓力測試工具之WebBench詳解
命令引數:
使用webbench測試elong,taobao,baidu的結果(可能受電腦效能、網路的因素影響):
ab - Apache HTTP server benchmarking tool
ab is a tool for benchmarking your Apache Hypertext Transfer Protocol (HTTP) server. It is designed to give you an impression of how your current Apache installation performs. This especially shows you how many requests per second your Apache installation is capable of serving.
ab命令會建立很多的併發訪問執行緒,模擬多個訪問者同時對某一URL地址進行訪問。它的測試目標是基於URL的,因此,既可以用來測試Apache的負載壓力,也可以測試nginx、lighthttp、tomcat、IIS等其它Web伺服器的壓力。ab命令對發出負載的計算機要求很低,既不會佔用很高CPU,也不會佔用很多記憶體,但卻會給目標伺服器造成巨大的負載,其原理類似CC攻擊。自己測試使用也須注意,否則一次上太多的負載,可能造成目標伺服器因資源耗完,嚴重時甚至導致宕機。
網上有些文章說安裝apache2後一般情況下在Apache的bin目錄下。我在Ubunut 14.04上測試,並不是這樣。要在Ubuntu上使用ab,需要apache2-utils。
ab的用法參考:ab - Apache HTTP server benchmarking tool,(總結)Web效能壓力測試工具之ApacheBench(ab)詳解
Siege
Siege is an http load testing and benchmarking utility. It was designed to let web developers measure their code under duress, to see how it will stand up to load on the internet. Siege supports basic authentication, cookies, HTTP and HTTPS protocols. It lets its user hit a web server with a configurable number of simulated web browsers. Those browsers place the server “under siege.”
Siege(英文意思是圍攻)是一個壓力測試和評測工具,設計用於WEB開發這評估應用在壓力下的承受能力:可以根據配置對一個WEB站點進行多使用者的併發訪問,記錄每個使用者所有請求過程的相應時間,並在一定數量的併發訪問下重複進行。 Siege 支援基本的認證,cookies, HTTP 和 HTTPS 協議。
Siege的安裝和用法可以參考:Siege Manual,Using Siege to Tune Apache on GNU/Linux,(總結)Web效能壓力測試工具之Siege詳解,壓力測試工具siege的用法
我的Ubuntu 14.04上的使用如下:
測試本地的tomcat的測試結果:
Apache JMeter
Apache JMeter是一個專門為執行和伺服器裝載測試而設計的、100%的純Java桌面執行程式。原先它是為Web/HTTP測試而設計的,但是它已經擴充套件以支援各種各樣的測試模組。它和用於HTTP和SQL資料庫(使用JDBC)的模組一起運送。它可以用來測試靜止資料庫或者活動資料庫中的伺服器的執行情況,可以用來模擬對伺服器或者網路系統加以重負荷以測試它的抵抗力,或者用來分析不同負荷型別下的所有執行情況。它也提供了一個可替換的介面用來定製資料顯示,測試同步及測試的建立和執行。
JMeter的工作原理:JMeter可以作為Web伺服器與瀏覽器之間的代理閘道器,以便捕獲瀏覽器的請求和Web伺服器的響應,這樣就很容易地生成效能測試指令碼,有了效能測試指令碼,JMeter就可以通過執行緒組來模擬真實使用者對Web伺服器的訪問壓力,這與LoadRunner的工作原理基本一致。
下面是JMeter的官方簡介:
The Apache JMeter™ desktop application is open source software, a 100% pure Java application designed to load test functional behavior and measure performance. It was originally designed for testing Web Applications but has since expanded to other test functions.
What can I do with it?
Apache JMeter may be used to test performance both on static and dynamic resources (Files, Web dynamic languages - PHP, Java, ASP.NET, etc. -, Java Objects, Data Bases and Queries, FTP Servers and more). It can be used to simulate a heavy load on a server, group of servers, network or object to test its strength or to analyze overall performance under different load types. You can use it to make a graphical analysis of performance or to test your server/script/object behavior under heavy concurrent load.
What does it do?
Apache JMeter features include:
- Ability to load and performance test many different server/protocol types:
- Web - HTTP, HTTPS
- SOAP
- FTP
- Database via JDBC
- LDAP
- Message-oriented middleware (MOM) via JMS
- Mail - SMTP(S), POP3(S) and IMAP(S)
- MongoDB (NoSQL)
- Native commands or shell scripts
- TCP
- Complete portability and 100% Java purity .
- Full multithreading framework allows concurrent sampling by many threads and simultaneous sampling of different functions by separate thread groups.
- Careful GUI design allows faster Test Plan building and debugging.
- Caching and offline analysis/replaying of test results.
- Highly Extensible core:
- Pluggable Samplers allow unlimited testing capabilities.
- Several load statistics may be choosen with pluggable timers .
- Data analysis and visualization plugins allow great extensibility as well as personalization.
- Functions can be used to provide dynamic input to a test or provide data manipulation.
- Scriptable Samplers (BeanShell, BSF-compatible languages and JSR223-compatible languages)
JMeter的使用者手冊:http://jmeter.apache.org/usermanual/index.html
推薦一本書籍:《零成本實現Web效能測試》
JMeter的指令碼錄製工具:Badboy
Web Polygraph
Web Polygraph這個軟體也是一個用於測試WEB效能的工具,這個工具是很多公司的標準測試工具,包括微軟在分析其軟體效能的時候,也是使用這個工具做為基準工具的。很多招聘測試員的廣告中都註明需要熟練掌握這個測試工具。
Web Polygraph is a freely available performance testing tool for caching proxies, origin server accelerators, L4/7 switches, content filters, and other Web intermediaries. Polygraph's features include:
- high-performance HTTP clients and servers
- realistic HTTP, FTP and SSL traffic generation
- HTTP and SOCKS5 proxy support, with chaining
- HTTP Basic, NTLM, and Negotiate proxy authentication
- LDAP-friendly generation of user credentials
- flexible content simulation
- ready-to-use standard workloads for benchmarking
- powerful domain-specific configuration language
- portable open-source implementation
- friendly commercial support
TCPCopy
Although the real live flow is important for the test of Internet server applications, it is hard to simulate it as online environments are too complex. To support more realistic testing of Internet server applications, we propose a live flow reproduction tool – TCPCopy, which could generate the test workload that is similar to the production workload. TCPCopy consists of two components: the TCPCopy Client (tcpcopy) and the TCPCopy Server (intercept). The TCPCopy Client (tcpcopy) is deployed on the production system and it copies live flow data, does necessary modifications and sends them to the test system in real-time. The TCPCopy Server (intercept) is deployed on the test system and it returns necessary response information to the TCPCopy Client (tcpcopy). To the test server, the reproduced workload is just from end-users. Currently, TCPCopy has been widely used by companies in China.
TCPCopy has little influence on the production system except occupying additional CPU, memory and bandwidth. Moreover, the reproduced workload is similar to the production workload in request diversity, network latency and resource occupation.
Traditional architecture
Advanced architecture
Tsung
Tsung is an open-source multi-protocol distributed load testing tool
It can be used to stress HTTP, WebDAV, SOAP, PostgreSQL, MySQL, LDAP and Jabber/XMPP servers. Tsung is a free software released under the GPLv2license.
The purpose of Tsung is to simulate users in order to test the scalability and performance of IP based client/server applications. You can use it to do load and stress testing of your servers. Many protocols have been implemented and tested, and it can be easily extended.
It can be distributed on several client machines and is able to simulate hundreds of thousands of virtual users concurrently (or even millions if you have enough hardware ...).
Tsung is developed in Erlang, an open-source language made by Ericsson for building robust fault-tolerant distributed applications.
curl-loader
curl-loader (also known as "omes-nik" and "davilka") is an open-source tool written in C-language, simulating application load and application behavior of thousands and tens of thousand HTTP/HTTPS and FTP/FTPS clients, each with its own source IP-address. In contrast to other tools curl-loader is using real C-written client protocol stacks, namely, HTTP and FTP stacks of libcurl and TLS/SSL of openssl, and simulates user behavior with support for login and authentication flavors.
The goal of the project is to deliver a powerful and flexible open-source testing solution as a real alternative to Spirent Avalanche and IXIA IxLoad.
The tool is useful for performance loading of various application services, for testing web and ftp servers and traffic generation. Activities of each virtual client are logged and collected statistics includes information about resolving, connection establishment, sending of requests, receiving responses, headers and data received/sent, errors from network, TLS/SSL and application (HTTP, FTP) level events and errors.
Virtual clients are grouped together to the so-called batches of clients, performing the same sort of activities, like:
- authentication login;
- user activity simulation by fetching several URLs with configurable timeouts in between;
- logoff.
The tool can be easily extended to generate sftp, telnet, tftp, ldap, ssh, scp etc other application protocols, supported by the great libcurl library.
Grinder
Grinder是一個開源的JVM負載測試框架,它通過很多負載注射器來為分散式測試提供了便利。 支援用於執行測試指令碼的Jython指令碼引擎HTTP測試可通過HTTP代理進行管理。根據專案網站的說法,Grinder的 主要目標使用者是“理解他們所測程式碼的人——Grinder不僅僅是帶有一組相關響應時間的‘黑盒’測試。由於測試過程可以進行編碼——而不是簡單地指令碼 化,所以程式設計師能測試應用中內部的各個層次,而不僅僅是通過使用者介面測試響應時間。
LoadRunner(商業軟體)
LoadRunner 是一種預測系統行為和效能的工業標準級負載測試工具。通過以模擬上千萬使用者實施併發負載及實時效能監測的方式來確認和查詢問題,LoadRunner 能夠對整個企業架構進行測試。通過使用LoadRunner , 企業能最大限度地縮短測試時間, 優化效能和加速應用系統的釋出週期。企業的網路應用環境都必須支援大量使用者,網路體系架構中含各類應用環境且由不同供應商提供軟體和硬體產品。難以預知的使用者負載和愈來愈複雜的應用環境使公司時時擔心會發生使用者響應速度過慢, 系統崩潰等問題。這些都不可避免地導致公司收益的損失。Mercury Interactive 的 LoadRunner 能讓企業保護自己的收入來源, 無需購置額外硬體而最大限度地利用現有的IT 資源, 並確保終端使用者在應用系統的各個環節中對其測試應用的質量, 可靠性和可擴充套件性都有良好的評價。LoadRunner 是一種適用於各種體系架構的自動負載測試工具, 它能預測系統行為並優化系統性能。LoadRunner 的測試物件是整個企業的系統, 它通過模擬實際使用者的操作行為和實行實時效能監測, 來幫助您更快的查詢和發現問題。此外,LoadRunner 能支援廣範的協議和技術, 為您的特殊環境提供特殊的解決方案。
kylinpet(國產商業工具)
kylinPET(performance emulation tool,麒麟寵物)是一款功能強大的效能測試工具;介面友好、操作方便,資源佔用率低;自帶TCP/IP協議棧,支援虛擬大量IP;支援IPv4、IPv6的多種業務測試;支援WEB/WebService業務與Flex(HTTP)、IMS業務(SIP)、IPTV業務(RTSP/IGMP/MLD)、XMPP、Socket業務、資料庫、JMS、FTP/SFTP、WEB視訊(包括FLV/MP4/HTTP Live Streaming/HTTP Smooth Streaming/HTTP Dynamic Streaming)、WebSocket、JAVA;支援多種協議組合。
總結一些效能測試的做法
- 為了減少網路延遲和頻寬限制對效能測試的影響,最好在內網進行測試。
- 效能測試一般在優化前和優化後,測試兩輪,這樣能夠比較出優化前後的效果。
- 效能測試最好跑的時間長一些,觀察程式是否存在記憶體洩漏、資源未釋放等問題。
- 有的公司為了避免影響線上環境,只在測試環境做效能測試。我所在的公司效能測試一般線上下線上都會做,線上做效能測試可以瞭解生產環境的效能情況。