1. 程式人生 > 實用技巧 >20200726_java爬蟲_使用HttpClient模擬瀏覽器傳送請求

20200726_java爬蟲_使用HttpClient模擬瀏覽器傳送請求

0. 摘要

0.1 新增依賴

<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpclient</artifactId>
    <version>4.5.2</version>
</dependency>

0.2 程式碼

//1. 開啟瀏覽器  建立httpclient物件
CloseableHttpClient httpClient = HttpClients.createDefault();
//2. 輸入網址 HttpGet httpGet = new HttpGet("http://www.baidu.com"); //3. 傳送請求 CloseableHttpResponse httpResponse = httpClient.execute(httpGet); //4. 響應結果 HttpEntity httpEntity = httpResponse.getEntity(); //5. 解析結果 String result = EntityUtils.toString(httpEntity, "utf-8"); System.out.println(result);

1. 實操

1.1 新增依賴

1.1.1 找到 pom.xml 新增依賴

1.1.2 依賴程式碼

<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpclient</artifactId>
    <version>4.5.2</version>
</dependency>

1.2 新增 TestHttpClient 類

1.2.1 建立類檔案 com.aifu.TestHttpClient

1.2.2 新增程式碼

public static void main(String[] args) throws IOException {
    //1. 開啟瀏覽器  建立httpclient物件
    CloseableHttpClient httpClient = HttpClients.createDefault();
    //2. 輸入網址
    HttpGet httpGet = new HttpGet("http://www.baidu.com");
    //3. 傳送請求
    CloseableHttpResponse httpResponse = httpClient.execute(httpGet);
    //4. 響應結果
    HttpEntity httpEntity = httpResponse.getEntity();
    //5. 解析結果
    String result = EntityUtils.toString(httpEntity, "utf-8");
    System.out.println(result);
}

1.3 執行

1.3.1 點選綠標執行 或者快捷鍵 ctrl + alt +F10