JAVA网络爬虫(1)

1.环境搭建

**1.jar包:****httpclient-4.5.2.jar 和 httpcore-4.4.1.jar  下载地址: **https://download.csdn.net/download/hongfei568718926/11706234

注意版本对不对,如果版本不对的话可能出现以下异常

java.lang.ClassNotFoundException:org.apache.http.config.Lookup(报该错误的请使用上面指定版本的包)

2.editplus开发软件,或者其他

2.实现过程

爬虫实现过程相当于:客户端想获取服务器的数据的时候的过程

1.打开浏览器    CloseableHttpClient httpClient= HttpClients.createDefault();

2.输入网址      HttpGet httpGet=new HttpGet("http://www.baidu.com");

3.按回车键发起请求    CloseableHttpResponse response=httpClient.execute(httpGet);

4.解析返回的数据并显示 

 if(response.getStatusLine().getStatusCode()==200){
String content= EntityUtils.toString(response.getEntity(),"UTF-8");
System.out.println(content);
}

代码实现过程

1import org.apache.http.client.methods.CloseableHttpResponse; 2import org.apache.http.client.methods.HttpGet; 3import org.apache.http.impl.client.CloseableHttpClient; 4import org.apache.http.impl.client.HttpClients; 5import org.apache.http.util.EntityUtils; 6 7public class demo01 { 8 public static void main(String[] args) throws Exception{ 9 //打开浏览器创建HttpClient对象 10 CloseableHttpClient httpClient= HttpClients.createDefault(); 11 //输入网址,发起get请求//注意需要在网址上面加上http 12 HttpGet httpGet=new HttpGet("http://www.baidu.com"); 13 //按回车发起请求,返回响应,使用HttpClient对象发起请求 14 CloseableHttpResponse response=httpClient.execute(httpGet); 15 //j解析响应, 16 if(response.getStatusLine().getStatusCode()==200){ 17 String content= EntityUtils.toString(response.getEntity(),"UTF-8"); 18 System.out.println(content); 19 } 20 21 } 22} 23 24

运行结果

 

 

代码交流 2021