admin管理员组文章数量:1660819
本文按照下列项目来进行说明。
mysite2
- manage.py
- mysite2
- app01
1、打开今日头条，对网页进行分析并爬取
获取请求URL
分析网站的数据来源后。
开始构造headers，对及今日头条进行爬取，并把数据JSON格式化。
其中的Url对应的就是当前新闻内容的网址，Title对应的就是新闻的标题。
{"data":[
{
"ClusterId":7072942452532842023,
"Title":"沙特和阿联酋领导人拒接拜登电话",
"LabelUrl":"https://p26.toutiaoimg/img/mosaic-legacy/2b29200041b9c651e8148~cs_noop.png",
"Label":"hot",
"Url":"https://www.toutiao/amos_land_page/?category_name=topic_innerflow\u0026event_type=hot_board\u0026log_pb=%7B%22category_name%22%3A%22topic_innerflow%22%2C%22cluster_type%22%3A%2210%22%2C%22enter_from%22%3A%22click_category%22%2C%22entrance_hotspot%22%3A%22outside%22%2C%22event_type%22%3A%22hot_board%22%2C%22hot_board_cluster_id%22%3A%227072942452532842023%22%2C%22hot_board_impr_id%22%3A%222022030918321201021216216025C743EE%22%2C%22jump_page%22%3A%22hot_board_page%22%2C%22location%22%3A%22news_hot_card%22%2C%22page_location%22%3A%22hot_board_page%22%2C%22rank%22%3A%221%22%2C%22source%22%3A%22trending_tab%22%2C%22style_id%22%3A%2240132%22%2C%22title%22%3A%22%E6%B2%99%E7%89%B9%E5%92%8C%E9%98%BF%E8%81%94%E9%85%8B%E9%A2%86%E5%AF%BC%E4%BA%BA%E6%8B%92%E6%8E%A5%E6%8B%9C%E7%99%BB%E7%94%B5%E8%AF%9D%22%7D\u0026rank=1\u0026style_id=40132\u0026topic_id=7072942452532842023",
"HotValue":"6753999",
"Schema":"",
"LabelUri":{
"uri":"mosaic-legacy/2b29200041b9c651e8148",
"url":"https://p26.toutiaoimg/img/mosaic-legacy/2b29200041b9c651e8148~cs_noop.png",
"width":200,
"height":200,
"url_list":[
{"url":"https://p26.toutiaoimg/img/mosaic-legacy/2b29200041b9c651e8148~cs_noop.png"},
{"url":"https://p3.toutiaoimg/img/mosaic-legacy/2b29200041b9c651e8148~cs_noop.png"},
{"url":"https://p9.toutiaoimg/img/mosaic-legacy/2b29200041b9c651e8148~cs_noop.png"}
],
"image_type":1
},
"ClusterIdStr":"7072942452532842023",
"ClusterType":10,
"QueryWord":"沙特和阿联酋领导人拒接拜登电话",
"InterestCategory":["international"],
"Image":{
"uri":"tos-cn-i-qvj2lq49k0/a7e3f7e3e8c04c37bc7f88b2340ab999",
"url":"https://p6.toutiaoimg/img/tos-cn-i-qvj2lq49k0/a7e3f7e3e8c04c37bc7f88b2340ab999~cs_noop.png",
"width":0,
"height":0,
"url_list":[
{"url":"https://p6.toutiaoimg/img/tos-cn-i-qvj2lq49k0/a7e3f7e3e8c04c37bc7f88b2340ab999~cs_noop.png"},
{"url":"https://p9.toutiaoimg/img/tos-cn-i-qvj2lq49k0/a7e3f7e3e8c04c37bc7f88b2340ab999~cs_noop.png"},
{"url":"https://p3.toutiaoimg/img/tos-cn-i-qvj2lq49k0/a7e3f7e3e8c04c37bc7f88b2340ab999~cs_noop.png"}
],
"image_type":1
},
"LabelDesc":"热门事件"
},
{
}，2、在app01/views.py文件中添加一个函数用来爬取新闻并进行展示
#爬取今日头条的头条热榜，进行展示并附加链接
def news(req):
url = 'https://www.toutiao/hot-event/hot-board/?origin=toutiao_pc&_signature=_02B4Z6wo00f01yG9tdQAAIDCQrd1vxaJp9chmbFAAKpR4Dqk0c56dkhdlvNsoD3I03ygIjgUcxkM0VcFYKfO0a9iJRjnl1M9yxZvlq-pgzUXDOrpi1wKoYlCVC9.llzChJ7GmTYXIDMvE.c1a6'
headers = {
"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.135 Safari/537.36",
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9", }
res = requests.get(url=url, headers=headers)
data_all_dict = res.json()
data_lists = dict(data_all_dict)['data']
return render(
req,
'news.html',
{
"news_dicts":data_lists
}
)
3、在app01/templates文件夹下新建一个news.html文件
其中style='text-decoration:none;color:black' ，作用是去掉超链接的下划线，并让超链接的颜色变成黑色。再使用Django的模板技术，对新闻字典进行遍历输出。
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Title</title>
</head>
<body>
<h1>今日头条</h1>
<ul>
{% for news in news_dicts %}
<li>
<a style='text-decoration:none;color:black' href = {{news.Url}} target="_blank">{{ news.Title }}</a><br>
</li>
{% endfor %}
</ul>
</body>
</html>4、在mysite2/urls.py文件中构造url和函数的链接关系
path('news/',views.news)5、启动服务python manage.py runserver 0.0.0.0:8000，在浏览器中输入http://127.0.0.1:8000/news/，查看是否成功。
版权声明：本文标题：Django项目实践（爬取今日头条的头条热榜） 内容由热心网友自发贡献，该文观点仅代表作者本人， 转载请联系作者并注明出处：https://www.elefans.com/dongtai/1729881968a1216032.html， 本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌抄袭侵权/违法违规的内容，一经查实，本站将立刻删除。
发表评论