admin管理员组文章数量:1596199
一、分析需求
腾讯招聘网址:https://careers.tencent/search.html
分析获取url:
提取数据的方法:
使用json转字典
二、代码实现
import requests
import csv
url = "https://careers.tencent/tencentcareer/api/post/Query"
headers = {
"user-agent": "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36",
}
params = {'area': ' cn',
'attrId': ' ',
'bgIds': ' ',
'categoryId': ' ',
'cityId': ' ',
'countryId': ' ',
'keyword': ' ',
'language': ' zh-cn',
'pageIndex': ' 1',
'pageSize': ' 10',
'parentCategoryId': ' ',
'productId': ' ',
'timestamp': ' 1602211262824'}
def parse_json(url, params={}):
"""解析url,得到字典"""
response = requests.get(url=url, headers=headers, params=params)
return response.json()
def get_position(data):
"""获取职位数据"""
item = {
"postion_name":"",#职位名称
# "postion_department":"",#职位部门
# "postion_location":"",#职位所在地
# "postion_country":"",#职位所在国家
# "postion_category":"",#职位类别
"postion_responsibility":"",#职位职责
"postion_url":"",#职位url
}
data_list = data["Data"]["Posts"]
for data in data_list:
item["postion_name"] = data["RecruitPostName"]
item["postion_responsibility"] = data["Responsibility"]
item["postion_url"] = data["PostURL"]
save(item)
def save(item):
"""将数据保存到csv中"""
with open("./腾讯招聘.csv", "a", encoding="utf-8") as file:
writer = csv.writer(file)
writer.writerow(item.values())
def start():
for i in range(1,618):
params["pageIndex"] = i
data = parse_json(url,params)
get_position(data)
if __name__ == '__main__':
start()
三、运行结果
版权声明:本文标题:Python爬取腾讯招聘信息 内容由热心网友自发贡献,该文观点仅代表作者本人, 转载请联系作者并注明出处:https://www.elefans.com/dianzi/1724850683a993840.html, 本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌抄袭侵权/违法违规的内容,一经查实,本站将立刻删除。
发表评论