Python crawl a platform short video method

preamble

The text and images in this article from the network, for learning, communication purposes only, does not have any commercial purposes, if there is any problem, please contact us in a timely manner in order to deal with.

Basic development environment

Python 3.6

Pycharm

Use of related modules

import os
import requests

Just install Python and add it to the environment variables, and pip install the relevant modules you need.

I. Identification of needs

Crawl the video content of the Funny Fun section.

II. Website data analysis

First of all, we need to make it clear that the way the good-looking video site loads is lazy loading, which requires you to slide down the page to load the new content.

The loaded out content contains the audio playback address as well as the title.

The content is relatively simple, just look at the code

import os
import requests

url = '/videoui/api/videorec?tab=gaoxiao&act=pcFeed&pd=pc&num=20&shuaxin_id=1612592171486'
headers = {
  'user-agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36'
}
response = (url=url, headers=headers)
json_data = ()
videos = json_data['data']['response']['videos']
for index in videos:
  title = index['title']
  play_url = index['play_url']
  video_content = (url=play_url, headers=headers).content
  path = 'video\\'
  if not (path):
    (path)
  with open(path + title + '.mp4', mode='wb') as f:
    (video_content)
    print('Saving:', title)

This article on Python crawl a platform short video method is introduced to this article, more related Python crawl short video content please search my previous posts or continue to browse the following related articles I hope you will support me more in the future!