preamble
websocket is a network transport protocol. It allows full-duplex communication over a single TCP connection. For this reason, websocket makes it easier and more efficient for clients to communicate with servers.
What is websocket
websocket is a standalone protocol created on top of TCP. The protocol was born in 2008 and became an international standard in 2011. One of its main features is - full duplex, i.e. once a connection is established, either the server or the client can actively push messages to the other.
Before websockets, websites that needed to implement push technology used polling, where the browser would send requests to the server at regular intervals. The disadvantage of this model is that the browser needs to keep sending requests to the server, which consumes a lot of bandwidth resources. The newer Comet technology also allows for two-way communication, but it still requires repeated requests, and the long HTTP connections that are common in Comet also consume server resources.
Based on the above, HTML5 defines the websocket protocol, which can better save server and bandwidth resources. And realize efficient real-time communication. Currently, all browsers support it.
Principles and mechanisms of websocket communication
Although websocket is a new protocol, but it can not be separated from http alone, when the client builds a websocket instance, and to the server to connect, will first initiate a http message request. This tells the server that it needs to switch its communication protocol to websocket.
If the server supports the websocket protocol, it will switch the protocol to websocket and return a response message. The return status code is 101, which means that the request for protocol switching has been accepted, and the data can be transferred next.
Websocket accomplishes the handshake protocol with HTTP because of good compatibility, default ports are 80 and 443, and the handshake phase is not easily blocked by firewalls.
Features of websocket
- Low overhead, when the server and client exchange data, the protocol packet header contains less information
- High real-time, the protocol uses full-duplex, compared to the http request client initiates the request, the server can only respond to the model, the delay is significantly lower!
- Good compatibility with HTTP, default ports are 80 and 443. handshakes use the HTTP protocol and are not easily blocked by firewalls
- Supports text and binary data transfer
- Support for custom extensions, users can implement their own customized sub-protocols
- Maintains a long connection between the server and the client through a heartbeat mechanism
A small example of building a real-time log trace
The server opens a service that listens to logging scripts, the service will limit the range of paths allowed to access (to prevent hackers from taking advantage of program vulnerabilities and scanning the entire server); the server parses the client's request and returns the message content of the logs to the client; the server sends a heartbeat detection to the client at regular intervals, and disconnects if it does not receive a response from the client
The server-side core program code logic is as follows
with open(file_path) as f:
# Read the log file with the specified number of lines (NUM_LINES) for the first time and send it to the client
content = ''.join(deque(f, NUM_LINES))
content = (content, full=False)
await (content)
# If a client is found to have a tail request, perform a tail log trace
if tail:
# First create a heartbeat time to initiate this request
last_heartbeat = ()
while True:
# Each time you tail the server's latest log entry, return it to the client
content = ()
if content:
content = (content, full=False)
await (content)
else:
await (1)
# Detect whether this request has exceeded the maximum heartbeat detection time from the last request, if so, initiate heartbeat detection.
if () - last_heartbeat > HEARTBEAT_INTERVAL:
try:
await ('ping')
pong = await asyncio.wait_for((), 5)
(f"pong:{pong}")
if pong != 'pong':
raise Exception()
except Exception:
raise Exception('Ping error')
else:
last_heartbeat = ()
else:
await ()
The client is very simple, listen to the server log file, found that there is a new log generated by the output log or directly display the log in real time on the front-end page. Accordingly, if you need to listen for a long time, then when the server sends a heartbeat detection signal over, you also need to respond to the heartbeat feedback.
The client-side core code logic is as follows
async def consumer_handler(websocket: WebSocketClientProtocol) -> None:
async for message in websocket:
log_message(message)
if message == "ping":
await ("pong")
async def cousume(hostname: str, port: int, log_file: str, tail:bool=True) -> None:
websocket_resource_url = f"ws://{hostname}:{port}{log_file}"
if tail:
websocket_resource_url = f"{websocket_resource_url}?tail=1"
async with (websocket_resource_url) as websocket:
await consumer_handler(websocket)
def log_message(message: str) -> None:
(f"Message: {message}")
A log production file is simulated here
The code logic is as follows
import os
from loguru import logger
class LoggerExtend(object):
# Name of the directory where it is stored
folder = '../logs'
def __init__(self, filename, folder=None):
= folder or
if not ():
()
= + '/' + filename
(, rotation="100 MB")
@property
def get_logger(self):
return logger
if __name__ == '__main__':
logger = LoggerExtend((__file__).replace(".py", ".log")).get_logger
import time
while True:
("Hello aaa")
Finally, start the log production program → server-side program → client-side program in sequence
After the log production file is started, it runs as follows
Server-side startup program running, no runtime logs generated
At this time, start the client program, the running effect is as follows
For the full code, go to GitHub.
/hacksman/l...
Log production program path:
common/logger_extend.py
Server-side program path:
websoctet_lab/log_server.py
Client program path:
websoctet_lab/cousumer_log_view.py
Above is python and websocket build real-time log tracker steps in detail, more information about python build real-time log tracker please pay attention to my other related articles!