Live Streaming Video Chat using python

In this blog we will going to create a video chat between two system without audio using a python library cv2 , So lets start with some of the python library which we will be using in these process.

CV2

OpenCV-Python is a library of Python bindings designed to solve computer vision problems.
cv2.imread() method loads an image from the specified file. If the image cannot be read (because of missing file, improper permissions, unsupported or invalid format) then this method returns an empty matrix.

Syntax: cv2.imread(path, flag)

Parameters:
path: A string representing the path of the image to be read.
flag: It specifies the way in which image should be read. It’s default value is cv2.IMREAD_COLOR

Return Value: This method returns an image that is loaded from the specified file.

Note: The image should be in the working directory or a full path of image should be given.

All three types of flags are described below:

cv2.IMREAD_COLOR: It specifies to load a color image. Any transparency of image will be neglected. It is the default flag. Alternatively, we can pass integer value 1 for this flag.
cv2.IMREAD_GRAYSCALE: It specifies to load an image in grayscale mode. Alternatively, we can pass integer value 0 for this flag.
cv2.IMREAD_UNCHANGED: It specifies to load an image as such including alpha channel. Alternatively, we can pass integer value -1 for this flag.

Socket

This module provides access to the BSD socket interface. It is available on all modern Unix systems, Windows, MacOS, and probably additional platforms.

A socket library is a library that implements sockets so that you can use them in your program for Internet communication. … You’d use that API in your Java program, and the JVM will use the socket library that the operating system provides to do the actual communication

Pickle

Python pickle module is used for serializing and de-serializing python object structures. The process to converts any kind of python objects (list, dict, etc.) into byte streams (0s and 1s) is called pickling or serialization or flattening or marshalling. We can converts the byte stream (generated through pickling) back into python objects by a process called as unpickling.

Why Pickle?: In real world scenario, the use pickling and unpickling are widespread as they allow us to easily transfer data from one server/system to another and then store it in a file or database.

Precaution: It is advisable not to unpickle data received from an untrusted source as they may pose security threat. However, the pickle module has no way of knowing or raise alarm while pickling malicious data.

Struct

The struct module in Python is used to convert native Python data types such as strings and numbers into a string of bytes and vice versa. What this means is that users can parse binary files of data stored in C structs in Python.

It is used mostly for handling binary data stored in files or from network connections, among other sources.

Let’s see the code of our app and try to understand it.

sender.py

import socket,pickle,struct
import cv2
s=socket.socket(socket.AF_INET,socket.SOCK_STREAM)  #TCP connection
print("Enter Your IP")
ip=input()
print("Enter Your port")
port=int(input())
print("Enter Your Partner IP")
pip=input()
print("Enter Your Partner port")
pport=int(input())
s.bind((ip,port))
s.listen(5)  #5 is backlog meaning it can accept 5 connections at a time
cap = cv2.VideoCapture(0)
while True:
    client_socket,addr = s.accept()
    print('GOT CONNECTION FROM:',addr)
    if client_socket:
        vid = cv2.VideoCapture(0)
        
        while(vid.isOpened()):
            img,frame = vid.read()
            a = pickle.dumps(frame)   #serialize frame to byte data
            message = struct.pack("Q",len(a))+a #pack each frame #Q is 8 bytes
            client_socket.sendall(message)
            
            cv2.imshow('TRANSMITTING VIDEO',frame)
            key = cv2.waitKey(1) & 0xFF
            if key ==ord('q'):     #press q to exit
                client_socket.close()

Here, we are using two extra libraries to serialize the video data frames and pack the data to send to client.

“Pickling” is the process whereby a Python object hierarchy is converted into a byte stream, and “unpickling” is the inverse operation, whereby a byte stream (from a binary file or bytes-like object) is converted back into an object hierarchy.

The struct module in Python is used to convert native Python data types such as strings and numbers into a string of bytes and vice versa.It is used mostly for handling binary data stored in files or from network connections, among other sources.

struct.pack()

Syntax: 
struct.pack(format, v1, v2, ...)

Return a string containing the values v1, v2, … , that are packed according to the given format (Format strings are the mechanism used to specify the expected layout when packing and unpacking data).The values followed by the format must be as per the format only, else struct.error is raised.

When an application puts a socket into LISTEN state using the listen syscall, it needs to specify a backlog for that socket. The backlog is usually described as the limit for the queue of incoming connections.

Receiver.py

import socket
import cv2,pickle,struct
s=socket.socket(socket.AF_INET,socket.SOCK_STREAM)   #TCP connection
print("ENter your IP")
ip=input()
print("ENter your PORT")
port=int(input())
s.bind((ip,port))
print("ENter your streamer's IP")
ip1=input()
print("ENter streamer's PORT")
port1=int(input())
s.connect((ip1,port1))
cap=cv2.VideoCapture(0)
data = b""
payload_size = struct.calcsize("Q")   #8 bytes
while True:
    while len(data) < payload_size:
        packet = s.recv(4*1024) #4k buffersize
        if not packet: break    #if no data, then break
        data+=packet            #if data comes,append to data until it becomes>=8k
    packed_msg_size = data[:payload_size]   #first 8 bytes have packed msg
    data = data[payload_size:]   #after initial 8 bytes, frame data present
    msg_size = struct.unpack("Q",packed_msg_size)[0] #unpacking packed messaage
    while len(data) < msg_size:            
        data += s.recv(4*1024)
    frame_data = data[:msg_size]
    data  = data[msg_size:]
    frame = pickle.loads(frame_data)
    cv2.imshow("RECEIVING VIDEO",frame)
    key = cv2.waitKey(1) & 0xFF
    if key  == ord('q'):
        break
s.close()

GitHub Link:
https://github.com/ksp220798/Live-Video-Streaming-using-python.git

Thanks For Reading