Creating A Console Video Player With A Catch. Part 1

Intro

I’ve always wanted to make a video player. I think it’s an interesting challenge, with many chances of creative optimizations.

This will be a two parter.

• How I handle client/server logic and implimenting my own video file format.
• Benchmarking and optimization.

The Idea

The idea here being that I will create a video player, entirely working in the console (or terminal, depending on who you ask). It will consist of a backend and a frontend. The backend is pretty much just a websocket in a comsole application, that sends and recieves data and requests to the frontend. The frontend is also just a websocket in a console application that prints to the terminal.

The backend loads the video file, prepares it so it can be transported more easy, then waits for the client to connect. Once the client connects, a header file will be sent to the client, discriping how the video should be displayed. Once the header is loaded, the client will request the content of the video and then render the video.

General Implementation

There is no real reason for me to use a server client program, but I think it’s a nice breath of fresh air in terms of the projects I do. Plus I haven’t done any websocket programming in a while.

I have implimented my own filetype, since that would be easier to work with. At the top of the file, there is a header of colors that each pixel reference. After the header, it’s basically just a 2D array of ints.

Server Implementation

Note: This is the unoptimised implementation. When the server starts, it loads the video to be played, then determins the runtime to play the video.

Afterwards, it awaits further instructions from the client.

When the client requests a hearder, it sends the hearder as a Dictionary<string, string>, since it’s just key/value pairs of indexes and hex colorcodes.

After sending the header, it will send each frame, one by one, to the client.

This is the very center of the server. It concatinates the individual color indexes from the video file, adds a newline, and continues until the video height. Then get the bytes and sends it over tcp.

if (response[0] == 'c')
{
    for (int j = 0; j < TerminalVideoDescriptor.Height; j++)
    {
        for (int k = 0; k < TerminalVideoDescriptor.Width; k++)
        {
            videoFrame += _terminalVideo.Frames[frame][j][k];
        }

        videoFrame += '\n';
    }

    echoBytes = Encoding.UTF8.GetBytes(videoFrame);

    _handler.Send(echoBytes, SocketFlags.None);

    frame++;

    continue;
}

Client Implementation

Note: This is the unoptimised implementation. When the client starts, it will connect to the server, and request the header for the video. After getting the header, it will save the header in memory, and request the frames from the video.

This is the central part of the client side. It connects to the client, and asks for content (aka a frame from the video). Once it has the frame, it will render the frame and shutdown and dispose the client. This is a stateless tcp connection after all.

_client = new(_ipEndPoint.AddressFamily, SocketType.Stream, ProtocolType.Tcp);
_client.Connect(_ipEndPoint);

if (!firstRun)
{
    SendMessage(_client, c);

    _displayHandler.Render(ReceiveResponse(_client));

    _client.Shutdown(SocketShutdown.Both);
    _client.Dispose();

    continue;
}

With each frame recieved, the client calls _displayHandler.Render(frame);

The way I do color is not the fastest. As a matter of fact, just implementing showing color added 12ms to render each frame, lol.

To get color in the terminal, I have to print the frame, character by character. The inner most line prints the █ character (\u2588) to the terminal, and .Pastel() extension method applies color.

string[] temp holds all color indexes. And .GetColor(_header) is an extension method that takes in the index and the header, and then returns the hex color for .Pastel() to use.

The frames are newline terminated. So each line in the frame ends with a newline character.

public void Render(string frame)
{
    Thread.Sleep(milliSecondBetweenFrames);
    Console.Clear();

    string[] temp = frame.Split('\n');

    for (int i = 0; i < TerminalVideoDescriptor.Height; i++)
    {
        for (int j = 0; j < TerminalVideoDescriptor.Width; j++)
        {
            Console.Write("\u2588".Pastel(temp[i][j].GetColor(_header)));
        }
        Console.Write('\n');
    }
}

Conclusion

Each frame takes 11 milliseconds to display at the fastest, and sometimes spiking to 45. Not great if the player should play with a minimul fps of 30.

Implementation Custom Video Format

It’s actually quite easy. The first line is all of the colors used in the video, the second line is just to tell the server that the header ends here. =0 is frame 0, which follows by a block of ints. Next frame will then start with =1

0#FFFFFF:1#000000
@
=0
12121212121212121212121212121212121212121212121212121212121212121212121212121212
12121212121212121212121212121212121212121212121212121212121212121212121212121212
12121212121212121212121212121212121212121212121212121212121212121212121212121212
12121212121212121212121212121212121212121212121212121212121212121212121212121212
12121212121212121212121212121212121212121212121212121212121212121212121212121212
12121212121212121212121212121212121212121212121212121212121212121212121212121212
12121212121212121212121212121212121212121212121212121212121212121212121212121212
12121212121212121212121212121212121212121212121212121212121212121212121212121212
12121212121212121212121212121212121212121212121212121212121212121212121212121212
12121212121212121212121212121212121212121212121212121212121212121212121212121212
12121212121212121212121212121212121212121212121212121212121212121212121212121212
12121212121212121212121212121212121212121212121212121212121212121212121212121212
12121212121212121212121212121212121212121212121212121212121212121212121212121212
12121212121212121212121212121212121212121212121212121212121212121212121212121212
12121212121212121212121212121212121212121212121212121212121212121212121212121212
12121212121212121212121212121212121212121212121212121212121212121212121212121212
12121212121212121212121212121212121212121212121212121212121212121212121212121212
12121212121212121212121212121212121212121212121212121212121212121212121212121212
12121212121212121212121212121212121212121212121212121212121212121212121212121212
12121212121212121212121212121212121212121212121212121212121212121212121212121212
12121212121212121212121212121212121212121212121212121212121212121212121212121212
12121212121212121212121212121212121212121212121212121212121212121212121212121212
12121212121212121212121212121212121212121212121212121212121212121212121212121212
12121212121212121212121212121212121212121212121212121212121212121212121212121212
12121212121212121212121212121212121212121212121212121212121212121212121212121212
12121212121212121212121212121212121212121212121212121212121212121212121212121212
12121212121212121212121212121212121212121212121212121212121212121212121212121212
12121212121212121212121212121212121212121212121212121212121212121212121212121212
12121212121212121212121212121212121212121212121212121212121212121212121212121212
12121212121212121212121212121212121212121212121212121212121212121212121212121212
12121212121212121212121212121212121212121212121212121212121212121212121212121212
12121212121212121212121212121212121212121212121212121212121212121212121212121212
12121212121212121212121212121212121212121212121212121212121212121212121212121212
12121212121212121212121212121212121212121212121212121212121212121212121212121212
12121212121212121212121212121212121212121212121212121212121212121212121212121212
12121212121212121212121212121212121212121212121212121212121212121212121212121212
12121212121212121212121212121212121212121212121212121212121212121212121212121212
12121212121212121212121212121212121212121212121212121212121212121212121212121212
12121212121212121212121212121212121212121212121212121212121212121212121212121212
12121212121212121212121212121212121212121212121212121212121212121212121212121212