TikTok Time Warp Scan (p5.js)

Introduction

At the time of writing, one of the most popular filters on TikTok is the time warp scan. If you aren’t familiar with what the filter does, check out this video:

After seeing the filter for the umpteenth time, it dawned on me it would be a fun little challenge to try and recreate this filter in p5. The resulting little project was a lot of fun to code so I thought I’d share. This tutorial assumes some basic knowledge of Javascript and p5.js. This project also won’t really focus on UI enhancements and will focus solely on building the filter. In addition, normally I’d split some of the code into more files, but in this case its just a simple tutorial so I want to keep things compact. If you just wanna see the code, check out this repo: https://github.com/qscacheri/time-warp-scan-p5.

Setup

Let’s start by creating our only two files:

-index.html
-sketch.js

Next, let’s add some boilerplate html to our index.html file:

Let’s also add our basic p5 functions to our sketch.js file and create a full screen canvas:

Grabbing the Camera Feed

The first thing we need to do is grab the camera feed with p5.

First let’s define our variable that will hold the p5 Capture object. The p5 Capture object is responsible for…capturing. Next, we’ll actually create our capture object in the setup function. The capture object will automatically create a video element and display our capture, but since we want to manipulate our video, we don’t want that, so we hide it. Lastly, we will want to flip our webcam so it is mirrored.

Lastly, let’s display our video on the canvas. We flip the image here so that the image will be mirrored.

By this point you should being seeing yourself on the canvas!

Note: Make sure you are using localhost or some other secure host or the camera will not work

How it Will Work

Before we start coding the meat of the filter, let’s briefly discuss what the code will actually do. Essentially what we need to do is to grab a column of pixels where the height of the column is the height of the video, and width is some small value, maybe 1 pixel. We store these pixels as an image so that we can draw it to the screen overtop of our capture display. Next, we move to the next column and grab another image that we draw to the screen. This process repeats, grabbing columns and drawing them, until we’ve moved across the whole video.

TimeWarpScan Class Setup

Let’s create a class that will represent our filter. We know some of the properties we will need immediately: we need to keep track of the position of the line moving across the screen, and whether or not the filter should start. We first need a variable stripRatio that will define the width of each column as a percentage of total width. This is very important because our capture width and display width are different, so they need to be scaled accordingly. Let’s add a flag for when we’ve finished the scan. Our position member will be initialized to 1 and go to 0. We do this because we are flipping the screen so we have to move right to left. Lastly, let’s instantiate our class and call our empty draw method.

Before we start our video processing, let’s quickly make it so when we click the spacebar, we activate the filter. Add the following function to our sketch:

Where the start function is:

Video Processing

Now we get to the awesome video processing!

First, let’s add a utility function that will give us the height of our display (accounting for aspect ratio) Then, we add another class that is responsible for storing an image column, storing its X position as a percentage, and drawing it to the screen. Remember, our images will initially be a different size than our display so we need to scale each column accordingly. Our TimeWarpScan class will hold an array of these objects, so let’s add a property to our constructor. We also need to clear this array so that when restart the filter we have an empty array:

Next, let’s add our function for doing the actual processing:

1. If the filter is not active, we return
2. If the data for a video stream has loaded, and we haven’t reached the end, we capture!
3. We grab a section of our video starting at the current position, with a width of stripRatio * capture.width
4. We push a new TimeWarpImage object to our array, passing in the captured image and our current position
5. We move our position to capture the next column
6. If we’ve reached the end stop
7. Draw our images to the screen!

Lastly, we need to fill out our draw function in the TimeWarpScan class. In addition, let’s add a bright green indicator line to show the progress of the scan.

Conclusion

There is definitely some room for improvement, but in the interest of keeping the tutorial short and simple I’ll leave that to you. Having the option of scanning vertically would definitely be a plus, as well as a better handling of restarting the scan. Overall, I’m happy with how the scan itself turned out and is definitely fun to play around with.