Back
Dec 13, 2022

How to create a timelapse video from frames

One of our projects that we are working on is an application that allows you to track the building progress from the construction site using data received from CCTV cameras. The user can view the last received frame, timelapse video for the entire data collection period, compare frames taken at different times, overlay a 3D model on a real frame from a construction site, etc.

Overview 

So, today we’ll tell you how to create a video timelapse from a sequence of snapshots and provide customers with video playlists optimized for browser playback.

Before start: advantages of a video playlist over a single-file video

Unlike a single-file video, a video playlist consists of a playlist file and segments. Segments are just chunks of a whole video, and the playlist file describes the playback order and duration of the segments. 

The video playlist has undeniable advantages:

  1. It is an optimized solution for playing large videos in a web application.
    As we already mentioned, the video playlist consists of video segments. This gives great flexibility in playback: users can start watching the video from the moment they want, skipping the unwanted part, as soon as the first required segment is loaded.
  2. The video playlist is much easier to extend than the single-file video.
    If you need to add to the existing video playlist, you don't need to open a large video file and edit it. You just need to generate a segment and add it to your playlist file. This can greatly reduce server requirements and processing time.

Obviously, the choice between these two options for our case is beyond doubt.

Generate video timelapse

In order to generate a video timelapse from frames, we need:

  1. The frames from which the timelapse is generated.
  2. ffmpeg.

ffmpeg is a great and very flexible tool for working with video and audio, and it can do everything we need. You can download it here and read the documentation here.

ffmpeg \
-y \
-r 25 \
-pattern_type glob -i "*.jpg" \
-vf "scale=w=1280:h=720:force_original_aspect_ratio=decrease" \
-c:v libx264 \
-preset ultrafast \
-tune zerolatency \
-force_key_frames "expr:gte(t,n_forced*1)"
-hls_time 5 \
-hls_segment_filename playlist_segment_%d.ts \
-f hls playlist.m3u8

 

Let's dive into the set arguments:

-y
Overwrite output files without asking.

-r 25  
Set framerate (25 frames per second).

-pattern_type glob -i "*.jpg" 
Take all .jpg files in the local directory to create a video playlist.

 -vf "scale=w=1280:h=720:force_original_aspect_ratio=decrease"
Set video resolution 1280x720, automatically decrease resolution if needed.

-c:v libx264: 
Set output codec.

-preset ultrafast: 
Choose codec preset from veryfast (best speed) to veryslow (best quality).

-tune zerolatency 
Set optimization for fast encoding.

-force_key_frames "expr:gte(t,n_forced*1)"
Force a key frame every 5 seconds.

-hls_time 5 
Cut segments by a duration equal to 5 seconds.

-hls_segment_filename playlist_segment_%d.t
Set mask of segment filename (%d is replaced by segment index).

-f hls 
Set HLS protocol and output format. 

playlist.m3u8
Set the name of the playlist file.

After running the terminal command, ffmpeg will collect all the .jpg frames in the current directory in order and generate a video playlist:

  • playlist.m3u8 – actually, playlist-file;
  • one or more segments (depending on the number of frames provided) - playlist_segment_{1|2|3…}.ts.

Done!

Update video timelapse with a new segment

So, we have already figured out how to create a timelapse video. But what if you need to supplement an existing timelapse?

You can get all the images and generate another video playlist with a new segment. In some cases, this can be a very resource intensive problem. It was the same in our case: we store all the frames and video playlists in the S3 storage, and each time it would take a very long time to get all the frames and the existing video playlist, just to add a new segment. However, there is a less resource-intensive way that will also save us from having to store images for already generated playlist segments.

Let's first see what the playlist file we got in the previous chapter looks like.

Actually, the playlist file is essentially a text file, so we can open it with any text editor:

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:5
#EXT-X-MEDIA-SEQUENCE:0
#EXTINF:1.520000,
segment_0.ts
#EXT-X-DISCONTINUITY
#EXTINF:1.520000,
segment_1.ts
#EXT-X-ENDLIST

Let's figure out what it consists of:

#EXTM3U 
The file header indicating extended M3U format and must be the first line of the file.

#EXT-X-VERSION: 
Indicates the compatibility version of the playlist file.

#EXT-X-MEDIA-SEQUENCE: 
Indicates the sequence number of the first URL that appears in the playlist file.

#EXT-X-TARGETDURATION: 
Specifies the maximum segment duration.

#EXT-X-DISCONTINUITY: 
Indicates discontinuity between the preceding and following segments.

#EXTINF: 
Track information and other additional properties.

#EXT-X-ENDLIST: 
Indicates that no more segments will be added to the file.

As we can see, the playlist file for the most part describes the sequence and duration of the segments. Perfect! You probably already guessed that we can add it ourselves, without using ffmpeg.

In this case, we can generate a new segment as a separate video playlist using the almost identical command from the previous chapter. It differs only in that it will only generate one segment.

ffmpeg \
-y \
-r 25 \
-pattern_type glob -i "*.jpg" \
-vf "scale=w=1280:h=720:force_original_aspect_ratio=decrease" \
-c:v libx264 \
-preset ultrafast \
-tune zerolatency \
-hls_segment_filename new_segment.ts \
-f hls segment_playlist.m3u8

And to update the main video playlist, you just need to insert the segment's metadata into the main playlist file. You may get it from the segment's playlist file or generate it manually.

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:3
#EXT-X-MEDIA-SEQUENCE:0
#EXTINF:1.520000,
segment_0.ts
#EXT-X-DISCONTINUITY
#EXTINF:1.520000,
segment_1.ts

#EXT-X-DISCONTINUITY
#EXTINF:1.520000,
new_segment.ts

#EXT-X-ENDLIST

And that's it! It works very simply: if there is an entry for a segment in the playlist file and the segment file is available at the specified path, it is played.

With S3 file storage: segment URLs

In the examples above, we used the segment filename to tell the video player which segment to play. In this case, the video player will search for the segment in the same directory/URL as the playlist file itself.

#EXTINF:1.520000,
segment.ts
#EXT-X-DISCONTINUITY

Basically, the highlighted line is the path to the segment, so we can conditionally specify any path, like this:

#EXTINF:1.520000,
C:\Video\segment.ts
#EXT-X-DISCONTINUITY

Or even like this:

#EXTINF:1.520000,
https://www.domain/video/segment.ts?any_arg=any_value
#EXT-X-DISCONTINUITY

The last example can be useful if you store the video playlist in private S3 storage: you can use pre-signed URLs as paths to segment in order to avoid difficulties in accessing segments when playing the video playlist in a browser.

Conclusion

Today we figured out how the video playlist works, how to create the timelapse from frames, and how to deal with it in the context of a web application. Take care of yourself and stay tuned!

Additional links:

Online test-bed for video playlists: https://hls-js.netlify.app/demo/
Video playlists with React: https://www.npmjs.com/package/react-player
m3u(8)-playlist syntax: https://docs.fileformat.com/audio/m3u/

Subscribe for the news and updates

More thoughts
Nov 29, 2022Technology
React Performance Testing with Jest

One of the key requirements for modern UI is being performant. No matter how beautiful your app looks and what killer features it offers, it will frustrate your users if it clangs.

Sep 21, 2020Technology
How to Optimize Django ORM Queries

Django ORM is a very abstract and flexible API. But if you do not know exactly how it works, you will likely end up with slow and heavy views, if you have not already. So, this article provides practical solutions to N+1 and high loading time issues. For clarity, I will create a simple view that demonstrates common ORM query problems and shows frequently used practices.

Jan 12, 2017Technology
Making Custom Report Tables Using AngularJS and Django

In this article I will tell you how to create an interactive interface with a widely customized visual look and different filtering to view reports.

Dec 1, 2016Technology
How to Use Django & PostgreSQL for Full Text Search

For any project there may be a need to use a database full-text search. We expect high speed and relevant results from this search. When we face such problem, we usually think about Solr, ElasticSearch, Sphinx, AWS CloudSearch, etc. But in this article we will talk about PostgreSQL. Starting from version 8.3, a full-text search support in PostgreSQL is available. Let's look at how it is implemented in the DBMS itself.

Sep 23, 2010Technology
Dynamic class generation, QuerySetManager and use_for_related_fields

It appears that not everyone knows that in python you can create classes dynamically without metaclasses. I'll show an example of how to do it.So we've learned how to use custom QuerySet to chain requests:Article.objects.old().public()Now we need to make it work for related objects:user.articles.old().public()This is done using use_for_related_fields, but it needs a little trick.

Feb 18, 2010Technology
Absolute urls in models

Everybody knows about permalink, but it's usually used only in get_absolute_url. I prefer to use it for all related model urls.class Event(models.Model):# ...@models.permalinkdef edit_url(self):return ('event_edit', (self.pk, ))And then in template:<a href="{{ event.edit_url }}">Редактировать событие</a>