User:Quantimoney/GSoC2021/Final Report

Video Sequence Editor Improvements - Individual strip previews and modification indicators - Final Report

Links

• Design task
  
• Differential (Code)
  
• Development Branchsoc-2021-vse-strip-thumbnails
  

Summary

Two deliverables were proposed at the start of the project: strip thumbnails and modifier indicators, such that majority of the coding period would be focused on the strip thumbnails work, and if time permits the second goal would be worked on. After discussions with the mentor it was decided that would be better to focus on the strip thumbnails, improve performance and reduce bugs as much as possible for it to have a better chance of being included into the master branch. The first half of the coding period involved setting up the drawing function for showing the thumbnails and implementing the caching system. The second half involved setting up the caching job to work in the background. The proposed goal for strip thumbnails has been successfully achieved, with quick caching and drawing, and efficient storage.

Details

The design task was initially split into different subtasks, each for the drawing, caching and the parallel thread for caching. This was good for seeing progress but the final implementation has different components using each other's functions and therefore a single diff is uploaded.

Drawing Function
The thumbnails are visible on adding a movie strip or an image sequence into the Sequencer. There is a visible indication of the strips being loaded into cache and drawn.

The visual look of the thumbnails was changed from what was proposed, after taking in feedback in the form of a poll. The look was taken similar to colour strips, with the handles present as translucent rectangles over the images. This gave a consistent look for the Sequencer strips.

There were many changes in the form of tweaks and performance improvements from the initial implementation. The thumbnails are drawn to represent the original source footage. It appears to be on a different layer below the handles, so that the strip handles cut over the thumbnails. The thumbnails show the frame present at the left edge of the image. The thumbnails match the aspect ratio of the source, and do not show the colour grading or post-processing of footage.

The images follow the translucency on overlap operation and slide with the strip, on a strip slide operation. This gives a natural look and feel to the strip. Toggles are provided in the overlap panel for toggling thumbnails.

Caching Function
The caching of the thumbnails is done, using the caching system of the VSE. The images are first downsized to have a maximum size of 256 on either side, and match the aspect ratio of source. This is then stored. The caching system loops over the entire set of visible possible images for the sequence. The limit is 5000 images for thumbnails specifically, and when this limit is crossed, the non visible, not necessary images are removed from cache.

Caching parallel Job
The entire caching is done parallel to main thread, called when there is a call to draw all strips. The sequences are added to a hash and then sent to the job, that cycles over the hash performing caching for each. This is faster as the main thread isn't blocked on looping operation. The job is called whenever there are no images to obtain from cache or when view is changed.

Job called each time there is view change as images aren't in cache

Future Scope

This feature could be implemented for scene strips. This requires the rendering of images first and is very different from the movie clip/image sequences. The images currently loaded could be retained when there is a change in view, or there is no image present in cache. This will stop showing the blank strips when images aren't present in cache to load from, and will be less jarring to the user. The way of choosing the frame to be shown could be improved for more accurate representation of source footage. This does require the intelligence to detect the scene in the source and act accordingly.