GSOC 2020: Regression Testing Framework
Calra on blender.chat, devtalk and developer.blender.org
Regression testing helps in quickly detecting if any new features or patch break existing functionality. This helps the developers to test their patch while they are still working on it as opposed to testing during the last phase, by having automated testing at their disposal.
Right now, the way to add a test object is only using Blender itself, this is easy when there are less tests or when we would like to customize the mesh at every step. While this can work, adding a framework which can automatically add blend objects and give them befitting names as well will make testing more beginner friendly.
- Writing framework for Automatic blend file generation.
- Improving framework for testing Mesh modifiers.
- Adding framework for Bone Constraint testing / Compositor automated testing.
- Developer documentation.
Stretch Goal: In case there is some time left, testing bone constraints as mentioned in https://wiki.blender.org/wiki/GSoC/Ideas_Suggestions under Regression Testing.
Automatic blend file Generator
The idea is to break the testing in 3 parts:
- Generation of bare essential blend objects using Python.
- Manually tweaking (changing the default values) as per user requirement inside Blender.
- Executing the test.
We already do the last 2 steps, now it may seem contradictory that more time will be expended but consider a case where there are a lot of tests, I think it will make testing more efficient. Or if one would like to remove the Step 2, and generate the blend file with a line of Python.
- Generation of blend file using Python entirely (with all the properties applied).
- Executing the test.
The above stated is one of ways we can achieve automated naming but the end result would be more or less the same. The process would also involve compiling a list of meshes from the already existing set of tests.
Automatically generating a blend file automatic generation of the blend object. naming of test and expected objects. adding them to a Collection.
Improving Mesh Modifiers Framework
Extending framework for
- Physics modifiers
- Curve Modifiers
Cover remaining Mesh modifiers Revamping the code to give a TestName to each test. Adding a sort of “Beautify” function that will revamp the already existing blend files, e.g. spacing in the blend file for consistency.
Compositor Automated Testing
Some of the work for Compositor tests has been started by Habib Gahbiche (D6334), and I would like to build upon the suggestions by Sergey (similar to Cycle’s tests). Working on the already developed code - one approach would be to generate blend files with nodes. Shifting the testing framework into a separate file. Generating node setup in the blend file using the “compositor.py”.
- Making a separate script for testing framework
- Adding sample tests
- Object / Bone Constraints
This framework will mostly mimic the framework used for testing of Mesh modifiers. I would initially build the framework keeping “Transform” constraints in mind and would extend accordingly for “Tracking” and “Relationship”.
I have submitted two patches one for Deform modifiers (D6620), and the other for Simulate modifiers (D7017). D7017 consists of tests for Cloth and SoftBody, I will add tests for the remaining Physics modifiers.
As per the Academic calendar, the semester were supposed to end on 8th May but due to the Coronavirus outbreak, there is a degree of uncertainty as all colleges and universities are suspended. There wouldn't be much interference during the GSoC program.
If I finish Deliverable 1 and 2 together, I would like to work on the framework for both Bone Testing along with Compositor Testing. For date by date details:
Hey! I am Himanshi Kalra, a student of ABV-IIITM Gwalior, India. I am in my 3rd year pursuing a dual degree in Btech (IT) + MBA. I have been using Blender for more than 2 years now, my journey began by learning from Andrew Price’s Beginner Tutorial Series. I tried my luck at animation but I am better at modeling ;)
Although I started with Blender because it was open-source (read as free) but eventually I fell in love with it and wanted to contribute towards making it better. I familiarized myself with the codebase of Blender and submitted my first patch D5610. I fixed a few minor bugs (D5744, D5867), submitted a patch for testing deform modifiers (D6620), and am currently working on above-mentioned D7017 (Simulate modifiers test).
I have studied C, Data Structures and Algorithm Analysis as a part of my 1st year curriculum. Eventually coding in C++ for competitive programming and have learned Python for Open-Source and ML (Machine Learning).