User:HimanshiKalra/PythonTests

Introduction

The current framework supports tests for-

• Mesh Modifiers
  • Generate
  • Deform
  • Physics
• Curve Modifiers
• Operators

Note-The scope of these tests is limited to modifiers.

There are some exceptions and limitations as each modifier is a bit different, some requires to switch modes (Edit <--> Object), some are just random and non-reproducible, and some output an Empty Mesh, hence its difficult or not possible to test them.

Note The current framework support is only for "mesh" and "curve" modifiers, compositor testing is not supported, nor are the new object types like Volume.

Need for Automated Testing

Automated testing improves software quality. It detects bugs very early in the development process and makes fixing them very cheap. It can also detect unintentional changes. Downside is it is more code to maintain and increases building time (e.g. if tests are ran as part of building in buildbot). So it is important to keep them as short and as fast as possible to minimize false alarms and therefore minimize maintenance effort.

How to get started with Automated Testing?

Follow the starting instructions on this page which shows how to run python tests- Python Tests

You can skip the ctest part for now if its a bit too much and come back to this page.

Quick Steps in case you couldn't follow :

1. Start blender by typing blender in the command prompt.

Note: blender should be set in your environment variables in the Path variable, the Path consists of the path/to/blender where your blender.exe file is located. OR

1. Try typing blender.exe (method 1 is preferred).

Open the command prompt and the enter the following command.

Note- the path given here of the python test file is relative, you need to navigate using the cd command inside of blender-git/blender

   blender --background -noaudio --python tests/python/bl_pyapi_mathutils.py  -- --verbose

Or another example with absolute path would be

  blender -b c:\blender-git\lib\tests\modeling\modifiers.blend --python c:\blender-git\blender\tests\python\curve_modifiers.py -- --run-all-tests -- --verbose

The Output would look something like this if they pass:

Comparing...
Sucesss!

Addition of new tests

The current tests for Mesh Modifiers are located in tests/python/modifiers.py file. Navigate to this file and add another test for "Array" Modifier.

Steps to follow:

1. Navigate to the supporting blend files which are located in the lib\tests\ folder.
2. Open the file modifiers.blend which is located in the modeling folder.
3. Open the blend file and add 2 Cubes.
4. Rename them to "LearnTestCubeArray", and the second one "LearnExpCubeArray" respectively.
5. Now open the python file and add this line in the tests list

 "LearnTestingArray", "LearnTestCubeArray", "LearnExpCubeArray",
[ModifierSpec('array', 'ARRAY', {})]],

Note: To have a better understanding for each parameter, navigate to tests/python/modules/mesh_test.py and look at the comments in "ModifierSpec" class, this will give a clearer idea of what's possible.

Different parameters sent in the test instruction

1.  --verbose 

    : First set your environment variable. Type `set BLENDER_VERBOSE=1` in the command prompt (cmd). Note, it doesn't work for Windows Powershell.

2.  --run-test 

    : To run a single test for visualization and comparison purpose in Blender.

3.  --run-all-tests 

    : To run all tests.

Advanced Steps for Automated Testing

How Automated Testing works?

Good question, the trick is you don't have to manually add and apply the modifier in Blender at all, on the "LearnExpCubeArray" that is the expected object, you can but then that is doing it manually. So here are the steps-

1. After you add the test and expected object, no need to apply the modifier to the expected object.
2. Set the BLENDER_TEST_UPDATE flag to 1, and it will update the expected object for you.
3. There is a good chance that the test will fail for the first time, if you look closely at the console, then get updated and then should pass.
4. The file will get saved with the updated expected object, so the next time you run the test, it will pass in first go.

Note - Failing of a test doesn't necessarily mean its a bug.

Changing Parameters of the Added Tests

Turn on the Python tool tip in the Preferences in Blender. You can now hover the setting you want to change and look at the attribute name. For example, let's increase the count in the Array modifier. Then the code will look like this -

 "LearnTestingArray", "LearnTestCubeArray", "LearnExpCubeArray",
[ModifierSpec('array', 'ARRAY', {'count': 4})]],

Note- The setting name in Blender and actual setting might be different, that's where the tooltip will help you.

Points to remember

• Try to keep the meshes as simple as possible.
• Make sure the tests pass after the BLENDER_TEST_UPDATE flag is set to 0.

Future Scope

Adding tests for new modifiers and extending the framework for new object types.

FAQ

What to do my test is failing?

Use Ctrl+F and find a similar test, look closely at the comments and see if its a corner case.

Its still failing? Let's dig deeper...

For Windows: Set the BLENDER_TEST_UPDATE flag=1 in the environment variable, so now when you run the test, the "expected object" gets updated automatically, if all's good it should pass.

If it still fails, you can manually investigate by running a single test using the --run-test <test_name>, see the example below

blender c:\blender-git\lib\tests\modeling\modifiers.blend --python c:\blender-git\blender\tests\python\curve_modifiers.py -- --run-test CurveWeld -- --verbose

Here you will find a new object called "evaluated_object", you can try applying the modifiers to it and see if its the same as the expected object you added.

In case you are interesting in testing operators, they follow a similar approach.

What to do when getting errors in console?

Try slowly reading through it, it could be a typo in parameter name, so the exception thrown would be an AttributeError and some common programming mistake.

Where are the Physics modifiers' tests located?

Each Physics modifier has been given a separate python file and blend file, the python file name usually starts with physics_<modifier_name>, so you can test them as well, have a look at already added tests to see how to send the parameters, they use nested dictionary to send nested parameters. The supporting blend files are in physics folder.

Deform modifiers have a common blend file for Deform tests of both Mesh and Curves, it contains mostly the one that requires user input, which has been achieved by animating the object.