User:Ankitm/GSoC 2020/Daily Reports

Week 12
Weekly Summary User:Ankitm/GSoC_2020_Weekly_Reports

August 23

 * Read multiple lines with "\\n" line breaks.
 * Cleanup: Mark unused variables.
 * Silence mesh_validate verbose warnings.

August 22

 * Use `blender::StringRef` instead of `std::string_view`.
 * Use remove_prefix for easy transition to StringRef.

August 21

 * For polygons with holes:
 * Commit as it is ngon-tessellate.
 * Fix tessellate crash due to wrong Vector size.
 * Add newly creates faces to imported faces and dissolve edges.
 * Edit FaceElem list outside polygon creation code.
 * Move object creation code from constructor.
 * Remove unused uv vertex index offset.

August 19-20

 * Kept working on `ngon_tessellate` and adding the new polygons to the mesh.

August 18

 * Fix overlapping nodes in the nodetree.
 * Silence normal being zero warnings in mesh validate's verbose mode.
 * Cleanup: use const, remove single use variables.

Working on writing `ngon_tessellate` and `tessellate_polygon` in C++ for ngon polygons with holes in them.

August 17

 * Parse vertex normals: vn lines.
 * Decouple getting smooth groups from export parameters
 * Export vertex normals if smooth shading is enabled.
 * Move file open errors to constructors.
 * Add a public member function in MaterialWrap.

Week 11
Weekly summary: User:Ankitm/GSoC_2020_Weekly_Reports#Week_11_August_10_-_15

14 August

 * Apply axes transform to exportable Nurbs.
 * Work on separating material data collection and MTL writers. Fix bugs in MTL import too.

13 August

 * Fix usemtl line not being written.
 * Use function pointer; remove redundant variables.
 * Export packed image file names as if they're in current directory.
 * Use const; move constructors to implementation file.
 * Fix several material import bugs, remove wrong assert.

12 August

 * Fix several material import bugs, remove wrong assert..

11 August

 * Fix loose edge export & import, and offsets..
 * Fix vertex offsets for MEdges and MLoops..

10 August

 * Catch out of range exceptions for stoi/stof..
 * Add vertex offset due to Curves' vertices also..
 * Fix new Geometry creation with empty initial object..
 * Use const& instead of pointers in Mesh class..
 * Correct geometry creation code with Curves..
 * Accept 3D UV vertex coordinates.
 * Create new Geometry instance at the start..

Week 10
Weekly Summary User:Ankitm/GSoC_2020_Weekly_Reports

August 7
Minor cleanups
 * Remove deleted header from CMakeLists.txt..
 * Use pragma once as per style guide..
 * Fix crash due to OBJ files with no `o name` line..

August 6

 * Don't add null Image* to the p-BSDF node..
 * Avoid .data while creating a std::string from StringRef..
 * Replace std::string_view with StringRef..
 * Remove OBJ prefix from containers..
 * Cleanup: Renaming, comments..

August 5

 * Cleanup: clang-tidy warnings
 * Load image and add it to image texture node.
 * Add created material to the object.
 * Replace enum and its functions with Map for maps

August 4
Committed material creation prototype & fixed several issues later on.
 * Cleanup: Use MutableSpan, add asserts.
 * Initial material creation support..
 * MTL: release shader_output_ node instead of bsdf_..
 * MTL: use mtllib for importing MTL files..
 * Use Map's lookup instead of direct pointer to MTLMaterial.
 * Fix bsdf_ being null; asserts for SOCK_RGBA.

August 3
Holiday

Week 9
Weekly Summary: User:Ankitm/GSoC_2020_Weekly_Reports

July 30

 * Add MTL parser.
 * Off-topic (Master) Enable header auto-complete suggestions in Xcode. This turned into a standardised way of dealing with Xcode's include file paths.
 * Off-topic Respond to a new developer on a bug fix and the devtalk thread

July 29

 * Add fallback value in string to number conversions.
 * Tested some Xcode attributes used for header search paths with a final goal of enabling auto-complete suggestions,

July 28

 * Updated design doc: https://docs.google.com/document/d/17Uzl47OljjoKgaMbukiLHUVGQP220lPTmPS-atb65mw/edit

July 27

 * Discussed naming conventions with mentors.
 * Worked on MTL parser.
 * Off-topic: checked out ClangBuildAnalyzer on Blender (lite build).

Week 8
Weekly Summary: User:Ankitm/GSoC_2020_Weekly_Reports

July 24

 * Support importing vertex deform groups..
 * Keep vertex deform group names and def_nr in sync . To handle the same deform groups appearing in the file multiple times, a deduplication is required and order of occurrence of deform groups is also important. So use `std::vector` instead of `Map`.
 * Use VectorSet instead of vector for def_group. . The commit used `std::vector` which is slow due to linear searching. Jacques Lucke suggested using `VectorSet` for faster lookups. Also `StringRef` is used to avoid copying.
 * Fix curve not being visible in Object mode. . Thanks to Howard Trickey for debugging this!
 * Cleanup: silence clang-tidy warnings; comments..

July 23

 * Move Object creation code in Mesh creation class. . This helps adding deform groups and materials to the `Object` in one place and away from the code that adds that `Object` to a Collection.
 * Rename exportable_meshes to exportable_as_mesh . (This was long due.)
 * Refactor: Conform to style guide, add documentation..

July 22

 * Worked on Refactor: Conform to style guide, add documentation, which got committed on July 23rd..
 * Set active NURBS of a curve; set object group..
 * The NURBS bug mentioned yesterday is still there.

July 21

 * Support NURBS curve import . BUT the vertices are visible in edit mode, but the black spline is not visible in edit mode. Committed the rest of the code since it was working to some extent. Will try to fix it next day.
 * Fix build errors: avoid `uint` with Vector's `size` function. . Style guide was updated to discourage `uint` usage and `{}` initialisation doesn't allow narrowing conversions. There's still a lot of code in OBJ exporter that uses `uint`, that's for a Friday. This commit was only for removing `uint` with `Vector.size`.

July 20

 * Move vertex (and UV vertex) list outside `OBJRawObject` so that objects that are defined (`o curvename`) after the list of vertices is written can access them using relative indices.
 * Kept working on importing NURBS curve.

Week 7
Weekly Summary: User:Ankitm/GSoC_2020_Weekly_Reports

July 17

 * Use MutableSpan instead of raw pointers in string to float converter
 * Support meshes with edges with or without polygons.
 * Cleanup: Rename variables, use {} initialisation.

July 16

 * Importer: Support smooth shading for polygons.
 * Importer: Fix misaligned UV vertices of a mesh
 * Move stoi into a function; Add exception handling
 * Fix UV crashes and redundant loops in v1//vn1 case.
 * Exporter: fix vertex normals not being exported when smooth shading is enabled. Calculating smooth groups before exporting normals was the solution.
 * Exporter: Fix freeing invalid pointer in ~OBJMesh
 * Exporter: Fix UV indices being edited by the writer.

July 15

 * Add UV vertices to the mesh. . This needs some corrections though: the coordinates are correct, but UV edges are not matched to edges in the mesh. Also spent some time trying to find the problem of the assert: `(uniform), function GPU_batch_uniform_4fv,` it was present in has been fixed before.
 * Exporter: Fix UV vertex indices off by one.
 * Cleanup single use variable, comments, formatting
 * Responded to some build error and a bug fix help post on Devtalk.

July 14

 * Use stoi and stof instead of >> for faster parsing. . Time taken in string to float conversion is less now as compared to `>>` operator. But the cost is in splitting the string and allocations made for several substrings. Overall this is faster. OBJ_file_import_using_stoi.png
 * Parse UV coordinates and normals.
 * Directly create Mesh without using intermediate BMesh . It lowered the peak memory usage by half. Also made mesh creation faster.
 * Refactor: Keep mesh_from_raw_object in a class

July 13

 * Refactor: Keep mesh_from_raw_object in a class
 * Rename files: exporter -> ex, importer -> im. . It will be reverted and files moved into folders though.
 * Break code into mesh, objects, and reader files.

Week 6
Weekly Summary: User:Ankitm/GSoC_2020_Weekly_Reports

July 8, 9, 10

 * Parsed and stored the file contents in a temporary, OBJ-local `OBJRawObject` structure. It is not the Blender object, rather an OBJ object with the data from the OBJ file.
 * Successfully created a mesh in the viewport and added it to an Blender object. Also added the new Blender object to a collection in viewport.
 * So far the mesh has vertex data only. Texture coordinates and normals are not there yet.
 * Initial profiling shows ~70% time being spent in parsing and storing the data in `OBJRawObject`. Rest ~30% is spent in making a Blender object from a Raw one. So there's a huge room for optimization. See attached image.
 * Also set up clang-11 and its leak detector since clang-10 in Xcode and in  doesn't have the lsan library. Needed to edit and enable @rpath.

July 7

 * Started working on the importer. UI is already there, worked on file IO and looked at Jacques Lucke's code for the same. https://github.com/JacquesLucke/blender/compare/master...obj-import-experiments

July 6

 * Set up clang-tidy on macOS. Had to build llvm from source, and remove extra executables from PATH to avoid conflicts with other compilers.
 * Got carried away a little bit and experimented a little bit to get clang-tidy to work with Xcode, using a compilation database (JSON) with clang-tidy etc. Finally settled with using Ninja as the generator in a new folder, solely for checking clang-tidy warnings.

Week 5
Weekly Summary: User:Ankitm/GSoC_2020_Weekly_Reports

July 3
(It was like a code quality day for me.)
 * Lots of tiny changes:
 * Adhering to recently updated style guide Style_Guide/C_Cpp. Specifically, `_` in the end of private class members, combined namespace, reorder class members, use `const` wherever suitable.
 * Remove documentation comments from headers and keep only at implementation. Reduces duplication, and reading a class easier.
 * Break functions doing different things into smaller ones.
 * Better comments.

July 2

 * Fix : Skip writing image texture in MTL (`map_Kd` ) if image filepath is empty.
 * Support exporting smooth groups for smooth shaded meshes.
 * Worked on feedback received by mentors for evaluation. Google Doc. Since the comments were on commits not posted for review, and I don't receive comments on my branch commits, the doc was made.

July 1

 * Support vertex group to which a face element belongs.	. It took some followup commits to make it working properly.

June 30

 * Build error: Replace nodes_with_idname with nodes_by_type, namespace BKE > bke
 * Cleanup: simplified conditionals, remove warnings on linux
 * Use unique_ptr to avoid early destruction of new meshes.
 * Remove axis transform defines and use existing enums
 * Working on exporting vertex groups.

June 29

 * Null checks
 * Check if there is any material to write.
 * no image is present in the texture node
 * uv vertices with no UV map.

Week 4
Weekly Summary: User:Ankitm/GSoC_2020_Weekly_Reports

June 25

 * Refactored to separate `OBJMesh` and `OBJNurbs` classes in separate files.
 * Trying to keep MTL writer in C++ itself: wrote object's material name and material library filename in OBJ file, create MTL file.
 * Need to extract required properties from the Principled-BSDF node. If it is not present, default hardcoded values are being written.

June 24

 * Clean up OBJNurbs to be consistent with OBJMesh. Also, Nurbs surfaces are exported after being converted to mesh. Nurbs curves are exported in form of control points if "export curves as nurbs" is checked.
 * Fixed a bug: duplicated UV vertices and indices.
 * To reuse the MTL writer in export_obj.py : Testing with registering an operator in Python and calling it from C++, and also sending some strings to be operator upon.

June 23

 * Update in response to the review. Goal was to move more variables to private access in OBJMesh & OBJWriter and expose functions only.
 * Finished Nurbs support. Now just need to refactor it like OBJMesh.

June 22

 * Cleaned up and put up for review.
 * Worked on exporting curves as nurbs.
 * Replaced `std::vector` with `blender::Vector` proposed in

Week 3
Weekly Summary: User:Ankitm/GSoC_2020_Weekly_Reports

June 19

 * Worked on exporting curves as meshes.
 * Started working on refactor to reduce memory allocation and copying the mesh data into it. Rather at the time of writing, relevant info should be obtained on-the-fly.

June 18

 * Committed mesh triangulation feature.
 * Renamed occurrences of `objects` with `ob_mesh` or similar to indicate that mesh objects are being handled. It is necessary to differentiate it from curve objects.
 * Made changes in response to the review in

June 17

 * Made writing UV coordinates optional (default true). By extension, the faces are now written as `f v1//vn1 ...`
 * Face normals are optional too (default true). Faces are written as `f v1/vt1 ...`
 * Added scaling factor. (default 1.000)
 * Worked on triangulation of meshes. Need to fix a memory leak though.

June 16

 * Slow day today. Worked on axes transform (up/forward) and exposing it in the exporter preferences UI.

June 15

 * Allowed all frames to be exported, instead of 0-1000. Added animation checkbox to make single frame export intuitive.
 * Destination filename is edited only when multiple frames are being exported.
 * Found and installed XcodeClangFormat for easy formatting inside the IDE itself. (Off-topic: Plugins were deprecated in Xcode 8 and extensions are now the way to go.)

Week 2
Weekly Summary: User:Ankitm/GSoC_2020_Weekly_Reports

June 12

 * Finished work on multiple objects to be exported in the same file.
 * Need to test its accuracy. Will commit tomorrow.

June 11

 * Not a lot of work today unfortunately.
 * Tested {fmt::fprintf} library as compared to `fprintf`. Saw only 10-15% speed up.
 * Pushed work upto UV coordinates..
 * Fixed a out of bounds crash on windows in.

June 10
Todo
 * Finished exporting non-duplicate and exhaustive list of UV coordinates and also stored their correct indices in `Polygon`s.
 * Tested it by exporting with C++ exporter and importing it back with python importer and comparing the texture map with that of original object.
 * Minor review update in.
 * Export multiple objects in the same file, multiple frames (different files).
 * Axes transform.

June 9
Todo Next day `vt1 u v` `vt2 u v` `f */vt1/* */vt2/*`
 * My initial attempt at UV coords was wrong. The list was incomplete, since it was not iterating over the whole `me_eval->mloopuv` list; there were duplicates since a mesh vertex can be in multiple places in the texture map.
 * Asked about it in chat and found `BKE_mesh_uv_vert_map_create`. Took me a lot of time to adapt it to my requirements.
 * Ended the day at merely listing non-duplicated list of vertices.
 * Now that we have a list of all UV vertex coordinates, we need to match every 3D vertex of all polygons with that list. So todo is to add correct indices, `vt1`, `vt2`.. and store it in every `Polygon` member to export it like:

June 8

 * Updated soc-2020-io-performance to improve readability inside loops using iteration variables. Minor namespace cleanup. and
 * Merged those changes in review branch & created.
 * Searched for averaging function, to replace `face_normal_axis_component`, but only found some functions in cycles utilities. However calculating face normals using vertex position can be done using cross product of two edges. That function is there. Downside is wasted memory in `data_to_export->mvert[i].no`.
 * Worked on UV coordinates. Testing for correctness at the end of the day.

Week 1
Weekly summary: User:Ankitm/GSoC_2020_Weekly_Reports

June 5
In the table below, we're checking the time taken by `wm_handler_fileselect_do` function. Release build.
 * Correctly transformed vertex normals according to object's transformation matrix. Still need to incorporate axes transformation matrix.
 * Also calculated face normal from its vertices' normals inlined while writing them to the file. It's a dirty averaging function.
 * This removed the need of `BLI::Vector face_normal_index;`. Since `vn` and `f` lines are written by looping over the same `polygon`s, their indices would be the same.
 * This also ensures that the memory allocated in `MVert *mvert` is used better. We were discarding all `data_to_export->mvert[i].no` arrays. Now we don't. They're used to calculate face normals on the fly.
 * It'll be more clear when I'll send it for review.
 * Added object's name in the file.
 * Encountered non-freed memory while testing both of these exporters^. Since that has been fixed in master, most probably with the commits between 25th May & 28th May, I'll merge master once.
 * Did a comparison between the old & new exporters. Release configuration, lite build + opensubdiv, default cube with 6+3 subsurf. See images: F8586987 F8586984

Todo on Weekend:
 * Texture coordinates.
 * Try to add multiple objects. An object can be a made a class instance with `data_to_export` its struct. But we'll discuss.

June 4

 * Review update on and :
 * Renamed obj to wavefront_obj in filenames to remove confusion with blender's data types. Also renamed C++ headers to `.hh`. moved the code to `wavefront_obj/intern/` leaving only `wavefront_obj/IO_wavefront_obj.h` externally.
 * Improved comments in structs & functions.
 * arc diff wasn't the best tool for getting two sequential commits reviewed. So there was some branch juggling, diff files creation & resets. Now onwards, I'll stick to getting one change reviewed at a time → Merge it to soc branch & repeat.
 * Got familiar with Xcode's Address sanitizer. (also fixed a small heap buffer overflow).
 * Tomorrow: I'll be working on transforming vertex normals, _obtaining_ face normals from vertex normals, write object name to the file. _Writing_ face normals to file is a on halt waiting for

June 3

 * Finished major changes in `io/obj/*.cc` files.
 * Uploaded and  which will be committed to the soc-2020-io-performance branch after review.
 * Experimenting with "multiplying obmat and mvert.co while writing to the file" vs "multiplying earlier & keeping its copies" as per Howard's comment D7917#191581. But first I need to find appropriate method to transform normals. After data structure for polygons is finalised, I'd be able to fix its issue with wrong vertex_index.
 * Also updated project plan to give more time and spread to such refactoring and bug fixing. Squeezing PLY and STL.

June 2

 * Received review from mentors on the commit.
 * Worked on resolving the straightforward bits, renaming files (.h → .hh), writing comments & proper naming.
 * Will finish the changes on the next day for review before committing it. That involves finding how to transform face normals according to global transform, using `BLI_*` instead of STL, breaking up functions into smaller, specific ones.

June 1

 * Cleaned and prepared a commit for the work done in the past 4 days. See.
 * The UI blocks and dummy files were added in week 0. After that, OBJ Exporter now exports a single, selected object's, vertices, vertex normals, and faces. Need to verify it with complex geometries. The axes transformation (`Z_up`, `Y_forward`) is not there yet.
 * Tried to get ASan to work on macOS, but failed. It's not supported, see the docs. gperftools' heap-profiler & heap-checker work on simple standalone scripts, not in Blender. Instruments.app's leak detector crashes after detecting a leak that happens without even doing anything.