MOSAIC feature ramblings - part 8

Part 8 - The Groups Setup tab:

The "Groups Setup" tab allows you to use code fragments and atmosphere shaders with groups of objects instead of just one at a time. This can prove especially helpful when dealing with large and complex scenes with lots of custom fragment work. The exporter achieves groups in RenderMan by wrapping every object in a group in a single AttributeBegin/End block, allowing custom code to be entered at the beginning or end of the block. This ensures that any custom code entered in the block is pushed and popped off the render state so they will not affect anything else outside of the group.

The exporter handles objects in groups in a very simplistic manner, it cycles through each group exporting each object in the group only once. This means that multiple or nested group setups are ignored and each object is only used in the first group it belongs to.

Also note that lamps must be declared before all geometry (unless dealing with area light geometry), so lamps will not be apart of any groups (however lights can still be put in groups for the material GR: control).

The group selection menu will only recognize groups that have something in them (empty groups are ignored), and groups cannot use RIBsets (due to some internal complexities). The "Group Block" archive menu has options unique only to groups. By default this option is set to "Ignore Group" for all new groups and must be set to "Attribute Block" before the group will be exported (although the objects in the group will still export but not as apart of a group).

Let's illustrate a simple example of using groups for scene setup. Let's say you're browsing the 3Delight docs and see a SSS example you want to try.

The example shows some RIB source using an attribute like this:

Attribute "subsurface" "scattering" [ 2.19 2.62 3.00 ] "absorption" [ 0.0021 0.0041 0.0071 ] "refractionindex" 1.5 "shadingrate" 16 "scale" 0.2

You read that this attribute simply needs to be placed before any geometry you want to use SSS. To do this in MOSAIC you would simply create a new text file and name it something like "cf_sss" and copy the code in it, then select it in the "Object Begin Code" menu in the "Geometry Setup" tab for the object you want to use SSS, simple! Now let's say you like the effect but want to experiment with 30 objects switching SSS on and off them individually, but you don't want to have to turn "cf_sss" code fragment on and off each one individually. This is where groups come in. You can assign the objects to a group and then attach the "cf_sss" to that group's "Group Begin Code", using Blender to add and remove objects from group without ever having to touch MOSAIC making experimenting much faster :)

Now let's outline the controls for this tab:

Navigation management:

Here's a breakdown of the controls in this group:

• Select Group -This will show a list of available groups for selection, once selected all controls in this tab will update.
  

Code management:
This group of controls is for attaching user created code fragments (text file with the "cf_" prefix in their names) to the beginning and end of different RIB blocks. As with the previous group, this group of settings is available in various configurations for all the remaining tabs.

Here's a breakdown of the controls in this group:

• Group Block - This menu allows you to specify whether to export the RIB code into an attribute block or completely ignore the group.


• Group Begin Code - This allows the insertion of custom code fragments into the beginning of the group attribute block before anything MOSAIC exports.


• Group End Code - This allows the insertion of custom code fragments into the end of the group attribute block after anything MOSAIC exports.
  

Shader management:
This group of settings is available in various configurations for all the remaining tabs and is for selecting shaders for the selection.

Here's a breakdown of the controls in this group:

• Atmosphere Shader - This will select an atmosphere (fog) shader that will only affect the objects in this group.


• Notes - This will display a popup of any notes attached to shader fragment by the author (usually containing information about which Blender controls are used by this shader).
  

Well, that's it for the groups tab... next up is the "Geometry Setup" tab :)

Thanks for reading, WHiTeRaBBiT

MOSAIC feature ramblings - part 3

Part 3 - The MOSAIC Settings tab and shader libraries:

The "MOSAIC Settings" tab gives you a wide range of settings for defining exactly what and how MOSAIC exports. It allows you precise control over which RenderMan binaries are called and with which parameters, which RiSpec elements are exported in the RIBs, which default display codes are used, which file extensions are recognized, where projects are exported, it even allows you to change the prefixes used to recognize code fragments internally. Everything set in this tab is saved to the Blender registry so they are retained for all new projects.

To help make setting up MOSAIC for the most common RenderMan systems easy, there's also a preset menu at the top of the tab that automatically fills-in all controls based on your selection.

Before beginning the outline of the controls I also want to describe shaders and shader libraries. RenderMan shaders are simply text files written in a C like programming language that describe light, surface, displacement, volume and image shading to the renderer. Before a renderer can use these source files they have to be compiled into something the renderer understands.

To use shaders in your project you'll either need to load shader source files into Blender's text editor (MOSAIC will automatically see and compile these files) or put the shader sources and/or pre-compiled shaders in a folder and add a shader library to point to it the "Add Shader Library" at the bottom of this tab allows you to do this and will do a couple of things:

• It will add that path to the "searchpath" Option in the exported RIB files making all compiled shaders in that path accessible to all RIBs.


• It will automatically list any compiled shaders in that path in the "Generate Shader Fragment" utility.

To create a library just click the "Add Shader Library" button and then navigate your file system to the path that contains your shaders. When done MOSAIC will add each library to the bottom of the tab. Once you create a shader library you will notice several things from the interface:

• You can enable and disable each library, allowing you to limit which shaders are available to the "Generate Shader Fragment" utility (if you group your shaders in folders by type then this can be used to limit selections to certain shader types).


• You can compile all shader sources in this path. This is extremely helpful when dealing with large numbers of shader sources. A great way to get started using this quickly is to add libraries for each folder in the "shaders" folder that comes with MOSAIC and compile each one and you'll have a whole slew a new shaders to play with :)


• The last control lets you delete each shader library entry.
  
  Note : This only deletes MOSAIC's reference to the shaders,
  not the shaders or paths themselves.

Now that we have an idea about what this tab does let's outline what each control does.

Renderer Presets:
This allows you to select which renderer you want to automatically setup.

Renderer Binary:
This allows you to specify how you want the renderer to be called.

Note : You can use the <RIB> token to tell MOSAIC to insert the RIB file call in-between custom parameters otherwise the RIB file is appended to the end.

Shader Compiler:
This allows you to specify how you want shaders to be compiled.

Note : You can use the <SHADER> token to tell MOSAIC to insert the shader file call in-between custom parameters otherwise the shader file is appended to the end.

Texture Optimizer:
This allows you to specify how you want textures to be optimized.

Note : You can use the <TEXIN> and <TEXOUT> tokens to tell MOSAIC to insert the texture file calls in-between custom parameters otherwise the input and output files are appended to the end.

LatEnv Optimizer:
This is the same as above but allows you to specify special optimizing commands for optimizing latitude/longitude maps.

Shader Info:
This allows you to specify how shader info is called. This calls a special binary that comes with most RenderMan system that will display verbose shader parameters on the console when passed a compiled shader. MOSAIC's "Generate Shader Fragment" uses this utility to generate shader fragments automatically.

Note : The fragment utility only understands BMRT style info output, the default for most info utilities but some require special parameters to produce this kind of output.

Shader Extension:
Which extension does this renderer use for compiled shaders?

RIB File Extension:
Which extension does this renderer use for RIB files (usually always .rib)?

Texture Extension:
Which extension does this renderer use for optimized maps (usually always .tx)?

Fragment Filters:
This allows you to change which prefix MOSAIC uses to identify text fragments with. You can use this to change each of the 8 fragment types individually (if maybe an existing prefix interferes with your naming conventions). In practice you should probably never mess with these unless in an emergency.

File Display:
This allows you to specify the default file display code used for creating new scenes.

Note : You can use the <###> token for inserting frame numbers in file names and also add a '#' to the beginning of the code to disable this in the scene's display list.

Frame Display:
This allows you to specify the default framebuffer display code used for creating new scenes.

Note : You can use the <###> token for inserting frame numbers in file names and also add a '#' to the beginning of the code to disable this in the scene's display list.

Depth Display:
This allows you to specify the depth file display code used for creating new shadow maps.

Note : You can use the <###> token for inserting frame numbers in file names and also add a '#' to the beginning of the code to tell MOSAIC this renderer does not support shadow maps.

Append Display:
This allows you to specify the appended depth file display code used for creating new occlusion maps.

Note : You can use the <###> token for inserting frame numbers in file names and also add a '#' to the beginning of the code to tell MOSAIC this renderer does not support occlusion maps.

Deep Display:
This allows you to specify the deep depth file display code used for creating new deep shadow maps.

Note : You can use the <###> token for inserting frame numbers in file names and also add a '#' to the beginning of the code to tell MOSAIC this renderer does not support deep shadow maps.


Exporter Switches:
This will popup a dialog box full of toggles that control the exporter states for export of all RIB structures and features. This can allow you to almost completely disable all internal RIB structures making it possible to get MOSAIC to work with even "loosely" compliant renderers.

Embed Above Settings In Project:
This very handy feature lets you embed all the above settings directly into the current blend file instead of in the local Blender registry. This makes it possible to distribute your MOSAIC settings along with the blend file. Once enabled any changes made to the above settings will only effect the embedded registry and will not effect the Blender registry. Once un-toggled, the embedded registry is destroyed and the Blender registry is used restored.

Export Directory:
This is the directory that all MOSAIC projects will be exported into, default is "/tmp" just like Blender's export directory.

Add Shader Library:
This will add multiple shader library paths to aid in shader management (see above description for more details).

Well that's it for this rambling, next post will focus on the "Project Setup" tab as well as discuss how render passes work in MOSAIC.

Thanks for reading, WHiTeRaBBiT

MOSAIC feature ramblings - part 2

Part 2 - The MOSAIC Actions tab and export process:

The "MOSAIC Actions" tab is the first tab in the interface (expanded by default) and is designed for quick access to the most common export related actions. It consists of several action buttons as well as a series of toggles for quick control of the export process. These toggles are meant to speed up exporting by controlling what gets exported as well as aid in building RIB packages for external use. As such the toggle states are only temporary and are not saved in registry nor do they control internal exporter states (those can be controlled in the MOSAIC Settings tab).

Now that I've defined what this tab is for let's talk a little about how MOSAIC exports. RenderMan compliant renderers read text files that use the .rib extension that contain RiSpec descriptions of the scene, objects, lights, meshes, materials, etc. The exporter creates these RIB files by cycling through everything in the project (datablocks) and translating them into text according to their MOSAIC settings (datablock properties), using defaults if there are no settings. This works very well in that the exporter doesn't need to maintain the entire scene in memory separately but instead just reads it from Blender.

These text files are exported in the folder specified by the export directory button in the MOSAIC Settings tab and the project folder specified in the Project Setup tab. The default export directory is "/tmp/" (like Blender) and the default project folder is "Mosaic", so the files will be exported to "/tmp/Mosaic". If the folder does not exist it will be created, also note that all projects will export to the export directory but each project can have a different project folder so its possible to export and render multiple projects at the same time (if you give them different project folders). The exporter will then create several folders in the project folder:

• Project Folder - contains main RIBs, renders and batch/shell scripts


• Archives - contains all RIB archives


• Cache - contains temporary export data such as depth files and shader info


• Maps - contains textures, optimized maps, and generated maps


• Shaders - contains shader source and compiled shaders

The Archives folder also contains the following folders which are self explanatory:

• Cameras


• Geometry


• Lights


• Materials


• Objects


• Scenes

RIB files have a feature called read archive that allows one RIB to call another RIB, by default the project is setup to use read archive (which is why the project is broken down into several folders). The basic RIB flow is: the project starts as a main RIB in the project folder which calls a scene RIB in the Archives/Scenes folder which in turn calls RIBs from the other folders as needed. MOSAIC attempts to use instancing through this mechanism to minimize the number of necessary RIBs as much as possible. It's possible to use other archiving methods by selecting "Read Archive", "Delayed Archive", "Instance Object" and "Inline Code" where available (see the RiSpec for differences between these). The "Inline Code" selection is used to tell MOSAIC to "inline" the code in the next connecting RIB in the chain, for instance if the entire project is set to "inline" then everything will be exported into the main RIB. These archiving methods can be mixed however you like :)

Now that we know the basics of where and how the project is exported to RIB let's outline how the action controls effect the export process.

(R)ender Current Frame/Passes:
This will render the current frame as well as any render passes using the current frame.

Render (A)nimation/Passes:
This will render an animation using all frame ranges to all passes or Blender's Sta to End if no passes exist.

(P)review Selected Object/s:
This handy feature will only render the current selections from the same perspective as your last used Blender 3D view window using all scenes, lights and any data already created for scene (such as shadow maps, occlusion maps, photon maps, brick maps, etc). This is great for zooming in or looking behind an object for tweaking independently.

Purge Export Project Folder:
This will completely remove the project folder and everything in it effectively "cleaning" it. This is handy for preparing a RIB package (where you don't want anything unnecessary in it), or purging any potentially unused RIB files (from changing object names etc). This is also helpful because sometimes the exporter will get confused when old unused RIB files begin to clutter the project folder.

Note : MS Windows likes to lock folders for no apparent reason, so if you get an error it usually means that everything in the project folder is removed but the project folder itself wasn't (which is fine).

Render RIB Files:
Call the renderer to render the RIB files after export.

Generate RIB Files:
Generate the RIB files to render, this is handy if you already have the RIB files or you're tweaking them by hand and just want to render them.

Export Render Passes:
Whether or not to include the render passes during export. This is handy if you've already exported the passes in an animation but only want to export the current scene.

Update Auto Passes:
If on, MOSAIC will automatically rebuild auto pass scenes using their filters before exporting, this insures that changes to the beauty scene will be updated in auto map scenes. It's sometimes handy to turn this off if you don't want the content of an auto map to update.

Export SL:
Lets you controls whether .sl and .h shader files in Blender's text editor are exported.

Compile SL:
Do you want to compile any shader sources exported ? This is handy when you just want the sources for a clean RIB package or have already compiled the shader and don't want to do it every time.

Export Maps:
Do you want to export any images in Blender's image editor ? Which images are exported and how they are optimized can be controlled form the Project Setup tab.

Optimize Maps:
Do you want to optimize the exported images ? This is handy when you just want the sources for a clean RIB package or have already optimized the images and don't want to do it every time.

Only Select:
This feature tells the exporter to only generate RIB for the selected objects. This is primarily meant to exclude complex objects that have already been exported that may be very slow to export. No matter what's selected the underlying scene RIBs as well as lights and material RIBs are always exported.

gzip RIBs:
This simply compresses the text RIB files with GZip. Many RenderMan renderers automatically support GZip compressed file input, this can greatly reduce project file size which is very important for large projects rendered over a farm.

Generate Log:
This will generate stdout and stderr output logs called OUTPUTLOG.TXT and ERRORLOG.TXT for the renderer, shader compiler and texture optimizer. These logs will be placed in the project folder for the renderer, Shaders folder for compiler, and Maps folder for the optimizer. This is of course handy for debugging problems from the command line or sending error logs to developers.

Batch Scripts:
This is mainly intended to aid in distributing RIB packages. Depending on which kind of scene you're exporting you can end up with a lot of main RIBs and it becomes impossible to know which ones to render in what order for the passes. This generates a Windows .bat batch script and a Unix .sh shell script that can automatically render the RIBs in the right order.

(H)elp:
Starts Blender's script help browser (just has a few MOSAIC web sites for online references).

(Q)uit:
I think you can guess this one ;)

One last note, a very handy benefit of using RenderMan for your projects is RIB's universality and the availability of tools and information. This allows you to package your scenes for distribution for many purposes, such as sending a test project for a bug report or analysis, or packaging a scene to be sent to a render farm, or testing a scene under a wide variety of renderers and features, or even sending a project for help on what you're doing wrong. If you send the blend file only then only a Blender user can communicate with you, but if you send RIB packages then many people can communicate with you (including Blender users). To create a fresh RIB package for universal distribution the process would be:

• Be sure that any shaders the project needs are either available to where you're sending the RIB package or include their sources in Blender so they'll be exported.


• Click "Purge Export Project Folder" to completely clear any existing data


• Make sure to ENable "Generate RIB Files", "Export Render Passes", "Update Auto Passes", "Export SL", "Export Maps", "Generate Log" and "Batch Scripts".


• Make sure to DISable "Render RIB Files", "Compile SL", "Optimize Map".

Once exported compress the project folder and distribute the archive anywhere you need.

Phew, this was a long one... next post will be about the settings tab and how MOSAIC deals with render presets, export toggles and shader libraries.

Thanks for reading,
WHiTeRaBBiT

Another update in MOSAIC's recent progress

Greetings everyone!

Just thought I'd post an update on several things just added to MOSAIC :)

After a few conversations with Syoyo and Tenzin as well as questions asked over the last couple of months, I have added more direct control of some of MOSAIC's internal exporter. In particular there is now a dialog in the "MOSAIC settings" tab that has a series of toggles for turning on/off some of the structural RIB export stuff. At this point the only things that can't be turned off in export are RiFormat and RiAttributeBegin/End which I think is the bare minimum valid RIB you could have. Also these toggles are tied into the preset system so they can be automatically set depending on what renderer is selected. Because of this I have added Angel, Gelato, jrMan and Lucille to the preset list. I tried adding PBRT and LuxRender but unfortunately they are using a heavily modified version of RIB that's not translatable (unless someone know about a translator out there).

I have Angel and jrMan setup like BMRT except for their render call so they can use BMRT's compiler and shader info.

Gelato can read standard RIB files (with a added module) but unfortunately uses its own GSL instead of RSL, If you use Gelato you'll need to translate MOSAIC's built in shaders with a utility you can get with Gelato or just use the shaders provided with Gelato. Also I have Gelato's shader extension added to the preset so you can include Gelato GSL source in Blender just like you can with RSL :)

Here's the Killeroo test scene (with shaders already translated) rendered with Gelato:



And here's the file if you want to play with it (warning 16meg file!)
http://www.dreamscapearts.com/Public/gelato_killeroo.blend

For comparison I decided to try and reproduce this scene in Blender's internal renderer. I used high res SDS with displacement modifier and normals. It was soo slow I ended up leaving the ground flat because it was too costly, this took 10.6 hours! I think this clearly shows the benefit of micropoly displacement :)



And here's the file if you want to play with it (warning 16meg file!)
http://www.dreamscapearts.com/Public/blender_killeroo.blend

And last but not least, because of the new exporter and preset controls I was able to get Lucille working in MOSAIC. I was having some problems with the meshes rendering properly but did manage to get this simple render ;)



Here's the project file
http://www.dreamscapearts.com/Public/lucille_killeroo.blend

I already have everything setup to render so hopefully this will help you with Lucille's development Syoyo ;)

Thats it, thanks for reading my rantings!!

>WHiTeRaBBiT<

Labels

Labels to apply, so posts can be searched by topic.

Blender to Renderman tools :

blenderman, neqsus, btor, blenderpixie, blenderlucille, ribkit, ribber, sler, svat, ribmosaic, blendergelato, blenderpbrt, luxblend

Renderers :

prman, bmrt, entropy, 3delight, rdc, air, angel, aqsis, pixie, lucille, gelato, pbrt, luxrender

RenderMan terms :

rib, rsl, shader, dso, shadeop

This post might be edited to add more.