Tuesday, 31 October 2017

Coming to Firestorm soon... A couple of new features for builders and non-builders alike.

There is a new Firestorm slowly working its way through the pre-release testing process, there's lots of goodness in there that I'll not cover here but I'd like to highlight a couple of features that I have added for builders but which I feel will help other users as well.

Physics view while editing objects.

Regular Readers of my blog will know that I love the physics view in "Render Metadata" for its valuable debug capability. You may recall that I "fixed" a subtle bug in that which prevented us correctly debugging certain "thin wall" problems and that change will be part of the new release. The debug physics view has a couple of major drawbacks, the first is that switching to it is clumsy and slow, requiring that you have the developer menu enabled, the second is that everything gets coloured whether you like it or not, making it difficult to see what is going on.

To address this I have added the ability to show a physics outline for the object that you are editing.
The functionality is accessed from the features tab in the edit dialogue and is represented by an "eye" next to the physics selection combo box.

Clicking on the eye will toggle it on and off, showing a representation of the physics shape as shown in the example here.

Once this mode has been toggled on, it will remain enabled when editing subsequent object unless toggled off;  it will, however, revert to being off after the next login. This is deliberately done to prevent it confusing users who enable it in error and forget how to disable it.


Mesh information in the object panel.

When you edit an object you will be used to seeing information in the object panel. On the left-hand side, the properties common to all objects. The position, scale and rotation.

On the right-hand side details specific to the type, the hollow, cut and taper etc for prims, special features like dimple and hole size on a sphere or torus respectively. For sculpt maps you'd often see the rainbow hues of the sculpt map texture, but for Mesh, nothing. In fact, worse than nothing because you'd get some greyed out leftover fields.

In the new release, however, I have started to address this and have added a Mesh specific information panel that shows you a couple of extra details about the object and also allows you to artificially see what it will look like at a distance by selecting the Level Of Detail.


What does it all mean?

The new physics mode operates in a very similar manner to that given in this more technical blog from a few days ago. In essence, the shape tells you exactly where your avatar and other objects will collide with the object. The colour tells you when the physics "cost" is very high. 

The Mesh info panel tells you how detailed each of the levels of detail in the model are by listing the number of triangles used to make ir, it then allows you to preview the LOD models. 

OK, but really, what does it all mean? Why do I care, I don't even build stuff?

For non-builders there are a few notable benefits. 


Fix those pesky "stuck LODs"

Firestorm "LODv iew" FTW from Beq Janus on Vimeo.

Sometimes, perhaps you've been cam shopping, taking photographs or maybe even perving on your neighbours, then when you "snap back" to your room you find that your clothes are not rezzing properly. There is now a "simple" fix.


1) Find your inventory window and select the worn tab
2) right click the item, and select edit. 
3) Click the object tab, and "edit linked"
4) click the broken item
5) It magically repairs itself

But that's not simple enough I hear you say... We agree and so by the time the new Firestorm is ready for release we hope to have simplified this trick into a one-click fix in the menus courtesy of our lead developer Ansariel.

Why can't I rez on my mesh table/floor/bed.

You know the feeling, you are late to a party, you grab that awesome Gacha "rare" from the reseller on MP. Of course it comes boxed. So you quickly drag it on to the floor....your brain catches up, but it is too slow to stop and your finger releases.... "Can't rez here" says the little pop up. What's worse, your dress is gone and won;t come back until you relog, it may take longer. 

We've all done it. Rezzing items can be a little hit and miss at times but did you ever stop and wonder why? The answer is often simply "laziness". Your special outfit just got eaten by the Linden void because the designer of your house didn't give it a proper floor plane. But it is not just floors, chairs and all manner of objects that you might conceivably drop an item on weren't made with a proper physics shape and even though it tries it's best neither the viewer nor the sim can work out where to place your item and ultimately it gives up. Well now you can find out where the safe spots are.

The physics shape view let's you see how the simulator sees the items when it is trying to work out where to place things, where you walk and how things collide. 

This short video shows the problem with a simple table (deliberately badly made by me!)
Physics View Mini Tip from Beq Janus on Vimeo.

Why do I sink in this floor? Why do I keep falling off the stairs

Of course, physics is used to decide where you can and cannot walk, and even for the non-builder, sometimes it can help to understand why your feet pass through the stairs.


Friday, 27 October 2017

Blue sky thinking - Physics view explained

Introduction

I found I could say things with color and shapes that I couldn't say any other way — things I had no words for. - Georgia O'Keefe
In previous blogs and in various group conversations, I've often discussed the physics overlay renderer of the Second Life Viewers, typically found in the Developer menu under the, perhaps confusingly named, "Render Metadata" menu.
Physics view in all its glory
The Render Metadata menu is a wonderful set of tools that help debug various issues. You may recall the "LOD display" which tells you which Level Of Detail is being shown, for example. They are mostly unnecessary to the average SL user but occasionally, as builders, and educators/mentors, we find that a couple of them can help us a lot at certain times; and primary amongst these, to my mind, is the physics view.

The physics view is a great example of "metadata rendering"; the use of a visual overlay (render) to convey additional information (metadata). So what does it do? Well essentially, it overlays new objects into your view to communicate non-visual aspects of the scene. in this case the physics shapes, but..what does it all mean?

Where do we find it?

Before we get too far ahead, let's take a moment to find the menu. I will, of course, use Firestorm but the feature should exist in all graphical viewers that have any relation to the Linden Lab viewer.

You will need to have access to the developer menu
The simplest way to do this is via that excellent preferences search.
Ctrl-p will bring up the preferences dialogue. Then in the search box type "developer".
Tick the highlighted option and hit OK, you should now have the developer menu.

Now, simply click on the "Developer" menu and locate the "Render Metadata" item, and pick "Physics shapes" from the resultant list.


To turn it off again, just reselect the item from the list.

Decoding the view

When you click on the option, you are immediately plunged into a strange parallel dimension; what you see will naturally depend on where you are, and what is around you but it will almost certainly take on a very blue look. It is often worth moving to an area clear of clutter when testing this feature.

New rough approximations of your objects will now have been rendered; your meshes and prims will be blue, or various shades of orange and red, some will have pencil sketched triangles across their surfaces and peculiar coloured boxes will appear. Every part of this is potentially useful information if only you know how to read it. There are two basic representations, the colours and the shapes, we'll look at each in turn.

Colours

First of all, I will correct the received wisdom of forums and blogs. For a long time most people, myself included, had considered that the colourisation represents the physics weight of the object; however, that is not actually the case. In fact, the view displays colour based on the overall object cost, i.e. the inworld LI. The below image demonstrates this clearly, and no this is not new, or a change in behaviour, it was created like this in 2010 and the assumptions have been made ever since.



With a few exceptions then, the hue of the faces shown in physics view represents the object cost of that object as a colour spectrum. Red indicating an excessively heavy load on the viewer/physics engine, through to amber, and blue.


The mechanism by which these are coloured is not entirely straightforward. Firstly the cost is normalised along a logarithmic scale. By default the threshold used is 50, anything over that value is considered high. A plot of this normalisation looks like this.

Anything with a normalised weight of <0,5 (which as we can see from the graph is an actual weight of a little under 35) will be rendered as a mix between the Low colour (blue-green) and the Medium colour (orange). For anything heavier than those the colour is a mix between the Medium and the High (Red) thresholds, giving a range between orange and angry red.

Note: For those interested in the details, the mix is a linear interpolation of the RGBA values of the two thresholds. The colours and thresholds can be changed, and I will briefly cover this at the end of this blog (if I remember), but it should be considered as an advanced use only. So these are really telling us where our heavy physics costs are, especially useful in a multipart build where an unexpected legacy prim has become linked with Mesh and resulted in an unforeseen and undesired spike in the LI.

Shapes

The shapes are where this view excels. When you are trying to understand why your Mentee's mesh wall is not allowing you to pass through the doorway, a quick flick into "Physics view" can often tell you the answer. Even those of us who use them frequently often fail to realise is that there is more information in the display of these than first meets the eye.

The representation is similar for both prim and mesh; similar but not quite the same. Unlike a mesh, a prim has no user-defined physics shape, yet the effective physics shape can be quite complex. Consider a twisted, cut spiral cylinder, used as a staircase. The users expect the physics to work accurately. As a result, there are a couple of optimisations and a default case which can if misused, be a problem.

The way that a shape is shown is linked closely to the type of physics that it represents.

Four prims a cube, sphere, cylinder and torus with various parameters affecting their physics shape


Case #1 The HAVOK Physics shape.

For the general case of a unmodified prim, it will be assigned a highly efficient physics hull that closely matches its shape. There are three possible physics shapes, Box, sphere and cylinder. These shapes are a special case of "hulls" a physics shape used by the Havok engine. These provide for extremely efficient collision detection by simplifying the mathematics required to detect collisions.

An example that seems contrary to many who are used to thinking of mesh shapes, is that the cheapest, most efficient physics shape is a sphere. This is because collision detection of a perfect sphere needs only two values, the centre and the radius, any plane that passes closer than the radius, must be intersecting with the sphere.

Note that complex prims such as the Torus shown top right, do not have an efficient representation and simply fall through to the second category.

For Mesh there is an extension of the simple form, it is known as the base hull and is the shape that you interact with when "convex hull " is enabled. A base convex hull is defined for all meshes and for the most part, appears as if a "cling film" wrapping has been stretched over the object.

These shapes are typically rendered with a semi-transparent tone and no visible edges.

Case #2 almost all of the rest

If a prim cannot be represented by a Havok shape then it will be converted to a "simple" triangular mesh representation. The rules that dictate when a physics primitive is swapped for a mesh are a little too detailed for a blog like this. But with experimentation in Physics view, you can see that typically a cube that is hollowed will "switch" to mesh physics at about 20%. I tend to think about it in terms of "is the hole big enough for something to fall through" (though that is not the real test :-) ).

This is also the typical (not quite a default) case for Mesh physics. A Mesh object must have a physics decomposition provided. There are 3 types, the base hull discussed above which is the actual default. If nothing else is provided then the base convex hull will be used. The second type is a triangular mesh derived from one of the visible mesh models or uploaded as a dedicated mesh (I highly recommend the latter). The third is an analysed mesh, which we will cover later.

A triangular mesh physics shape can be very efficient it allows you to fully represent the shape, with any holes and other shapes. It is important however to be very economical with the mesh used, keep the resolution of the physics as coarse as possible, to improve performance and lower cost.

It is worth bearing in mind that it is a side-effect of the triangular mesh physics and the physics cost calculation that gives rise to the super-heavyweight prims in certain circumstances.

These shapes are typically rendered with a darker outline showing the mesh triangle edges. 

Case #3 rare and getting rarer

From time to time you may see a magenta wire-frame cube 50% the size of the prim. This is effectively an error report. It means that for one of a number of reasons the viewer cannot display the physics shape. With Prims there is typically only one reason for this, the prim has been forced to use Convex Hull and the user is running a viewer that does not have the capability to decode that hull into a visible object. At the time of writing it is quite common in 64-bit third-party viewers such as Firestorm to show all convex hulls (both for prim and mesh) as magenta boxes. 

Why does this affect 64-bit viewers?

The reason behind the limitation is straightforward once you know about it, the Havok physics engine used by second life is a commercial product, Linden Lab kindly provide a special library that supports a limited subset of Havok capabilities but until very recently, the Lab did not have a 64 bit version of their viewer and did not supply a 64-bit compatible version of the Havok Library. Almost all physics functions in Second Life are handled on the server, however, there are a handful of services required by the viewer and the library provides these. The one that most builders notice is the lack of options in the "analyse" menu on Mesh uploads, but another is the ability to convert a "convex hull" into a mesh shape that can then be rendered in physics view. Thus, to keep this short(ish) when you ask a 64-bit viewer to show a convex hull it says "I can't" by showing you a magenta wire-frame.

The next version of Firestorm will be fully Havok enabled and the vast majority of instances when the Magenta boxes appear will be banished to history.

Case #4 The very small object

This case is a safety net. As we observed in case #3, the equation for calculating the physics cost has an often unexpected side-effect for the uninitiated. The logic behind the calculation is clear enough, the Havok physics engine needs to do less work when it hs handling fewer shapes as is shown in this simple animation from a previous blog.
Smaller triangles more work. Which triangle did the sphere hit?

The earlier blog post gives more details on this and describes the conditions under which the Lab implement a cut-off to prevent the asymptotic behaviour. The mechanics we can leave aside, the result though, for our purposes is a simple box physics that wraps around the small objects, removing holes and details completely. For completeness here, I will repeat the conditions (but keep in mind that at the time of writing physics view will lie.)
IF object is a prim AND has more than 2 dimensions less than 0.2mTHEN force the shape to a convex box.ELSIF object is a mesh AND has one dimension less than 0.5mTHEN force the shape to a convex box   
These are shown without edges and are in some ways indistinguishable from (deliberate) hulls 

Case #5 Hulls analysis and fixed cost physics (not shown in image)

The fifth type of physics is an analysed hull based physics. To produce this the viewer takes a user-specified mesh physics shape and applies an algorithm to it to construct a series of 3d "capsules" called hulls. These are Havok primitives, optimised for the engine to process and can in some cases be more efficient than the equivalent triangular mesh. For reasons I have never fully investigated, the cost of an analysed mesh is frequently higher than the equivalent triangle based mesh, a problem exacerbated by the lack of Havok in 64-bit viewers. For this reason, alone people often avoid it and yet it has some very specific benefits. An analysed mesh has a fixed cost, it does not suffer from the increased physics cost when scaled down and it does not have the dimension based cap imposed so is immune to the "thin wall" problem.

These are deliberate hulls and are shown the same as #1 above (and #4)
There's some good news to come on physics views but for now, I've written more than enough.

See you soon

Love 

Beq

x






Friday, 20 October 2017

My life in lamps...

As I noted in my previous blog I finally got around to creating a set of items for sale. The Antique Radiators were the "sale item" for the Decennial event but I also needed a hunt item. The theme of the hunt is Timeless, the event and the hunt celebrate 10 years of Fallen Gods in Second Life, so I wanted something that also meant something in the theme of time and growth. The item I chose will be familiar to those who have known me for years or who have read my articles in Prim Perfect.

Back in 2007, I made myself a lamp, based on a rather lovely Arts and Crafts period design from the Roycroft movement. A simple but elegant design that could work as a low prim lamp for my undersea refuge in the Vernian Sea in New Babbage.






The very first version is shown here on the left. A gloriously extravagant 8 prims, 2 of which were a bizarrely flickering animated candle flame where the bulb should be. The rim texture cut from a photo of the lamp in RL and recoloured to look as though it was lit up.



As SL and I both grew, I rebuilt it in around 2008 using "the latest technology", reducing the 8 prim lamp to a glorious 3 prim sculpty based model (right).

The lamp was more or less identical in form to the prim version and created using an Inworld sculpting tool called SculptCrafter which in its way (despite being made by a different person) was a forerunner to tools like Mesh Studio, converting prims to sculpts, which it spat out in chat as a uuencoded text stream which you pasted into your preferred decoder to get the expected sculpt texture. A really neat solution to not needing an external service in fact.





Then in 2012, I was asked to write articles for Prim perfect, demonstrating that Mesh and prim building were not necessarily at odds, the articles built upon a blog post I wrote on inworld mesh creation for which I built the version on the left. The model was created using the excellent and highly under-rated Mesh Studio. The project was a little artificial I added heavy metal fluting to the sides of the stand giving it a more Art Deco feel than theo original RL piece having challenged myself (and the tools) to improve upon the 3LI sculpty in both looks and land impact. The lamp was also used to explain generated LOD models and how handmade LOD models are essential to really efficient and well-structured meshes.

The result was a credible (I think) demonstration of the tool, taking a detailed 110 prim model with all my metal fluting and additional details and producing an efficient 1LI Mesh with Strong LOD characteristics.

It's been five years since that project; we've seen the introduction of many new features, materials being perhaps top amongst them. And I have started to confront my personal daemons, tackling my insecurities around artistic ability to improve my texturing skills, face down the fear of Blender and it's mythically horrendous UI (hint: it really is not anything like as awful as people will lead you to believe, it just takes a little practice). And so it seemed fitting, that to celebrate 10 years of my friend Alia's wonderful creations, with a nod to my own 10 year anniversary too, that I should once again update the lamp.

The 2017 Roycroft style lamp is available as a hunt prize during the timeless hunt and may become available later from the Verdigris store, in a variant form.

At 1LI, we now have a lamp that retains much of the hammered effect detail of the Roycroft originals. The detailing employs Normal maps for those with Advanced Lighting, but also bakes and additional Ambient occlusion to the texture to give some of the effects to those without that feature.
Materials are used for the bakelite bulb holder as well as the metallic lamp itself, and the mica panels use the emissive mask feature of the diffuse texture to give a more realistic glow when the lamp is turned on. It was important of course that in an item of this scale it retained its shape and volume as the LODs decayed, which became a little war of attrition when the lamp nudging over the 1LI threshold.

The Decennial event and the excellent Timeless hunt are still open and I would encourage you to pay a visit.




Thursday, 5 October 2017

Verdigris by Beq - OMG I have a store!!

It has only taken me 10 years to get around to doing this; but I finally have a store, with a name and actual "stuff" in it. Well not much stuff, yet, but I'm working on it. It will ultimately live in my old factory build in Babbage Canals

So what brought about this monumental event? To be honest no one thing, I've been gathering steam towards this for a while, but all those normal excuses like Fantasy Faire, joining the Firestorm team, RL School and family etc have been a convenient cover for my laziness.

Then at the end of August, I got a notecard from Alia Baroque. Alia, as Faire lovers will know, is a crazy, wonderful designer who is best known for Fallen Gods skins, his awesome Fantasy Faire regions and err... Flamingoes. It turns out that Fallen Gods have been making skins for ten years now and Alia decided to celebrate as only Alia can, by promoting his friends through the Decennial shopping event and the Timeless hunt organised by Alia and his wonderful partner Sonya.


And so it was, with a little coaxing from Sonya and Alia I committed to taking a stand at the event and to participating in the hunt. This commitment threw me into my usual dilemma of confidence. "What on earth would I make that anyone else would want?"

In recent times a lot of my building, all of my building really, has been focussed around region scale designs with a limited mass market appeal. So after much soul searching and gnashing of teeth I fell back upon my early roots, making smaller items, for my own use, looking to produce artefacts from the Art Nouveau and more generally Victorian and Edwardian eras. The result was two items, the first was a prize for the timeless hunt, an item both brand new and yet completely familiar to those who know me. I'll talk about that in a separate blog as the hunt is still in its early days. The second item is the first official Verdigris product, a range of radiators based historic designs and presented in a number of material finishes.

Initially available only at the Decennial event itself these Radiators will later appear in my inworld store, and on Market Place, hopefully in time for winter.

Each radiator is mod/copy and thus can be scaled to your heart's content, It is 1LI at its standard size and remains below 5LI even when scaled to larger proportions.

You can find my stand and those of the many awesome creators that are celebrating Alia's ten year anniversary at Decennial


Bye for now, love 

Beq

X



Friday, 28 July 2017

A (con)Vexing problem - Fixing doors becomes a little easier.

Anyone who has been around Mesh building groups for any amount of time will be familiar with the anguished cries of "Why won't my door work?". One of the most annoying aspects of trouble shooting that problem is about to become a thing of the past.

I covered the "door no working" topic in this blog post last year. Those of us who advise/mentor know what to check for. The first thing is to make sure that they uploaded a physics shape. "Have you switched the physics shape to Prim?", they will typically do this and mostly it fixes things, but then there is still a reasonably significant proportion who return to say "I've done that and it's still not working."

The next step, typically, is to ask what the dimensions of the mesh are. Quite often the poor beleaguered builder will look quizzically at you, conclude you are mad and perhaps go off on some crackpot detour led astray by another helpful but ill-informed soul, they will often end up dropping into physics view and concluding that they are right and that the blocked door is a bug. They are half right, there is a bug, but it is not in the "blocked door".

Who debugs the debugger?

How to find physics view in Firestorm (shoudl work on most viewers)
When we are looking for issues with physics that go beyond the basic checks it is often the case that those who are at least aware of the advanced menus will take a look at the Render Metadata menu, and the Physics shapes view in particular.

Physics shape view is a special display that encodes a lot of information about the physics in the scene. You can use this to examine the physical representation of the region and in particular of the object that you are interested in.However, herein lies the issue. There is a bug, it has been around for a number of years and many of us have raised Jira bug reports against it in the past. With my more recent switch to become a developer on the Firestorm viewer, I was able to spend some time to work out why there was a bug here and what the underlying issue was. You can read the Jira BUG report I raised for the technical background. 

The case of the invisible wall

A typical example, perhaps the most typical example, is a creator who has uploaded a mesh wall with a door in it. The creator sets a custom physics shape in the uploader and remembers to set the phsyics type to "Prim" yet still the doorway refuses to let her pass. She concludes that she must have uploaded it badly and looks at the physics representation using physics shape view. 

This is what she sees.


"What the heck? The physics shape looks perfect why won't it let me through?"
The problem is that the viewer is lying to us. What it thinks the physics should look like is not in agreement with what the server is enforcing.

So what is going on and why is the viewer lying?

Size Matters (sometimes)

If you've read my previous blogs you'll know that there are two distinct ways of representing a physics shape (or physics volume to use a more correct term), hulls and triangles. The key difference is that a triangular physics shape (mesh) changes its value based upon the scale, and to make matters more confusing, it changes inversely with the object scale. this is counter intuitive to the uninformed.

Why does physics cost grow as the objects get smaller?

When you upload a mesh you can define the physics shape that will be applied to the object.
The default gives you no user defined physics at all, in this situation all that the object has when rezzed in world is the "convex hull" and "none" option. We are not going to concern ourselves with this today.

If you look at the physics tab of the upload dialogue you find a selector that allows you to choose a mesh model to be used for your physics.

This is a very important step and often overlooked by Mesh creators.
The Mesh I am using in the example here is one of my bugs from this year's Fantasy Faire. It has around 20000 triangles in the form shown (High LOD) and was designed to be very large (40m wingspan). It would undoubtedly be an awful choice for the physics shape. One of the basic rules of physics uploads is to keep things very simple, the reason is plain; if you want to reduce the lag on the sim and in your client then minimising the work that has to be down to determine when someone has collided with your object is essential.

So how does a collision get detected?

Whenever another object comes within the radius of my object then SL has to decide whether a collision has occurred, this also applies for rezzing and sitting on objects.
If I were to use the High LOD model as my physics shape then every one of those 20000+ triangles is potentially intersecting with a part of the other object. Special optimisations segment the space between objects to reduce the number of checks that need to be made but in the end, it resolves down to individual triangles. the smaller the triangles the more work has to be done to determine which, if any, it will intersect with and the higher the overhead on collision detection.

In the following animation ask yourself "Which of the triangles is the ball touching?"

Which Triangles is the ball touching?


In order to encourage well-constructed Mesh assets, the Second Life land impact algorithm includes a physics weight component. The algorithm itself is not fully public and the documentation incomplete and inaccurate but the principle is clear. The more "skinny" triangles in the object the more your physics cost will penalise you.

The obvious side-effect here is that as you make an object thinner in one dimension the triangles will become thinner and as such will increase the physics cost. The smaller it gets the higher the physics cost. In theory, as width tends to zero the physics cost tends to infinity. This is where Second Life steps in and takes control. In order to prevent the physics cost from going asymptotic Second Life has code that forces an object to have the physics of a simple box once its dimensions go below a certain limit and as many of you will have known from the outset, that is exactly what was affecting our little mesh door. The rule that the viewer enforces is as follows:-

IF object has more than 2 dimensions less than 0.2m
THEN force the shape to a convex box.

"Wait a minute, that's not right" I hear you cry, and yes you are right, our wall was more than 0.2m in all three dimensions. That's because the viewer does not actually enforce the phsyics. The server does and physics view, simply draws what the viewer "thinks" the phsyics is. But it is wrong.

The rule that the server enforces is different.

IF object is a prim AND has more than 2 dimensions less than 0.2m
THEN force the shape to a convex box.
ELSIF object is a mesh AND has one dimension less than 0.5m
THEN force the shape to a convex box  

so here is our answer, the server is far less tolerant of thin meshes than the viewer thinks causing the viewer to display the normal mesh shape when the server has already given up.

Making things right, or "you cannae change the laws of physics, captain"

The big question though is, which one is more correct?
The simple answer is the server rules are right because those are the ones that are enforced, but the 0.5m limit on walls is a pain in the rear, nobody wants thick walls and while analysed mesh is a solution, very few know about this and the second life wiki still recommends non-analysed!
For this reason, I have been lobbying for solution 2 on my Jira to be considered. Fingers crossed we may get the ability to make thinner Mesh walls.

In the meantime, I have updated Firestorm to correctly reflect the enforced laws of physics. This should greatly improve the use of the physics view as a debugging tool.

The following video shows the new changes in operation and now a blocked door looks blocked until the dimensions change. It should be in the next release of Firestorm.




Another overly long and exhaustively explanatory blog. I hope it explained a few aspects of physics and one of the (soon to be ex) annoyances.

Love

Beq
x








Sunday, 9 October 2016

Not as simple as it looks...

A little bad news

In my last post on "bug hunting", I talked about the LOD decay issue and a potential fix. I said in the blog that "subject to QA" it would be coming to you soon and sure enough QA found an issue. It turns out, rather a large issue given that as a result of my fix a lot of mesh textures get utterly corrupted. Needless to say, that patch was backed out. Thanks to Ansariel Hillier and Whirly Fizzle from the Firestorm development team for spotting and doing some initial troubleshooting of it for me while I was overseas.

It is not all bad news

The good news is that I have submitted a new patch that uses a completely different method. It is a method that I shied away from first time as I felt it was too close to the critical path, i.e. a part of the code that has significant impact on the overall performance, and chose to make changes far earlier. I am happier with the new solution, it is simpler, more elegant in some ways, and overall improves the performance ever so slightly by avoiding some convoluted checks when calculating the LOD for Mesh.

This patch has also highlighted the rather bizarre dependency that a mesh has on the legacy prim definitions and adds weight to my request for documentation on why this is done. Once I have cleared a few other loose ends up I may well circle round and have a look at this again.

On a related note. Complexity and confusion

With the recent introduction of complexity limits to the viewer has confused some people and highlighted certain bad practices in Mesh creation. However, due to a range of different "features" it is still not really as reliable as anyone would hope. Part of this is down to the fact the rigged mesh does not have its LOD calculated in the same way. 

In fact, for worn, rigged mesh the LOD is not based upon the radius of the object but is instead tied to the bounding box of the avatar that wears it. This is a quite deliberate choice, explicitly mentioned in the viewer code,  and means that if you wear a Mesh body and Mesh head and Mesh hands, that they ultimately LOD swap at the same time, rather than the hands decaying first (being the smallest) then the head and later the body but it also means that the complexity calculations need to consider this (which they do not at present). This means that those super high poly onion skin mesh head that melt down your graphics card in a busy sim, are only getting a complexity score based on the apparent scale and yet they are visible based on a much larger scale. Once you realise this...you can then read my Jira here because it is actually worse than that, rigged meshes are being given a far far easier ride that they deserve.

All this being said, it is what it is and changing this stuff is hard because it breaks things. So the Lab are looking at pulling together all the oddities in the LOD and complexity calculations and reviewing them to see what if anything can be done to make the complexity numbers more meaningful and usable. There is no timescale for this, just an intention to review things at the moment. I for one will be keeping a keen eye on progress.

Love 

Beq
x

Saturday, 1 October 2016

Bug hunting - Fixing an ancient LOD issue

Those of you who may have read my blog post in July, "Tell your friends - An old bug that people really ought to know about." might be interested to know that I have made a lot of progress towards fixing it.

TL;DR

The long-standing bug discussed in the blog (see link above) that impacts the way that certain Mesh objects decay their LODs has been identified and fixed. I will be submitting the patch to Firestorm and other TPVs where applicable and back to the Lab so that (if it is accepted) we can be rid of this pain in the posterior.

Introduction

After having suffered and grumbled at this bug for a long time I decided to bite the bullet and for the first time since about 2009, build and debug the viewer. After a week or so of digging around and working out how V3 viewers hang together, I now have a solution to the problem we observe, but it still needs to go through QA and of course the thing I cannot know...why did it do that? More of that later...

The basic problem

Here is the basic problem as we originally observed it.

Any mesh object with either 3 or 4 texture faces will crumple earlier than an identically sized mesh with only 2 texture faces.

But in fact it is worse than that (a little).

As I dug into this problem it turns out that this is not a problem that affects 3 and 4 materials only, it just affects them worse. Meshes with 5, 6, 7 & 8 material faces will also collapse earlier than the comparable 1 and 2 material versions. The following image will illustrate

The image shows 8 identically sized columns. Under normal circumstances, one would expect these to look identical and change LOD at the same distance from the viewer. The textual display above each shows the following:
Dist:                   The distance of the object from the viewer.
Biased Radius:   An adjusted radius based upon a biasing algorithm, the source of our woes.
Visual Radius:    The "True" Radius defined by the bounding box of the object.
LOD:                  The Level Of Detail currently shown. 3=HIGH, 2=MED, 1=LOW, 0=LOWEST

The display is a bug fix/enhancement of my own and if accepted will appear in a future viewer. It is a change to the existing Render Metadata -> LOD Info which is basically broken on all existing viewers. I should also note that I use the term radius here, because not only is it the term we use inworld when examining the LOD equations, but it is also used in the code, all of which is despite the fact that it is not the radius at all but the long diagonal of the bounding box!

As you can see, Objects 1 & 2 are still at LOD 3, even though their distance from the camera is marginally more than the others. Further scrutiny of the hovering figures shows that the Biased Radius is 5.22 compared to 4.35and 0.42. Objects 3 & 4 have collapsed to LOD 0, with a biased Radius of just 0.42 they had little hope of remaining visible. While all the others have decayed to a slightly withered LOD2.

But why does this happen? To understand this we need a little implementation detail.

What does a mesh look like on the inside?

SL is often criticised and even rubbished for the way it does things, but if I am really honest I have a great deal of admiration for the general architecture. For a system design 15 years ago it has managed to grow and adapt and shown remarkable durability. The code certainly bears many battle scars and the stretch marks of its adolescence glare an angry red under scrutiny but the fact that it has gone from super optimised prims through to industry standard Mesh, growing as and when the technology of its users was best able to adopt it is very impressive. 

Second Life has achieved this longevity through a series of "cunning plans" which have extended the capability without altering the infrastructure drastically. all the objects in your inventory have a top level structure which is basically a legacy prim, extensions have been variously grafted on to this but leave behind the traits of the original prims. This means that, even though they are unused, a mesh has a slice, taper and cut setting as well as many others.

These top level prims also denote what type of prim they are, cube, sphere, cone, etc. and Meshes are no different. The basic shape is determined through two parameters the PATH and the PROFILE. Thus a sphere has a PATH and PROFILE of CIRCLE, while a cylinder has a PATH of LINE and a PROFILE of CIRCLE. Sculpts came along later and are indicated by the presence of a sculpt parameter block on the end of the Prim. Perhaps surprisingly Mesh is denoted as a type of Sculpt with the "SculptType" is set to the value 5 representing Mesh.

This allows the "cunning plan" that the settings for a sculpt can be reused. In a traditional sculpted prim, the SculptID holds the asset server UUID of an image that defines the sculptmap. In a Mesh the same field is used to hold a UUID of the underlying Mesh. It is important to note here that the Mesh that you upload is given this UUID that is the "child" of the Mesh object. You never actually get to see or know the underlying Mesh asset ID inworld.

So we now know that a Mesh is really a legacy prim, denoted as a sculpt, whose map is redirected to a Mesh defintion. So let's see where it goes wrong.

LODScaleBias and the legacy impact.

My first task in trying to fix this bug was to start to map out the viewer. It has been at least 6 years since I last looked at the viewer code and back then I was only really building it for my own purposes. The code has all the hallmarks of mature and much patched and extended code and is a bit of a rat's nest at times, but nestled deep inside the nest is a function simply called calcLod()
This function, along with the name, also was home to the output for the Render Metadata->LOD Info function.

The Render Metadata services are a set of great tools for builders and developers who are trying to understand a problem, those who have read this blog in the past will be well aware of the physics display. The LOD Info display has been a bugbear of mine for some time, I have never been able to work out what it was displaying, It would show a number that would typically not change with the LOD display and was to all intents and purposes useless. It turns out that is exactly what it is. At some point in the past it appears to have been borrowed for some other purpose and upon examination had nothing to do with the LOD at all. The damning evidence was a commented out remnant of the original call. My first "fix" of the expedition was, therefore, to make this function more useful, the new display is shown on the left. I will be submitting that patch separately.

Back to our friend calcLOD().
I won't post the code here, it is too long but the function does what you would expect it to do given its name but the devil is in the detail.

BOOL LLVOVolume::calcLOD()
{
F32 radius;
F32 distance;

if (mDrawable->isState(LLDrawable::RIGGED))
{
// if this is rigged set the radius to that of the avatar              
}
else
{
distance = mDrawable->mDistanceWRTCamera;
radius = getVolume() ? getVolume()->mLODScaleBias.scaledVec(getScale()).length() : getScale().length();
}
.....etc etc
}

There are a couple of interesting diversions in this function, the first we covered above, the second is a special clause for rigged attachments which deliberately adjusts their LOD scale to be that of the avatar that is wearing them. This is the subject of a Jira and is likely to come under scrutiny in the current quest to improve complexity determination.

However it is the code in bold and further highlighted that we care about. What is this LODScaleBias? Our amended LODInfo display proves that this is the culprit. The BiasedRadius of a 3 face Mesh is shown on the left and can be compared to the same mesh with 6 material faces shown in the example above. 0.42 when the true radius is 8.7, no wonder the thing crumbles. 

Digging deeper we can identify where the LODScaleBias vector is initialised. 
BOOL LLVolume::generate(){
...snip...
    mLODScaleBias.setVec(0.5f, 0.5f, 0.5f);
...snip...        
    if (path_type == LL_PCODE_PATH_LINE && profile_type == LL_PCODE_PROFILE_CIRCLE)
    { //cylinders don't care about Z-Axis
        mLODScaleBias.setVec(0.6f, 0.6f, 0.0f);
    }
    else if (path_type == LL_PCODE_PATH_CIRCLE) 
    {    
        mLODScaleBias.setVec(0.6f, 0.6f, 0.6f);
    }

 ...
So here we have it.

"Cylinders don't care about Z-Axis"

The code above sets up the bias. The default bias is <0.5, 0.5, 0.5> and I'm feeling rather stupid now because having said previously that Radius is not really the radius...if you take the long diagonal and half it then of course you do have the radius (the radius of a sphere that encloses the bounding box, at least.) We then get to the code in bold red. Here we find that if the legacy prim has a linear path and a circular profile then it must be a cylinder, 
The image to the left shows my hand drawn annotation of what those two parameters mean. Anyone who worked with prims will most likely understand the terms.

This does pose a couple of questions, the most obvious of which is:-
"Cylinders don't care about Z-Axis" WHY!!!!?

There seems no logic to explain why a cylinder would be set to LOD quicker. Clearly, when used as a column it results in a high number of long thin triangles but does that really warrant such punishment? I have enquired with a couple of Lindens to see if we can get some clarification on the history of this.

Noting the <0.6, 0.6, 0.0> when applied to our example mesh columns give a Radius of 0.42 we can confirm that this is , as had been suspected, how out poor Meshes are being evaluated, and so the second most obvious question is:-
Why is my Mesh arbitrarily being branded as a cylinder? 
Again there seems no rhyme nor reason to the 3 and 4 material face meshes being treated this way. If the lab responds with an answer to either of these I will post a blog to share the info.

Having determined why we have this issue we need to go and find out where. At first, this seemed a daunting task. Somewhere in all the viewer code was a single line or two that was initialising these parameters incorrectly. I decided to start at the very beginning. The beginning for any asset is when it gets sent from the server to the client, a little hunting and we find a function that is called to process and unpack an update message for an object. In this code, I found the point at which the parameters are unpacked and placed some additional logging to print out the settings. 

Lo and behold the viewer is not to blame at all. The Object is already tainted before it arrives. This means that something is happening on the server and it would seem to be deliberate. 

How can we assume it is on the server? 

We have in previous blogs examined the Mesh Asset upload format and can note that there is no room in there for the legacy parameters. Moreover, that asset is the data that is referenced as the "SculptId". The Containing/parent prim is different, it is created on the server side, presumably during the validation of the upload process, the initilisation of the parent object must be assigning default values based on certain consistent criteria and as such results in the problem. As with the above, I have asked the lab whether they can confirm the reason for this, primarily so that we can understand if there are any side effects.

Having noted that Meshes are already tainted I added code to list out the types of Mesh and using a conveniently empty sim on the beta grid Aditi I created my series of 8 "identical" meshes.
the result can be summarised as follows.

# faces
PATH
PROFILE
BIAS
1
CIRCLE
CIRCLE_HALF
<0.6,0.6,0.6>
2
CIRCLE
CIRCLE
<0.6,0.6,0.6>
3
LINE
CIRCLE
<0.6,0.6,0.0>
4
LINE
CIRCLE
<0.6,0.6,0.0>
5
LINE
EQUALTRI
<0.5,0.5,0.5>
6
LINE
SQUARE
<0.5,0.5,0.5>
7
LINE
SQUARE
<0.5,0.5,0.5>
8
LINE
SQUARE
<0.5,0.5,0.5>

That is the end really. With the fix in place the Meshes quickly resolve and the new LOD Info display confirms that the Bias is no longer unfairly having some meshes. As for side-effects, we only modify at run time, and nothing is ever saved back to the server. Moreover I have implemented this to be configurable and should any issues arise it could be easily disabled. 

So what next? 

I am no cleaning up the code to remove or at lesat comment out any of the debug logging I used. I will then create a submit a patch to Firestorm. Having spoken to Oz Linden, I have been asked to sign a contribution agreement, this is a form that protects the Lab (and thus all of us) from me giving code and then claiming some licensing later.Once I have that in place the lab can accept my change and would then consider it. So that means that subject to QA and testing to follow hopefully we can put this bug to rest once and for all. 

It leaves a few loose ends. 

Why does a Cylinder ignore the Z? I just want to know.
Why does the server do this and will/should the server-side get fixed?
Would fixing this on the server make a difference to the SL Map

That's all for now, I shall leave you with an animation of the FIX in action,

Love

Beq
x

Labels