Download

Online

Gallery

Blog

  Index  | Recent Threads  | List Attachments  | Search
 Welcome Guest  |  Register  |  Login
Login Name  Password
 

Sweet Home 3D Forum



No member browsing this thread
Thread Status: Active
Total posts in this thread: 31
Posts: 31   Pages: 4   [ Previous Page | 1 2 3 4 | Next Page ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 26263 times and has 30 replies Next Thread
mazoola
Advanced Member
Member's Avatar

USA
Joined: Mar 25, 2015
Post Count: 153
Status: Offline
Reply to this Post  Reply with Quote 
Re: What am I doing wrong?

Hans -

That's surprising -- but good to know.

In my tests, I've found that oversampling the image has a major effect on reducing white spots. I've also found a default minimum AA of 1 tends to exacerbate the problem, as it seems the uber shader at 4 samples per pixel tends to introduce, rather than minimize, abrupt pixel-to-pixel transitions.

Why the difference? Well, it could be I'm not testing under real-world conditions. Cecilia's problem report started me thinking about the problem, but as I didn't have a scene handy that displayed white spots, I had to force them to appear by setting high shininess values and positioning the viewer for maximize reflections. In addition, there *is* a resolution-dependent component to the problem, and my test renders have been generally low-res. I've just started generating some higher-resolution tests, so there's a chance they may conflict with my earlier findings.

Finally, it very well could be there are multiple situations that may cause white spots, and the ones in your image result from a different sort of, well, infelicity than the ones in mine.

(Actually, that was next-to-finally. wink ) Finally, while there shouldn't be any difference in how SH3D performs under different OSes, there's always a chance the respective VMs vary slightly. While I doubt this is the case, the is at least one significant difference in the results we're getting: I have not been able to render any image with an antiAliasing.max greater than 4. Inevitably, my attempts end prematurely, with no error generated -- the same way they do if a scene contains an invalid model. (Actually, that's not entirely true: Earlier this morning, I did abort an attempt myself; I was running the 5.2 beta under Windows 10, attempting to render a test scene at 500 x 352 pixels with aa.min = 0 and aa.max = 5, and I cancelled the attempt after an hour of constant CPU utilization > 95% had yet to yield even a single bucket of output.) I've had similar [lack of] luck under both XP and Win10, attempting to render either my 'real' plan (currently ~270 Mb) or a test scene (~ 24 Mb) created specifically for this purpose -- so there is something about 1024 samples per pixel my PC doesn't like.

(BTW, I think you'll find 5 is the largest permitted value for aa.max; that is, while larger values are accepted, SunFlow quietly sets them down to 5.)

In any case, I believe I've found sufficiently odd behavior with 4-times oversampling (that is, an anti-aliasing value of 1) to recommend it not be used. (For instance, under some circumstances, I've seen reflections present at (aa = 0) and (aa = 2) completely disappear when (aa = 1). I've also found that with (aa = 1), SH3D/SunFlow sometimes has trouble determining which side of a two-surfaced plane is active. (For example, using Sketchup create, say, a cube. Apply one texture to the outside surfaces and a different texture to the inside. Export from Sketchup and import into SH3D. When rendered with (aa.max = 1), there is a good chance the interior texture will appear on at least one side of the cube. This will not occur at any other anti-aliasing level.)

Both of these issues appear to be problems with the SunFlow engine itself -- although I guess there conceivably could be a problem with the way Emmanuel converts SH3D objects into triangle meshes. I hope to have some sample scenes available in a day or two; assuming others can duplicate my findings, and given that development and maintenance of SunFlow ended several years ago, the obvious fix/workaround would be to change SH3D's default aa.min for Q4 rendering to something else. Frankly, I find SunFlow's threshold for triggering adaptive anti-aliasing -- that is, the amount by which a pixel must differ from its neighbors to trigger aa.max over-sampling -- to be conservative enough I would argue (aa.min = 0) to be perfectly acceptable. (Again, though, I need to run more tests at higher resolutions before I'm willing to go on record as having said that.)

I've also found one instance in which SH3D appears mistakenly to render what should be two identical scenes differently, which I believe to be an unintended side-effect of Emmanuel's attempts to resolve some of the earliest reports of white spots, which at the time appeared could be linked to use of the 'uber' shader for external renders. As a result, he appears to tweak rendering settings slightly based upon time-of-day. This can result in a scene that renders one way during 'local' daylight hours looking very different when rendered with a 'night' setting -- even if nothing else in the scene has changed and none of the scene's illumination comes from the virtual sun.

In any case, though, I realize I have quite a bit more testing still ahead of me before I can state any of this with conviction. Thank you greatly for ruining my hypothesis wink -- it only means we are getting closer to a solution. (Seriously, though, thanks for giving this a shot. I'm sure I'll have more to ask of you before I'm done....)

Maz
[Jan 27, 2016, 5:58:48 PM] Show Printable Version of Post    View Member Profile    Send Private Message [Link] Report threatening or abusive post: please login first  Go to top 
hansmex
Advanced Member
Member's Avatar

Netherlands
Joined: Sep 26, 2009
Post Count: 3948
Status: Online
Reply to this Post  Reply with Quote 
Re: What am I doing wrong?

Maz,

A few thoughts on hiQ rendering.

1 - I have suggested some time ago that the Q1 through Q4 default settings be changed to something more useful.
Q1 = as currently Q1 or Q2
Q2 = current Q3 or something close
Q3 = current Q4 or something close
Q4 = really hiQ settings

What plays in the background, is that the average computer has become quite a bit more powerful in the last few years. We could use that extra power to have better rendering without having to wait for days.

2 - Sunflow
Albeit long abandoned, I think it's still a powerful rendering engine. I remember reading (a long time ago?) that Emmanuel had plans to use a different rendering engine. I have no idea if that plan is still alive.

3 - Standard file
It would be a great idea to have a (few) standard file(s) that causes lots of white spots, has all kinds of transparency, etc, so we can all experiment with different settings on one and the same file.

My own test renderings were done under W10, with 6 cores, 8GB of memory, and the installed version of SH3D v5.2 beta1.

In a distant past, when photo rendering was introduced in SH3D, a lot of testing was done by several people using different settings and filters. I still may have a backup of some old SH3D versions made for testing purposes with different filters, such as Blackman-Harris and/or Mitchell. The current defaults in Q3 and Q4 are a result of testing done in that period (if they haven't been changed in the mean time). In old forum threads that info should still be available.

Hans
----------------------------------------
Hans

new website - under constuction
hansdirkse.info
[Jan 27, 2016, 8:39:49 PM] Show Printable Version of Post    View Member Profile    Send Private Message [Link] Report threatening or abusive post: please login first  Go to top 
mazoola
Advanced Member
Member's Avatar

USA
Joined: Mar 25, 2015
Post Count: 153
Status: Offline
Reply to this Post  Reply with Quote 
Re: What am I doing wrong?

Hans -

Thanks for the additional info and pointers. I'd actually spent quite some time last summer reading through the old posts on tweaking SunFlow, and more recently I've begun to dig into the source code for both SH3D and SF. (It's amazing what one can accomplish where there are unpleasant tasks to avoid!) I hadn't really thought much about the white spot problem, though, as it's been relatively uncommon in my scenes *except* when it appeared as a side effect of a broken model. (It took me a while to realize how big an issue Sketchup's 'two-faced' nature can be when importing SU models into SH3D -- *and* how many of the models from the 3DWarehouse display this problem.) However, once I started playing around with anti-aliasing and discovered the default Q4 setting could cause the program to fail at correctly identifying surface normals, I began to wonder if all of these symptoms might be related. Until your message, I was pretty sure they were -- now, I need to crawl back into my office and ruminate some more.

Thanks again,
Maz
[Jan 27, 2016, 10:18:26 PM] Show Printable Version of Post    View Member Profile    Send Private Message [Link] Report threatening or abusive post: please login first  Go to top 
hansmex
Advanced Member
Member's Avatar

Netherlands
Joined: Sep 26, 2009
Post Count: 3948
Status: Online
Reply to this Post  Reply with Quote 
Re: What am I doing wrong?

Maz,

I knew I had forgotten something: textures.

It's a long known problem that some models from the Sketchup website cause problems when rendering in SH3D. There has been a nice exchange on the forum on that issue as well.

There is a not too complex solution, namely to apply the texture to both sides of the affected faces. And if I can do that in Sketchup, than everybody can.

Also, under Options you should set the export settings to double faced.

H
----------------------------------------
Hans

new website - under constuction
hansdirkse.info
[Jan 27, 2016, 11:37:58 PM] Show Printable Version of Post    View Member Profile    Send Private Message [Link] Report threatening or abusive post: please login first  Go to top 
Puybaret
Expert
Member's Avatar

France
Joined: Nov 7, 2005
Post Count: 9141
Status: Offline
Reply to this Post  Reply with Quote 
Re: What am I doing wrong?

Thanks for all these tests and sharing your nice results.
As Mazoola mentioned it, the current SunFlow settings come from numerous tests done some time ago (thanks again to db4tech for the hours he spent on this). Meanwhile, a few bugs were fixed in SunFlow and maybe, it would be time to review again these settings, in particular how shininess is handled. The problem is that SunFlow isn't maintained anymore so I'm quite reluctant to invest too much in this time consuming task.
I also always wondered how fast SunFlow is running, compared to other free solutions like YafaRay, because another issue of using an unmaintained piece of software is that it won't be optimized in the future. Anyone tried to render the same scene with Sweet Home 3D and Blender for example?
----------------------------------------
Emmanuel Puybaret, Sweet Home 3D developer
[Jan 28, 2016, 8:46:52 AM] Show Printable Version of Post    View Member Profile    Send Private Message [Link] Report threatening or abusive post: please login first  Go to top 
Puybaret
Expert
Member's Avatar

France
Joined: Nov 7, 2005
Post Count: 9141
Status: Offline
Reply to this Post  Reply with Quote 
Re: What am I doing wrong?

under Options you should set the export settings to double faced.
From recent experiments, the double face option should be avoided when you export to OBJ format from Sketchup if you want to reuse your models in Blender. Such OBJ models can be imported correctly in Sweet Home 3D but Blender doesn't always import them correctly, reorienting some faces. Probably a bug in Blender OBJ import handler.
The double face option isn't a problem when the exported format is DAE/Collada.
----------------------------------------
Emmanuel Puybaret, Sweet Home 3D developer
[Jan 28, 2016, 4:53:34 PM] Show Printable Version of Post    View Member Profile    Send Private Message [Link] Report threatening or abusive post: please login first  Go to top 
Ceciliabr
Advanced Member
Member's Avatar

Denmark
Joined: Jul 7, 2013
Post Count: 619
Status: Offline
Reply to this Post  Reply with Quote 
Re: What am I doing wrong?

Sure, I will.

Cec
[Jan 29, 2016, 2:41:19 PM] Show Printable Version of Post    View Member Profile    Send Private Message [Link] Report threatening or abusive post: please login first  Go to top 
mazoola
Advanced Member
Member's Avatar

USA
Joined: Mar 25, 2015
Post Count: 153
Status: Offline
Reply to this Post  Reply with Quote 
Re: What am I doing wrong?

Hans -

I don't think I was very clear in my earlier comment. As I see it, there are two separate issues regarding the way SunFlow (and, thus, SH3D) handles surface normals in Sketchup-originated models. One is relatively trivial; the other, probably not.

1. Ambiguous meshes

The first is the problem to which Emmanuel alluded [um, in a comment I can no longer locate] a while back -- namely, when importing a model where different textures have been defined for either face of a surface, it's not always possible to determine the appropriate surface normal for a given triangle. As a result, the imported object can end up a patchwork quilt of triangles displaying one or the other face -- which can have a nice effect if one is rendering an actual quilt, but for other objects, not so nice.



This is the more trivial of the two issues -- in part because, as you note, an easy workaround exists (at least for models originating in Sketchup), but mainly because the scope and extent of the problem is understood: Only models with a specific design quirk are involved.

2. Unpredictable results

I'm more concerned with the other surface normal problem, where an object, when rendered a second time under different anti-aliasing settings, can end up with surfaces that display the supposedly hidden face.

As an example, these two images were rendered of the same scene using identical lighting, objects, camera position, and the like. The only difference is the image on the left was rendered with SH3D's default settings for highest quality, including a minimum anti-aliasing value of 1 and a maximum value of 2, while for the image on the right an aa.min of 0 and an aa.max of 4 were used.



Note the texture displayed for the tabletop. The image on the right is correct, displaying the cherry wood texture I applied to that portion of the model using SH3D's 'Materials' override. The image on the left, though, displays the model's original texture over most of the tabletop. (Fortunately, the two textures are oriented perpendicular to each other, making the shift from one to the other easier to see.) Incidentally, also note the areas displaying an incorrect surface normal also reflect light incorrectly: The large specular reflection in the foreground is missing, while secondary reflections of light initially reflecting off the chair back at the rear of the image and the frontmost wineglass, are clipped.

In this case, I see the surface normal failure as being more of a symptom than a self-contained problem. While the first issue could conceivably be a correct rendering of an incorrectly imported mesh, this second problem can only be caused by a bug in the SunFlow engine. Emmanuel doesn't pass objects to the SunFlow API differently based upon the amount of over-sampling; neither does SunFlow process them differently. Finally, differing rates of over-sampling should not cause SunFlow to read surface normals differently: If the mesh has been loaded as an incorrect patchwork of faces, the same incorrect patchwork should be displayed at any anti-aliiasing setting.

While in the case of Sketchup-originated objects this problem can be ameliorated by the same workaround provided for the first failure, I am less comfortable with accepting this as a true resolution, primarily because in this case the scope and extent are not definitively known. While the unreliable surface normal problem may be limited to models with the same sort of two-textured surface that triggered the first issue, the underlying bug could easily manifest itself in other ways under other conditions.

For instance, as the 'white spot' problem.

White spots

Admittedly, I don't know the white spot problem is caused by the same broken routine that causes the anti-aliasing-dependent surface normal issue, I believe there are strong enough suggestions of a link to merit additional investigation. Furthermore, based on the results of my tests, I believe it may be advisable to modify SH3D's default anti-aliasing parameters for highest quality (Q4) renders when using the 'uber' shader.

I first came across the anti-aliasing-dependent surface normal problem while investigating the relationship between over-sampling and the anomalous white spots that occasionally appear in Q$ rendering with SH3D. While this in and of itself does not prove a definite link between the two issues, I had several reasons to suspect some sort of involvement with SunFlow's anti-aliasing mechanism. In my experience:

  • White spots clearly are not, strictly speaking image artifacts -- for instance, they don't appear randomly (think noise or, in the hardware realm, a stuck pixel), nor do they appear without spatial relevance to underlying scene elements (as would, say, a JPEG compression artifact).
  • White spots larger than a pixel or two in diameter typically aren't literally 'white' spots. Instead, when viewed at high-enough resolution (for instance, in the foreground), they look like any other strong specular highlight: a strong reflection at the half-angle, solely the color of reflected light, progressively fading (and increasingly incorporating color of the underlying texture) outward from the center.
  • Unlike normal specular reflections, white spots appear either where there should be no such reflection or with far more intensity than they should. As such, one might more accurately think of them as scene artifacts: tiny disturbances in the faux universe where we imagine the scene exists.
  • White spots typically appear on reflective textured surfaces; only very infrequently on reflective colored surfaces; and -- to date -- never on (as we call them on this side of the pond) matte surfaces.
  • They more frequently involve textures with discontinuous transitions between adjacent colors (as opposed to, say, a smoothly transitioning gradient), and they typically appear to be centered on a transition border.

Items 1 through 3 suggest white spots result from a flaw that involves calculating the 3D architecture of a scene, as opposed to one that involves mapping a 3D representation onto a 2D image. (For example, this would be the equivalent of SunFlow's anti-aliasing miscalculating the component values making up an over-sampled pixel versus making a mistake in how it averages the multiple components together.)

Items 4 and 5, to me, suggests the reason the miscalculation typically appears as discrete spots instead of a larger field or area of too-high reflectivity is an artifact of SunFlow's adaptive anti-aliasing, which toggles between a standard sampling level (aa.min) to a [supposedly] more accurate level (aa.max) whenever a pixel varies more than a threshold amount from the pixels surrounding it. This shift from one sampling level to another would cause anomalies to appear at transition points within a texture -- as they do -- and to appear as 'island' anomalies -- as they do -- if the way the broken algorithm breaks is dependent upon or influenced by the degree of anti-aliasing in effect for a given point -- which we've seen it is, at least as regards correctly identifying surface normals.

The sample rate also influences the amount and positioning of white spot anomalies, as shown in the following image. Its source is a single scene definition, rendered using five different sampling levels. In SunFlow, adaptive anti-aliasing can be disabled by setting both aa.min and aa.max to the same value, resulting in every pixel in the image being rendered at the same sampling rate.



As you can see, significant white spots appear at a setting of (aa = 1), which causes 4 rays to be shot for each pixel. A setting of (aa = 2), or 8x over-sampling, reduces the number and size of the anomalies somewhat, with settings of (aa = 3) (16x) and (aa = 4) (128x) further improving the image. [In theory, 1024x over-sampling using (aa = 5) should result in even additional improvement. While SunFlow appears to support sampling values from -4 to +5, in my tests any attempt to exceed 128x over-sampling inevitably failed.]

Caveats

I should note I've only performed limited tests, and some of what I've found hasn't necessarily held true for you and other users. It's certainly possible that rendering at different resolutions, different camera configurations, or different lighting models could lead to very different results. However, I have to assume the underlying behaviors will be consistent, regardless of the specific expression. I will say what I've found in my tests is in line with my day-to-day experience with SH3D.

Possible changes

Again, much of this post is smoke and mirrors, conjecture based on assumption. While I've broken open the SunFlow source, I've only barely scratched the surface of understanding the code. (My last real exposure to anything having to do with low-level graphics programming was a brief stint in 2007 documenting PS3 APIs, while the last time I seriously thought about ray tracing was nearly 25 years ago.) It's an open question whether I'll be able even to identify areas for additional study, let alone pinpoint the source of the troubles.

However, fully aware my attempts are based on little more than half-assed investigation and superstition, I've been experimenting with modifications to the default Q4 parameters. So far I've had good results with using Enko Nyito's Advanced Photo Rendering Properties plug-in to change the desired values of antiAliasing.min and antiAliasing.max from the default values of 1 and 2, respectively, to a .min of 0 and a .max of 4. (I also find I prefer the Mitchell filter to the default box filter, as it seems to result in fewer artifacts. One place this is especially noticeable has been in the depiction of vertical blinds.)

Setting antiAliasing.min to 0 disables anti-aliasing on many, perhaps most, pixels in a typical scene; in theory, doing so should increase blockiness, resulting in a lower-quality render than would be achieved at SH3D's Q4 defaults. However, in practice I've not found this to be the case, which I believe to be for two reasons.

First, my tests suggest the uber shader at a 4x sampling rate (set by an aa.min or aa.max of 1, and the current Q4 default) introduces the greatest number of errors: Not only does the surface normal bug appear only with 4x over-sampling, but the rate also leads to the worst white spot performance. As a result, it appears that forcing over-sampling for every pixel risks degrading the image for any pixel less than [threshhold] different from its neighbors.

Second, SunFlow's default adaptive anti-aliasing threshold of 10% difference is an extremely conservative one -- I'm more accustomed to values nearer 30%. As a result, essentially every pixel whose appearance would benefit from anti-aliasing is anti-aliased, without the risk of introduced errors or the performance hit resulting from forced anti-aliasing of pixels below the threshold. [This can be tested by rendering a scene at (aa.min = 1, aa.max = 4) and at (aa.min = 4, aa.max = 4) and comparing the resulting images. In my experience the two have been identical or very nearly identical.]

Admittedly, there is something of a performance hit at (aa.max = 4) -- or there is, at least, on my antiquated system. However, as the white spot tests earlier showed, there is a significant-enough improvement in quality to merit the additional processing time. You may find an aa.max of 3 or even 2 sufficent for many scenes, and for preliminary renders I often use (aa.min = 0, aa.max = 2).

Conclusions

I hope this comment, while overly wordy, better explains my concerns over anti-aliasing and the two surface normal issues. In fact, I suspect, in the end, there will prove to be only one such problem, and we'll find there is nothing wrong with the way Emmanuel's code exports meshes to the render engine. Below, you'll find the same rendering of a bread plate I used earlier to illustrate the pathwork effect causes by ambiguously oriented triangles. Beside it you'll find another rendering of the same scene -- but this time at a different sampling rate.



As you can see, a number of triangles display a different surface normal depending upon anti-aliasing value. Accordingly, fixing the second failure type will likely correct the first one, as well.

And one final quirk -- one that, although technically still caused by a glitch in SunFlow, may in part owe its behavior to implementation choices made within SH3D. Below you'll find two renderings of the table setting scene similar to those cited earlier showing how an incorrect face may be displayed for a surface, depending upon the sample rate.



---except that, here, default Q4 anti-aliasing settings are used for both. The only difference is the left-hand image was rendered with a 'local time' of 11:25 a.m., but the right-hand image uses a time of 11:25 {b]p.m.. Notice the different faces displayed on what should be identical images -- even though the scene is entirely shielded from the effects of any 'natural' light.

From my quick dash through the SH3D source, though, it appears that in an effort to reduce the prevalence of white spots on exterior renders under some conditions, Emmanuel chose to tweak certain render parameters based on time-of-day. Conceivably, the discrepancy in face displayed may be an unintended side effect of this earlier fix.

Maz
[Feb 5, 2016, 4:58:49 AM] Show Printable Version of Post    View Member Profile    Send Private Message [Link] Report threatening or abusive post: please login first  Go to top 
okh
Advanced Member




Joined: May 12, 2013
Post Count: 1545
Status: Offline
Reply to this Post  Reply with Quote 
Re: What am I doing wrong?

Thanks for this. Useful and very instructive reference. ok
[Feb 5, 2016, 8:32:03 AM] Show Printable Version of Post    View Member Profile    Send Private Message [Link] Report threatening or abusive post: please login first  Go to top 
hansmex
Advanced Member
Member's Avatar

Netherlands
Joined: Sep 26, 2009
Post Count: 3948
Status: Online
Reply to this Post  Reply with Quote 
Re: What am I doing wrong?

Maz,

Thank you for your exhaustive story. Although most of it is beyond my grasp, I get the general drift of what you're saying.

I did a few tests of my own concerning the double faced problem.


I proudly present my herd. They wandered over from the Sketchup Warehouse.
I deleted the farmer's wife and the underground and exported all cows individually. Once as standard DAE (right), once with two sided faces (left).
For me the result was amazing. It used to be so, in my experience, that exporting with double sided faces solved the problem of "reversed triangles". Not in this case. All cows on the left have skin problems. On the right, three cows appear healthy.
Whoever made this model, seems to have applied textures in two different ways.

Maybe we should ask Sketchup to adapt their texture policy?

Hans
----------------------------------------
Hans

new website - under constuction
hansdirkse.info
[Feb 7, 2016, 6:53:13 AM] Show Printable Version of Post    View Member Profile    Send Private Message [Link] Report threatening or abusive post: please login first  Go to top 
Posts: 31   Pages: 4   [ Previous Page | 1 2 3 4 | Next Page ]
[ Jump to Last Post ]
Show Printable Version of Thread  Post new Thread

    Get Sweet Home 3D at SourceForge.net. Fast, secure and Free Open Source software downloads
   
© Copyright 2006-2024 eTeks - All rights reserved