The Maximum deviation optimizes using a threshold on how much the user allows InstaLOD to deviate away from the original surface. This deviation is set in world-scale units, however, takes many Feature Importance
parameters into consideration. The Feature Importance settings are typically best kept on their default.
A maximum deviation of 0.01
should be sufficient in most cases for real-world-scaled objects to maintain most curvature and medium-sized details. If the object's curvature or shading already starts breaking with this deviation, it could be that the object is scaled too small.
If it is a small part within a larger assembly, results like this can happen, so an even smaller deviation might be necessary.
Deviation: 0.01 vs Deviation: 0.001
To make sure that details are maintained perfectly, you can also optimize the model to e.g. 25% first and then vertex-paint the parts using the blue color channel and use that color set as optimizer weights to optimize the object further to your desired result. This will optimize the details to 25% of the original part but optimize the rest of the model further.
For more information on vertex painting please refer to this article: How to Preserve Topology During Polygon Optimization.