3 ms·
1e-3 is a pretty loose error bound if you're dealing with double-precision. I usually use tolerances in the region of 1e-5 to 1e-10.
by jcranmer 6y ago
1e-3 is a pretty loose error bound if you're dealing with double-precision. I usually use tolerances in the region of 1e-5 to 1e-10.
- nitrogen 6y agoIt's also better to use significant figures or a relative (as opposed to absolute as computed above) error if you're working with small values.