Your argument applies in the case where you do not distribute the GPL'd library - as zerohp points out, opinions differ on this case, and it hinges on whether y

Your argument applies in the case where you do not distribute the GPL'd library - as zerohp points out, opinions differ on this case, and it hinges on whether your compiled program would be considered a "derivative work". However, if you distribute the GPL'd library, then it's a lot more clear-cut. "Derivative work" doesn't enter into it anymore - to distribute the GPL'd library itself you must have a license to do so. If the license you're relying on there is the GPL, you must abide by its terms, including those about linking.