Efficient trajectory planning algorithms are a crucial issue for modern autonomous underwater vehicles. Classical trajectory planning algorithms in artificial intelligence are not designed to deal with wide continuous environments prone to currents. Furthermore torpedo-like underwater vehicles are strongly nonholonomic. A novel Fast Marching based approach is proposed to address the following issues. First, the FM* algorithm is proposed to extend the A* algorithm to the continuous domain. Second, underwater currents are taken into account thanks to an anisotropic extension of the original Fast Marching algorithm. Third, the vehicle turning radius is introduced as a constraint on the curvature of the optimal trajectory for both isotropic and anisotropic media. Furthermore, a dynamic version of the Fast Marching algorithm called DFM is developed to efficiently replan trajectories in dynamic unpredictable environments. The overall Fast Marching based trajectory planning method has been tested on simulated underwater environments and validated on a real experimental platform in open water.