In this paper, we present a new feedback guidance algorithm for autonomous docking maneuvers in the cislunar environment. In particular, we propose a closed-loop optimal guidance algorithm that is capable of taking path constraints and collision avoidance into account while being on a Near Rectilinear Orbit (NRO) around the L2 Lagrangian point in the Earth-Moon system. The algorithm is based on the Lyapunov vector field guidance where the acceleration command is derived from a desired velocity vector field. We use reinforcement learning to learn the shape of the field as a function of the state of the system, allowing for increased flexibility in terms of constraint shapes and better performance in terms of fuel consumption with respect to classical Lyapunov vector field guidance.