The double quantum coherence (DQC) signal in nonlinear spectroscopy gives information about the many-body correlation effects not easily available by other methods. The signal is short-lived, consequently, a significant part of it is generated during the pulse overlap. Since the signal is at two times the laser frequency, one may intuitively expect that the pulse overlap-related artifacts are filtered out by the Fourier transform. Here, we show that this is not the case. We perform explicit calculations of phase-modulated two-pulse experiments of a two-level system where the DQC is impossible. Still, we obtain a significant signal at the modulation frequency, which corresponds to the DQC, while the Fourier transform over the pulse delay shows a double frequency. We repeat the calculations with a three-level system where the true DQC signal occurs. We conclude that with realistic dephasing times, the pulse-overlap artifact can be significantly stronger than the DQC signal. Our results call for great care when analyzing such experiments. As a rule of thumb, we recommend that only delays larger than 1.5 times the pulse length should be used.