Acoustic emission (AE) is a key method in the field of fatigue, both as a high-performance non-destructive testing (NDT) tool for structural monitoring and as a refined approach to physical investigation. To tackle the challenge of exploring new signals and recognizing sources based on both statistical and physical analyses in fatigue experiments, a relevant tool for representing time series, known as a Deep Scattering Network (DSN), has been developed. A DSN shares a similar architecture to that of deep convolutional neural networks: it implements a cascade of convolutions using wavelet filters, a modulus function, and pooling operations. Combined with dimensionality reduction techniques and clustering methods, unsupervised classification of time series reaches particularly high performance, as signal representations called Deep Scattering Spectra are locally invariant to translation while preserving transient phenomena such as attack and amplitude modulation. Depending on the dimensionality reduction method and network parameters, Deep Scattering Spectra of the continuous recordings are used either to detect recurrent transient signals of short duration or to cluster the entire input signal by focusing on the background wavefield. Thus by applying this method to continuous AE recordings at high sampling rate, obtained during fatigue tests on steel and aluminum alloys, we demonstrate that the background signal contains relevant information. This information can be extracted -even at very low signal-to-noise ratios-, grouped into distinct source mechanisms (separated from mechanical and electronic noise), and investigated to provide in-depth insights into fatigue crack activity (e.g., closure, crack-tip plasticity, etc.). Fatigue acoustic signatures are identified and distinguished within each fatigue cycle and correlated with loading and strain measurements throughout the duration of the experiments to track the long-term evolution of the sources. The acoustic content of these fatigue signatures is characterized using deep scattering spectra and frequency- or time-based descriptors, enabling a typology of acoustic fatigue signatures to be proposed.