This study investigates whether macaque monkeys develop abstract internal models from sensory sequences, focusing on the ventrolateral prefrontal cortex’s role in encoding visual sequences. Through recording prefrontal neurons as monkeys viewed visual sequences and mismatches, it was found that these neuronal populations encode both the specific images shown and their abstract sequential structure without requiring any overt task. Notably, the prefrontal cortex forms representations of sequence structure, serial order, and image identity within distinct yet overlapping neuronal subspaces. These representations update quickly after exposure to a mismatch sequence and can differentiate mismatches in sequences of varying complexity. Importantly, the representations can generalize across different sequences that share the same repetition structure. This research highlights the spontaneous ability of prefrontal populations to form detailed internal models of visual sequences, encompassing both specific and abstract information, which underscores the complexity of cognitive processing in the primate brain.