Understanding how we represent objects within scenes is fundamental to perceiving and interacting with our world. Scene grammar, a hierarchically structured set of predictions regarding objects and their spatial proximity and functional relationships, plays a crucial role in this process. Objects within a scene are grouped into clusters, or “phrases,” which are organized around anchor objects, predicting the identity and location of other objects within that phrase. Building on prior research conducted with adults in our lab, we set out to investigate the development of object representations in children aged 5-10 years. Using a visual triplet odd-one-out task, we examined how children’s similarity judgments are influenced by scene hierarchy under two task conditions: one where similarity is judged without specific instructions and another where children had to base their judgements on the actions associated with the three objects. Our findings revealed that children rated objects from the same scene as most similar in both conditions. However, phrasal structure did not influence similarity judgments, which were instead driven by the frequency of object pair co-occurrence. Notably, while the phrasal structure did not affect similarity ratings overall, children’s similarity ratings increased for same-phrase objects as they aged. The only difference between task conditions was children rating objects from different scenes as less similar in the action-based task compared to the no-task. In summary, while children are sensitive to scene categories and integrate scene knowledge into their object representations early on, a more fine-grained phrasal structure seems to continue developing as they age.