Multiferroic oxides have attracted much attention in recent years due to their potential for controlling magnetism with an electric field and ferroelectricity (FE) with a magnetic field. Most multiferroics, however, typically display relatively weak coupling between these parameters and rarely operate near room temperature. This has motivated researchers to design and fabricate layered oxide heterostructures, with the goal of enhancing the magnetoelectric (ME) coupling between individual ferroelectric and magnetic layers while also increasing their operating temperature. However, further progress in this area requires a deeper understanding of the underlying mechanisms and timescales limiting ME coupling in both bulk and heterostructured multiferroics. In the past few years, we have demonstrated that ultrafast optical and terahertz (THz) spectroscopy is a unique tool for probing the interplay between FE and magnetic ordering in canonical bulk multiferroics such as HoMnO3 and EuYMnO3 as well as in multiferroic oxide heterostructures. For example, these studies have revealed a long-lived, photoinduced enhancement of the FE polarization in a FE/ferromagnet (FM) heterostructure, and have also shown that femtosecond optical pulses can induce transient magnetoelectric coupling in a FE/FM heterostructure, with implications for high speed magnetoelectric devices. We have also used THz pulses to directly probe low energy modes in these materials, allowing us to determine the timescales governing energy transfer from electronic to magnetic degrees of freedom. Overall, our studies demonstrate that ultrashort optical pulses can help unravel the mechanisms underlying magnetoelectric coupling in multiferroic oxides, with the potential for all-optical control on femtosecond timescales.