The metal-poor stars in the bulge are important relics of the Milky Way's formation history, as simulations predict that they are some of the oldest stars in the Galaxy. In order to determine if they are truly ancient stars, we must understand their origins. Currently, it is unclear if the metal-poor stars in the bulge ([Fe/H] < -1 dex) are merely halo interlopers, a unique accreted population, part of the boxy/peanut-shaped bulge, or a classical bulge population. In this work, we use spectra from the VLT/FLAMES spectrograph to obtain metallicity estimates using the Ca-II triplet of 473 bulge stars (187 of which have [Fe/H] < -1 dex), targeted using SkyMapper photometry. We also use Gaia DR2 data to infer the Galactic positions and velocities along with orbital properties for 523 stars. We employ a probabilistic orbit analysis and find that about half of our sample has a >50 per cent probability of being bound to the bulge, and half are halo interlopers. We also see that the occurrence rate of halo interlopers increases steadily with decreasing metallicity across the full range of our sample (-3 < [Fe/H] < 0.5). Our examination of the kinematics of the confined compared to the unbound stars indicates the metal-poor bulge comprises at least two populations; those confined to the boxy/peanut bulge and halo stars passing through the inner galaxy. We conclude that an orbital analysis approach, as we have employed, is important to understand the composite nature of the metal-poor stars in the inner region.