The UK buys most of its imported oil from outside of the EU, which means that if Britain loses access to the single market when it formally leaves the EU, there is a chance that it could negotiate better deals with oil rich countries in the Middle East.
If the pound returns to a stronger level in the future, and Britain has its own deals with countries in the rest of the world for buying oil, it could see a less expensive price for oil and therefore lower prices on the forecourts.
The UK still produces a lot of oil itself, mainly from the North Sea oil fields. With a stronger ability to export and a lower-valued pound, Britain could actually generate more profit and be a more attractive proposition for investment in oil. This could also ultimately mean lower prices for UK drivers.