Sajid Javid has told Sky News he wants to remove the PCR test requirement for travellers returning from some foreign countries "as soon as I possibly can".
The health secretary said he was aware of the cost for families holidaying abroad and that the measure should not be in place "for a second longer than is absolutely necessary".
Mr Javid said he had asked officials to remove the rule "at the moment we can".
"We have got a huge number of defences; of course we still want to remain very cautious, and there are some things that – when it comes to travel for example – there are some rules that are going to have to remain in place," Mr Javid told Sky News' Trevor Phillips on Sunday.
"But the PCR test that is required upon your return to the UK from certain countries, look, I want to try and get rid of that as soon as I possibly can.
"I am not going to make that decision right now, but I have already asked officials that at the moment we can, let's get rid of these kind of intrusions, the costs that generates for families, particularly families just trying to go out and holiday.
"We shouldn't be keeping anything like that in place for a second longer than is absolutely necessary."
Meanwhile, the health secretary also told Sky News he did not "like the idea" of vaccine passports and hoped to "avoid" introducing them.