Six out of ten broadband customers endured slow service or had their connection stop working altogether last year, according to Citizens Advice.
It wants the government to appoint a telecoms consumer advocate to help promote the case of fed-up customers.
The advocate should be included in the upcoming Consumer Green Paper, it said.
Almost a quarter of the 4,000 people surveyed for Citizens Advice said internet problems had disrupted their work or study.
The report said that customers on average spend 2.4 hours trying to sort out problems with their service.
"People now rely on their broadband and mobile connection for the day-to-day running of their lives," said Gillian Guy, chief executive of Citizens Advice.
"Yet the majority of people continue to face significant disruptions that can waste their time
A government spokesman said broadband was "no longer a 'nice to have', but a modern necessity, and we all know how frustrating it is when it doesn't work".
He said a raft of measures would help the regulator Ofcom protect consumers, including receiving compensation when their service falls short.
In November Ofcom announced that households who receive poor service from their telecoms provider would get automatic compensation.
From 2019 they will get £8 a day if a fault is not fixed, paid as a refund through their bill.
Providers will also have to pay £5 a day if their broadband or landline is not working on the day it was promised.
Ofcom estimated that as many as 2.6 million people could benefit from the new rules.
The government is also pushing the telecoms industry to provide a faster service for everyone.
In December it promised that homes and businesses would have a legal right to demand faster broadband speed by 2020.
Under the plan, broadband providers would have to provide a minimum speed of 10 Mbps to anyone requesting it, subject to a cost threshold.