According to The Daily Star, a woman was also among those killed.
Hillol Roy, an inspector of Gowainghat police, said four workers went to the area to extract stones from a trench. Suddenly, a landslide hit the pit, leaving them dead on the spot.
The injured were rushed to the hospital for medical assistance.
The workers had dug the trench beyond the permissible limit.
NGOs working in the area said that several people work at stone quarries in Sylhet without any safety gears.
In 2017, around 30 workers were killed during stone extraction in five Upazilas of Sylhet.
UNO Bishwajit said workers' safety is a serious issue, but the task force has only been working to stop the illegal stone extraction.
"We have put red flags in the area and banned all stone extraction for now to avoid further casualties. We are also trying to trace the employers of the victims to take legal action against them," he added. (ANI)