Immunotherapies - which harness the body's own defences to fight tumours - can clear even terminal cancer in a small proportion of patients.
However, a small study by the University of Texas found those harbouring a more diverse community of gut bugs are more likely to benefit.
Cancer Research UK said understanding gut bugs had "great potential".
The human body is home to trillions of micro-organisms - estimates suggest our own tissues are so heavily outnumbered that our bodies are just 10% human.
And a growing wealth of studies shows these microbes can influence our immune systems and have been implicated in auto-immune diseases and allergies.
Immunotherapies are one of the most exciting breakthroughs in treating cancer. They work by taking the brakes off the immune system to help it to attack tumours more easily.
The research group compared the gut bacteria in 23 patients who responded to the therapy and 11 who did not.
Dr Jennifer Wargo, a melanoma surgeon and scientist, told the BBC News website: "We found a night-and-day difference in the diversity of bacteria species in the faecal samples."
The study, presented at the National Cancer Research Institute's Cancer Conference in Liverpool, found Ruminococcus bacteria in much higher levels in those that responded to treatment.
It suggests that it may be possible to boost the effectiveness of immunotherapy by altering the balance of bacteria in the gut.
'Scratch the surface'
Procedures such as a trans-poo-sion - a transplant of faecal matter containing beneficial bacteria - are already used as a treatment for some diseases.
Dr Wargo added: "It is hugely plausible I think - we still need to dig a little deeper, but I think we're on to something.
"I think it really does shape our body's immune response as a whole and to cancer."
It is not yet clear if the differences in bacteria are the cause of the better response.
People with diets containing more fruit and vegetables tend to have a richer set of gut bugs, so it is possible that it is those with a healthier lifestyle that respond better to therapy.
"It might point to a healthy diet increasing your chances, which I think would be a great message," she added.
Sir Harpal Kumar, chief executive of Cancer Research UK, said: "Our bodies are filled with trillions of bacteria, and we are just beginning to scratch the surface of understanding their great potential.
"It's really interesting and exciting to see new evidence emerge on the close connection between the immune system and the bacteria living in our guts. As this, and several other studies, have shown, manipulating these bacteria could be exploited in future to help patients respond better to treatment."