Consider all those people who intentionally or unintentionally permit actions from the religious community that they disagree with, yet remain silent since they see the people as being 'on the same team'.
I think it's precisely here that the argument runs into difficulty. Silent inaction is not a product of religion, but of human nature.Of course, people of all stripes are guilty of silent inaction. But these are large numbers, where the point is made by statistics, which I don't have. I will assume there is a far greater percentage of "silent inaction" types within the religious community than from a secular community.
When Hitler rose to power, Germany was a scientific, philosophical and technological powerhouse, arguably the premier nation in all these fields. Yet this nation sat silent while Hitler espoused National Socialist propaganda and embarked on systematic 'genocide' of not only Jews, but also Gypsies, homosexuals, communists and religious opponents.
When Stalin and Mao embarked on separate pogroms in their officially atheist nations, the masses sat silent.
In the campaign against slavery, did the leadership arise from the secular or religious world? Wilberforce, Thomas Clarkson and the Quakers would suggest that Christians were active in significant ways.
Please feel free to study the available evidence rather than taking my thoughts at face value. Some germaine topics may be: charitable giving, volunteerism in causes of social justice, the current slave trade, child prostitution, and care for the marginal in society (inc. prisons).