I'm more concerned with the ones starting riots, violence, blocking traffic. When I think of the extremist left I'm not really worrying about the stoner who dreams of a world where he never has to work again. I'm more worried about the guy brining a knife or some fireworks to a protest to use on people.
Or even the more benign people who are using force to take away peoples right to speech I often see (like sabotaging sound equipment or similar.