it sounds like at least some of them could be criminals.
It's a grey area. I wouldn't say that they're involved in anything heavy, there's definitely nothing organized about them. It's in fact the very lack of being organized that makes some of them unruly.
To be fair, I did things too, especially in my teens and in my early twenties. But some guys I know in their forties still routinely have boisterous tendencies in the weekends, with substance abuse and everything. I mean, come on, when does it end?
It's like they never snapped out of it and just do this on auto-pilot and that gives the parties that they throw or attend something uncanny and soulless. Hard to explain that one, but maybe you get the idea.
I also struggle with parties but I do feel safe when I am with a few good friends who I know don’t do drugs or anything silly like that
I also don’t get friendships that depend on drinking -
I'm glad that you hang with what sounds like a sensible crowd.

And what you say there about friendship dependency based on drinking, you've hit the bullseye right there.
At some parties I have brought a bunch of 0.0% alcohol beers with me. I do like the taste of beer, I just don't enjoy getting drunk of them. So those alcohol free beers seem like a good solution, right?
But every time I've been bringing them with me, I get called certain things for bringing such drinks, and they encourage me to just go for some real beers.
This is something that alkies do. They bring you down to their degenerate level, so that they're not alone in this.