So I guess that’s actually several questions, and they each have different answers.

Why does combat feature heavily in D&D? It doesn’t. Or at least, not necessarily. How much or little it features is dependent on your DM.

Ok, so why has it historically been featured heavily? Because of D&D’s lineage. The game evolved mechanically from wargames, where combat was the whole thing, and thematically from works like Conan the Barbarian and Tolkein, where fighting monsters featured prominently.

Why so many types of monsters, then, if works like The Hobbit only had a half dozen or so? Because The Hobbit is a single story, whereas D&D is a framework for creating lots of stories. Maybe one short campaign or a campaign arc has as many monsters as a Tolkein story, but then you go on to the next arc, the next campaign, and you need something new. You can obviously recycle lots; orc bandits are different from orc soldiers are different from orc cultists. But with (tens of?) thousands of games going on continuously, year after year, there’s always a demand for new content to slot in, and monster design is often a handy thing for DMs to outsource. Hence, there are a lot of kinds of monster because there is demand for them.

Are you asking why there are so many kinds of monsters, or why monsters appear so frequently in the campaigns you’ve played in?

That doesn’t work for 40k, to my understanding. It’s a miniatures combat game

Addendum to the “Are you sure you want to do that” bullet: if a player ever does something that seems nonsensical to you, ask them what they expect to achieve by doing that. Understanding their motivation is often what resolves the miscommunication and/or allows you to steer them towards a better way to do what they’re trying to do.

I read once that the earliest edition(s?) didn’t have Rogue as a separate class, that everyone would be searching for traps and such. And when Rogue was added with the explicit ability to detect traps, it caused a crises because suddenly that implied that no one else had that ability.

Nope, no, that’s encouraging their behavior. Now your player thinks you’re giving them a quest to earn enough money to play out their brothel scene.

You absolutely do not have to RP this. You can say “No.” You can say “Ok, you go off and do that, what’s everyone else doing?”