I don't have the latest AAA games so may not be relevant. I don't seem to have issues though in skyrim (around 50 mods), cod mw, cod ghosts, farcry 3, crysis2 etc.
Dx12 is supposed to support more even threading and is also a means for the hardware to talk to each other more directly. Lower driver overhead means less overhead for weaker cpu's so they can focus more on the game while the gpu handles the graphics. That's why lower power cpu's tend to benefit more greatly from dx12 than say an i5 or i7. I3's, pentium's, fx cpu's would generally benefit moreso.
I've not played bf1 personally though others testing of dx11 vs dx12 shows dx12 to have issues. It's not true for all games so I'd venture to say it's how it was implemented in this specific game title rather than a testament to dx12 in general. Any game has the potential to be a rockstar or a flop in terms of performance depending on how it's coded and what it consists of in graphics. Updates can also be released which work to correct various flaws. When looking at performance reviews it's important to make note of when it was reviewed, which patches if any were applied etc. Otherwise it could be tested upon first release and revisiting that review several months (and several updates/patches) later could yield different outcomes.
Dx12 performance seems to be a bit all over the place. It could very well depend on a game by game basis, some tests done are theoretical but don't pan out in real world cases. Dx11 seems to use more than just 3 cores, all the games I play are dx11 and my cpu shows all 4 cores fully loaded. Not just 2 or 3 with 1 core sitting idle. Some tests done by pcworld have shown dx12 performance doesn't increase much past 6c/12t and isn't a huge increase beyond that of 6 cores (no ht).
http://www.pcworld.com/article/3039552/hardware/tested-how-many-cpu-cores-you-really-need-for-directx-12-gaming.html
Here's an article that deals with clock speed as well as dx11 vs dx12 and amd vs nvidia.
https://www.hardocp.com/article/2016/04/19/dx11_vs_dx12_intel_cpu_scaling_gaming_framerate/5
The higher the clock speed the less performance difference between dx11 and dx12, the lower clock speeds the more benefit (throwing back to weaker cpu's gaining more from dx12 due to reduced overhead). At least with nvidia, with amd cards they showed more benefit from dx12. The article is also a year old, doesn't included the latest gpu's or the latest drivers. Take it with a grain of salt. It's hard to get finite data when the data is perpetually changing.
Multiplayer does have a large impact, performance won't be the same from time to time if gameplay is different. The game effectively becomes different. One game with 12 players coming from the left, 3 from the middle and 7 on the right is going to be different than single player campaigns where you have a more predictable 8 enemies coming from the left every play through. That's why benchmarks are often done with campaign mode or built in benchmark modes, not multiplayer the way many people play.
In order for a test to be confirmed or results to be deemed accurate you need the fewest variables possible. That's basic scientific approach. Multiplayer is full of variables by nature so difficult to bench.
I can't really answer which scenarios would cause 100% usage on an i5 to be ok and for fps to drop heavily in other scenarios. It's about data that needs to be processed, so long as there's data to be processed the cpu will keep working. If enough data is processed in time, fps should be smooth. If the game contains more data than an i5 can process by the time it's needed it may cause hiccups noticed in fps drops. That gets too far into the details of how each game is coded for me to answer.
It's a bit like asking why one webpage loads faster than another that's identical in appearance to it without knowing what's going on in the background. Are the images different sizes? Is the internet connection the same? Is one written in plain html, html+css, javascript, is there flash being used to achieve simplistic things that css could handle? At first glance they might appear identical, behind the scenes the coding could be vastly different. More streamlined efficient coding along with lightweight images or even repetitive images (things already in memory) vs inefficient bloated methods can achieve the same effect at reduced cost in terms of processing power and internet bandwidth. Games are similar.
Speculation is difficult, it's very possible that future games and hardware will move to higher core count options. When is the better question. Obviously cpu's have moved from single core to dual, quad, hex and octa core for various needs. Speculation that 6 and 8 core cpu's would be needed for common applications and gaming were made 4, 5, 6yrs ago with fx cpu's being so inexpensive (comparatively).
Every year people say well console xyz uses lots of cores so pc games must too and the logic doesn't exactly play out that way. Consoles have to use multicore cpu's at lower speeds to keep temps down. Imagine the cooling needed for an i5 even using a stock cooler, there just isn't room in a console. They went slower cores and wider core count. Pc games operate differently and even a dual core + ht i3 or 7th gen pentium can outperform an xbox one or ps4. Apples and oranges.
Big moves tend to happen more slowly, dx12 was promised to be game changing years ago and it's still in its infancy. 64 bit processing was out long before programs were able to fully utilize it or even need it. Browsers being some of the slowest adopters of 64bit. If you're a game dev you have to consider the desire to move forward along with the user base. If your game runs like crud on anything less than 6 or 8 cores, you're cutting out athlon x4, fx 4xxx, pentium, i3, i5 users. That's a pretty large percentage of hardware owners to suddenly snub and cutting your customer base by a thick chunk so it's better to ease into requiring more cores. Even 'cheap' ryzen 8 core cpu's are running $330-550. The ryzen 5 series are 6c/12t and supposed to be $220-250. If that becomes the min requirement what are budget gamers supposed to do? Many are still buying sub $200 cpu's to game with for budget builds. It's just my personal speculation, nothing more, but making the requirements/cost too high in the pay to play of gaming means excluding folks who can't afford all the luxury. Imagine if every game suddenly became VR only, you had to have a $300+ cpu, $450+ gpu, couple hundred worth of VR headgear and other components or forget it - a lot of folks would say 'forget it'. Prices are coming down for sure but 6core cpu's still aren't 5 for a dollar.