Intel Xeon E5-2680 v4

And that CPU is just too slow, it would be better to have a newer 6 core than one with lots of old slow cores.

On top of that, Broadwell is 5th Gen Intel. This is again old where it lacks many features for newer software. Adobe Premiere for example wants at least 6th Gen Intel. Windows 11 wants at least 8th Gen Intel. While this can be manually overrided, it can still cause software issues.

Today a newer 6 core CPU like a 12400F or 5600 will outperform it in multicore performance: https://www.cpubenchmark.net/compare/2779vs4681vs4811/Intel-Xeon-E5-2680-v4-vs-Intel-i5-12400F-vs-AMD-Ryzen-5-5600 Because each core is nearly twice as fast, and it throttles less. So if something uses every core, the newer CPUs are slightly faster. But if something uses 1-6 cores only the newer CPUs are vastly faster.

I strongly wouldn't recommend it. It's a 10 year old chip from Q1' (January-March) 2016: https://www.intel.com/content/www/us/en/products/sku/91754/intel-xeon-processor-e52680-v4-35m-cache-2-40-ghz/specifications.html Typically you can expect PCs to last 7-10 years reliably, and anything beyond that is luck. Most of the motherboards are recycled in China where they take apart server boards and recreate them as economy PC boards. These use old parts, so reliability isn't known.

The second one I thought maybe a xeon e5 2680 v4 would be alright for multitasking since it has more cores and the rest would be pretty much the same + a rx 580 but I've heard pretty bad things about xeons going around.