Re: Forgive my ignorance but...
Like much of science, it really comes down to Occam's razor. We know matter exists, we know gravity exists, and we have a theory that explains how they behave with incredible accuracy in the vast majority of cases. We then make a new observation, and our existing theory doesn't quite seem to work for some unknown reason. We now have two options. One, we can throw out all our existing theories and come up with a new, significantly more complicated, one with a whole pile of unknown or arbitrary parameters that need to be carefully chosen specifically to explain this one new observation. Or two, we can assume that there's a bit more stuff out there than we previously thought, at which point the existing theories continue to work perfectly. And then we make another new observation and this time we need yet another even more complicated theory to replace everything, or we can just assume that there's exactly the same amount of extra matter and see that our existing theories still continue to work. And so on.
People love to complain that it looks like endlessly adding complications like epicycles, or arbitrary fudge factors to fix things, but it's actually the exact opposite. By far the simplest explanation for a whole raft of different observations is that there just happens to be a bit more matter out there than we can easily see. All other attempted explanations are far more complicated, contain far more arbitrary assumptions made specifically to explain one or two observations while also contorting themselves to explain why all the observations made previously failed to see anything unusual, and ultimately fail to actually explain the vast majority of other observations anyway.
Specifically to your complaint, the thing about adding an arbitrary fudge factor is that you'd expect it to be different every time. But what we actually see is that we have to include exactly the same factor for every single observation no matter how its made or at what scale. Every individual galaxy we look at seems to have about 5 times more mass than expected. Every cluster we look at seems to have about 5 times more mass than expected. The CMB says the entire universe has about 5 times more mass than expected. At some point it stops being a factor added just to make things work, and instead becomes a ton of different lines of evidence all telling us that maybe there actually is 5 times more mass out there than we expected.
A final thing worth considering is that when you have a bunch of complicated equations describing how thing work, there are relatively few ways things can be added or changed while having the whole thing continue to work in a consistent manner - you can't just chop bits out or add new parameters in wherever you like and have it all still make sense. For example, Einstein famously called the cosmological constant his greatest mistake, because at that point he thought it wasn't necessary and therefore looked like he'd just added a fudge factor. But even before we discovered the universe was expanding he was absolutely correct to include it, because it's actually the only thing that could have been added to his equations. In fact, from a certain point of view he'd have been wrong not to include it, even if it had turned out to be zero in practice (that's a bit of a mathematcian vs. physicist argument).
The same is true for dark matter. It's not just adding a random number to make things work. It's changing one of the very few parts where it actually makes sense to change anything. Things like modified gravity theories invariably end up horrifically complicated precisely because you can't just change a couple of bits in general relativity and still have it make sense, instead you have to come up with a whole new theory that mostly still gives the same answers except in a few specific cases. Total mass, on the other hand, is essentially a free parameter as far as theory is concerned, set only by our epirical observations of how much we can see. So changing it really isn't arbitrary at all, it's one of the parts that exists specifically to make things work. We pick a value that makes things work based on our observations. Previous observations gave one number, newer observations say it's actually a different number. That's far simpler and less arbitrary than trying to come up with convoluted explanations for how things actually work in a completely different way but just happen to behave how we thought in every observation up to a certain point.