> unless you keep the poisoning attack strictly inaccessible to the public
This is likely impossible. As the in-vogue breed of model extraction methods ("distillation attacks") demonstrates, you can infer the underlying training and/or fine-tuning of a model with a series of carefully constructed prompts.
Another name of model poisoning? Adversarial fine-tuning.
If we take a completely utilitarian and amoral viewpoint, the insiders are selling their material, non-public information. The rest of the market participants are buying. From the latter's utilitarian perspective the former are providing a valuable service and getting paid for it. I'm pretty sure there was a legal term for such sales activity...
The only people playing fair are those who don't know how to cheat well enough.
Problem is not just NVD issuing inflated scores. That's their workaday MO. They are required to assume the worst possible combination of factors.
The real problem is that CVSS scoring is utterly divorced from reality. Even the 4.x standard is merely trying - and failing - to paper over the fundamental problems with its (much needed!) EPSS[ß] weighting. A javascript library that does something internal to software and does not even have any way of doing auth is automatically graded "can be exploited without authentication". Congrats, your baseline CVE for some utterly mundane data transformation is now an unauthenticated attack vector. The same applies to exposure scoping: when everything is used over the internet[ĸ], all attack vectors are remote and occur over the network: the highest possible baseline.
This combination means that a large fraction of CVEs in user-facing software start from CVSS score of 8.0 ("HIGH") and many mildly amusing bugs get assigned 9.0 ("CRITICAL") as the default.
Result? You end up with nonsense such as CVE-2024-24790[0] given a 9.8 score because the underlying assumption is that every software using 'netip' library is doing IsLocal* checks for AUTHENTICATION (and/or admin access) purposes. Taken to its illogical extreme we should mark every single if-condition as a call site for "critical security vulnerabilities".
CVSS scoring has long been a well of problems. In the last few years is has become outright toxic.
ß: "Exploitability" score.
k: Web browsers are the modern universal application OS
As pointed out in the oil crisis article[0], the reduction that led to the 1970's oil shock was about 5%. Effectively eliminating 15% to 20% of global production capacity is going to be a pretty damn big deal.
That implies that either the auth is too heavy (possible, ish) or their systems don't degrade gracefully enough and many different types of failures propagate up and out all the way to their outermost layer, ie. auth (more plausible).
Disclosure: I have scars from a distributed system where errors propagated outwards and took down auth...
I have now witnessed first hand what the unexpected benefits might be. I expected CC to be a boon to overburdened teams, because it's now possible to spend $2 on compute and have it write a mostly-one-off tool that nobody would ever otherwise have the capacity or time for.
Sure, that's happening too, but to a lesser degree than I thought. CC with a number of "enterprise integrations" (really: corporate MCPs) is a pretty hefty force-multiplier for operations teams. "Go summarise and dissect this weird client request for me. Documentation is spread across at least $THESE_ENTERPRISE_DATA_SILOES." Saves a bunch of time pinging the different people across continents who happen to know intimate details. That was not entirely unexpected.
It's the technically minded but not necessarily otherwise technical people who keep surprising me in weird and wacky ways. People are building themselves and their immediate peers disposable dashboards. Who needs a service to pull data for a real-time display when CC can collect the necessary information and construct a local, static HTML file with all the info neatly in one place? I'm sure there will be a hangover because the compute cost for doing these in JIT fashion will surely feel like death by a thousand cuts at some point, but the ability to really quickly validate whether certain types of data aggregations are useful is proving to be ... a positive development.
I disagree about the ease of maintaining the software, though. You still need the skills to really understand what the code is doing, and with the original "why" possibly lost in the adrenaline haze, the maintenance effort floor has shifted.
Phased roll-out. You first introduce a version that still accepts all extant inputs but will actively warn that there are characters that will be removed in a future release.
Then you wait. Then you roll out a version where the new functionality is flipped on by default, but where you still allow to explicitly toggle to the old one. Then you wait some more.
And then - only then - you roll out a release where the old functionality has been removed entirely.
That’s dangerous. Apple fooled me with the iOS 26 glass theme, it’ll be a while before I install another major update from them. I know many people still on iOS 18. I doubt many of them will update until either Apple fixes their UI/UX or they upgrade to an Android.
Meh, I think you keep the old keyboard and set a password expiry. New passwords use the new keyboard. Or, if you're in a rush to remove the old code, _after_ next login you require password replacement and use the new onscreen keyboard from then.
And perhaps also introduce an upgrade blocker, as the keyboard app notifies the system of a situation that would be unsafe to upgrade to newer releases
Hah, if you ever used the N800 media player you will have been exposed to a tiny bit of my code. Some of the UI polishes and usability tweaks were mine. (Well, someone else had figured out they needed to be done and the bug landed on my lap...)
"/* Here be dragons */" in a particularly hideous d-pointer punching chain must have been a surprise for whomever eventually picked it up.
This is likely impossible. As the in-vogue breed of model extraction methods ("distillation attacks") demonstrates, you can infer the underlying training and/or fine-tuning of a model with a series of carefully constructed prompts.
Another name of model poisoning? Adversarial fine-tuning.
reply