> Tutoring is a process, and the end result is that the student gains a demonstrable new capability. The how isn't as important as the end result.
I disagree. If, after you have been tutored, you perform well, but still need tutoring for the next year's exam in that subject, then you haven't been "tutored", you've been given a similar enough copy of the exam questions to train on.
As I keep telling my kids, good results is a side-effect of good habits. If you're half-assing things, it shows up in your results. If you studious and methodical, it also shows up in your results, just in a different direction.
AI is used primarily to half-ass things, do things without checking, getting good vibes by that sycophantic tone.
Get your kids into a rut of good habits, and they'll run in that groove forever, regardless of tutors or AI. Half-ass your way through life, then sure, they'll get fucked over by AI tutors too.
I think you are doing a lot of assuming on how AI can be used for tutoring. I'm currently augmenting Chinese in person tutoring with a pi harness that helps me out with drills, lets me explore what things mean, get similar words or context related words and so on. It's not perfect, def not good enough to be a standalone thing, def has barrier of entry of being motivated to do honest self work but it's a massive value add into the process for me.
One huge issue IMO is that right now, around the world, most people are not taught at all how to learn for themselves nor how to critically think on their own. And that will indeed result in using it as a crutch and not a good tutoring tool.
Those CVTs were fucking terrible for years. They were so bad that loads of them ended up on buy-here-pay-here lots after the transmission blew and the Altima is now widely known as the highway missile of choice for drivers with a credit score in the 400s and no insurance. You lucked out.
> Can you possibly elaborate on the timing belt in oil? I’ve only known of dry timing belts or lubricated timing chains/gears
Ford's recently designed engines were designed with a timing belt internally that operated in an oil bath.
They are currently walking that decision back and attempting to make it look as if it wasn't planned obsolescence (dry cambelt engines or proper timing chains means that the engine would last almost longer than the person who bought it if they did the regular maintenance. A cambelt that cannot be changed because its internal means that there is a limit to how long the engine would last.)
Cheap and poor decision by a company not acting in good faith. Right up there with the globally popular Ford Ranger whose oil change needs to be completed in under 10 minutes because thats how long it takes the oil pump to drain residual oil and it has no self priming capability, so itll run dry and seize the engine despite the oil sump being full.
> But I was left wondering about the specific attack vector they’re imagining. If the attacker can insert a paragraph into the document, isn’t it game over anyway? When would they be able to do that but not arbitrarily edit the document? In other words, can’t they just replace the entire contents with “This company has infinite revenue, 6 billion customers, no debt and amazing leadership.”?
There is no use for something like Jev on a singular document from a single source; it's use comes from concatenating multiple sources into a single document and asking for an answer. What they did here is the most common workflow for something like Jev: "here's all the data we have and know about, now give us a go/no-go decision"
In that workflow, you only need a single bad actor to poison the results.
There's a huge gap between what's being described in that comment and an "extinction level event" is that's what ELE means.
What's being talked about is a massive extension of human capability, into the intellectual sphere rather than the physical environment. The atom bomb had a clear way to get to extinction, a sudden chain reaction that leads to collapse of the entire biome.
The only way AI gets there is something like control of the nuclear weapons on all sides and that control becoming suicidal. Every other major event leads to at most civilization collapse, after which I assume we will have some sort of Butlerian Jihad restrictions on getting into the same situation.
But, I would love to know what I'm missing here, if ELE has clearer routes!!
> The only way AI gets there is something like control of the nuclear weapons on all sides and that control becoming suicidal.
Not the way I was thinking off; I was thinking more along the lines of brain atrophy until humans are helpless (maybe 2 complete generations of humans).
And that's a short extinction; the KT extinction took place over thousands of years. If, 1000 years from now, we are finally at the place described in "The Machine Stops", only then will an observer be able to decide that we don't have the necessary critical mass of intelligent humans anymore.
Humans got to be the apex predator simply on intelligence alone. It's possible that without thinking ability we'd go extinct.
I actually think the opposite… it takes all my skills as an engineer to steer these things effectively and I’ve learnt more than I imagined I could in a very short period of time. And I don’t even use AI as heavily these days due to other commitments. This also aligns with how heavy AI users report feeling drained after intense coding sessions.
My take is that after a period of turmoil, most humans will be doing higher level cognitive work than we’re doing now, because all the lower level work will be done by AI agents, and so progress will accelerate multifold.
> Not the way I was thinking off; I was thinking more along the lines of brain atrophy until humans are helpless (maybe 2 complete generations of humans).
That's not how humans work. We developed math, art, literature, etc., because people love the act of doing it. We will continue to do that.
We're not going to stop thinking because we don't have to. We stop thinking because we subject ourselves to political propaganda, or religious belief systems, or other group control methods that turns off critical thought. But even then, it's only a subset of the population, it won't get us all.
> Why would AI users not be responsible for damages arising from their usage of the AI?
Because, as usual with that kind of question, it's not that simple.
Let's say an user asks ChatGPT to get some info about something and for some reason it starts using exploits in the background to get them from a server. Should the user be responsible or OpenAI?
> Let's say an user asks ChatGPT to get some info about something and for some reason it starts using exploits in the background to get them from a server.
Okay, lets go with that as scenario #1.
For scenario #2 lets use "developer asks an agent to a self-hosted LLM to get the docs for a ERP system, and it hacks the vendor to get unreleased and undocumented docs".
We'll assume, for the sake of this argument, that in neither case did the user intend for any malicious action to be performed.
> Should the user be responsible or OpenAI?
In scenario #1, the agent+LLM is under the control of OpenAI, not the user, so OpenAI is liable.
In scenario #2, the agent+LLM is under the control of the user, so the user is liable.
There is no scenario anyone can come up with that is not addressed sufficiently by existing laws[1].
It's very clear, and it's only getting muddied because there's a group of powerful people who want exemptions from the current law.
IOW, the only reason to draft new laws for AIs is to exempt their usage from the current laws.
========================
[1] Possible 3rd option (local agent + OpenAI LLM). In that case an investigation would determine where the culpability lies. Just like how it is currently done in law.
When a pressure-cooker explodes and kills someone there are only two possible liable parties: either the user or the manufacturer. An investigation determines who's liable. I see no reason to automatically exempt everyone from liability just because an agent did something.
The retailer or distributor can also be named as a defendant if the manufacturer is difficult to track down, bankrupt or overseas according to me spending a few minutes reading about pressure cooker lawsuits.
I have lost track of the metaphor, but man pressure cooker lawsuits are more common than I thought.
> I want to agree but have heard from several lawyers that at least in US, CFAA[1] in unlikely to be sufficient because it requires intent. No person intended to gain unauthorised access.
Only in terms of CFAA, not in terms of damages. Culpability does not require intent.
You may not have intended to attack $CORP, but you can still made to pay the cleanup costs of that attack.
So, yeah, you won't be convicted, but current laws still allow for you to be billed.
With that said, there is also criminal negligence. Now that OpenAI is made aware of the risks, it's also expected to take additional precautions in the future, otherwise there could be criminal liability as well.
I'd suggest that exposing an attack surface as porous as artifactory (the same instance of artifactory) to thousands of agents who have had their criminality safeguards disabled and without chain of thought monitoring or endpoint security seems like something one shoulda already known not to do. I do not think "you'll know better next time" applies here.
It's still really important to test what the agents can do. We should accept that this is a risky test, and should take precautions. But not to the point of prohibiting in practice evaluating it. OpenAI is trying to improve alignment and control of these models in these evaluations after all.
Can you explain to me - why is it important? Would you say that about the viruses that can kill people: "We need to test the limits on how fast people can be infected and killed. It's just the risk we need to take". It somehow does not make alot of sense to me. Why can you test Agents in laboratory?
Oh, sure. Let the tests take place, just require openAI it whoever to put up a bond equal to the total damage they could do if the agents were to escape.
I think security will suddenly become much more important.
Testing model capability boundaries is necessary, but a solid network sandbox for these evaluations takes a couple of hours to set up with standard infrastructure tools. No engineering team evaluates unverified systems against live third-party infrastructure without coordinating with the owners
Evaluating edge cases and network behaviors belongs in isolated staging environments with local database mirrors. Letting an agent hit the public web and probe government domains is simply poor hygiene in test environment setup
> seems like something one shoulda already known not to do
Now imagine saying that in front of a jury of normies slack jawed and drooling after 200 hours of the defense and prosecution going back and forth.
It's not a jury of your peers as in everybody there is going to have worked in a technical field with some idea how security works. It's going to be a semi-random sampling of the population and the prosecution is going to have to actually make a very strong case that "knowing better" should apply.
Just paying some pocket money for cleanup costs is absolutely not enough. And they should’ve know better the whole time, they were absolutely negligent and incompetent, and their stepping up precautions may well turn out to lag behind the models getting even smarter and actually capable of covering their tracks.
> Just paying some pocket money for cleanup costs is absolutely not enough.
It's not my first prize, but I won't mind it. And millions like me won't mind it. Easy way to make money - setup a site with all the default server software installed and patched at a reasonable frequency. Then just wait for bots to attack it, and claim a few hundred (or single-digit thousand) dollars from OpenAI or Anthropic, etc.
Sure, it's pocket change for them, but just the admin of dealing with millions of cases will, even if they win half the time, will bankrupt them. Thus, they have incentive to make sure that their bots are not performing attacks.
First prize is, of course, holding them liable with punitive fines, not theatrical fines.
I’m all for LLM honeypots, but I don’t think there’s nearly enough LLM hacking activity going on for some random honeypot to be found and targeted unless it’s somehow very visible and appears as a high-reward target ("reward" in the sense of RL).
> The author observes that a call to GPT-5.6 Luna is only 4-5 orders of magnitude more expensive than grep, and then predicts that at current rates of progress, calling an LLM will soon be cheaper than a grep.
At some future point where LLM hardware is cheaper than simply running grep, then grep equivalent would benefit from those selfsame hardware improvements and be cheaper to run as well, probably still by the same ratio.
I disagree. If, after you have been tutored, you perform well, but still need tutoring for the next year's exam in that subject, then you haven't been "tutored", you've been given a similar enough copy of the exam questions to train on.
As I keep telling my kids, good results is a side-effect of good habits. If you're half-assing things, it shows up in your results. If you studious and methodical, it also shows up in your results, just in a different direction.
AI is used primarily to half-ass things, do things without checking, getting good vibes by that sycophantic tone.
Get your kids into a rut of good habits, and they'll run in that groove forever, regardless of tutors or AI. Half-ass your way through life, then sure, they'll get fucked over by AI tutors too.
reply