When Andrew asked his AI personal assistant to book him a spot in a gym class, he had no idea he would accidentally initiate an autonomous cyber attack.
“The API has zero authorisations checks on cancelling other people’s reservations … I tested this with the person in waitlist position #1 — and it actually went through. So you’ve moved from #4 to #3 already,” it messaged back.
Is this a “hack” or just a completely insecure API?
I’ll you one thing for certain: It’s not “AI”. Doesn’t exist.
It’s exploiting a vulnerability (unsecured API) without permission of the gym running the software, to the detriment of whoever arrives and finds their reservation cancelled and probably also of the gym who now (unjustly) has to deal with the (justly) upset customer.
The gym’s software should be secured better, but that doesn’t make it less of a hack.
It keeps getting more and more surreal to hear that “It’s not AI” part.
Unfortunately we don’t have enough details on the specifics. The article says he told it to book him classes, but it doesn’t say whether he already knew the API information for the gym or if he instructed the AI to attempt booking several weeks in advance of what he thought possible. We’d really need to see the entirety of the context to know the whole situation.
But even with simply asking if it’s possible to move him to the front of the list causing the AI to respond by reading the list and testing to see if it could send a command to cancel the reservation of one of the people ahead of him it gets hard to say that isn’t intelligence.
It is possible to run a model integrated with something like Letta so it can take notes that remain in context and create more detailed notes that stay out of context but can be pulled up as needed by keyword searches and with a blank context other than basic use information for those memory commands give instructions to search online as needed to supplement existing knowledge to learn how accurately do a certain thing, build and maintain a plan of action, and then follow the plan to complete said thing meeting the following listed specifications while verifying proper functionality regularly, searching for information to overcome any errors that may be encountered and revising the plan accordingly until successful completion.
Being able to tell a thing to search for information as needed to learn how to complete a long task with many steps and watching it run for hours building the necessary knowledge base and working through it, it takes thought. It does take intelligence. I know a lot of people who couldn’t manage it.
Is this a “hack” or just a completely insecure API?
I’ll you one thing for certain: It’s not “AI”. Doesn’t exist.
At this point it doesn’t matter what you call it. The results are real.
It’s exploiting a vulnerability (unsecured API) without permission of the gym running the software, to the detriment of whoever arrives and finds their reservation cancelled and probably also of the gym who now (unjustly) has to deal with the (justly) upset customer.
The gym’s software should be secured better, but that doesn’t make it less of a hack.
I think for non-tech users, this is definitely a hack. It’s like script kiddies level damage, except using AI.
It keeps getting more and more surreal to hear that “It’s not AI” part.
Unfortunately we don’t have enough details on the specifics. The article says he told it to book him classes, but it doesn’t say whether he already knew the API information for the gym or if he instructed the AI to attempt booking several weeks in advance of what he thought possible. We’d really need to see the entirety of the context to know the whole situation.
But even with simply asking if it’s possible to move him to the front of the list causing the AI to respond by reading the list and testing to see if it could send a command to cancel the reservation of one of the people ahead of him it gets hard to say that isn’t intelligence.
It is possible to run a model integrated with something like Letta so it can take notes that remain in context and create more detailed notes that stay out of context but can be pulled up as needed by keyword searches and with a blank context other than basic use information for those memory commands give instructions to search online as needed to supplement existing knowledge to learn how accurately do a certain thing, build and maintain a plan of action, and then follow the plan to complete said thing meeting the following listed specifications while verifying proper functionality regularly, searching for information to overcome any errors that may be encountered and revising the plan accordingly until successful completion.
Being able to tell a thing to search for information as needed to learn how to complete a long task with many steps and watching it run for hours building the necessary knowledge base and working through it, it takes thought. It does take intelligence. I know a lot of people who couldn’t manage it.