Chat GPT and other AI guff

Leroy Ambrose said:

cantersaddick said:

cafcpolo said:

Dazzler21 said:

Leroy Ambrose said:

I tell you what, we can have a good old laugh at AI, but the implications of it - in my field at least are pretty terrifying. 

What field are you in, I've also seen it used in my field, Business Analysis but the amount of mistakes makes it completely unworkable. It takes twice the time due to corrections. 

Yup! I've spent the better part of this month reworking code deployed by a data analyst that was clearly knocked up by ChatGPT. Full of bugs and slow as fuck in our data warehouse. Their solution was to spent an extra £10k a month on more resources and licenses. Maybe just learn to code instead 🙄 

We're finding this too. It can't do analysis. Even simple questions it can't get write even when lots has been invested in training it. 

📢 Irony klaxon... 😏

Leroy Ambrose said:

I'll give you a personal example of the current ludicrousness around AI. A lot of this might go over peoples' heads, but you'll get the basic gist of it, which is all that's really important.

Three weeks ago, I get a frantic message from one of the development directors at work on a Friday night. he's a lovely bloke, and intelligent with it. So my natural inclination is to believe him when he tells me that one of the API integrations between his application and part of the security stack that I manage is broken. I spend about 20 minutes troubleshooting it (there are a LOT of things that *could* be wrong, so I need to check all bases). Eventually, I put it down to a client-side issue (which is 'his' side of the equation), as I've literally ruled out anything it could be. He replies back telling me it's not at his end - it's DEFINITELY on 'my' side. So, to humour him, I regenerate the access token that he's using (exact same rights as the current one) and provide it. He replies to say it still isn't working. That's when I twig (after he copies the output from his prompts) that's it's not actually HIM, I'm talking to, but Claude (the AI our developers use for pre-development work, feature design etc). I ask him to send me the prompt history - and he gets a little indignant when I say that the AI is prone to hallucinations, and VERY persuasive about them. Eventually I figure out what's wrong (the AI is asking for the wrong field when attempting to enumerate records). I tell him to change the prompt accordingly, and it, of course, now works. The AI's messages back to him the entire time have been comprehensively certain that the issue was with the 'other side' - and comprehensively wrong. The kicker? Once he changed the prompt and started receiving  results, the response was:

"SUCCESS!! The access issue has been resolved, and I can now see results..."

No mention of the fact that it weas talking absolute bollocks for an hour leading up to this, or an acknowledgement that the information it had provided (gleaned from its LLM's experience of generic API calls, rather than the specific API in question) was wrong.

But here's the real crux of the issue. This is an intelligent man - he's been a software engineer for decades. Done his time on multiple iterations of products, probably been through four or five different dev stacks in his working life, getting to know all of them inside out. He's been working with APIs for the best part of ten years. He knows them better than I know them (my experience tends to be related to the security aspects of them only). Yet because Claude 'told' him that it was right, he didn't even stop to think that one of the basic things you do with an API is get the endpoint and method correct when calling it.

Now imagine that the new iteration of developers doesn't have any of the past context he's got - built up over decades. They're not learning software development from scratch, they're learning vibecoding.

We're fucked.