… I have a local LLM that I run as a primarily a coding assistant, mostly for GDScript.
I’ve never like, submitted anything as a potential commit to Godot proper.
But dear lord, the amount of shennanigans I have had to figure out just to get an LLM to even understand GDScript’s syntax and methods properly is… substantial.
They tend to just default back to using things that work in Python or JS, but… do not work or exist in GDScript.
Like one recurring quirk is they will keep trying to use ? ternary instead of if x else(if) y constructions.
That or they will constantly fuck up trying to custom sorting properly, they’ll either do it syntactically wrong, or, just hallucinate various kinds of set/array methods and properties that don’t exist in GDScript.
And its a genuine stuggle to get them to comprehend more than roughly 750 lines of code at the same time, without confusing themselves.
It is possible to use an LLM to be like, hey, look at this code, help me refactor it to be more modular, or, standardize this kind of logic into a helper function… but you basically have to browbeat them with a custom prompt that tells them to stop doing all these dumb, basic things.
Even if you tell them in conversation " hey you did this wrong, heres how it actually works ", it doesnt matter, keep that conversation going and they will forget it and repeat the mistake… you have to have it contstantly present in the prompt.
The amount of babysitting and constantly telling an LLM the number of errors it is making is quite substantial.
It can be a thing that makes some sense to do in some situations, but it is extremely, extremely far away from ‘Make a game for me in Godot’, or even like ‘Make a third person camera script’.
You have to break things down into much, much more conceptually smaller chunks.
At the risk of drawing the ire of people…
… I have a local LLM that I run as a primarily a coding assistant, mostly for GDScript.
I’ve never like, submitted anything as a potential commit to Godot proper.
But dear lord, the amount of shennanigans I have had to figure out just to get an LLM to even understand GDScript’s syntax and methods properly is… substantial.
They tend to just default back to using things that work in Python or JS, but… do not work or exist in GDScript.
Like one recurring quirk is they will keep trying to use ? ternary instead of if x else(if) y constructions.
That or they will constantly fuck up trying to custom sorting properly, they’ll either do it syntactically wrong, or, just hallucinate various kinds of set/array methods and properties that don’t exist in GDScript.
And its a genuine stuggle to get them to comprehend more than roughly 750 lines of code at the same time, without confusing themselves.
It is possible to use an LLM to be like, hey, look at this code, help me refactor it to be more modular, or, standardize this kind of logic into a helper function… but you basically have to browbeat them with a custom prompt that tells them to stop doing all these dumb, basic things.
Even if you tell them in conversation " hey you did this wrong, heres how it actually works ", it doesnt matter, keep that conversation going and they will forget it and repeat the mistake… you have to have it contstantly present in the prompt.
The amount of babysitting and constantly telling an LLM the number of errors it is making is quite substantial.
It can be a thing that makes some sense to do in some situations, but it is extremely, extremely far away from ‘Make a game for me in Godot’, or even like ‘Make a third person camera script’.
You have to break things down into much, much more conceptually smaller chunks.